Multitasking in real mode?

[Prochamber]

Hey,
I'm planning to develop a multitasking system for my hobby operating system TachyonOS.
https://code.google.com/p/tachyon-os/

What is the best way to implement multitasking in real mode?

Notes:

• My operating system runs in real mode.
• I can use instructions for 386+ CPU.
• I do not want to switch to protected mode, 'nuff said.
• I already have basic extended memory management.
• I want to keep binary ORG points.
• I want to keep programs separate from each other.
• I don't want to over-complicate programs.

Would using unreal mode or segments or copying code between extended memory help? I'm read the OSDev Wiki but it mostly covers protected mode and doesn't have many examples, so I'm a little confused.

Thanks in advanced for any help you can provide!

[brain]

Well, if all your programs were bytecode, e.g. .NET CLR or Java or pascal P-Code, then you could virtualise each process and get around the segmented ugliness of realmode, without switching to protected mode but still giving a virtual environment to each process, equal to or potentially better than x86 protected mode, depending on design.

I assume you have a really good reason for not switching into protected mode so i won't evangelise that.

On the downside, a bytecode system will be noticeably slower unless you use JIT compilation.

What are your thoughts on this?

[Prochamber]

Hmmm...
That's an interesting idea, certainly not one I had thought of.
I already have quite a sophisticated BASIC interpreter, perhaps if I could create some kind of BASIC 'byte code'?
I could modify my interpreter to accept values instead of commands.

[brain]

Hmmm...
That's an interesting idea, certainly not one I had thought of.
I already have quite a sophisticated BASIC interpreter, perhaps if I could create some kind of BASIC 'byte code'?
I could modify my interpreter to accept values instead of commands.

I also have quite a sophisticated BASIC interpreter and am creating something much like i outlined earlier just without bytecode, with direct interpretation. Perhaps an exchange of code might be in order

[AJ]

I do not want to switch to protected mode, 'nuff said.

Would you consider using v8086, even though that would mean some of your kernel in protected mode but maintaining a real mode userspace? After all, you do seem willing to consider unreal mode...

Cheers,
Adam

[Prochamber]

Tell me more about 'Virtual 8086 mode'.

• Where can I find some good examples of this mode?
• Do I have to rewrite all my real mode interrupt handlers?
• Will userspace code be unaffected? I have a lot of binary and source compatibility to worry about.
• Will user programs need to be recompiled?
• Which parts of my kernel need to be in protected mode?

Thanks!

[bluemoon]

Where can I find some good examples of this mode?

Manuals & Google.

Do I have to rewrite all my real mode interrupt handlers?

You need to install protected mode interrupt handlers, but then you may delegate most of the work back to the original handler by switching to v86 mode.

Will userspace code be unaffected? I have a lot of binary and source compatibility to worry about.

If the userspace application does not use any dirty hack, it should not be affected. This is the whole idea of v86 mode - to emulate real mode.

Will user programs need to be recompiled?

It depends on how your program loader works and how the application interface with the kernel, usually it is not required.

Which parts of my kernel need to be in protected mode?

Everything except those you shifted/delegate out to other mode. Note that some of them are required by having the cpu to run protected mode itself - like interrupt and exception handlers; and some of the kernel feature may be easier to do with protected mode - like memory management.

[Brendan]

Hi,

I assume you have a really good reason for not switching into protected mode so i won't evangelise that.

This is a false assumption. There is only one good reason for not switching into protected mode - it's called long mode.

Sadly, lazy beginners that don't know better see the BIOS as a way of avoiding the need to write device drivers; and use this as an excuse to waste years developing a piece of crud, while failing to learn anything about developing a real OS. It's unethical not to (attempt to) prevent these beginners from heading down this dead-end path; especially now that the entire industry is shifting to UEFI.

Tell me more about 'Virtual 8086 mode'.

• Where can I find some good examples of this mode?

You can find examples in the 1980's (a lot of DOS extenders used it because DOS was a lame piece of puke). All the application developers abandoned real mode at the first opportunity they got (e.g. Windows 95) so you probably won't find any good virtual8086 code written in the last 2 decades.

• Do I have to rewrite all my real mode interrupt handlers?

No, your real mode interrupt handlers can continue being lame pieces of puke.

• Will userspace code be unaffected? I have a lot of binary and source compatibility to worry about.
• Will user programs need to be recompiled?

No, all your user-space code can also continue being lame pieces of puke too.

• Which parts of my kernel need to be in protected mode?

All of code that aren't lame pieces of puke would need to be in protected mode.


Note: I'm sorry if this post seems harsh; but I'm getting fed up with necrophiliacs (people who seem to be obsessed with things that died a long time ago, like real mode).


Cheers,

Brendan

[Mikemk]

Note: I'm sorry if this post seems harsh; but I'm getting fed up with necrophiliacs (people who seem to be obsessed with things that died a long time ago, like real mode).

You had me fooled - I thought you had a cold.

[Prochamber]

Hi,

I assume you have a really good reason for not switching into protected mode so i won't evangelise that.

This is a false assumption. There is only one good reason for not switching into protected mode - it's called long mode.

Sadly, lazy beginners that don't know better see the BIOS as a way of avoiding the need to write device drivers; and use this as an excuse to waste years developing a piece of crud, while failing to learn anything about developing a real OS. It's unethical not to (attempt to) prevent these beginners from heading down this dead-end path; especially now that the entire industry is shifting to UEFI.

Well, I wasn't really expecting any insults. Firstly THERE IS NOTHING WRONG WITH BIOS!
Is it so wrong that I want to get my operating system off the ground without worrying about writing low level drivers for everything? If I had started writing in protected mode I would not have been able to start at all. I know NOW how to write device driver, I didn't when I started. I have written drivers for some devices but I will slowly replace BIOS drivers in a way that they remain compatible this takes times and will not happen overnight.
Also, I'm pretty such UEFI will still support legacy BIOS so it is not a "dead-end path". Even if there is not BIOS support I can still run in an emulator or write my own imitations of BIOS drivers.

Tell me more about 'Virtual 8086 mode'.

• Where can I find some good examples of this mode?

You can find examples in the 1980's (a lot of DOS extenders used it because DOS was a lame piece of puke). All the application developers abandoned real mode at the first opportunity they got (e.g. Windows 95) so you probably won't find any good virtual8086 code written in the last 2 decades.

• Do I have to rewrite all my real mode interrupt handlers?

No, your real mode interrupt handlers can continue being lame pieces of puke.

• Will userspace code be unaffected? I have a lot of binary and source compatibility to worry about.
• Will user programs need to be recompiled?

No, all your user-space code can also continue being lame pieces of puke too.

I can't read any more of this because you persist in insulting everything I have created. I know what protected mode is and I have written code for it. I am not a beginner and yes, I understand what I am getting into. Why don't you and the rest of your protected mode elitists go off to some other forum where you can write your "real operating systems" and leave the rest of us alone.
@Bluemoon, @Brain, @AJ Thanks for your help.

[M-Saunders]

Hi!

Sadly, lazy beginners that don't know better see the BIOS as a way of avoiding the need to write device drivers

I think this is a bit harsh and unfair. Sure, there are some people who pop in here and claim that they're writing a "Windows killer", and it really turns out to be merely a glorified real-mode bootloader. We see this all the time. And sure, it's ludicrous to even consider writing a proper, serious, mass-market OS in 16-bit real mode assembly today.

But BIOS-based OSes have their uses. For budding OS developers, being able to spend time on interesting things, like implementing system calls, loading programs and writing interpreters, rather than poking bytes into the PS/2 controller, is attractive.

You could argue that doing these things in real mode asm teaches you very little, because the same techniques aren't used in advanced OSes. But I don't think that's the issue -- for instance, learning old-school C is still good, even if, later on, you use a much broader feature set in C++/C# etc. programs. Lots of game coders started out hacking asm on 8-bit platforms, and while those skills may not be directly relevant today, they still learnt a lot from the experience.

I've heard all the arguments, and I agree with many of them. The BIOS is dead, it's old, it's crummy, it has bugs, and nobody should use it. But while it's still here, it provides a base for writing a very simple OS -- getting people on the road to "serious" OS development. When I started, I was happy that I could get the BIOS to do the "boring" stuff while I experimented.

Six years later, with over 50,000 downloads of my OS, and hundreds of emails from people grateful that I got them into OS development, I don't regret anything. So let's be fair. Real mode isn't sensible for modern OS development, but its handful of benefits can still bring lots of potential OS developers.

And Brendan, Prochamber is a very smart guy. He has done a lot of work on the MikeOS BASIC interpreter, system calls, and added a mouse driver etc. He's made a lot of stuff that people actually find useful and interesting to learn from, so whether or not that's in real mode is, for me, immaterial. He might be working with old-school tech, but he's actually getting people interested in OS development, and that's to be applauded I reckon.

Mike

[DavidCooper]

If we're all forbidden from using the BIOS without being branded as fools, how many of the users of this site would bother to build anything? It isn't just the BIOS: you can't build a "proper" OS without getting on top of all the USB stuff either, and that's such a mountain to climb that most people would simply give up the whole idea of starting to build anything. As soon as you start tinkering with USB it's likely to disable your use of the BIOS in various ways, so it's a nasty barrier to progress, but it's a lot easier to contemplate taking it on once you've already built a BIOS-dependent OS rather than starting out with nothing, and it's also the case that the bulk of the OS you have already written will (if you've designed it carefully) still be used after you've freed yourself from the BIOS.

The BIOS is sitting there on a billion machines which will still be usable for the next decade or more, so there is nothing wrong with using it as a stepping stone. I can use the BIOS-dependent OS I've already built to make it much easier to build all the functionality that will ultimately free me from using the BIOS, and indeed I have already built an AGI system (still not complete, but it's nearly there) on top of my BIOS-dependent OS which will at some point be able to help write the rest of the OS. Deadlines come and go, but they were always just hopes and guesses: things take how long they take and you can't take shortcuts, but my health problems are receding now (the trick is to program while using a treadmill instead of sitting in a chair - chairs are lethal bastards which seriously will try to kill you) and I'm gradually finding that I'm able to work longer and longer hours again, so I'm confident that by the time Brendan has finished the build of his OS, mine will be able to wipe the floor with it in every imaginable way, quite in addition to overthrowing all governments and taking over the running of the world. Who is going to look like a fool then? Someone who has manually written dozens of device drivers which will soon become obsolete as the continual deluge of the new goes on replacing the old? No: it's a race that can't be won by anything other than large companies or AGI.

[Brendan]

Hi,

I assume you have a really good reason for not switching into protected mode so i won't evangelise that.

This is a false assumption. There is only one good reason for not switching into protected mode - it's called long mode.

Sadly, lazy beginners that don't know better see the BIOS as a way of avoiding the need to write device drivers; and use this as an excuse to waste years developing a piece of crud, while failing to learn anything about developing a real OS. It's unethical not to (attempt to) prevent these beginners from heading down this dead-end path; especially now that the entire industry is shifting to UEFI.

Well, I wasn't really expecting any insults. Firstly THERE IS NOTHING WRONG WITH BIOS!

Wrong. Hardware is capable of reading from disk while reading from floppy while sending/receiving packets on the network while playing audio while blitting data to video while transferring data from web cam; while all of the CPUs are still free to get real work done. As soon as you use BIOS for any of this it turns into a "one piece of hardware at a time" joke.

Worse, when you finally do get around to writing a real OS it makes it hard because you've designed everything around "land of puss" and don't have the framework needed for anything decent. For example, you might be tempted to write a decent ethernet driver, but you can't use MSI like you should because that'd break everything, and don't have decent power management because the BIOS sucked and couldn't use it anyway, and don't have reasonable IO priorities because the BIOS is too crappy to use them, etc.

Even more worse; the only "drivers" you actually gain by turning your OS into a crippled mess are things like SATA/PATA controllers that have standardised interfaces; where writing a small number of drivers covers a large range of hardware. Things like ethernet and sound and GPUs (where you'd need to write a large number of drivers for a small range of hardware) aren't supported by the BIOS. Basically you only "gain" (if you can call it that) where it's doesn't matter much and you don't gain anything where it does matter.

Is it so wrong that I want to get my operating system off the ground without worrying about writing low level drivers for everything?

Yes, it is wrong. You're not getting your OS off the ground, you're digging a hole to bury the OS in.

If I had started writing in protected mode I would not have been able to start at all.

You wasted time doing things wrong, and then started learning how to do things right. You could've of skipped "doing things wrong" to save time.

I know NOW how to write device driver, I didn't when I started. I have written drivers for some devices but I will slowly replace BIOS drivers in a way that they remain compatible this takes times and will not happen overnight.

How does "replace BIOS drivers in a way that they remain compatible" help? You want drivers to be good, not "compatible with puss".

Let's build a car. We'll start with a dead horse, and slowly replace pieces one at a time without breaking compatibility. You start designing a pump to replace the mouldy old heart, and I'll start trying to design some sort of filter to use instead of the maggot infested kidneys. This will be the best car ever!

Also, I'm pretty such UEFI will still support legacy BIOS so it is not a "dead-end path". Even if there is not BIOS support I can still run in an emulator or write my own imitations of BIOS drivers.

Now that Microsoft requires UEFI (for their hardware compatibility logo thing), the only excuse motherboard manufacturers have to bother with BIOS is old OSs. This mostly means Windows XP, which is probably the only OS that consumers still care about that doesn't support UEFI. We may have less than 5 years before BIOS isn't supported on new hardware.

Running inside an emulator sounds like a dead-end path to me. Writing your own "BIOS imitation drivers" doesn't make sense (it'd be easier to write native drivers instead).

I can't read any more of this because you persist in insulting everything I have created. I know what protected mode is and I have written code for it. I am not a beginner and yes, I understand what I am getting into. Why don't you and the rest of your protected mode elitists go off to some other forum where you can write your "real operating systems" and leave the rest of us alone.

At the moment, you may hate me for insulting your work. That's fine (I really don't mind). Just don't forget to thank me later when you realise I was right; even if "later" means "several years of mistakes that you eventually regret because you didn't want to listen now".


Cheers,

Brendan

[Brendan]

Hi,

Six years later, with over 50,000 downloads of my OS, and hundreds of emails from people grateful that I got them into OS development, I don't regret anything. So let's be fair. Real mode isn't sensible for modern OS development, but its handful of benefits can still bring lots of potential OS developers.

Are you helping these people, or providing a distraction that postpones any meaningful progress while giving them false hope?

The shortest path between 2 points is a direct line; and doesn't begin with a pointless detour in the wrong direction.

If we're all forbidden from using the BIOS without being branded as fools, how many of the users of this site would bother to build anything?

People with enough enthusiasm to get anywhere will build something.

I can use the BIOS-dependent OS I've already built to make it much easier to build all the functionality that will ultimately free me from using the BIOS, ...

It's been 2 and half years and you're still dreaming about being "ultimately free from using the BIOS"? You should've been free from the BIOS 2 years ago.

Of course your case is special - you're writing an intelligent hex editor that's going to develop an OS by itself.


Cheers,

Brendan

[brain]

I started out on writing a real mode OS many years ago (read as 13 years ago). At the time this served my need to get something to work, and to get straight into writing a shell, and loading programs etc.

The reason i abandoned this was not because the BIOS was crap, the OS i was writing was only like DOS anyway. The reason i abandoned it was because i started the project in assembler, and did not see myself writing hundreds of thousands of lines of assembler just to level the playing field with windows 3.1.

What i am trying to say here is, use the right tools for the job. If it suits you to use the BIOS to prototype your OS, then sure, do so, it will allow you to produce a prototype faster in most cases. Just remember that a prototype is built to throw away, don't waste too much time on it, and when you've proven you can do what you need to do, move on to the proper design. In my case, a proper design came over a decade later and was in long mode, which didnt even exist back then in 2000.

The OP is in this case maintaining and extending a fork of an existing OS (MikeOS) which was written in assembler and in real mode. These are perfectly valid points to me for not wanting to switch to protected mode. For such an OS, switcing to protected mode means essentially a huge rewrite of the OS, in which case, why fork an OS as it will take just as much work to make one from scratch? On that, i rest my case.

For some, it is also the journey that matters and not the destination. If it takes someone ten years to write an OS they could have written in two years, and they make several mistakes along the way, release several broken prototypes and have several false starts, what does that matter to us if they enjoy the process and it is their hobby? After all, aren't we writing hobby OSes? If on the other hand they don't enjoy it, they have the wrong hobby anyway, and should go back to knitting, or something.

That's my four cents worth. (0.02 plus current rate of inflation)...