How do I setup multitasking for my diy os?

[JustforOSDEV250]

Ive put together a simple OS using c and assembly I have a shell and some other functions set up but I cant seem to figure out multitasking. I saw something about saving the current process and starting another one or something like that but how can I actually use it. Thank you any answers would be appreciated and if I need to supply more info just tell me.

[bzt]

Hi,

How about
Multitasking Systems (about the theory)
Kernel Multitasking (an example implementation in C)

In a nutshell (I'm taking about one CPU core only): multitasking does not execute multiple tasks at once, it just gives you that illusion. It's running one process at any time, then a task switch happens (see below), which saves the process state (all registers and the address space pointer), calls the scheduler to pick a new process, then it loads the state of that new process, and returns control to the new process. From the process' point of view, nothing have happened. For example:
1. process A calls a special function (usually called yield())
2. task switch returns from the yield() call to process B
3. process B then calls again yield()
4. task switch returns from yield() to process A
5. from process A's point of view, it has called yield() which returned; it knows absolutely nothing about process B was running in the meantime
This is called cooperative multitasking, as the process cooperatively has to call yield() to pass control over.
There's another one, pre-emptive multitasking, the only difference is, here yield() is called by the timer interrupt handler, so that processes do not need to actively do anything. It is also common that when a process gets blocked (from reading an empty pipe for example), then the yield() is called implicitly, to allow other processes to run and put something in the blocked process' input buffer for example.

I would suggest to do some digging first: read articles on the topic, and check out how educational OSes have done that (start with simple ones, like Minix or xv6). When you have a clue what's going on and why, then move to more complex examples (like BSD or Linux). Only after these steps should you start your own implementation.

Cheers,
bzt

[JustforOSDEV250]

Another noob question but how do I compile the switch.s file. Sorry.

[Octocontrabass]

You should be able to use your cross-compiler, the same way you compile any .c or .S file in your OS. The file extension is case-sensitive, so you might need to name it "switch.S" and not "switch.s".

[Octacone]

https://wiki.osdev.org/Kernel_Multitasking Is an example on how NOT to write your multitasking code.
Please don't try to follow/copy it, multitasking is something quite crucial and should not be taken from somewhere else.
It is best if you try to understand it yourself.
I would suggest reading this instead: https://wiki.osdev.org/Brendan%27s_Mult ... g_Tutorial It is quite well written and has everything to get you started.
Also pre-emptive multitasking is a way to go, IMHO.
Don't bother with scheduling algorithms yet, just write a simple round robbing scheduler to get you started.
If you find it hard to understand at first, don't worry, this subject is not a small one.
Also for the love of God write you main switching code in Assembly. You can write everything else in C/C++ but register switching and stuff should be done in Assembly, to save yourself (and us) from endless bug hunting.

[JustforOSDEV250]

I looked at all of the suggested articles but none of them are what I am looking for. I want to make my own multitasking system I know what I have to do just not how to do it for example I don't know how to save the current task and determine the next one. Thank you for all the help so far!

[Schol-R-LEA]

Would you mind telling us more about what you are stuck on or confused by? A few things which might help include:

• What is your target hardware (x86 PC; ARM-based single-board computer such as Raspberry Pi; AVR Arduino; some other SBC or microcontroller such as the HiFive or Omega Onion2; something else)?
• Assuming it is a PC, are you running in 16-bit real mode, 16-bit protected mode, 32-bit protected mode, or 64-bit long mode? Conversely, if it is an ARM, MIPS, or RISC-V, what brand and model of SBC are you using, and are you running in 32-bit or 64-bit mode?
• How is your system booting? Did you Roll Your Own Bootloader, or are you using a existing loader such as GRUB, U-Boot, or Boot Magic, or (if on PC or SBC newer than 10 years old) using the UEFI loader directly?
• Again assuming an x86 system: if you are using UEFI and/or GRUB, did you change the Global Descriptor Table, the kernel process's Local Descriptor Table, and Interrupt Descriptor Table, or are you using the settings created by the boot loader? (Other ISAs have similar tables for managing memory protection, but they all handle them differently, and may called them by different names; even in x86 Real Mode, you'd need to set up the Interrupt Vector Table sooner or later.)
• Are you following any specific model of OS design (execution monitor, Monolithic Kernel, Microkernel, Hybrid Kernel, Exokernel)?
• Which method of scheduling do mean to use, cooperative multitasking or (much more likely) preemptive multitasking?
• Assuming preemptive scheduling, do you have an interrupt handler set up for the system timer? (On a PC, this would be the Programmable Interval Timer.)
• Do you have one or more specific Scheduling Algorithms in mind to implement?
• Do you have a plan as to how you will expose System Calls to user processes? Most newer x86 OSes use SYSENTER/SYSEXIT, but there are other ways which may depend on how you are handling the scheduling(and the type of CPU you are targeting).

By the way, have you ready the sticky thread "Links and Advice for New Members" yet? You may find some of your answers in the pages linked there, or at least a place to start on it.

Also, do you have a version control repo which you could share with us? It would help if we could see what you've developed already, so we could have a bit more reference on you need to do next. If you don't have one, I would recommend that you drop everything and set one up, using the VCS and host of your choice. It is hard to overemphasize how important version control can be, and how much trouble it will save in the long run.

[bzt]

I looked at all of the suggested articles but none of them are what I am looking for.

Are you sure?

I don't know how to save the current task

Well, Breandan's Multi-tasking Tutorial is terrible in this regard, as its "switch_to_task" routine does not actually save the state, only partially a few registers. But the other wiki page, Kernel Multitasking demonstrates how to do that in the "switchTask" function.

There are basically two schools here. One is saving the state into a designated area (the wiki page for example, see task->regs), while the other simply pushes everything on top of the process' stack (Brendan's code saves ebx, esi, edi, ebp as such). Both has advantages and disadvantages, but not many, in general they are pretty much the same. Anyway I would recommend to save ALL registers in one place, so that you'll have an exact snapshot of the process' state (useful when dumping).

determine the next one.

As I've said, that's the job of the scheduler. Before the low level task switch code is called, you place the next task in a variable (the wiki page's code receives it in the "to" argument, Brendan's code uses a global "current_task_TCB" variable).

As Octacone suggested, the scheduler can be as simple as a round-rubin picker. Assuming you have an array of processes, then the index would be:

Code: Select all

int numtasks; // number of tasks (processes)
int currtask; // index of the array, current pid

currtask = (currtask + 1) % numtasks; // pick the next one

Obviously this is an extremely minimal version. You have to take into account that processes may terminate, and therefore the array may have empty elements.

Cheers,
bzt

[JustforOSDEV250]

What I don't understand is how to actually save the process and schedule a new one. My OS is targeted towards x86 architecture and I am using 32 bit protected mode I think. I'm using GRUB to boot the OS and I am using a monolithic kernel design. I would prefer to use preemptive multitasking and I don't have a system timer set up although I can probably work on that. I think I want to use Priority-Based Round Robin but it doesn't matter a whole lot and I think ill stick with SYSENTER/SYSEXIT. Sorry for the late reply and thank you for telling me about [wiki]https://forum.osdev.org/viewtopic.php?f=1&t=33787[/wiki]!

[bzt]

What I don't understand is how to actually save the process

You store all registers into memory (including cr3).

and schedule a new one.

You restore registers from memory.

I am using 32 bit protected mode I think.

Hold on, how come that you don't know? That's not something you "think". In OS dev you have to know things for sure. That's why it is the hardest task a programmer can do. It's not for anybody, an average programmer can't cope with it.

Cheers,
bzt

[Schol-R-LEA]

To repeat what I said earlier, do you have a public VCS repo or something similar, where we could view the code? It would make things much easier

Also, are you following any tutorials (aside from the two discussed earlier), and if so, which ones? As the wiki and the forum frequently point out, a 'tutorial' in OS dev has to be viewed as a rough guide, not a step by step process to follow. To be frank, there are simply no complete tutorials which don't have some sort of flaws, and most are at least somewhat outdated as well.

[JustforOSDEV250]

I do have a public repo https://github.com/PersonHumanDevelops/Unosh and Ive been following a couple of tutorials including https://littleosbook.github.io/ and a youtube tutorial witch most of my code is from https://www.youtube.com/watch?v=rr-9w2gITDM its not the best as it only shows keyboard input, printing and etc. Also sorry for the late reply again.

[Schol-R-LEA]

OK, that gives us more to work with, especially since several of the others here are already familiar with that book (I've been meaning to go through it but haven't taken the time, I probably should get to that now).

I'll take a deeper look at it soon, after I've gone over both the book and video and in question and taken a deeper look at your code, but in the meanwhile I can tell you a few things:

• The code does appear to be for 32-bit protected mode, which fits with the book you've mentioned.
• Both the GDT and IDT you currently have are pretty much bare skeletons; they are a place to start, but it looks as if you haven't done much to change them from what the tutorials show. In particular, you currently have all of the interrupt service descriptors (in isr.c) pointing to identical stubs which simply halt the CPU. Populating the interrupt descriptors with suitable handlers will be one of the steps you'll need to take before proceeding to a process scheduler. You'll need to handle all of them eventually, but for now the most significant of these probably are:
  • Programmable Interval Timer. This will be necessary to give the timing for your scheduler, either through a 'tick' (a regular interrupt at a fixed interval) or as scheduled by your kernel (for a 'tickless' scheduling approach).
  • Keyboard. As things stand, your kb.c function readStr() works through polling the hardware buffer, rather than waiting for an interrupt. While this will work for a single-tasking system (poorly, but it will work), it isn't going to be suitable for a preemptive multitasking system - not only does it rely on busy-waiting (i.e., the system is actively waiting on the keyboard input, and can't do anything else in the meanwhile), it also means anything typed in when the system isn't actively waiting will get silently dropped. You'll need to enable the keyboard interrupts after the interrupt handler is in place, as well (the fact that it isn't enabled is the only reason why your kernel isn't halting very time you press a key).
  • General Protection Fault, Stack Fault. These present a bit of a quandary right now, as they usually mean killing the process which raises them - except that until you have a scheduler running, the only process is the kernel itself. For now, you may need to leave them as they are.
  • Page Fault. This generally means you need to shuffle memory around, or change how you have virtual addresses mapped to physical ones. While your kernel will have ample physical memory to use and you won't have any user processes yet, you may be surprised what sort of situations can trigger one of these. This goes hand in hand with setting up a more complete GDT, as well as the LDTs for the individual processes later down the road. You will want at least a basic page fault handler from the outset, though you'll probably need to revamp it later if you add things such as paged virtual memory and shared libraries.
  • Divide by Zero, Invalid Opcode, Integer Overflow, and Out of Bounds exceptions. While it is unlikely that you will need these any time soon, it is worth getting them out of the way once you have a scheduler running, to avoid forgetting to do it later on. In most cases, all it will entail is halting the offending process and removing it from the scheduling queue and schedule a different process. If it occurs in the kernel itself, or you don;t have a scheduler yet, well... in that case, halting the system may be you're only choice until you can figure out a more elaborate error-handling approach.
• The repository as it is now includes the object files (in obj/), the kernel binary (in unosh/boot/) and the bootable image file (in the main directory). For a number of reasons, it is undesirable to include these in the repository itself, if only because it would lead to these binaries getting downloaded when they aren't really needed when working on the source code. I would recommend adding a .gitignore file to the repo root so as to avoid this, one such as this should do nicely for a start:

  Code: Select all

  *.bin
  *.o
  *.iso
  *.bak
  *~
  *.so
  *.a
  

• I would recommend changing the variable declaration for exception_messages[] to extern string exception_messages[32]; to avoid conflicts. TBH, I'm surprised this isn't causing a problem as it is.

[kzinti]

• I would recommend changing the variable declaration for exception_messages[] to extern string exception_messages[32]; to avoid conflicts. TBH, I'm surprised this isn't causing a problem as it is.

What conflicts would that be? Both are equivalent since he is defining exactly 32 entries... I do agree with putting an explicit length of 32 because explicit is better than implicit, but it really makes zero differences in this case.

[Schol-R-LEA]

• I would recommend changing the variable declaration for exception_messages[] to extern string exception_messages[32]; to avoid conflicts. TBH, I'm surprised this isn't causing a problem as it is.

What conflicts would that be? Both are equivalent since he is defining exactly 32 entries... I do agree with putting an explicit length of 32 because explicit is better than implicit, but it really makes zero differences in this case.

Because it is a global variable declaration and definition in two different places in the same source stream for isr.c (once when the header is included, and again in the source file proper). To make it a bit more explicit what I would expect to happen, what the compiler would see when compiling isr.c is:

Code: Select all

string exception_messages[32];

... followed in that same file (as far as the compiler is concerned, since in principle it doesn't know what the pre-processor has already done) by:

Code: Select all

string exception_messages[] = {
    "Division By Zero",
    "Debug",
/* ... */
};

This should by all rights have triggered a duplicate definition error.

Even if that weren't the case, you always want to declare variables and constants in headers with extern anyway; while it wouldn't be a problem if the header is only included in one place, it would become a problem as soon as two different, separately compiled source files are linked, as they both would contain the definition of the variable (not just the declaration, as would be the case for an extern).