starting threading, no more threads, other questions...

[Joshw]

Hello OSDev-ers!

OK, I've been without internet forever, so my OS has been progressing slowly with whatever I can do. I'm really desperate to know exactly what you guys think, I want to know the best way to do some things. I will start by telling you how far I am. I've made my boot sector and boot loader; It loads the kernel from a dynamic elf, so it knows its symbols, to 0xC0000000, using paging. My kernel has heap management, text video, and other small things. I've started to make sure all of the system calls and everything are thread safe. I can switch contexts; so far, each process has only one context, but I'm planning on making it so that each process has at least one thread. So I will start on my questions, they are kind of messy; These are the questions which I have been waiting to be answered and could not answer myself, I'm desperate! So here's what comes off of the top of my head:

1. Scheduling - Right now, the loader jumps to the kernel, so there is only one thread. The scheduler starts running after one process is running, and at least one process must be running. If there are no more threads, there is no context to switch to. And what if I don't want to start any threads before I run the scheduler? What if all threads are waiting for I/O? I can't think of anything else other than having a dummy low-priority thread or something. Any better ideas?

2. Timing - What kind of timing would you suggest for shceduling? Now, I have a big problem in Bochs with a high PIT frequency, so I don't know if there is a frequency that I should shoot for... is 15ms good enough for a timeslice? What about performance applications? How is this good enough, how does it work well on Windows?

3. Process space - I am not totally sure if my address space arrangement is the best way to do it, if you have any better ideas, or have seen better ways of organizing it, please tell me:
(This gives 2.5GB to the process. Do you think I should limit it to 2GB for safety, i.e. if someone uses a signed pointer?)
-- PROCESS SPACE
0x00000000-0x9FFFFFFF Unused by system, available for process
0xA0000000-0xAFFFFFFF Dynamic libraries
0xB0000000-0xBFFFFFFF thread stacks, IPC shared memory, disk buffers, etc.
-- KERNEL SPACE
0xC0000000-0xCFFFFFFF Kernel image and physical memory map
0xD0000000-0xDFFFFFFF Kernel heap
0xE0000000-0xEFFFFFFF Reserved for drivers
0xF0000000-0xFFFFFFFF Default kernel stack, various mappings (video memory, etc.), page tables

ALSO - The current method I came up with to synchronize the kernel space between processes is this:
When a process is created, copy the kernel PDEs into the process' PDT.
If another process adds a PDE to the kernel, it is not yet mapped into existing processes,
but if another process tries to access that PDE, it gets refreshed by the page fault handler.
PDEs are not allowed to be UNMAPPED, because it would not be unmapped in the other processes --
PDEs in other processes (in kernel space above 0xCxx...) only get updated when ADDING PDEs, not when removing them.
Is this the best way to do this? I have though about using Global pages but had decided not to for processor compatability... but I don't really think that should be an issue.

4. C library - If I implement newlib, I give a system call like sbrk() that malloc will use. Will this limit the way that a process allocates memory, I mean, will all libraries have to use malloc to allocate memory? If not, for example, a different allocator uses sbrk, then the two heaps are really using the same memory area for the heap, which is not good (_end - brk).

5. Dynamic unloading... I have basically up until now thought that I should probably not suppory dynamic unloading of drivers and libraries. Do any operating systems do this? It would probably be difficult to do right... Should I be thinking about supporting this? If I were, what would you suggest, buddy blocks?

ANYWAY I really feel like I need some input and that I may not be doing well. I'm sorry for the long post, and of any of the questions seeming stupid (or like it's not a question). I really appreciate your input and help. -Josh

[ru2aqare]

1. Scheduling - Right now, the loader jumps to the kernel, so there is only one thread. The scheduler starts running after one process is running, and at least one process must be running. If there are no more threads, there is no context to switch to. And what if I don't want to start any threads before I run the scheduler? What if all threads are waiting for I/O? I can't think of anything else other than having a dummy low-priority thread or something. Any better ideas?

I believe that almost all OSes implement the same idea (at least Windows does). The idle thread clears unused memory pages and conserves energy by halting the processor (hlt instruction or putting it into some low-power mode).

[Joshw]

Thanks for the quick reply! Yes, it's called the zero-page thread in windows, I've seen it mentioned before in the Windows SDK. It just seems kind of like a quick hack to me. That's why there was a doubt, I thought there might be a better way. So anybody got anything else? It's nice of you all to welcome me into this forum.

[bewing]

1. The Idle Task is so common that I can capitalize it's name, and everyone knows what I mean. I figure that there will be so many low priority daemons waiting to defrag the disk, test hardware, look for SETI signals, etc -- that my Idle Task will never run -- but I still have one. It's a HLT statement, in a loop, built right into my scheduler code. The scheduler spawns it as the very first "job" on the system.

2. I'm shooting for an OS that has some millisecond resolution realtime features -- so I figure I'm going to try to shoot for a 50 microsecond timeslice. My OS is built around the concept of threads not using complete timeslices/sleeping/blocking, all the time. But, as I understand it, most OSes aim for something like a 10 millisecond timeslice. I think it depends on how many threads you expect to have running concurrently.

3. Just so long as the concept flows nicely when you expand to bigger address spaces (64 bit!) -- anything goes, I'd think.

4. I haven't gotten that far, yet.

5. If you expect uptime of a year or more (which seems like a reasonable number), I'd think some sort of library garbage collection would be needed over that timeframe. I wouldn't think that Load-On-Demand would be too hard to implement. If the library/whatever gets forcibly unloaded, and somebody needs it again -- they can reload it again.

[Joshw]

I thought I just replied but it doesn't show, so I guess I'll write it again...

1. Just curious, when you say "built into the scheduler code," do you mean that the scheduler code creates the idle task? Because if it were really handled in the interrupt, what if you just erased the context of the last task on the system? What context is there to run the idle task? Unless you write something like this...

Code: Select all

a:
idle = false;
if (next_thread)
    switch_to_thread(next_tread);
else
{
    idle = true;
    cr3 = idle_cr3;
}
if (previous_thread_killme)
    erase_context(previous_task)
if (idle) { while (!next_thread) hlt(); goto a; }

2. I'm scared to too high a frequency because bochs gives me a spontaneous GPF with error code 0x13B.

3. I guess I was mainly asking this because of question 5.

4. Me neither, I just want to make sure I have the idea straight for design!

5. What about address space fragmentation?

EDIT: Oh, and when I tried to reply earlier, I said thanks for the reply, you're really helpful!

[piranha]

If you create a task structure containing all the task information, then on each interrupt the task struct is loaded, and the info is set.

The Idle Task (in my kernel) is part of all the tasks, and is initialized with the scheduler_init() function. So, yes the idle task is pretty much build in. The scheduler sets up the idle task on init.

For me, the idle task is what is running from the start. When the scheduler starts, the EIP of the idle task is set as the current EIP, and then the rest of the kernel init continues. Once interrupts start, the scheduler switches to the idle task and to nothing else (because no other task is running). After the init process is forked, the idle (kernel) task comes to a:

Code: Select all

while(1)
{
      switch_task(); //This can be replaced by a hlt(); or idle processing
}

-JL

[Joshw]

OK so question 1 makes alot more sense