Loading processes in (new) virtual address space

[hailstorm]

Hi all,

I have been thinking for some time now about a problem. To say it melodramatically: I am facing a wall and I can't see trough it, nor can I turn around. I guess I am developing some kind of tunnel-vision, and I can't get my head straight about this so-called problem.

Ok, here is the situation: until now, I developed a kernel that supports protected mode, using a gdt with 5 entries (null, kernel code, kernel data, user code, user data), all spanning the full memory range. The IDT has been implemented, the pic and pit have been initialized. A made my own version of the buddy system for physical page management (yes, I use paging and this works, that is, my kernel doesn't crash on page allocation and page mapping). Kernel memory (kernel heap) is being managed using a variation on Dougs Lea allocator.

The next thing for me to do is task management. I can grasp anything that has to do with tasks except for one thing: How to start a new f*cking task, or process. Here is how I think about it: as soon as a process makes a system call to startup a new task, a new address space has to be created. That is something I can code, no problem. But somehow I think that the way I create the basic environment for the new task is bothering me now in how to continue.

Here is how I think I should do it (keep in mind that the last pagedir entry always points to itself):
1: Find a free pagedir entry in kernelspace part of the current pagedir
2: Allocate a physical page and put the phys. address in the found entry, this pagetable will be used as the new pagedir of the new process
3: Calculate the virtual address of this new pagedir.
4: Copy at least the kernel pagetables in the new pagedir
5: From this point, I can allocate pagetables for this new process and put them (that is, the physical address) in this new pagedir
And from here, I block totally. What if I want to load the code, data and bss sections of this new process in memory? I can't access the pages behind the allocated pagetables and I guess I can't just overwrite the value in CR3 because I can think of some serious problems this might cause.

Can anybody give me some insights or directions? How did any of you solve this, or, which way did you choose?

Thanks in advance,
Hailstorm!

[frank]

Well what I do is that after I have set up the kernel page directory entries for the new task I set the start location of the new task to Load_Process which is a function that reads the file off of the disk and places it in memory and everything. Note it is a kernel function that is visible in all address spaces. Then I do a retf to userspace at the entry point of the application. This way I don't have to do any temporary page table mappings, just the initial 1 to set up the page directory.

[hailstorm]

Hmmm,

That is something I did not think of and at first sight, it seems a workable solution for my problem. But that would mean that a process switch has to occur first; but I think this would work nicely.
But just out of curiousity, is it also possible to load the process using the current process' address space, without messing up memory to much (I guess not) ?

Frank, thanks for your answer, this will help me coding the task-loading part of my kernel. Though I am wondering why I couldn't make up this solution...

[AJ]

But just out of curiousity, is it also possible to load the process using the current process' address space, without messing up memory to much (I guess not) ?

It is possible, and this is the way my first test kernel did it. The main problem with this is that you then have to save CR3 on every task switch as well as loading it (or prevent all interrupts every time you start a new task). The reason?

Code: Select all

void setupnewtask()
{
    createnewtaskspace();
    switchtonewtaskPD();
                                            //<-----------X
    loadnewtaskexecutable();     
 
    addtasktoschedulelist();
    restorethisPD();
                                            //<-----------Y
    return;
}

Ok - sorry for the odd function names, but it's just pseudocode to give you an idea. Just before X, we switch to the page directory of the new task. Now, supposing your scheduler interrupt fires at any point between X and Y - it will leave the parent task in an uncertain state. The parent task thinks it has switched to the new task memory space in order to initialise the new task. But when this parent task gets re-scheduled, its proper PD will be loaded in to CR3. This is a Bad Thing as it means that it will be loading some of the new task's data in to its own memory space and some in to the intended memory space.

This means that between X and Y, you either need to guarantee no scheduling (something that will, in effect, freeze other processes) or you need to read and save CR3 every time a task is outgoing - you don't need the extra overhead of this on each task switch.

All that leads to what Frank suggested - create the new PD and task, set that task's entry point somewhere in the kernel and set up the new task within its own memory space.

Cheers,
Adam

[sigler]

This means that between X and Y, you either need to guarantee no scheduling (something that will, in effect, freeze other processes) or you need to read and save CR3 every time a task is outgoing - you don't need the extra overhead of this on each task switch.

Why can't you do something like this:

Code: Select all

void setupnewtask() 
{ 
    createnewtaskspace(); 
dword oldcr3 = curTask->cr3;
curTask->cr3 = new_cr3;
    switchtonewtaskPD(); 
                                            //<-----------X 


    loadnewtaskexecutable();      
  
    addtasktoschedulelist(); 
curTask->cr3 = oldcr3;
    restorethisPD(); 
                                            //<-----------Y 
    return; 
} 

Now, no change is required to the task switch code

--
Sigurd Lerstad

[Brendan]

Hi,

The way I do it is:

• - create a new page directory (mapped into kernel space)
  - copy kernel page tables into the new page directory
  - setup scheduler data (thread priority, etc) and set the initial EIP to a "start process" kernel function
  - (optionally) wait or block until the new thread sets some sort of status to return
  - return to caller

After this I've got a new thread/process running in kernel space. When the scheduler gives it a time slice, the kernel sets up "user space" (loads an executable file, etc), and then "returns" to CPL=3. All of this happens from within the new address space though, so there's no hassles with paging, etc.

For error handling, for my OS everything is asynchronous - the spawning thread doesn't block waiting for status from the spawned thread. Just before the kernel "returns" to the new process/thread it sends a status/error message to the first thread. This means the "spawnThread()" function might return "OK" and you might be sent an error (e.g. "failed to load executable file") afterwards.

For synchronous OSs, the "spawnProcess()" function would block until full status can be returned. In theory this is easier for applications programmers to use but worse for performance. For example, a user interface might lock up for several seconds because the thread decided to spawn a process (e.g. where the executable file for the new process happens to be a large file on a slow file system). To avoid this problem application programmers might want to spawn a new thread that spawns the new process; but this makes it harder to use than an asynchronous function, has higher overhead, and most application programmers won't do it (it's either too much hassle, or they simply don't realise the problem is there).


Cheers,

Brendan

[hailstorm]

Thanks for your answer Brendan, the insight in how to handle the loading of the process relatively to the process that started the new process.

Well, now having it all clear, it is time for me to write this load-process routine and fd-driver, dma, fs (fat12 for now), so I can *really* implement my first loading routine.

Pfff. I get tired even by thinking about it...

[AndrewAPrice]

What I do is have a process manager that runs in it's own thread. Usually this thread is sleeping, but if a job gets added to the process manager's queue ("Spawn this process" - that's all it's used for right now), the process manager's thread is sleeping. The process that sent the request usually sleeps until the process manager has completed it's job - but it's possible in a multi threaded application to do other stuff while the process is loading.

When my scheduler switches to one of these special kernel threads (the process manager, the vfs manager, etc) it actually switches to the memory context of the process it is working in. That way, really large requests (e.g. load 20MB from a file) is actually processing in a separate kernel thread with the data being loaded into the process's address space, allowing the process and all other processes to continue doing what ever it wants to do.

This does have a (fairly insignificant) performance penalty. For example, to load a program into memory the system has to do something along the following lines:

- Parent program requests program to be loaded.
-- JUMP TO KERNEL LAND --
- Kernel adds request to process manager's queue. Wake process manager.
-- JUMP TO USER LAND --
- Parent program requests to go to sleep.
-- JUMP INTO KERNEL LAND --
- Remove program from 'awake' list.
-- WAIT FOR CONTEXT SWITCH INTO PROCESS MANAGER --
- Set up new process's memory. Request file from VFS manager. Wake VFS manager. Send process manager to sleep.

begin of loop:
-- WAIT FOR CONTEXT SWITCH INTO VFS MANAGER --
- Request part of the file from the disk driver. Wake disk driver. Send VFS to sleep. (Requests are broken up into 128kb chunks, so multiple read/writes can occur at the same time).
-- WAIT FOR CONTEXT SWITCH INTO DISK DRIVER --
- Load part of file from disk. Wake VFS.
if not fully loaded go back to beginning of loop (VFS will only request parts of the file, e.g. the ELF header first, then the program headers, then the data, etc).

- Wake process manager.
-- WAIT FOR CONTEXT SWITCH INTO PROCESS MANAGER --
- Add new process to scheduling algorithm. Wake parent process. If nothing else to queue, send process manager to sleep.

As you can see, a lot of work is involved. I call my kernel a Polylithic Kernel (some guy made up Stereolithic so I'll make up my own name). It's not a fully monolithic kernel, nor is it a micro-kernel. These special threads aren't really part of the kernel (a la monolithic) and they don't exist in their own address (a la micro-kernel), they sort of just go up to other processes and say "what's up, can I hang with you?"

Oh, and I'm thinking of making my process manager front end like something along the lines of psDoom. Imagine a Sims like scenario - sleeping processes lie lazily in bed, were as busy processes are busy at their computers (maybe something more analogue - a hall of typewriters!). Sleeping processes only go to bed if they haven't been processing for a while, otherwise they will just stop typing for a second, or go get themselves a drink, go to the toilet, etc. You can tell when your system will be over crowded and slow, because your house will be overcrowded and there won't be enough processors for your poor little Proms (Processor Sims) to type on.