OSDev.org

Hello. I'm developing an x86 microkernel with minimalistic API (similar to L4). I decided to use Multiboot 2 modules to load user-space programs in memory because kernel knows nothing about filesystem paths. Those modules will be filesystem itself, process manager, memory manager (my kernel does not even use MMU and activate paging), etc. At startup my kernel have information from multiboot where the modules are placed in memory (physical addresses, I though). I can read ELF headers and jump to entry point after switching to user mode. However, I'm sure that I have to reallocate modules and add GDT entries for their code and data segments. What order shoud I do the things in for success? In particular:
1. Do I have to activate paging in early boot stage? Maybe this can be done from user-space memory manager via kernel API?
2. Do I have to implement multitasking in kernel? Maybe I can run process manager the first and transfer this responsibility to it?
3. Do I have to implement a timer in kernel? (I though yes because even Minix 3 didn't move this to user space).
Maybe, the main question:
4. How should the kernel be aware about memory map and module addresses? You can look at the screenshot to learn what information I have got from Multiboot bootloader. Do I have to reallocate something (maybe even the kernel) or create GDT entries for module physical addresses?

After discovering MINIX 3 source code I've learned something interesting. During it's boot process it calls two functions: architecture-dependent pre_init and architecture-independent kmain. The order of things have surprised me:

1. Paging is initialized in pre_init even before protected-mode descriptor tables are loaded;
2. Then kmain calls prot_init to load descriptor tables;
3. Suddenly, paging is reinitialized again during this. I don't understand what means 'Set up a new post-relocate bootstrap pagetable' in the comment before this code;
4. Then clock (timer?) is initialized;
5. As expected, process management is initialized then;
6. Finally, processes from the boot image are initialized. This is what I want to achieve in my kernel for modules loaded by multiboot, so multiboot will help to avoid creation of boot image, just as planned by the specification creators.

I hope, MINIX 3 source code will help me in my needs. I want to avoid paging right now to simplify the development because I don't fully understand how to do memory mapping with it. Any explanation will be helpful.

I have the timer working already, so it's time to write process management and run multiboot modules as user-space programs. I will just relocate modules in memory if needed.

Still the main question is:
How should the kernel be aware about memory map and module addresses? Do I have to reallocate something (maybe even the kernel)?

I have made some progress in switching to user mode. I have followed the http://wiki.osdev.org/Getting_to_Ring_3 tutorial. I push user-mode code and data selectors to stack, push the address of module loaded by GRUB and do "iret" (code: https://pastebin.com/pS3VyAVU). The module is plain 32-bin binary which makes a few system calls via "int 0x80", which just print the value of %eax to the screen (module: https://pastebin.com/Ps8KgH4R). You can see the line "[INFO] syscall: number 123" at the screenshot to be sure that it works. The modules also tries to call system call "exit()", but it also just prints the line "[WARN] syscall: process try to exit with error code 0, haha". Then the module tries to return doing "ret", however stack is not configured, so general protection fault occurs. GRUB have loaded multiple modules, but I can execute the first only.

Now I have to answer a conceptual question. In UNIX the only way to create a process is a fork() system call. Kernel runs the init process, which then forks itself into all other processes, which also can fork and so on. If init process exits, kernel panics. There is no such a thing as "kernel process" or "kernel thread".

The problem is that I'm trying to implement microkernel, which has no filesystem driver built in, so there are only two ways:
1. Make init process a "god", which has filesystem driver, process manager, memory manager, etc built into it.
2. Tell the init process about other modules and implement an API in kernel to switch to them.

I only accept the second solution. It is fully consistent with the idea of microkernel which only implements IPC (as I said in the first post, I want my kernel to have API similar to L4).

However, I still have to implement something like fork() system call in kernel, and also some system call to replace child's process memory with other module or just jump to it. I'm stuck on this. What I have to implement in kernel to fork the process? I have implemented task state segment, paging, page frame allocator. As I understand, I should copy the kernel's page directory and allocate a stack for each process (code and data are already loaded in memory by GRUB). I haven't found any tutorials or articles about this.

Yes, fork() (or a bunch of system calls that can emulate it) needs to be implemented in the kernel, even for microkernels.

There are multiple ways to implement that. If your kernel has a concept of POSIX processes, you can just directly implement fork() in the kernel. If the process abstraction is provided by a userspace program, that program needs to coordinate with the kernel in order to implement fork().

For example, in my microkernel, processes are managed by a POSIX server that sits in userspace. When a program calls fork(), the C library sends a synchronous message to this server (needs to be synchronous so that the POSIX server can accurately capture the caller's registers and stack; apart from execve(), kill() and a few other exceptions, this is the only message that needs to be synchronous). The server calls the kernel to clone the virtual address space of the calling process. After that, it calls the kernel again to create a new thread inside this address space. The server then sets up all registers of the new thread and unblocks the caller. In order for this mechansim to work correctly, programs have to tell the kernel how memory regions should behave on fork. Thus, my memory mapping system call takes an argument that determines if the kernel should copy-on-write, share or drop the mapping on fork().

Note that it would also be possible to emulate fork() by some "map memory into other address space" system call. There are some relatively complex design decision why I didn't do that. Basically, I don't what to make the concept of "memory mapping inside an address space" visible to user space in order to prevent user space from doing stupid things (like creating unbounded chains of copy-on-write dependencies). Let me know if I should discuss that in detail.

Korona wrote:The server calls the kernel to clone the virtual address space of the calling process.

As I understand, this creates a copy of process page directory? I am right?

Korona wrote:After that, it calls the kernel again to create a new thread inside this address space.

I completele do not understand what is "to create a new thread" in terms of x86.

Korona wrote:There are multiple ways to implement that. If your kernel has a concept of POSIX processes, you can just directly implement fork() in the kernel. If the process abstraction is provided by a userspace program, that program needs to coordinate with the kernel in order to implement fork().

I prefer user-space solution. I want kernel to be just a message queue. It will have concept of actors, which user-space process manager can build concept of processes on top of. However, for the beginning I may implement fork in kernel.

Korona wrote:When a program calls fork(), the C library sends a synchronous message to this server (needs to be synchronous so that the POSIX server can accurately capture the caller's registers and stack; apart from execve(), kill() and a few other exceptions, this is the only message that needs to be synchronous).

For the beginning I want all messages to be synchronous. Also I want to avoid scheduling. So the process is executed any long until it makes a system call, which is a synchronous receive or send_and_receive. This is the way I want to implement multitasking for now. You may say that this means no multitasking at all, doesn't matter. This model requires just to allocate separate stack for each process.

So now my main problem is that I don't understand how to set up stack before iret in syscall handler, so it's a low-level x86 trouble. Kernel doesn't know about processes. There is no data structures to take information about task state from.

I may admit that all tasks are trusted, so the only solution I see is to use stack to take information from it.

BraidenVasco wrote:As I understand, this creates a copy of process page directory? I am right?

Yes.

BraidenVasco wrote:I completele do not understand what is "to create a new thread" in terms of x86.

The kernel should have some struct to store a thread's state. This should include the registers and stack of the thread while it is suspended. Creating a thread more or less means creating an object of this struct.

BraidenVasco wrote:So now my main problem is that I don't understand how to set up stack before iret in syscall handler, so it's a low-level x86 trouble. Kernel doesn't know about processes. There is no data structures to take information about task state from.

Saving and restoring the registers is fairly simple. I don't have the time to write about that, but there should be plenty of information in the Wiki and in this forum.

I have a problem with paging again. I have kernel page directory which is a directory of 4MB pages with identity mapping. Let's say I want to clone it. Where can I place a new page directory? I can allocate 4MB page for it, but isn't it too big for 1KB data structure? I can allocate 4KB page for it, but I have no code to work with page tables. Should I write it? Where can I find the examples of all-purpose page directory manipualtion code?

I can't write it myself because I can't write tests. How can I write unit tests for architecture-dependent code, if tests run on host machine? Code which is not tested doesn't work at all, this is an axiom of test driven development. Previous things I've tested by printing messages on screen, this approach doesn't work anymore.

Don't forget that you need to allocate physical memory for page tables. So just choose a free 4K block and mark it as used so that your virtual memory allocator won't try to use it.