Microkernel Memory Management

[FlashBurn]

I write on a microkernel and as I wanted to design/write my memory management system, I went into several problems.

This is the memory map which looks the same for every process:

Code: Select all

0GiB-1GiB -> kernel space
1Gib-4GiB -> app space

This is the kernel space:

Code: Select all

0x3fc00000 -> PTE
0x3fbff000 -> PDE

1st, I want that the kernel uses paging, too. I mean normaly I would have mapped the memory for the kernel 1:1 to the physical memory.

Now I have the problem that I cannot access every PD with the addresses only. So I map the PDE and the PTE into a defined region of the memory (as shown above). So that I can access the PDEs and the PTEs of the actual process.

But this lead me into another problem. When I add a page for the kernel I have to update every PD of every process and this will be very slow So, I thought that when I use the same PDEs for the 1st GiB of every process that I needn?t to update when I alloc a new page for the kernel. But if I then need to alloc a new PT, then I have to update the PDs of every process

I have to admit that I don?t know every thing of a microkernel. But some put the MM into the kernel and some doesn?t. I put it into the kernel. But this means that an app has to run some ring0 code, if it wants to get some memory. Correct me if I?m wrong, but if you have a microkernel, the only thing that an app will ever do in ring0 is sending and receiving messages?!

My problems in a short:

How to alloc a new page w/o the need that the app has to run ring0 code?
How to solve the problem that I have to update the PDs of every process if I need to alloc a new PT for the kernel?

[Pype.Clicker]

one thing that could simplify your design is that each Page Directory could point to the *same* page tables for the kernel region. This way, when you want to give a new page to the kernel, it automatically appears in all the processes.

(oops you suggested that already). For the 'if i allocate a new page table for the microkernel' thing, i suggest you allocate all the kernel page tables right from the start ... This will make a grand total of 256 tables for system management.

Now the question is do you really need 1GB of kernel-level system-wide memory ? can't you work with less (say, 16MB) of such system-wide memory and have kernel-level per-process memory for some stuff ?

[Candy]

My problems in a short:

How to alloc a new page w/o the need that the app has to run ring0 code?
How to solve the problem that I have to update the PDs of every process if I need to alloc a new PT for the kernel?

My favorite solution that Bochs doesn't like: take 1 or 2 exact GB's for the kernel, and use PAE. That way, each GB has its own PDT, and you can just modify that one PDT. The PDPT has those one or two PDs permanently mapped, for each process the same.

You don't allocate pages without switching to ring0. You might keep a userlevel backlog that some userlevel program might help you with, but if you are going to map them, you either:

- Give up all system security (if you can write to the page table, who can stop you?)
- Or switch to ring0

[FlashBurn]

As I finished posting I also had the idea that I could write the 256 PDEs from the start, but his would wast ca. 1MiB of memory, but I?m playing with this idea - What is 1MiB!? - The solution that I only reserve e.g. 16MiB is not bad, but what is if you have many processes with many threads and many messages? If this comes true the 16MiB wouldn?t be enough very long!

[Pype.Clicker]

How to alloc a new page w/o the need that the app has to run ring0 code?

This cannot be done. Even if your process has a list of 'pre-allocated' frames and that it has a restricted access to a portion of a pagetable (through a segment, for instance), what would prevent it to put garbage or intentionnally random frame values in the page.

What you can do is allocate *virtual addresses* at the user level. You can also try to prepare more pages than what is actually required and reclaim unused pages when the system gets short on physical memory. If you allocate 256KB of stack instead of 4KB when the stack needs expansion, you have 64 less system calls.

Lazy-frees can also be used: you do not return memory to the OS as soon as the APP doesn't need it but instead try hard to re-allocate blocks that have just been freed.

[FlashBurn]

There is a way that an app cann alloc some memory without running ring0 code (except sending and receiving). It sends a msg and a thread which has access to the pd of the process then maps the page into the address space. But this would be to slow. I think I will use the idea where I use the 256 PDEs from the start.

[Pype.Clicker]

you'll only gain with that send/receive policy if you have a processor that is continuosly polling the memory manager queue while other processors do useful job. Sending a message to a process typically incurs more overhead than doing a system call.

I believe that even in a microkernel system, the page mapping control should be *within* the microkernel. At least give the ?K the opportunity to pick pages from a 'cache list' that some external process can fill and to map those pages somewhere in the current process's address space.

[BI lazy]

I second that. It is not as if the *whole* memory management system needs to be in Kernel space, but the Low Level Stuff (physical page management, paging stuff & process page table creation) has to be there - for the "pager" should be able to slip into process adress spaces.

You can of course put the whole process adress space management into kernel land too and have an easy life - but we don't want to have an easy life, do we? *gg*

[Therx]

although PAE is pretty useless for getting 64gb of ram as no hardware supports it. It could be useful for managing the virtual memory space. As it adds an extra table above Page Dirs and changes page dirs to mapping 1gb each. This way you can only interchange one at a time or what ever. I recommend you read up on it in the Intel manual 3

Pete

[Candy]

although PAE is pretty useless for getting 64gb of ram as no hardware supports it. It could be useful for managing the virtual memory space. As it adds an extra table above Page Dirs and changes page dirs to mapping 1gb each. This way you can only interchange one at a time or what ever. I recommend you read up on it in the Intel manual 3

Pete

That's not the most useful part I've seen. The best use I could think of was:

Map the kernel from 2GB-3GB. Make it global (PGE) and add that page directory in all process spaces. Put the page table/directory in the top 8M of space. Put the other thread-related/process-related stuff in pages and stuff those in the top GB too. Stuff some stacks & stuff along with that.

Now create process spaces each with that 1GB from 2-3 shared, and from 3-4 for their own. Now the kernel doesn't miss caches for no reason (processes that are kernel-bound are really going to speed up, such as graphical stuff & interactive stuff) and you can modify all the process related stuff in the top GB.

Each process has 2GB left for own good, not counting stack (yes, stacks are now free concerning its address space), it greatly simplifies stack allocation & process allocation and it simplifies mapping kernel pages.

What more could you want?

[FlashBurn]

Thanks to all for you help. I solved the problem of above (hope so). But now I have a new

I have a function for allocating a physical 4KiB page and I have a function which maps this page into the process?s address space. My problem now is how to find a "free" page in a PT of the process?s PD. I could search every PT for a "free" page but I think that this would take to long! I thought that it may be would be a good idea to use a bitmpa for that?

[Candy]

Most user programs want it at the end of what they have now. Keep info on the blocks that are free (sugg: linked list or single block) and put it in there.

[Pype.Clicker]

it is possible to create lists and 'fit lists' (listing all the continguous zones of N pages with 2^i < N < 2^(i+1)) right in the page table, so that you can usually allocate fast, with controlled fragmentation and do not require additionnal memory to manage it.

However, for most of the job, you do not allocate a "single" page table, but rather large chunks (one chunk for the program, another chunk for a shared library, etc) or allocate in either a bottom-up (for the heap) or top-down (for the stack) manner ...