Generalized virtual memory management

[JoeKayzA]

Hi!

In the last time, I have made up my mind about how I'm going to handle virtual memory mappings in my os, and I've come up with some design thoughts that I wanted to share with you and open for comments, critics, suggestions and flames. ;D

The basic idea is to group certain pages of physical memory together to form a 'memory section'. A memory section provides an abstract view into memory, it has a specific length (which is a number of pages) and each of its pages is mapped either to physical memory, or to a page of another section. In the latter case, this can form a 'mapping hierarchy' (multiple sections are stacked on top of each other then).

To improve coherence of design, I thought that all physical memory should be represented by a special memory section, so when you map a page of physical memory, it works the same way as mapping a page of another section. Furthermore, the virtual address space itself should also be represented as a special section of its own. As soon as you map some page into it, it will be entered into the processor's page tables and be accessible through linear memory.

So that's the basic idea. I think with this design it should be easy to implement things like shared memory, swapping, COW and memory mapped files (these are all things that could be represented by memory sections then). I'm not yet sure about implementation details however, especially which data structures to use for keeping all the mapping data. I'm afraid that the memory overhead will get larger than I thought at first. :\

So what do you think? I'm curios whether someone of you has thought of a similar design, and how did you implement it? Comments and critics welcome. If you are interested in more details, let me know.

cheers Joe

[Colonel Kernel]

That sound exactly like memory management in L4. Instead of "sections", they call them "flexpages".

[JoeKayzA]

*[me=JoeKayzA]checks some L4 docs*[/me]

You are right, there are quite a lot of analogies, I didn't notice at first.

The main differences AFAICS are:

• An L4 flexpage must be mapped into an address space before you can forward the mapping to another one.
• I thought that memory sections can behave differenly depending on their implementation. It can be as simple as keeping a static mapping to some other sections, but as complex as dynamically allocating and deallocating physical (cache) memory on demand. I think you can implement something like this on L4 as well, but that requires seperate servers (for a microkernel, that's ok, but I'm not doing one )

To get a grip about the involved datastructures I could have a closer look at L4's internals though, thanks for that hint.

cheers Joe

[Cjmovie]

That sounds rather interesting, actually. Took 2 reads to get it, but cool.

Slightly off topic, but that made me think of an idea which may or may not already be implemented somewhere. Programs trying to eg. write data to a disk just write into a special section of kernel memory where no page is mapped, generating a PF. The PF handler will map a temporary page in place and notify the receiving driver, and map it into its address space.

Oh, and what's with the name flexpages? I'm kind of time-pressed to read the white pages at this time now though =/.

[JoeKayzA]

That sounds rather interesting, actually. Took 2 reads to get it, but cool.

THX

Slightly off topic, but that made me think of an idea which may or may not already be implemented somewhere. Programs trying to eg. write data to a disk just write into a special section of kernel memory where no page is mapped, generating a PF. The PF handler will map a temporary page in place and notify the receiving driver, and map it into its address space.

This way you could implement memory mapped files (you could do the same with a whole drive as well), why not. With slight modifications you could also use it for on-demand allocation of memory, as well as swapping: If the temporary page hasn't been accessed for some time, you simply write it back to disk and deallocate it. The next time an application needs to access it, a PF is triggered and a new 'temporary' page is allocated at its place.

Oh, and what's with the name flexpages? I'm kind of time-pressed to read the white pages at this time now though =/.

A flexpage in L4 terminology is a sequence of pages in one address space that can either be 'mapped' or 'granted' to another address space. 'Mapping' means that both address spaces share this memory area then, and the 'sender' has the right to revoke access from the 'receiver' at any time. 'Granting' means that the sender gives away the memory area and only the receiver of the flexpage can access it from this time on.

AFAIK this can be continued between as many address spaces as you like - process A can grant a flexpage to process B, B grants it to C, C maps it to D and so on...

cheers Joe