Troubles with Memory Management

[blacky]

Hello together,

I am stuck with the implementation of my memory management and believe that it requires a complete redesign. First, I would like to present the current state of the kernel and then I would like to state the troubles I am having with the memory management - In fact, I believe that I am missing some crucial details, which prevent further progress. I would be very grateful if we could discuss suitable memory management solutions in some detail. By the way: If you should discover any flaws in the following description, do not hesitate to mention it - as I am here to learn and hence open for any form of constructive criticism

Current state:
To boot the kernel, I chose to use two separate binaries: one for the bootstrapping code (ELF32) and one for the kernel code (ELF64). During startup, the GRUB2 bootloader loads the bootstrap image as "kernel" and the actual kernel as a "GRUB module". The bootstrapping code handles the initial initialization steps (setup of GDT, IDT, ...) and locates the GRUB module representing the actual kernel image.

Since paging is required to jump into Long Mode, I have implemented a rudimentary paging mechanism already in the bootstrapping code. It makes use of a statically allocated pool of page-sized blocks to initialize the 4-level paging structures (PML4, PDPT, PD, PT). (Please note that at this point, there is no active physical memory management yet, as I thought the kernel would be a good place for it). The bootstrapping code maps the LOAD segment of the ELF64 kernel image to the address 0xFFFFFFFFF0000000 (the bootstrapping code does not move the kernel image within the phys. memory) and jumps to this address after performing required steps for Long Mode initialization.

Troubles/Questions:
Now, in the actual kernel I would like to implement a decent phys. memory management complemented with the paging mechanism so that it would be possible to allocate additional memory and map it to any virtual memory space. The issues I am encountering are the following:

1) Find a good way to take over control of the active paging mechanism (initialized within the bootstrapping image) inside the kernel:
To achieve this, first, I would need to know the physical address of the current paging structures. I have read that it is common to map the last page directory entry (PDE) to the beginning of the Page Directory itself. This enables modification of the Page Tables through virtual memory in 32-bit architectures withouth PAE. In case Long Mode has been initialized, the PML4 structure has 512 entries pointing to 512 PDPTs and so on... This makes it harder to localize the page table structures through virtual memory. I guess what I am trying to understand is what would be a good way to localize these structures in Long Mode.

2) Find out the physical memory address of the kernel:
To be able to implement a phys. memory manager, I need to know where the kernel is located in phys. memory so that I can mark its memory location as occupied or simply as "not free". Since the kernel has been loaded into memory as a GRUB module, only the bootloader itself and the bootstrapping code knows its location address in memory. Or is there a way how the kernel may find its actual phys. start address? (Does the GRUB memory map contain the memory occupied by GRUB modules?).

As always: Thank you very much in advance

[Combuster]

1: For long mode paging, you would map a PML4 entry to itself, rather than a PDE entry

2: If your kernel is running, you know its start and end address with the use of symbols. The fact that you use ELF mean that you have to parse it, and subsequently it won't stay at the address GRUB loaded it at. If you don't have any other modules, there's nothing else to subtract from the memory map (which is always as the hardware reports it.)

[blacky]

1: For long mode paging, you would map a PML4 entry to itself, rather than a PDE entry

Ok, I see: This makes sense - I guess I tried to overthink this part a little: I will try it out, thank you

The fact that you use ELF mean that you have to parse it, and subsequently it won't stay at the address GRUB loaded it at.

Exactly: In my case (since I parse the ELF file) the best way of locating the kernel's phys. address would be probably by passing its address as argument during the Long Mode jump.

Thank you very much

[Brendan]

Hi,

By the way: If you should discover any flaws in the following description, do not hesitate to mention it - as I am here to learn and hence open for any form of constructive criticism

That sounds reasonably good to me.

Note that there may actually be quite a bit of processing for initialisation/sanity checking that's only done once, that can be shifted into your "kernel setup" module so that it's freed after the kernel starts and doesn't waste RAM after boot. For a simple example; I'd recommend converting the physical memory map that GRUB gives you into a different form - specifically, a form that guarantees there's no "zero length" entries, that entries are sorted in order and that there's no overlapping areas; where the "area type" field is converted to your own system that's a superset of the types the BIOS/GRUB and UEFI provide (so that you can have a boot loader for UEFI that converts UEFI's physical memory map into whatever your kernel expects without information loss). You could even go a step further here, and merge other information (e.g. the NUMA information from ACPI) into your version of the physical memory map, so that kernel can find all information about physical memory in the same place.

Of course your "kernel setup" module could also do a lot of other things; like pre-initialising data structures the kernel uses for physical memory management, virtual memory management and scheduling; detecting and starting other CPUs; getting a list of PCI devices; etc.

2) Find out the physical memory address of the kernel:
To be able to implement a phys. memory manager, I need to know where the kernel is located in phys. memory so that I can mark its memory location as occupied or simply as "not free".

Once you've mapped the PML4 into itself; you can use the page tables, page directories, PDPTs, etc. to determine the physical address of all page/s that the kernel is actually using (including everything your "kernel setup" module mapped into the initial virtual address space - e.g. including any data from multi-boot, and not just the kernel itself).


Cheers,

Brendan

[blacky]

I'd recommend converting the physical memory map that GRUB gives you into a different form - specifically, a form that guarantees there's no "zero length" entries, that entries are sorted in order and that there's no overlapping areas

Thank you Brendan: that is a very good advice! Following your advice, today, I have started with a custom representation of the phys. memory area; In my case, it is a structure containing a sorted doubly linked list of "free" memory regions and additional information, such as the kernel's physical location and size. This structure is placed at a dedicated memory location (making it easier to pass the address to the kernel), which will be freed as soon as the kernel has initialized the phys. memory manager.

Once you've mapped the PML4 into itself; you can use the page tables, page directories, PDPTs, etc. to determine the physical address of all page/s that the kernel is actually using

This is still on my TODO list: However, what makes me a little skeptical is the fact that the PML4 structure will remain at an address defined by the bootstrap code, however the pages for further paging structures (PTPT, PD, ...), which will be allocated as needed, will be possibly distributed across the entire physical memory space; Not that it would not work... However allocating space for all paging structures at once (in a way so that they are placed together) would not make sence, since this would be an overkill on a 64bit system with a 4-level paging hierarchy - and possibly only a small amount of phys. memory. I guess, I will need to learn to live with the fact that paging may be a little messy