Physical memory manager and grub memory map questions

[finarfin]

So i'm planning to start soon implementing my physical memory manager, i did it couple of times in the past, so at least i kind of know what to do.

But i just discovered that there is something that looks useful when doing a PMM, the grub memory map, that says what area of the memory are free to use, or what not.

But i did a search in several os projects and looks not many of them are using it, and looking at the wiki page https://wiki.osdev.org/Detecting_Memory ... p_Via_GRUB it points toward many drawbacks of this approach (but the header of the page says also that the article is little bit biased toward certain points of view), so my question is if it worth using it or not, the effort to read and parse it is not too much, and at least provide a good starting point to initialize properly the physical memory manager, with all the unavailable area marked as it is.

Then the other question is about the bitmap, more for confirmation, actually the kernel is configured to use 2mb pages (even if in the future i'm planning to make it flexible depending on user choice), so i assume that the bitmap must take it into account, since it is returning pages, every bit will then count for 2mb instead of 4kb (of course probably the best appoach is to make the bitmap size flexible too (it shouldn't be too hard) am i right?

Another question i have is regarding accessing page tables/dir data from the kernel, so i used the recursion method in the 32bit kernel mapping the page table itself to the last element, and was planning to do something similar, but since i mapped the kernel in the higher half, the pml4 entry for it is 511 (the last element), so this prevent me from using the last element of the pml4 page dir. What i'm wondering at this point is if there is a good positioning (convention) for the recursive mapping in 64bit or i just need to improvise (i was thinking btw to use in case 509th entry or something similar to keep all kernel stuff close in the addressing space), or is there a better/simplier technique that can be used with 64 bit kernels?

[Velko]

Regarding GRUB-provided memory map, I would say that it should be preferred method, unless there's a special need. You could check out Multiboot2, however.

In order to use BIOS functions you will have to drop out to Real mode. In case of UEFI, I think GRUB exits boot services, so that is not available either.

Regarding PMM bitmap: yes, each bit will represent 2 MiB of memory. The best approach, however, would be to support both page sizes. It might get a bit tricky (especially on de-allocations), but you can maintain 2 bitmaps, one for each size. Personally I find 2 MiB pages too large for efficient use, but it's your OS and your preferences.

Recursive paging works just fine in Long Mode. About the choice of entry #, I don't think there are any conventions, but I did exactly the opposite - used 256th entry. This way it is easy to find out addressed area by just looking at the pointer. Paging is one of the most trickiest parts of OS, so you may want to have as many debugging clues as possible.

[finarfin]

Regarding GRUB-provided memory map, I would say that it should be preferred method, unless there's a special need. You could check out Multiboot2, however.

I'm using multiboot2 header, so i suppose i just need to read data throught it i suppose.

Regarding PMM bitmap: yes, each bit will represent 2 MiB of memory. The best approach, however, would be to support both page sizes. It might get a bit tricky (especially on de-allocations), but you can maintain 2 bitmaps, one for each size.

Yeah i was thinking of making a mechanism of supporting both 4kb and 2mb pages. I was thinking of providing probably few Macros to do the computation that they will be expanded depending on the size selected by the user (that should be tricky but hopefully doable)

Personally I find 2 MiB pages too large for efficient use, but it's your OS and your preferences.

I stumbled across a post in the forum where someone did a proper computation and showed up that using the 2mb pages approach was wasting a lot of resources, said that at the moment i haven't chosen clearly what page size to use (luckily my code base is still pretty small and the change is going to be minimal), i used 2mb just because i was following a tutorial to boot a kernel in 64bit mode and it was using 2mb pages. but as said above i'm thinking of trying to support both, so that should not be an issue.

Recursive paging works just fine in Long Mode. About the choice of entry #, I don't think there are any conventions, but I did exactly the opposite - used 256th entry. This way it is easy to find out addressed area by just looking at the pointer. Paging is one of the most trickiest parts of OS, so you may want to have as many debugging clues as possible.

If i understood correctly you placed it halfway in the pagedir? Why it should be easiest at 256th instead of any other postion?
Yeah i know, when i studied the paging address translation and the recursion method, it took me long time to understand it!!!
And everytime i still have to compute a entry for some reason manually i have to redo it 3/4 times because i always miscount some bit XD

Thanks for the info!

[nullplan]

Another question i have is regarding accessing page tables/dir data from the kernel, so i used the recursion method in the 32bit kernel mapping the page table itself to the last element, and was planning to do something similar, but since i mapped the kernel in the higher half, the pml4 entry for it is 511 (the last element), so this prevent me from using the last element of the pml4 page dir. What i'm wondering at this point is if there is a good positioning (convention) for the recursive mapping in 64bit or i just need to improvise (i was thinking btw to use in case 509th entry or something similar to keep all kernel stuff close in the addressing space), or is there a better/simplier technique that can be used with 64 bit kernels?

There is: Map all memory linearly to 0xffff'8000'0000'0000. That way, going from physical to virtual address is only a single addition. You still have enough address space available to map your kernel logically to -2GB. And following physical address chains, whether for paging or MMIO, is way easier this way.

[Velko]

Why it should be easiest at 256th instead of any other postion?

It's not. The point was that instead of trying to keep things close, I distributed them in whole upper address space. Then, for example, if there's a page fault, I can just look at the faulting address and immediately (ok, it may take some time getting used to) tell if it is in paging area (0xffff800000000000), kernel itself (0xffffffff80000000) or somewhere else.

[finarfin]

Why it should be easiest at 256th instead of any other postion?

It's not. The point was that instead of trying to keep things close, I distributed them in whole upper address space. Then, for example, if there's a page fault, I can just look at the faulting address and immediately (ok, it may take some time getting used to) tell if it is in paging area (0xffff800000000000), kernel itself (0xffffffff80000000) or somewhere else.

Ah ok, i thought there was some specific reason for number 256 like that it was making the computation easier. Thanks