Memory managment

[8infy]

Hello everyone.

I've just started working on memory allocation/paging/etc for my OS and I have a few questions:

I've seen a few implementations of kernel heap and they all seem to have a hardcoded size pool of a few megabytes just above the kernel.
1. Is this OK? For example if I load the kernel at 1 megabyte physical, then I should have a few free megabytes above me, is it safe to assume that I do?
2. If not, how does one handle setting up kernel heap without any free memory at hand? (so basically chicken and egg problem).
3. Can the kernel use other physical ranges of memory after its done parsing the memory map (or rather is this a normal practice?) or is it limited with that hardcoded heap size?

If I understand everything correctly the right memory initialization order is:
init basic kernel heap of hardcoded size -> init paging and move kernel to 3GB (obv. can be any other location) -> parse memory map and detect free pages -> init allocator with free pages
Is this accurate? ^

Would really appreciate any other memory handling advice and how to do things right from experienced osdev people.

[AndrewAPrice]

Do you need a kernel heap before you setting up a physical allocator and paging?

I ask this because the paging structures (page tables, page directories, etc.) are 4KB - the size of a page. I allocate my boot stack and boot-time paging structures in the linker script, and I use GRUB as my bootloader, so I'd asssume GRUB would give me an error if it tries to load my kernel into memory that doesn't exist.
One of the first things I do is loop through the multiboot header that GRUB populates, and create a linked stack of free memory pages (I store the pointer to the next free page at the start of the previous page.) Now that I can request/release 4KB pages, I have all I need to set up my paging structures. (After paging is set up, I can release my boot-time paging structures back to the physical allocator.)

With your physical and virtual allocators set up, your malloc implementation can request a page to use as the heap. liballoc is popular because it's super simple to port.

[thewrongchristian]

Do you need a kernel heap before you setting up a physical allocator and paging?

I ask this because the paging structures (page tables, page directories, etc.) are 4KB - the size of a page. I allocate my boot stack and boot-time paging structures in the linker script, and I use GRUB as my bootloader, so I'd asssume GRUB would give me an error if it tries to load my kernel into memory that doesn't exist.
One of the first things I do is loop through the multiboot header that GRUB populates, and create a linked stack of free memory pages (I store the pointer to the next free page at the start of the previous page.)

Do you store the pointer in the page itself, or in some external management structure? I ask, because while your scheme has the benefit of O(1) allocation, it does require all your physical memory to be mapped into virtual memory if you store the pointers in the page itself.

Of course, that's not likely to be a problem for a 64-bit only OS, with vast swathes of VM at its disposal. My OS, on the other hand, is currently 32-bit (and I want to keep it 32-bit capable and not assume I can map all of HW memory in VM) and just as an added bonus, I'm trying to keep the OS mapped above 0xf0000000 on x86, to give me as much user address space as possible.

My page allocator is bit mapped, and so there is a small cost to scanning memory for free pages.

[AndrewAPrice]

Do you store the pointer in the page itself, or in some external management structure? I ask, because while your scheme has the benefit of O(1) allocation, it does require all your physical memory to be mapped into virtual memory if you store the pointers in the page itself.

The pointer is stored in the page itself.

You don't need to keep all the physical memory mapped into virtual memory, just the page you're currently operating on.

[thewrongchristian]

Do you store the pointer in the page itself, or in some external management structure? I ask, because while your scheme has the benefit of O(1) allocation, it does require all your physical memory to be mapped into virtual memory if you store the pointers in the page itself.

The pointer is stored in the page itself.

You don't need to keep all the physical memory mapped into virtual memory, just the page you're currently operating on.

Right, and the link is the next physical page number rather than a VM pointer?

I'll definitely put that one on the list to investigate.

[AndrewAPrice]

Right, and the link is the next physical page number rather than a VM pointer?

That's correct. I store the physical address for the next page inside the page. Then you can temporarily map it into memory to ready/write this address.