Kernel Heap Design Questions

[stillconfused]

Hello, I have some questions about my kernel heap, and related things.
Firstly, what should, and shouldn't I be using my kernel heap allocator for? During the boot process I initialize my kernel heap with a large block of memory, and then use this to allocate memory for all kernel space operations.
I'm writing this kernel for RISC-V, and I don't use a higher-half kernel design. My kernel has a trampoline design like Minix, or XV6 to transfer control between userspace and the kernel. Userspace processes have a stack/heap mapped at arbitrary high addresses, but the kernel stack/heap are identity mapped in kernel space, so that I can easily allocate memory for things like user process page tables, or VirtIO queues where I need a physical address. The initial block of userspace heap memory is allocated when the process is created, and can be grown/shrunk as needed. Are there any obvious flaws with what I'm doing? I'm trying to follow open-source examples, but I'm finding them hard to follow, or ill suited for one reason or another.

[thewrongchristian]

Hello, I have some questions about my kernel heap, and related things.
Firstly, what should, and shouldn't I be using my kernel heap allocator for?

You should us the heap for anything that cannot be statically sized at compile time.

In older kernels, tables of data structures like all the processes, were statically sized and compiled into the kernel image at build time, making them very difficult to change and tune (basically, at least a small re-compile and relink was required to change any statically sized table.)

Not only that, but the tables would have to be sized for the worst case, wasting memory in the average case.

Instead, just allocate your process structures (and any other dynamic structure) on your heap.

During the boot process I initialize my kernel heap with a large block of memory, and then use this to allocate memory for all kernel space operations.
I'm writing this kernel for RISC-V, and I don't use a higher-half kernel design. My kernel has a trampoline design like Minix, or XV6 to transfer control between userspace and the kernel. Userspace processes have a stack/heap mapped at arbitrary high addresses, but the kernel stack/heap are identity mapped in kernel space, so that I can easily allocate memory for things like user process page tables, or VirtIO queues where I need a physical address.

The problem with using 1:1 virtual/physical mapping for heap memory is that you have to size and allocate all your heap memory up front, whether you use it or not.

If you instead create your virtual memory space, without backing the memory with physical memory, you can allocate physical memory to your heap on demand.

Otherwise, you're little better off than with statically sized tables.

The other option is to not have a single contiguous heap, but instead use a slab allocator, with each slab sliced into equal sized memory slots. Then, you can new allocate physical pages when you need a new slab, and your heap need not be contiguous.

[stillconfused]

Thank you for your reply!

In older kernels, tables of data structures like all the processes, were statically sized and compiled into the kernel image at build time...

I started off doing things this way. I figured there might be some benefit in knowing ahead of time what your worst-case memory usage would be, but allocating every single structure (Process Control Blocks, Memory Space mapping structures, etc) ballooned in size very quickly.

The other option is to not have a single contiguous heap, but instead use a slab allocator, with each slab sliced into equal sized memory slots. Then, you can new allocate physical pages when you need a new slab, and your heap need not be contiguous.

I see the benefit of what you're talking about. My current heap implementation allows for non-contiguous zones, but it's not a perfect system.

Something else I'm having trouble with is the large amount of fragmentation caused by page-aligned allocations. My allocator creates an 8-byte allocation header before the allocated address, and if I'm allocating a large number of page-aligned structures on the kernel heap, I end up with a very fragmented memory space. I'm using these page-aligned allocations for structures that need to be mapped into a process' virtual memory space, such as their stack/heap. I've wanted to avoid having alternate allocators for special purposes like Linux has. Should I just use my PMM for allocating these blocks of page-aligned memory?

[thewrongchristian]

Something else I'm having trouble with is the large amount of fragmentation caused by page-aligned allocations. My allocator creates an 8-byte allocation header before the allocated address, and if I'm allocating a large number of page-aligned structures on the kernel heap, I end up with a very fragmented memory space. I'm using these page-aligned allocations for structures that need to be mapped into a process' virtual memory space, such as their stack/heap. I've wanted to avoid having alternate allocators for special purposes like Linux has. Should I just use my PMM for allocating these blocks of page-aligned memory?

Personally, I don't think anything large and page aligned should be using the heap, so yes, just use your PMM to get whole pages and map them as appropriate.

[stillconfused]

Thank you very much for your help! I really appreciate it.

[beredis]

Regarding your design, it’s important to ensure that the kernel heap is used efficiently to avoid memory bloat and fragmentation. The identity mapping of the kernel stack/heap is a common practice and should work well as long as you manage the virtual address space carefully. For the userspace processes, it’s good that you have a mechanism to grow/shrink the heap as needed. Just make sure that the allocation and deallocation are well-managed to prevent memory leaks or corruption.

[beredis]

where is my post?

[nullplan]

where is my post?

It was stuck in moderation for a couple of months...