How do modern operating systems manage the heap memory?

[LieutenantHacker]

I was reading up on pointer to pointers, dynamic memory allocations for User programs, and it got me wondering how the operating system keeps track of heap memory for reservation/reallocation/deallocation. I rummaged through some Linux kernel sources on kernel.org, and I can gather that the OS would use linked list data structures to pool out memory, but that's when some head scratching started.

If all local data within locally defined datums are allocated and placed on the stack, to what extent does the program keep track of the allocated memory?
I mean heap memory is dynamically pulled, and to the best of my knowledge the actual stack/process does not keep any reference to the dynamic bits from the pooled memory reserve, so the OS controls it internally within its memory management, or does the runtime thread instruct the OS on hoe to do this via system calls?

I know this is lame, but this, writing a bootloader in ARM v7, and contemplating structure of high level Java apps has perplexed me somewhat.

Is it the OS that allocates, manages, and pools the dynamic memory upon requests internally programmed to provide for the User program, or does a language runtime have more setup in regards to accomplishing this than the OS does by itself?

Like, is it a specific part of the kernel constantly pooling and reserving the pointed addresses for calling requests to use it dynamically via communication from the kernel code itself, or is it the, say, C runtime implementing linked lists, etc.?

[Owen]

The "heap" is implemented entirely by the project itself (or, more likely, a library used by said process: libc, kernel32, etc)

The kernel just hands the application pages. How the kernel keeps track of these varies enormously.

[LieutenantHacker]

So, but the app would then have to manage the allocated pages, correct?

My first guess was that this could be implemented by a kernel thread, however, proven more effective in userland.

But so how would the program do this through kernel32 or libc? I mean I know they are libraries that enable memory management and such, but how is it controlled by the process manually, i.e. keep track of the memory?

[dozniak]

man sbrk
man VirtualAlloc
man malloc
google dlmalloc

[LieutenantHacker]

Wow, thanks, dozniak!

Just what I was looking for.

I don't know why I never thought of considering the Windows API's VirtualAlloc function, sbrk or malloc.c.

[OSwhatever]

Basically, the user processes are responsible for handling their own address spaces and they can set them up almost how they want. Like laying ceramic tiles in your bathroom, you decide which colors they should have and where they should be. Your ceramic tile is usually 4096KB.

What you can read about is the "virtual memory area" in Linux, which is a virtual memory region intended for a certain purpose decided by the user application.

[sortie]

Also be sure to check out 'man mmap'.

[TylerH]

Maybe a little OT, but does anyone know of a memory management library that will let you give it the pages to use? I was working on a multiprocess app once where it would have been really nice to have a shared heap for everything. What I wanted to do was just mmap a shared anonymous file in the root process and let this heap be shared by all children (w/o cow). Obviously, I would have to manage race conditions, but that would have been acceptable. In the end, I couldn't find any library like I'm describing and deciding it was too much like work to write my own heap manager, so I quit.