Returning free memory to the OS
Posted: Mon Feb 16, 2015 10:46 pm
Both from what I've read and what I've observed, most runtime libraries don't tend to take any action to return large free sections of their heap to the OS. Rather, if a chunk of memory is freed, they hang on to it so that they don't have to grab more address space to fulfill future allocations.
In general, this is a fairly good policy. We don't want to make a system call to give up address space that we'll turn around and want right back before our process's timeslice is over. However, it has some downsides. If the system is forced to start swapping to meet the memory demands of the running processes, and is using a least-recently-used replacement strategy, or something that approximates LRU, it may well be the case that the least recently used page on the system is in the middle of a free chunk of the owning process's heap and thus contains only junk data. If our allocator just holds on to free memory without letting the OS know that it's not currently in use, then the OS will write that page to disk, wasting I/O bandwidth. What's worse, whenever memory in that page is next allocated and used, all that junk data will be read right back in to memory before the page can actually be used.
The solution I would propose is this: The kernel ABI should include some data structures that a userspace allocator can use to communicate to the kernel what pages that it had previously allocated are currently unused. When a process starts, the runtime allocator initializes these structures and makes a system call to tell the kernel where it's storing them (or the ABI can define a static address for the structures). When the kernel runs out of free RAM, instead of starting to swap, it first checks to see if any running processes have pages marked as free in memory. If so, instead of freeing the page frame for use by swapping the data contained in that page to disk, it simply discards the data and remaps that page to a copy-on-write zero-filled page, which makes the page frame available for use much more quickly. If the allocator allocates memory in that page again, the kernel just obtains a free page frame and uses it to fulfill the copy-on-write, rather than pulling useless data in from disk.
In general, this is a fairly good policy. We don't want to make a system call to give up address space that we'll turn around and want right back before our process's timeslice is over. However, it has some downsides. If the system is forced to start swapping to meet the memory demands of the running processes, and is using a least-recently-used replacement strategy, or something that approximates LRU, it may well be the case that the least recently used page on the system is in the middle of a free chunk of the owning process's heap and thus contains only junk data. If our allocator just holds on to free memory without letting the OS know that it's not currently in use, then the OS will write that page to disk, wasting I/O bandwidth. What's worse, whenever memory in that page is next allocated and used, all that junk data will be read right back in to memory before the page can actually be used.
The solution I would propose is this: The kernel ABI should include some data structures that a userspace allocator can use to communicate to the kernel what pages that it had previously allocated are currently unused. When a process starts, the runtime allocator initializes these structures and makes a system call to tell the kernel where it's storing them (or the ABI can define a static address for the structures). When the kernel runs out of free RAM, instead of starting to swap, it first checks to see if any running processes have pages marked as free in memory. If so, instead of freeing the page frame for use by swapping the data contained in that page to disk, it simply discards the data and remaps that page to a copy-on-write zero-filled page, which makes the page frame available for use much more quickly. If the allocator allocates memory in that page again, the kernel just obtains a free page frame and uses it to fulfill the copy-on-write, rather than pulling useless data in from disk.
