I was just looking at dlmalloc and wondering if using that in the kernel could bring about any noticeable gains in performance and memory usage. Before I go though too much trouble I have some questions.
1. Is it worth it?
2. What support functions do I need to have in order to get dlmalloc to work.
3. How do the functions from #2 work.
Thanks for your time
DLmalloc in kernel
Theoretically it's supposed to work with nothing but sbrk(). I looked into using it at some point, and after browsing it a bit, I didn't bother going to the trouble of actually getting it to work in my kernel.
It's a good malloc, so unless you plan to do lots of special case allocation in your kernel, I think it can be worth it. I wouldn't worry about it too much though, if your kernel already has a working malloc, unless your malloc performance actually seems like a problem.
Malloc is a relatively individual component after all, so you can always change it later, if it seems necessary.
It's a good malloc, so unless you plan to do lots of special case allocation in your kernel, I think it can be worth it. I wouldn't worry about it too much though, if your kernel already has a working malloc, unless your malloc performance actually seems like a problem.
Malloc is a relatively individual component after all, so you can always change it later, if it seems necessary.
The real problem with goto is not with the control transfer, but with environments. Properly tail-recursive closures get both right.
I am just hoping to find something that will be a bit faster and help with the fragmentation. This is what my kernel heap looks like now
and so on.
So how does this sbrk work? Am I correct in assuming that it just gives out page size chunks of memory? Also are the chunks it gives out required to be contiguous?
Thanks
Code: Select all
402 bytes used
500 bytes free
4655 bytes used
32 bytes free
So how does this sbrk work? Am I correct in assuming that it just gives out page size chunks of memory? Also are the chunks it gives out required to be contiguous?
Thanks
Code: Select all
BRK(2) Linux Programmer's Manual BRK(2)
NAME
brk, sbrk - change data segment size
SYNOPSIS
#include <unistd.h>
int brk(void *end_data_segment);
void *sbrk(ptrdiff_t increment);
DESCRIPTION
brk sets the end of the data segment to the value specified by
end_data_segment, when that value is reasonable, the system does have
enough memory and the process does not exceed its max data size (see setr-
limit(2)).
sbrk increments the program's data space by increment bytes. sbrk isn't a
system call, it is just a C library wrapper. Calling sbrk with an incre-
ment of 0 can be used to find the current location of the program break.
RETURN VALUE
On success, brk returns zero, and sbrk returns a pointer to the start of
the new area. On error, -1 is returned, and errno is set to ENOMEM.
CONFORMING TO
BSD 4.3
brk and sbrk are not defined in the C Standard and are deliberately
excluded from the POSIX.1 standard (see paragraphs B.1.1.1.3 and B.8.3.3).
SEE ALSO
execve(2), getrlimit(2), malloc(3)
Linux 0.99.11 1993-07-21 BRK(2)
The real problem with goto is not with the control transfer, but with environments. Properly tail-recursive closures get both right.
I use dlmalloc in my kernel, and only had to do minimal changes to it. Make sure you read the comments at the top carefully, as you will probably need to do several #defines to get it set up the way you want. In my case, I told it not to use sbrk at all, but to use mmap instead. I then made a simple mmap wrapper for the memory allocation function I already had. I also got rid of the dlmalloc code that tries to open /dev/zero, as that wasn't possible or necessary for my implementation. I also got rid of a section that tried to print statistics to the console or something; I can't remember for sure.
sbrk extends the heap. For example, if your dynamically allocated memory range was from 0xC0000000 to 0xC8000000, sbrk(0x20000) would make it so that you're now using 0xC0000000 to 0xC8020000.
mmap maps a new area of memory. It might fall directly after the current top of your heap, or it might not. mmap can also be used for memory-mapped files, but that's beyond what you'd be using it for in this case.
My memory manager doesn't guarantee that memory will all be allocated together, so that's why I took the mmap approach. If your kernel did make that guarantee, sbrk would probably make more sense.
mmap maps a new area of memory. It might fall directly after the current top of your heap, or it might not. mmap can also be used for memory-mapped files, but that's beyond what you'd be using it for in this case.
My memory manager doesn't guarantee that memory will all be allocated together, so that's why I took the mmap approach. If your kernel did make that guarantee, sbrk would probably make more sense.
you should be aware that dlmalloc does not give back memory in a very "sane" way
this is arguably not because of dlmalloc itself but rather a side-effect of how sbrk and friends work
basically, you will never get anything back if there's a used block left close to the break
while this is normally acceptable in a userspace program, it could cause some rather interesting situations in a kernel
consider this: alot of threads spawn and use up alot of kernel memory, say 200MB, all but the last one of these threads exit
this will, in theory, leave almost 200MB unused memory for dlmalloc that it wont give back simply because that last thread is storing something at the end of the block
that said, dlmalloc is an excellent choice for a first, testing/beta-stage memory allocator, its reliable and easy to glue into just about any system
this is arguably not because of dlmalloc itself but rather a side-effect of how sbrk and friends work
basically, you will never get anything back if there's a used block left close to the break
while this is normally acceptable in a userspace program, it could cause some rather interesting situations in a kernel
consider this: alot of threads spawn and use up alot of kernel memory, say 200MB, all but the last one of these threads exit
this will, in theory, leave almost 200MB unused memory for dlmalloc that it wont give back simply because that last thread is storing something at the end of the block
that said, dlmalloc is an excellent choice for a first, testing/beta-stage memory allocator, its reliable and easy to glue into just about any system
Well I got the dlmalloc up and running. I already think that there is a lot less fragmentation. I am not really woried about returning memory to the kernel just yet. I think that if you use mmap instead of sbrk than dlmalloc will be able to give back memory to the kernel, even if the areas that surround it are not free. Thanks to all of your responses.