OSDev.org

Posted: **Mon Apr 09, 2007 10:33 am**

I was just looking at dlmalloc and wondering if using that in the kernel could bring about any noticeable gains in performance and memory usage. Before I go though too much trouble I have some questions.

1. Is it worth it?

2. What support functions do I need to have in order to get dlmalloc to work.

3. How do the functions from #2 work.

Thanks for your time

Posted: **Mon Apr 09, 2007 11:39 am**

Theoretically it's supposed to work with nothing but sbrk(). I looked into using it at some point, and after browsing it a bit, I didn't bother going to the trouble of actually getting it to work in my kernel.

It's a good malloc, so unless you plan to do lots of special case allocation in your kernel, I think it can be worth it. I wouldn't worry about it too much though, if your kernel already has a working malloc, unless your malloc performance actually seems like a problem.

Malloc is a relatively individual component after all, so you can always change it later, if it seems necessary.

Posted: **Mon Apr 09, 2007 12:43 pm**

I am just hoping to find something that will be a bit faster and help with the fragmentation. This is what my kernel heap looks like now

Code: Select all

402 bytes used
500 bytes free
4655 bytes used
32 bytes free

and so on.

So how does this sbrk work? Am I correct in assuming that it just gives out page size chunks of memory? Also are the chunks it gives out required to be contiguous?

Thanks

Posted: **Mon Apr 09, 2007 12:47 pm**

Code: Select all

BRK(2)                     Linux Programmer's Manual                    BRK(2)



NAME
       brk, sbrk - change data segment size

SYNOPSIS
       #include <unistd.h>

       int brk(void *end_data_segment);

       void *sbrk(ptrdiff_t increment);

DESCRIPTION
       brk  sets  the  end  of  the  data  segment  to  the  value  specified  by
       end_data_segment, when that value is  reasonable,  the  system  does  have
       enough memory and the process does not exceed its max data size (see setr-
       limit(2)).

       sbrk increments the program's data space by increment bytes.  sbrk isn't a
       system  call, it is just a C library wrapper.  Calling sbrk with an incre-
       ment of 0 can be used to find the current location of the program break.

RETURN VALUE
       On success, brk returns zero, and sbrk returns a pointer to the  start  of
       the new area.  On error, -1 is returned, and errno is set to ENOMEM.

CONFORMING TO
       BSD 4.3

       brk  and  sbrk  are  not  defined  in  the C Standard and are deliberately
       excluded from the POSIX.1 standard (see paragraphs B.1.1.1.3 and B.8.3.3).

SEE ALSO
       execve(2), getrlimit(2), malloc(3)




Linux 0.99.11                     1993-07-21                            BRK(2)

Ignore the details about how it's implemented in Linux for user processes. I think there's also some info in our Wiki.

Posted: **Mon Apr 09, 2007 2:32 pm**

I use dlmalloc in my kernel, and only had to do minimal changes to it. Make sure you read the comments at the top carefully, as you will probably need to do several #defines to get it set up the way you want. In my case, I told it not to use sbrk at all, but to use mmap instead. I then made a simple mmap wrapper for the memory allocation function I already had. I also got rid of the dlmalloc code that tries to open /dev/zero, as that wasn't possible or necessary for my implementation. I also got rid of a section that tried to print statistics to the console or something; I can't remember for sure.

Posted: **Mon Apr 09, 2007 5:06 pm**

Yeah well. My own kernel provides morecore() for my malloc, so the sbrk() approach is easier for my case. Sure, you can use mmap() as well. Or both.

Posted: **Mon Apr 09, 2007 6:09 pm**

what is the difference between sbrk and mmap?

Posted: **Mon Apr 09, 2007 6:23 pm**

sbrk extends the heap. For example, if your dynamically allocated memory range was from 0xC0000000 to 0xC8000000, sbrk(0x20000) would make it so that you're now using 0xC0000000 to 0xC8020000.

mmap maps a new area of memory. It might fall directly after the current top of your heap, or it might not. mmap can also be used for memory-mapped files, but that's beyond what you'd be using it for in this case.

My memory manager doesn't guarantee that memory will all be allocated together, so that's why I took the mmap approach. If your kernel did make that guarantee, sbrk would probably make more sense.

Posted: **Sat Apr 14, 2007 12:25 pm**

you should be aware that dlmalloc does not give back memory in a very "sane" way

this is arguably not because of dlmalloc itself but rather a side-effect of how sbrk and friends work

basically, you will never get anything back if there's a used block left close to the break

while this is normally acceptable in a userspace program, it could cause some rather interesting situations in a kernel

consider this: alot of threads spawn and use up alot of kernel memory, say 200MB, all but the last one of these threads exit
this will, in theory, leave almost 200MB unused memory for dlmalloc that it wont give back simply because that last thread is storing something at the end of the block

that said, dlmalloc is an excellent choice for a first, testing/beta-stage memory allocator, its reliable and easy to glue into just about any system

Posted: **Sat Apr 14, 2007 5:26 pm**

Well I got the dlmalloc up and running. I already think that there is a lot less fragmentation. I am not really woried about returning memory to the kernel just yet. I think that if you use mmap instead of sbrk than dlmalloc will be able to give back memory to the kernel, even if the areas that surround it are not free. Thanks to all of your responses.

OSDev.org

DLmalloc in kernel

DLmalloc in kernel