MM questions

[UX]

Hi,

I'm in very early pahase of designing a primitive mm, and I'm struggling with some conceptual issues, hope you guys can help:

1.) when freeing the memory of some process, should mm parse every PT to fetch physical adresses of page frames, or maybe the pool of allocated pages should be kept somewhere in the process structure?
2.) How to allocate small quantitates of memory like byte, word, etc? I guess that page size is a bit of overhead...

Thank you.

[OSwhatever]

Hi,

I'm in very early pahase of designing a primitive mm, and I'm struggling with some conceptual issues, hope you guys can help:

1.) when freeing the memory of some process, should mm parse every PT to fetch physical adresses of page frames, or maybe the pool of allocated pages should be kept somewhere in the process structure?
2.) How to allocate small quantitates of memory like byte, word, etc? I guess that page size is a bit of overhead...

Thank you.

1. Usually you have a region list or tree that describes the virtual regions in a process. You don't have to parse the page table outside these regions.

2. I use 32 byte as smallest size in the general allocator in the kernel. Often you have smaller objects in vast quantities, then I create a dedicated slab cache for those objects in order to reduce fragmentation and overhead.

[UX]

1. Usually you have a region list or tree that describes the virtual regions in a process. You don't have to parse the page table outside these regions.

Not sure if I understood you correctly...so there should be some structure describing VM->PM mappings inside the task structure(arranged in list or tree like structure). How can I be sure that mm will give me the contignous block of memory which I can proclaim as region?

2. I use 32 byte as smallest size in the general allocator in the kernel. Often you have smaller objects in vast quantities, then I create a dedicated slab cache for those objects in order to reduce fragmentation and overhead.

OK, so did you just added the code for smaller objects or rewritten the whole heap manager code?

[UX]

Your frame allocator will give only stretches of pages to the process, it's up to malloc() implementation to allocate smaller quantities within these allocated pages.

Ah..OK...that will make things a bit more complex.

[OSwhatever]

1. Usually you have a region list or tree that describes the virtual regions in a process. You don't have to parse the page table outside these regions.

Not sure if I understood you correctly...so there should be some structure describing VM->PM mappings inside the task structure(arranged in list or tree like structure). How can I be sure that mm will give me the contignous block of memory which I can proclaim as region?

2. I use 32 byte as smallest size in the general allocator in the kernel. Often you have smaller objects in vast quantities, then I create a dedicated slab cache for those objects in order to reduce fragmentation and overhead.

OK, so did you just added the code for smaller objects or rewritten the whole heap manager code?

1. In a user process, in general the virtual space is divided up in regions. Usually these regions are text, data, io memory for example. If a page fault occurs within these regions the kernel needs to know what to do, that's what these region descriptors are for among other things. There is no VM->PM mapping in these structures, that's done in the page table. These structures can also used when the process is deleted so that the entire page table doesn't need to be parsed.

You can design the mm how you want. Usually you don't assume that memory is continuous in process heap. If you want to proclaim the region, you have to parse the page table (when deleting process). If you want to return pages when the process is running you should implement an unmap kernel call.

2. I suggest you read about the Linux slab allocator and slabs in general. Slabs (and its variants) are used in several kernels for small heap allocations. The slab allocator can be used for both general heap allocation and dedicated object allocation.

First hit Google: http://www.ibm.com/developerworks/linux ... allocator/

There are several good books about this subject. Understanding the Linux Virtual Memory Manager is a good start. Now Linux isn't the master solution to every kernel but it has several good design choices.

[UX]

These structures can also used when the process is deleted so that the entire page table doesn't need to be parsed.

I know how process mem. layout looks like...but I'm still confused on how does mm relese pages after the process is killed.
Physical mm must know which pages are released so that it can proclaim them as free. Now, as I understand, the only way it can be done is by parsing PTs(which is an overhead) or if the actual frame addresses for particular process have been cashed somewhere, no?

Understanding the Linux Virtual Memory Manager is a good start. Now Linux isn't the master solution to every kernel but it has several good design choices.

Linux kernel is quite a beast and don't feel like understanding it's mm design at this point. Now, I want to make manager as primitive as possible to get over the tehnical issues and later on when I fully understand them, worry about more efficient design, even if that means completley rewriting the code.

[gerryg400]

Now, I want to make manager as primitive as possible to get over the tehnical issues and later on when I fully understand them, worry about more efficient design, even if that means completley rewriting the code.

If you are looking for traditional style memory management (Posix style mmap and munmap) with user-side malloc then I would recommend reading the Gorman book. Memory management is not nearly as difficult as it first seems and you can waste 100's of hours writing code that you will eventually discard.

It took me a while to understand that the virtual memory in a process (or indeed the kernel) and the physical memory of the machine are completely separate resources and can be managed quite separately. Of course virtual memory and physical memory are bound together by the paging mechanism but this is generally a transient binding.

http://ptgmedia.pearsoncmg.com/images/0 ... n_book.pdf

[UX]

If you are looking for traditional style memory management (Posix style mmap and munmap) with user-side malloc then I would recommend reading the Gorman book. Memory management is not nearly as difficult as it first seems and you can waste 100's of hours writing code that you will eventually discard.

Maybe I'm really dumb, but this book is a overkill for me. ~800pages just for VM and this is 2.4 kernel series... I can stuck on any code examples for days.
I'm just a beginner, so rather reccomand me a color VM book.

Cheers!

[gerryg400]

If you are looking for traditional style memory management (Posix style mmap and munmap) with user-side malloc then I would recommend reading the Gorman book. Memory management is not nearly as difficult as it first seems and you can waste 100's of hours writing code that you will eventually discard.

Maybe I'm really dumb, but this book is a overkill for me. ~800pages just for VM and this is 2.4 kernel series... I can stuck on any code examples for days.
I'm just a beginner, so rather reccomand me a color VM book.

Cheers!

Did you look at the book ? The pages after page 211 are a code commentary. It's very light reading.