memory management:Bookkeeping of dealt out pages

[distantvoices]

Here is the problem:

I keep book of each and every page dealt out to any process in a linked list - currently stored in kernel heap. Due to some limitations, this list seems to be trashed at some point - I keep receiving pagefaults when releasing pages of a process - because the chain gets broken at some point.

I've tracked it down to a certain degree, then it became too much a hassle - because something gets overwritten - and I started to think different:

What about this:

set aside a dedicated region of memory for this page book keeping - inside kernel space of course - in the size of (nr of pages available*sizeof(bookkeepingelement)) rounded up to page alignment.

then, I'd allocate the needed pages somewhere behind the kernel heap, but outta way of it and guarded. I'd set aside two lists: one for the free page-entry-elements, one for the used ones. Currently, I don't bookkeep kernel memory this way, I just decrement the amount of available pages accordingly. Might change.

When I allocate pages: just pick the first off the pile of free pages, add it to the pile of used pages, add information about who uses it and increment reference count to one. If a page is referenced by more than one process - increase the reference counter and insert the process to the list of pageowners. (kinda reverse lookup)

Deallocating a page: remove the page from the list of used pages and simply add the element to the list of free ones.

This requires some dedicated allocating/deallocating mechanism, but that's nothing I care too much about *gg*

Upon init of the pile of free pages I'd just stroll throu memory as if it were an array and say:

Code: Select all

element[x].next=element[x+1];

some border checking included of course.

I admit, this method would require some memory to be preallocated, but on the other hand - it would keep the page bookkeeping away from malloc - which isn't that performant - mostatime.

I use this system to easily deal out and keep book of shared memory (which is a set of pages inside a pagearray associated to a named shared-memory-entity)

What do you think of it? could it be useable?

[Pype.Clicker]

so if i understood correctly, you'd have something like

Code: Select all

struct free_page_used_for_bookkeeping {
    struct free_page_used_for_bookkeeping *more_free_pages;
    int free_pages_left_in_here;
    phys_addr free_pages[1022];
};

free_page_used_for_bookkeeping* bookkeeper;

phys_addr allocate() {
    if (bookkeeper->free_pages_left_in_here) {
       return bookkeeper->free_pages[bk->left--];
    } else {
        struct ___* this=bk;
        bk=bk->next;
        return phys_addr_of_then_unmap(this);
    }
}

I'm just scratching at "list of processes that ownz the page" ... that seems excessive overhead to me. I'd rather try to have this information per *region* (e.g. list of process that share region X) -- with of course enough per-page info to know if it's actually shared or not and if it is, by how many processes.

But i admit that's something i still have to handle myself ...

[distantvoices]

have implemented the stuff yesterday evening in about half an hour and it rocks like hell. This quickly it 's been never before. *gg* Needs some testing of course.

[ranting]
and my launcher application (something like the task bar in windows) has some issues too - open/close as many applications as you want. It works. Then there remain some orphaned items in the "open windows"-popup. Et voila, one clicks on one such entry and it causes gui service to crap out. Not Nice but I'm sure rather easy to debug. *grmpf*
[/ranting]

@pype: You are right with the "list of processes that ownz page". It is kinda ... odd to do it this way and it has a bit of overhead. Especially when we do shared libraries (shared memory with code to call so to say), we gonna have lots of owner entries.

Mystran too has pointed out in his vmm paper, that regions (logical entities of memory which isn't actually mapped into an address space) are a good approach to solve the shared memory thing.

Hm ... let's have it evolve a bit. *gg* I'm in a mood to try out shared libraries.

[Pype.Clicker]

you may wish to look for a paper about "VMM management in BSD" too. I found it very enlightening. (that means, i felt the need to re-think my whole approach of VMM after reading it)

i'll try to doclocate it ...

no, found "Composing High-Performance Memory Allocators", but not the paper i wish to refer to... it must be @home...