Memory Manager

[Whatever5k]

Hey folks, I'm at the beginning of writing a memory manager - a hard and important step to do... .
I just want to clarify things:
The mm (memory manager) is the part of the kernel that actually deals with physical addresses. It's responsible for (de)allocating pages and mapping virtual addresses to physical ones.
First of all, I'll need to decide where I'll put the memory manager. Tim Robinson seems to put it right at the end of the kernel, that's fine with me. The allocator just looks for free memory (by looking into a bitmap or s.th. similar), marks it as reserved (in the bitmap) and returns the address to the user who can now use it, right? The Deallocator does the same, just the other way round
And the mapping depends on wether I have paging enabled or not, is that correct? Because if so, I'll need to create page tables etc. and enable paging. But that's it, isn't it. Nothing more to do, is there? Is that the "mapper's" job?

That was much to read
Is my comprehension of a Memory Manager ok (and is it modern enough)? Anything forgotten? Do you wanna add something?

best regards and thanks for your help,
A. Blessing

[K.J.]

The allocator just looks for free memory (by looking into a bitmap or s.th. similar), marks it as reserved (in the bitmap) and returns the address to the user who can now use it, right? The Deallocator does the same, just the other way round

Yes. Though if you are using paging, the allocator will not return the physical address of memory to the user. It returns the linear(apparent) address. If you choose not to use paging, it will return the physical address.

Since it seems you want to use paging, you will actually need two allocators, one for physical memory that returns physical addresses for whatever builds page tables, and one for the user that returns linear addresses.

But that's it, isn't it. Nothing more to do, is there? Is that the "mapper's" job?

Well... that's a very simplified paragraph you wrote, but that's basically it. Depending on whether or not you use paging, you will need to code a page fault handler. That's mostly it.

K.J.

[Whatever5k]

Well... that's a very simplified paragraph you wrote

Why is that very simplified? Can you add anything, have I forgotten something?

[Whatever5k]

I have a question concerning the model of the two Allocators:
Writing a Physical Memory Allocator is (more or less) easy. We have a linked list with used and unused blocks, and given the size we can easily find a fitting block for a request. Next, we return the *physical* address (it's the Physical Memory Allocator).

But what's about the Virtual Memory Allocator? This allocator is used by user programs, isn't it? So what happens when a user wants to have let's say 2 KB of memory? Does the virtual memory allocator call the Physical Allocator at first, then maps the Physical address into a virtual address and returns it to the user? Remember: I'm talking about a architecture with paging enabled!

So, is that the right way to do it? If yes, how do I map the Physical address into a virtual one?
Best regards,
A. Blessing

[Pype.Clicker]

user: "i need 2K of free mem"

user-mem allocator:
* find out 2K of free mem in user region (this is a malloc-like feature)
* check if this memory has already been allocated by the user process (it could be on a newly not-yet-ready page or on some page already used, or cross a page boundary ...)
* policy 1 : immedi4te allocation:

Code: Select all

foreach (unitialized page) page={frame:physicalManager.getFreeFrame(), present:1};

* policy 2 : del4ayed allocation

Code: Select all

 foreach (unitialized page) page = {present:false, valid:true, zero_on_pagefault:true};
on page_fault do {
   if (!page.present && page.valid && page.zero_on_pagefault) {
      page = {frame:physicalManager.getFreeFrame(), present:1};
      bzero(page.computelinearaddress(),page.PAGE_SIZE);
};

[Whatever5k]

Ah, I see...
So, if I want for example 100 kilobytes, a malloc-like function looks for 100 kb of *physical* memory and checks if we can use this area.
Next, the pages in this memory area are build by looking for free page frames and setting the present bit to 1.
Right?

And when a page fault occurs, the mm looks for some pages that can be swapped out and for free page frames and connects them, right?

[Slasher]

yep, you got the idea.
what happens in the VM manager is similar to your physical memory allocator.
once pages have been mapped into a users space, its up to malloc to allocate regions of these pages to the users as demanded. but is free regions run out malloc then requests more pages and adds these to the list of free pages(memory) available to the current task!
;D
haven't implemented my paging yet so alot of details have not be fully explored yet ;D

[K.J.]

Why is that very simplified? Can you add anything, have I forgotten something?

You didn't leave out anything, you just didn't cover the dozens of ways to do parts of it.

Remember that while the user allocator can allocate most any ammount of memory, the physical allocator is limited to 4kb/2MB/4MB units. A good malloc/free function takes this into account and realizes it will be given a 4kb/2MB/4MB amount of memory if it asks for less than that. So the malloc/free shares that amount of memory given by the physical allocator amoungst several malloc operations.

K.J.

[Whatever5k]

Alright, I understood the basic idea of a memory manager -
it consists of three parts: an allocator, a deallocator and a mapper.
The Allocator and Deallocator consist theirselves of a Physical and a Virtual one. The physical one is used for holding track of which memory blocks are used and which not. These is realized with a bitmap or a stack. The virtual allocator is used by the user programs, it searches for enough free memory, looks for free pages and free page frames, connects them and returns the virtual address.
The mapper is responsible for mapping virtual into physical addresses. This is done with paging. The mapper also has to handle page faults.

Ok, got it. But I still have one basic question:
How do I realize having two address spaces: a kernel address space and an user address space?
How is that done? How are the different address space realized?

best regards,
A. Blessing

[pskyboy]

I think its done by just setting the access bits on the page table entries.

Also i was thinking of using a link list for my physical memory managment, has anyone got any reasons why this would be a bad idea.

cheers
Peter

[Whatever5k]

By the way, what happens if the kernel wants to have more memory? Which allocator does he call, the physical or the virtual one?

[pskyboy]

I would guess it depends what your trying to do but normally it will probably be the physical one, this way you can keep all the kernel stuff in a linear run of memory.

Peter

[Slasher]

hello,
the kernel uses the malloc/free functions as the users but must be one that sets its page in the page table as SYSTEM not USER.
But safer to allocate say 4mb,8mb etc at the start of all page directories to the KERNEL ie allocate 8mb to kernel which is mapped to the beginning of all page directories (if you are using more than one) that way you will only need to call malloc only when you run out of space.
also its a good idea to use memory below 16mb cause of DMA and co.