Memory manager

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
sfpiano
Posts: 8
Joined: Sat Apr 01, 2006 12:00 am

Memory manager

Post by sfpiano »

I've been reading along with Tim Robinson's tutorial:
http://www.osdever.net/tutorials/pdf/memory1.pdf

and I think I understand what I'm supposed todo, I just have no clue how to do it. The first thing I tried to do was determien my memory size. I'm using grub, so I'm trying to use the multiboot structure like so:

Code: Select all

STACKSIZE equ 0x4000

start:
    mov	esp, stack+STACKSIZE
    push eax
    push ebx
...
SECTION .bss
align 32
stack:
    resb STACKSIZE
When I print out the values for the base mem and extended, I get 639, and 31744 respectively; to me those numbers seem off.

Then I believe I'm supposed to extend the kernel memory, put a pointer at the end of the actual kernel itself, and reserve the rest of that block for the kernel.

After that, I need to setup paging which I was going to use a stack for.
User avatar
xenos
Member
Member
Posts: 1121
Joined: Thu Aug 11, 2005 11:00 pm
Libera.chat IRC: xenos1984
Location: Tartu, Estonia
Contact:

Re: Memory manager

Post by xenos »

That numbers are normal, indicating that you have 32MB RAM. They arise as follows:

You have 31MB (= 31744kB) of upper memory, just as GRUB says. The lower 1MB of memory contain the BIOS ROM, video memory etc. There should be 640kB remaining. 1kB is the extended BIOS data area (EDBA), leaving you with 639kB lower RAM.
sfpiano
Posts: 8
Joined: Sat Apr 01, 2006 12:00 am

Re: Memory manager

Post by sfpiano »

Ok, but I have 1Gb of RAM on my computer, so why is it only giving me 32 Mb? Is the only way to do that using int 15h? If so, I tried

Code: Select all

mov ax e820h
int 15h
but I don't know how to send the value of ax to my kernel.
User avatar
JAAman
Member
Member
Posts: 879
Joined: Wed Oct 27, 2004 11:00 pm
Location: WA

Re: Memory manager

Post by JAAman »

warning: it is dangerous to us 'total memory' numbers for larger amounts of ram -- it is quite posible (i have seen it) for hardware to be mapped into the middle of your ram (esp as you cross large boundaries -- like 1GB)

it is possible (i dont know) that grub (or your BIOS) caps this number to 32MB, for safty reasons, or that there is a gap in physical memory -- if you check the map (int15-e820 -- grub also passes this map to you if your BIOS supports it) it may show the full ram (and perhaps there is a gap at 32MB mark?)

personally, if the BIOS doesnt support e820, i dont allow myself to use more than 64MB regardless of how much other detection methods tell me there is


grub only supports those numbers for compatability with old computers (which dont support e820) if e820 is supported, you should use the map instead (grub does pass a copie of the e820 map to you -- though i dont use grub, so i cant tell you where it is)
Da_Maestro
Member
Member
Posts: 144
Joined: Tue Oct 26, 2004 11:00 pm
Location: Australia

Re: Memory manager

Post by Da_Maestro »

All the computers that I've seen with a gap have had the functionality in BIOS to close the gap and have continuous memory.

I think it's something to do with compatability with OS/2 but I'm not sure
Two things are infinite: The universe and human stupidity. But I'm not quite sure about the universe.
--- Albert Einstein
User avatar
xenos
Member
Member
Posts: 1121
Joined: Thu Aug 11, 2005 11:00 pm
Libera.chat IRC: xenos1984
Location: Tartu, Estonia
Contact:

Re: Memory manager

Post by xenos »

You can't just call int 15h because it's a real mode interrupt and GRUB already sets up protected mode. So the best way is to use the memory map passed by GRUB. Have a look at http://www.gnu.org/software/grub/manual/multiboot/ for details.

BTW, just a stupid question: You are not using bochs or another emulator, are you?
sfpiano
Posts: 8
Joined: Sat Apr 01, 2006 12:00 am

Re: Memory manager

Post by sfpiano »

Yes, I'm using Bochs, and hey look, my config file says megs:32 ... didn't even think of that.

Anyway, I thought the total memory was what was available for the system to use, so why would grub or anyone else cap that amount?
User avatar
JAAman
Member
Member
Posts: 879
Joined: Wed Oct 27, 2004 11:00 pm
Location: WA

Re: Memory manager

Post by JAAman »

it prob doesnt, however it could for safty purposes -- as i said, it is dangerous to rely on that value, as it doesnt account for mem-mapped IO or device memory (both of which map into the normal memory address space, and can be dangerous to write to -- the largest of these is the video memory -- but there is also other memory and IO in the address space) usually the devices will be mapped high in the address space, but if you have larger amounts of RAM, you may overlap -- where the same address maps to both system RAM and another device (and many BIOS will include this in the total RAM -- though some will cap the RAM at this point to avoid this problem, not all will)
Last edited by JAAman on Wed Apr 05, 2006 11:00 pm, edited 1 time in total.
ktulu1115
Posts: 1
Joined: Sat Apr 08, 2006 11:00 pm

Re: Memory manager

Post by ktulu1115 »

Correct me if I'm wrong, but isn't E820 supposed to list the "reserved" and otherwise unsafe areas of RAM - ie: where memory and io-mapped devices lie? I thought that was the whole point of the code.

Now obviously on an emulator like Bochs it won't report the same results as real hardware.
Da_Maestro
Member
Member
Posts: 144
Joined: Tue Oct 26, 2004 11:00 pm
Location: Australia

Re: Memory manager

Post by Da_Maestro »

E820 reports everything including:
- Free memory
- Reserved memory
- Some IO devices
- ACPI Reserved Memory
Two things are infinite: The universe and human stupidity. But I'm not quite sure about the universe.
--- Albert Einstein
User avatar
JAAman
Member
Member
Posts: 879
Joined: Wed Oct 27, 2004 11:00 pm
Location: WA

Re: Memory manager

Post by JAAman »

yes, it does:

Now obviously on an emulator like Bochs it won't report the same results as real hardware.
which real hardware? thats the point -- every system you test on is likly to produce a different map -- there will be different results on every system you test, bochs is no exception, as it emulates a system, its results will be for the specific system which it emulates -- which will be different than any other system, whether it be another emulator or a 'real' system


if you reread my post, you will find i was specifically talking about the total-memory value, not e820, and i said that this is the reason you should use e820 (or the grub map -- which is an exact copy of the e820 map) rather than the total-ram value, which can be dangerous -- as it [the total-ram value] doesnt account for holes, mem-mapped IO, device memory, etc. and the author of this thread is using the total system memory value rather than the e820 map
gmoney
Member
Member
Posts: 106
Joined: Mon Jan 03, 2005 12:00 am
Location: Texas, Usa

Re: Memory manager

Post by gmoney »

while we're on the subject of memory managment, how would you add a memory manager to a kernel. I've add malloc and free to my kernel already but i know i need more if you can help i would value it greatly.
Just because your phone is "smart" doesn't mean the user is... ijs
User avatar
carbonBased
Member
Member
Posts: 382
Joined: Sat Nov 20, 2004 12:00 am
Location: Wellesley, Ontario, Canada
Contact:

Re: Memory manager

Post by carbonBased »

A form of malloc and free is all that should be needed to support the kernel heap, really. Some would say a microkernel doesn't even require these... I, personally, still have a managed kernel heap.

You'll probably also want to implement code to make use of the pager to map memory, and make use of the exceptions it will generate on unmapped pages to perform (disk) swapping.

Lastly, you'll need some way for an application to get a chunk of memory. The app can manage it, itself... using its own form of malloc/free implementation, if it wishes.

--Jeff
User avatar
JAAman
Member
Member
Posts: 879
Joined: Wed Oct 27, 2004 11:00 pm
Location: WA

Re: Memory manager

Post by JAAman »

malloc and free are not usually part of the kernel

rather they are part of the user library (which is linked with the application at compile time)

malloc and free, interface with kernel functions only when neccessary:

example:

the application will call malloc to allocate 100 bytes, the librarys malloc implementation will make a syscall asking for a large block of memory (usually 4KB), it will then allocate 100 bytes, next time it is called, it will allocate more only if it runs out of memory in its internal allocation, free will add to the libraries internal allocation and rarely will it call the kernel to truely free the memory (again, full blocks at a time) -- the syscalls will always deal with full pages (the only exception is that in some implementations, the kernel needs malloc and free for its use in kernel land -- but is not availible to user applications)

an example of a posible malloc/free implementation:

the application will call malloc to allocate 100 bytes, the librarys malloc implementation will make a syscall asking for a large block of memory (usually 4KB), it will then allocate 100 bytes, then it will place a marker at the begining, indicating that the next 100 bytes are in use, then a marker after the 100 bytes indicating that the rest of the block is free

then it will return the address immediately after the first marker (which is the begining of the 100bytes)

when free is called, it will look at the address just before the address specified for the marker, telling it how much to free, then it will look ahead to the next marker and if it is also free, it can combine them (just add the length of the block to the start to find the next header -- looking behind is a little trickier unless you also store a back pointer)

it will only make a syscall when it doesnt have a block large enough to satisfy the request (and free will only truely free when it has multiple complete pages and can be reasonable certain that it wont need that many again)



for starting your memory manager, you should start with a physical memory manager (which maps physical RAM into process address space -- on demand) then a virtual memory manager which allocates virtual memory (within the process address space) -- neither of these should ever deal with partial pages

for me (currently) i have no malloc/free for my kernel (most structures are 4k -- a complete page)


hope this helps
gmoney
Member
Member
Posts: 106
Joined: Mon Jan 03, 2005 12:00 am
Location: Texas, Usa

Re: Memory manager

Post by gmoney »

thanks for the help but i was looking for more like a tutorial or something, i have a topic under development so post it there
Just because your phone is "smart" doesn't mean the user is... ijs
Post Reply