Where can I find physical memory mapped in protected mode?

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
yuki

Where can I find physical memory mapped in protected mode?

Post by yuki »

... or something! I have my basic kernel working: GDTs, IDTs, ISRs, IRQs and the like are setup, however I wish to begin memory management. I've been searching the 'net for a while, but I can't find anything related to where I can find my /physical memory/ mapped. One comment was that it begins at 0x0, however - let's say we have 256MB of memory - that equates to 0x10000000 as the upper bound. It also read that we had to skip the area between 0xa0000 and another... but I don't think it'd be like that.

Is there a simpler way - can we ask something to hand us a page of memory, or ... etc.

Thanks for your help,
Yuki.
AR

Re:Where can I find physical memory mapped in protected mode

Post by AR »

Physical memory isn't "mapped" anywhere, physical memory is physical memory, starts at 0 goes to 4GB, of course addresses beyond the amount of RAM can't be used for storage. However, there ARE reserved areas which are where things like the BIOS, Video Memory, PCI Memory, PCI boards, etc are mapped in physical memory. These can be (and usually are) mapped in the 4GB area where there isn't actually a RAM stick for.

There is no builtin allocator, you request the memory map from the BIOS which tells you which parts you can't use then you write a physical memory manager which converts this map into a bitmap and hands out chunks (Usually 4096 bytes, the same as a virtual page).

The only time mapping becomes involved is when you enable paging in the CPU, then you have to create Page tables which map virtual memory to physical memory. I suggest you look at the entries in the Wiki (Click the banner than says "MEGA-TOKYO.COM") and the Quicklinks
yuki

Re:Where can I find physical memory mapped in protected mode

Post by yuki »

Indeed, I have read through the Wiki a bit. So, does this imply that accesing 0x0 is actually accessing the first byte of RAM that exists? If so, how would I access the 753664th byte of RAM? (0xb8000 ^_^)
AR

Re:Where can I find physical memory mapped in protected mode

Post by AR »

Yes, without paging 0 refers to first byte of RAM.
To write 0xB8000 in protected mode (Assuming the Data Segment Register is 0 to 4GB and paging is off):

Code: Select all

mov [0xB8000], '!'
Or in C:

Code: Select all

*(char *)0xB8000 = '!';
Or in C++:

Code: Select all

*reinterpret_cast<char*>(0xB8000) = '!';
yuki

Re:Where can I find physical memory mapped in protected mode

Post by yuki »

Well, I know literally how to do it, otherwise the kernel would not be as developed as it is .. but 0xb8000 is the video-memory, no? Is it actually mapped as video memory in the RAM itself? [as in, if I picked apart the RAM and found 0xb8000, is it there?]

That is to say, has someone just decided to say, "0xb8000 is now for video RAM". I thought it was just that some video memory (located in graphics card, or elsewhere?) was mapped in at 0xb8000.

[edit]: Similarly, if I asked for 0xff800000, just because I've loaded my kernel into there does not imply I necessarily have almost 4GB of RAM ^_^
AR

Re:Where can I find physical memory mapped in protected mode

Post by AR »

That's the problem with physical memory, you don't physically know where it is. 0xA0000 - 0xBFFFF is in the video RAM.

As to why, unfortunately it's an inherited legacy. IBM in all its wisdom just decided to pick regions of things (memory, IO Ports) almost at random to scatter various hardware components. Apparently they didn't expect the 8086 to take off but now it has and we're stuck with their mistakes.

As how to tell where this stuff is, the physical memory map from the BIOS has 4 possible classifications of physical memory: Free, Reserved, ACPI Something and ACPI Something else. Reserved usually equals a memory mapped device (like video RAM for instance).

On a side note, yes there is (or at least should be) space on the RAM stick that represents 0xA0000-0xBFFFF, but it's effectively "dead" since attempting to write to that location goes to the video card.
yuki

Re:Where can I find physical memory mapped in protected mode

Post by yuki »

So then .. if I read from 0x1000000, for instance.. where does it come from? I seem to have 4GB of addressable space, but where does it all go, when I only have 256MB of RAM?
AR

Re:Where can I find physical memory mapped in protected mode

Post by AR »

Basically physical memory goes from 0 - However many MB of RAM you have. The memory mapped devices are called "memory holes" and are flagged as "Reserved" by the BIOS. Like I said, there is 4GB of addressable space, but for the purposes of storing data you can only use what is actually classified as "Free" by the BIOS.

To make up an ASCII diagram:

Code: Select all

---0-3FF---|---400-4FF---|---500-9FFFF---|---A0000-BFFFF---|---C0000-FFFFF---
 RAM (IVT)    BIOS in RAM         RAM            Video RAM       BIOS ROM
And so-on. 0x100000 goes on to the end of available RAM but may have holes. Beyond the end of the available RAM is either nothingness or Memory mapped hardware (My PCI Bus on this computer starts at the end of available RAM [1GB RAM - 0x40000000] and continues to the BIOS ROM at the end [0xFFBFFFFF])
yuki

Re:Where can I find physical memory mapped in protected mode

Post by yuki »

Aah. Really, it is senseful to start using memory from 0x100000, and only end up losing about 640K, I suppose. (aka "conventional memory" ?)

Thankyou very much! ^_^ Time is appreciated.

[ed]: I suppose you get an exception, or just nothing if you r/w to nothingness?
AR

Re:Where can I find physical memory mapped in protected mode

Post by AR »

I'm not sure what happens if you access nothingness as I've never tried myself, I imagine it would simply have no effect (The data goes down the memory bus, finds that there is nowhere to go so the data is dropped) or it might cause the bus to float. So if you write 0xAA to 0x20001000 [assuming it doesn't exist] and read it back you'll probably get either 0x00 or 0xFF.

Note that it may work correctly if the CPU puts it in the cache, if it does then it doesn't matter whether it exists or not, but when the cache is flushed it'll be lost anyway. Also be aware that if the bus does float, the data may be remembered by the bus itself until the next request for reading/writing comes in. Needless to say, you shouldn't rely on these things.
mystran

Re:Where can I find physical memory mapped in protected mode

Post by mystran »

Actually, last I read BIOS did map physical memory and MMIO areas and whatever into what we perceive as the physical memory, so the best way might be asking BIOS for the map?

Indeed, asking BIOS is a good idea also, because there might be holes in physical memory (many BIOS still have the OS/2 setting which forces a hole at 16MB or something like that), and there might be MMIO areas where it's not a good idea to go test if it's there either... (note that hole!=MMIO, they are two different things really, although your memory manager probably wants to skip MMIO areas as holes).

Personally I really don't remember the relevant BIOS services, because I happen to leave that part to GRUB which gives you the physical memory map if you ask for it. Somebody else here probably knows the details of how to get it.

But in short: forget about guessing, and ask BIOS. Really!
yuki

Re:Where can I find physical memory mapped in protected mode

Post by yuki »

Wouldn't that require one to do it before we enter protected mode? [for me, in the bootloader] In any event - which interrupt does it?
XStream

Re:Where can I find physical memory mapped in protected mode

Post by XStream »

Newer BIOSes - GET SYSTEM MEMORY MAP

AX = E820h
EAX = 0000E820h
EDX = 534D4150h ('SMAP')
EBX = continuation value or 00000000h to start at beginning of map
ECX = size of buffer for result, in bytes (should be >= 20 bytes)
ES:DI -> buffer for result (see #00581)

Return:

CF clear if successful
EAX = 534D4150h ('SMAP')
ES:DI buffer filled
EBX = next offset from which to copy or 00000000h if all done
ECX = actual length returned in bytes
CF set on error
AH = error code (86h) (see #00496 at INT 15/AH=80h)

Notes: Originally introduced with the Phoenix BIOS v4.0, this function is now supported by most newer BIOSes, since various versions of Windows call it to find out about the system memory. A maximum of 20 bytes will be transferred at one time, even if ECX is higher; some BIOSes (e.g. Award Modular BIOS v4.50PG) ignore the value of ECX on entry, and always copy 20 bytes. Some BIOSes expect the high word of EAX to be clear on entry, i.e. EAX=0000E820h. If this function is not supported, an application should fall back to AX=E802h, AX=E801h, and then AH=88h. The BIOS is permitted to return a nonzero continuation value in EBX and indicate that the end of the list has already been reached by returning with CF set on the next iteration. This function will return base memory and ISA/PCI memory contiguous with base memory as normal memory ranges; it will indicate chipset-defined address holes which are not in use and motherboard memory-mapped devices, and all occurrences of the system BIOS as reserved; standard PC address ranges will not be reported

...

Format of Phoenix BIOS system memory map address range descriptor:

Offset Size Description (Table 00580)
00h QWORD base address
08h QWORD length in bytes
10h DWORD type of address range (see #00581)

(Table 00581)
Values for System Memory Map address type:
01h memory, available to OS
02h reserved, not available (e.g. system ROM, memory-mapped device)
03h ACPI Reclaim Memory (usable by OS after reading ACPI tables)
04h ACPI NVS Memory (OS is required to save this memory between NVS sessions)
other not defined yet -- treat as Reserved

------------------------------------------------------------------------

Do a search for Ralph Brown's interrupt list, it has a huge amount of useful info.

Cheers.
AR

Re:Where can I find physical memory mapped in protected mode

Post by AR »

The BIOS information XStream posted is in the Wiki as well:
http://www.osdev.org/osfaq2/index.php/H ... f%20RAM%3F
It also includes direct probing and fall back interrupts if the newer ones aren't supported.

Yes, you have to ask the BIOS before entering protected mode
Post Reply