Grubs multiboot memory map

[McZ]

I have successfully read the memory map that grub loads for me (well that wasn't too hard ::) ) I have searched but I havn't found any good information about the memory map

I found this link about Multiboot specefication and it says:

The map provided is guaranteed to list all standard RAM that should be available for normal use.

so then I understand it would be the physical address space I can do anything I like to with?

[durand]

The memory map lists different types of memory regions: reserved, available, acpi stuff.. You should use the regions of memory marked as available (and maybe the acpi release ones if you don't want them). These are all physical memory regions.

Just remember that there are still memory mapped devices, modules, legacy code, etc, loaded into this memory. For example, 0xB8000 will be the text screen. The BIOS, ACPI, etc is sitting beneath the 1MB mark. Anything right above your kernel could be the modules that grub has loaded for you.

For the modules, you'll need to parse more of the multiboot structures to determine where they are and reserve those parts of memory. Reserve anything else you want to keep and then it's relatively safe to use the rest.

[JAAman]

i believe multiboot complient boot systems (or at least grub) simply pass the numbers from the BIOS (e820), so the types should be the same as those -- look that up in the RBIL

[McZ]

So then the physical addresses given by grub are free except those that I found here Physical memory layout

[Rob]

McZ: in my opinion there are a few problems with that page.

The EBDA (Extended BIOS Data area) does not have to begin at that address. On two systems here it begins at a different address (but in that area). There is a field in the normal BDA (which is in a fixed place) that indicates the EBDA segment.

Also the structures at the end of physical ram (ACPI etc.) do not have to exist at all or even be in that location. BIOS function E820 (if it exists) gives you details on which memory regions (above 1 MB) are unusable.

Personally I would simply discard the whole area below 1 MB (except when booting, of course) and simply do all my stuff above 1 MB, but others might see that as too much of a memory waste. In that case you need to check how much low memory actually exists (if I remember correctly that is BIOS INT 11h) and where the EBDA starts. Only use memory below the EBDA (and the end of low memory). And if you want to return to BIOS / DOS after you switch to protected mode make sure you leave the IVT + BDA in tact as well (otherwise you can overwrite that).

Again, I think it is just too much trouble in this day and age for the extra 630 - 640 KB of memory you are getting that way. Unless of course you are planning to run on old (ancient?) technology or with very low amounts of memory. Then every byte counts.

Also when using grub or E820 memory map make sure you check where *available* extended memory actually starts. I can imagine this might not be @ 1 MB in all cases.

[McZ]

This is the memory map information I get from GRUB

Code: Select all

Memory map address: 0x2C628, length: 48 bytes                                   
 size: 20 bytes, address: 0x00, length: 0x09FC00, type: 0x1                     
 size: 20 bytes, address: 0x0100000, length: 0x01F00000, type: 0x1              

but as far as I can understand I need the memory <1mb to be able to create a v8086 monitor for real-mode bios interrupts.

is it safe if I stay inside the 500-9FBFF area or will some EBDAs start before 0x9FC00?

EDIT: Is there a way to find out the physical address where GRUB has loaded my kernel, and the size of the kernel?

I read somewhere that the size/end of the kernel can be found by putting some kind of end symbol at the end of the kernel image, I can't find that again so I don't know how and how can I be sure that the symbol is unique what if that symbol apears in some other place in the kernel?

[JAAman]

no, iirc you only need the bottom 1MB virtual not physical for VMode

Grub loaded your kernel where you told it to!

the EBDA may, on some computers, be a different address: there is an address in the BDA that contains the start address (this was mentioned above, you must not have seen it?)

grubs memory map is simply a copy of the map returned by e820 -- and may contain types other than 1 -- not all of these are availible for use, and sometimes this map will contain addresses lower than 1MB, but many times it wont (however the addresses below 1MB are quite well documented)

The BIOS, ACPI, etc is sitting beneath the 1MB mark.

not quite acurate: the BIOS is actually located at 4GB-xMB (where x is the size of the BIOS ROM chip) -- the area directly below 1MB, is a copy of the BIOS, located in RAM -- and, afaik, recoverable -- that is, if you don't need the BIOS in RMode -- though i wouldn't recommend this, but this statement lacks acuracy

[Rob]

I would assume the EBDA could be below that. You can find a 16-bit word at 0040h:000Eh which is the segment where EBDA is located which you can convert to a physical address by doing a SHL xx, 4

0040h:000Eh is the same thing as 0000:40Eh or 40Eh as the physical address.

I would calculate how much low memory you actually need at start at 07C0h:0000h and calculate if it fits inside the gap left by EBDA / maximum amount of low memory)

I don't know GRUB myself, but you can at least look at where the current instruction is by doing:

call near FakeJump

FakeJump:
pop eax ; pop return address

Not sure if the call pushed anything else on the stack, but I don't think so with a near jump. If you place that at your kernel's entry point you have the current EIP in EAX.

I would assume GRUB also gives you information on where you are loaded etc. (you would need that to decide which memory is available for your memory manager). But I have no idea where you could find that.

[Rob]

JAAMan: what would happen if I told grub to load the kernel / modules at 8 MB and I only have 4 MB of memory?

[Brendan]

Hi,

McZ: in my opinion there are a few problems with that page.

There are a lot of problems with that particular page - IMHO it's mostly obsolete (it might have been less incorrect for 80386).

Specifically:

• - the EBDA size and the size of both the video ROM and BIOS aren't fixed sizes. Some computers are "headless" and have no video (or video ROM).
  - the area between the video ROM and BIOS has never contained "nothing". It has always been reserved for other "device specific ROMS" (SCSI BIOS, ethernet boot ROMs, etc) and for some systems this area may contain usable RAM (some OS's, like Windows 95, actually scan this area and use any RAM found within it).
  - for older computers (and possibly recent computers, including some new computers if the BIOS is configured for it) the area between 15 MB and 16 MB is a "memory hole" that was (and may still be) used for old ISA devices for memory mapped I/O. This area was most often used by ISA SVGA cards, but could be used by any ISA card (even an old ISA card plugged into a brand new computer).
  - NVRAM and ACPI do not use memory above 0xFEC00000 and never have. Instead memory at the top of RAM is used (for e.g. just below 0x10000000 for a computer with 256 MB of RAM).
  - typically there's an area below 0xFEC00000 that is used for memory mapped PCI/AGP devices (e.g. video linear frame buffer). The size of this area depends on the BIOS and motherboard (often it begins at 0xC0000000 or 0xE0000000). This means that for a computer with 4 GB of RAM, a "32 bit" OS will only be able to use 3 GB or 3.5 GB of it. Most "good" motherboards will map the spare RAM above 4 GB (an OS will need to use PAE, PSE36 or long mode to access it). Cheap motherboards may just waste this RAM instead.
  - the area from 0xFEC00000 to the bottom of the BIOS (which is actually just below 4 GB - all or part of it appears to be below 1 MB due to chipset tricks) is mainly reserved for local APICs and I/O APICs (if any).
  - RAM may be above 4 GB on newer computers. This isn't that common yet, but Vista hasn't been officially released yet either.
  - for servers, things can be even more complicated. NUMA is one problem (which includes simple "dual-socket" AMD systems) which may cause RAM to be configured in banks (e.g. 512 MB at 0x00000000 and another 512 MB at 0x80000000 with a big hole in-between). Also some servers support "hot-plug" RAM areas (which are typically above "installed RAM at boot", and also typically above 4 GB).

Personally I would simply discard the whole area below 1 MB (except when booting, of course) and simply do all my stuff above 1 MB, but others might see that as too much of a memory waste. In that case you need to check how much low memory actually exists (if I remember correctly that is BIOS INT 11h) and where the EBDA starts. Only use memory below the EBDA (and the end of low memory). And if you want to return to BIOS / DOS after you switch to protected mode make sure you leave the IVT + BDA in tact as well (otherwise you can overwrite that).

I think you mean "int 0x012", which returns the amount of usable RAM starting from 0x00000000. This excludes the EBDA - it's entirely safe for an OS to use all RAM returned (except for the real mode IDT, BDA, etc if you want to use real mode).


Cheers,

Brendan

[zyp]

If you tell GRUB to load your kernel above your physical ram it will refuse with an ?out of memory? message.

[Brendan]

Hi,

This is the memory map information I get from GRUB

Code: Select all

Memory map address: 0x2C628, length: 48 bytes                                   
 size: 20 bytes, address: 0x00, length: 0x09FC00, type: 0x1                     
 size: 20 bytes, address: 0x0100000, length: 0x01F00000, type: 0x1              

That is the broken/dodgy memory map you get from Bochs or QEMU (which both use the same BIOS image). All real computers return a "system area" below 1 MB and other areas (ACPI areas, the system area below 4 GB, etc).

is it safe if I stay inside the 500-9FBFF area or will some EBDAs start before 0x9FC00?

I wouldn't touch anything between 0x00080000 and 0x00100000 without checking first. There is no limit for the size of the EBDA (but up to 0x00080000 should be safe).

not quite acurate: the BIOS is actually located at 4GB-xMB (where x is the size of the BIOS ROM chip) -- the area directly below 1MB, is a copy of the BIOS, located in RAM -- and, afaik, recoverable -- that is, if you don't need the BIOS in RMode -- though i wouldn't recommend this, but this statement lacks acuracy

The area used by the BIOS below 0x00100000 is recoverable, but it's "chipset specific". For "PCI only" machines the area used for device ROMs (between 0x00C0000 and the bottom of the BIOS copy) is recoverable, and the area used for the primary video device's display memory mapping (from 0x00A0000 to 0x00C0000) is also recoverable.

Unfortunately, the BIOS uses SMM code for APM and ACPI "behind the OS's back" which will probably crash the system if you try to recover some areas. Specifically, the EBDA, the area from 0x00A0000 to 0x00C0000 and the BIOS's area below 0x00100000 is not safe to mess with (even though it's possible). Using PCI to shift device ROMs and then using RAM between 0x00C0000 and the bottom of the BIOS copy should be "safe" on modern motherboards. For an extra 192 KB or less, it's probably not worth the hassle though.

SMM sucks - it'd be nice to recover everything and have a single, clean, contiguous chunk of usable memory from 0x00000000 to the top of RAM...


Cheers,

Brendan

[Rob]

Here are some results from two of my machines.

::AMD Athlon machine::

Conventional memory in KB = 639
EBDA real mode segment = 0x9FC0

base = 0000 0000, length = 0009 FC00 -> available
base = 0009 FC00, length = 0000 0400 -> reserved
base = 000F 0000, length = 0001 0000 -> reserved
base = 0010 0000, length = 1FEF C000 -> available
base = 1FFF C000, length = 0000 3000 -> ACPI reclaim
base = 1FFF F000, length = 0000 1000 -> ACPI INVS
base = FEC0 0000, length = 0000 1000 -> reserved
base = FEE0 0000, length = 0000 1000 -> reserved
base = FFFF 0000, length = 0001 0000 -> reserved

::DELL Pentium 4 laptop::

Conventional memory in KB = 636
EBDA real mode segment = 0x9F00

base = 0000 0000, length = 0009 F000 -> available
base = 0009 F000, length = 0000 1000 -> reserved
base = 0010 0000, length = 3FEA A800 -> available
base = 3FFA A800, length = 0005 5800 -> reserved
base = FECF 0000, length = 0000 1000 -> reserved
base = FED2 0000, length = 0007 0000 -> reserved
base = FEDA 0000, length = 0006 0000 -> reserved
base = FFB0 0000, length = 0050 0000 -> reserved
base = FEC0 0000, length = 0000 1000 -> reserved
base = FEE0 0000, length = 0000 1000 -> reserved

So it seems Brendan is right that the total amount of conventional memory goes right up to the EBDA and it includes a reserved area below 1 MB. So no worries there. The E820 memory map stops at the right place as well.

Funny enough the newer DELL laptop has no ACPI memory blocks (though it, of course, has ACPI)

You can also see that the EBDA is in two different locations.

Brendan:

for older computers (and possibly recent computers, including some new computers if the BIOS is configured for it) the area between 15 MB and 16 MB is a 'memory hole' that was (and may still be) used for old ISA devices for memory mapped I/O. This area was most often used by ISA SVGA cards, but could be used by any ISA card (even an old ISA card plugged into a brand new computer).

Do you know if that shows up in the E820 memory map?

p.s. it was indeed int 12h instead of 11h, my bad.

[Rob]

is it safe if I stay inside the 500-9FBFF area or will some EBDAs start before 0x9FC00?

As you can see from my example above it can start below 0x9FC00 (physical). Simply do an int 12h and multiply the returned number by 1024 (shift left by 10 bits). Keep in mind that this might not fit inside 16-bit word. Use a 32-bit dword.

This returns the *physical* address which you need to stay under. So that should give you this physical memory range:

0x500 - (int 12h << 10)

[Brendan]

Hi,

Brendan:

for older computers (and possibly recent computers, including some new computers if the BIOS is configured for it) the area between 15 MB and 16 MB is a 'memory hole' that was (and may still be) used for old ISA devices for memory mapped I/O. This area was most often used by ISA SVGA cards, but could be used by any ISA card (even an old ISA card plugged into a brand new computer).

Do you know if that shows up in the E820 memory map?

It should. For example, my Pentium Pro (with the memory hole enabled) gives:

0x00000000 to 0x0009FBFF -> available
0x0009FC00 to 0x0009FFFF -> system (EBDA)
0x000E0000 to 0x000FFFFF -> system (BIOS copy)
0x00100000 to 0x00EFFFFF -> available
0x00F00000 to 0x00FFFFFF -> system (ISA device space)
0x01000000 to 0x07FFFFFF -> available
0xFFE00000 to 0xFFFFFFFF -> system (APIC and BIOS combined)

For comparison, if I set the "memory hole" BIOS option to "disabled" I get:

0x00000000 to 0x0009FBFF -> available
0x0009FC00 to 0x0009FFFF -> system (EBDA)
0x000E0000 to 0x000FFFFF -> system (BIOS copy)
0x00100000 to 0x07FFFFFF -> available
0xFFE00000 to 0xFFFFFFFF -> system (APIC and BIOS combined)

I wouldn't rely on this though - some BIOS's (with the memory hole enabled) might report something like this:

0x00000000 to 0x0009FBFF -> available
0x0009FC00 to 0x0009FFFF -> system (EBDA)
0x000E0000 to 0x000FFFFF -> system (BIOS copy)
0x00100000 to 0x00EFFFFF -> available
0x01000000 to 0x07FFFFFF -> available
0xFFE00000 to 0xFFFFFFFF -> system (APIC and BIOS combined)


Cheers,

Brendan