Multiboot 2 memory map

[TheLittleWho]

Hello, I am trying to get the memory map from GRUB using Multiboot 2 structure. I've succeeded to read the memory info tags, but there is something strange about it. Even if I give only 256 MiBs of memory to Qemu, it shows me some region of reserved memory at 0xFFFC0000, and that's outside of physical memory for sure. What's it's purpose and why do GRUB infrom me about it if it doesn't physically exist?

Here is some code

Code: Select all

    // Iterate over tags and collect info
    uint32 memoryEnd = 0x0;
    struct reservedArea {
        uint32 addr;
        uint32 len;
    } areasToReserve[256];
    uint32 areasNumber = 0;

    struct multiboot_tag *tag;
    for (tag = (struct multiboot_tag*) (mboot2Addr + 8);
         tag->type != MULTIBOOT_TAG_TYPE_END;
         tag = (struct multiboot_tag*) ((multiboot_uint8_t *) tag
                                        + ((tag->size + 7) & ~7)))
    {
        switch (tag->type)
        {
            case MULTIBOOT_TAG_TYPE_BASIC_MEMINFO: ;
                struct multiboot_tag_basic_meminfo *mem = (struct multiboot_tag_basic_meminfo*) tag;
                //kprintf("Basic memory area %x - %x\n",
                //        mem->mem_lower,
                //        mem->mem_upper);
                break;
            case MULTIBOOT_TAG_TYPE_MMAP: ;
                multiboot_memory_map_t *mmap;
                struct multiboot_tag_mmap *mmapTag = (struct multiboot_tag_mmap*) tag;

                for (mmap = mmapTag->entries;
                     (multiboot_uint8_t*) mmap < (multiboot_uint8_t*) tag + tag->size;
                     mmap = (multiboot_memory_map_t *) ((unsigned long) mmap
                            + mmapTag->entry_size))
                {
                    kprintf("Memory area starting at %x with "
                            "length of %x and type %x",
                            (unsigned) (mmap->addr),
                            (unsigned) (mmap->len),
                            (unsigned) mmap->type);
                    
                    memoryEnd += (uint32) mmap->len;

                    if (mmap->type == MULTIBOOT_MEMORY_RESERVED)
                    {
                        areasToReserve[areasNumber].addr = mmap->addr;
                        areasToReserve[areasNumber].len = mmap->len;
                        kprintf("%x %x\n", areasToReserve[areasNumber].addr , areasToReserve[areasNumber].len);
                        areasNumber++;
                    }
                }
                break;
            default:
                //kprintf("Unkown tag %x, size %x\n", tag->type, tag->size);
                break;
        }
    }

[iansjack]

That is memory reserved for hardware devices.

[no92]

What's it's purpose and why do GRUB infrom me about it if it doesn't physically exist?

All GRUB really does here is to pass on the memory map it obtained. The area is marked reserved, so you don't have to worry about it anyways.

[TheLittleWho]

Um, so I could just ignore the reserved memory even it doesn't phisically exists?
Also, if I give more than 3.4 GiB of memory to Qemu, there appears a new free memory area from 0x0 to 0x20000000 or higher. Why?

[Octocontrabass]

Um, so I could just ignore the reserved memory even it doesn't phisically exists?

It does physically exist, that's why it shows up in the memory map. It just happens that it's not usable RAM, so you can ignore it.

Also, if I give more than 3.4 GiB of memory to Qemu, there appears a new free memory area from 0x0 to 0x20000000 or higher. Why?

Because you're not reading the memory map correctly. The address and length are 64-bit integers, but you're handling them as 32-bit integers. It will make more sense once you see the correct numbers.

[TheLittleWho]

I'm sure that they are 32-bit integers. You can see in the multiboot 2 header.

Code: Select all

struct multiboot_tag_mmap
{
  multiboot_uint32_t type;
  multiboot_uint32_t size;
  multiboot_uint32_t entry_size;
  multiboot_uint32_t entry_version;
  struct multiboot_mmap_entry entries[0];  
};

[Octocontrabass]

I'm sure that they are 64-bit integers.

Code: Select all

struct multiboot_mmap_entry
{
  multiboot_uint64_t addr;
  multiboot_uint64_t len;
#define MULTIBOOT_MEMORY_AVAILABLE		1
#define MULTIBOOT_MEMORY_RESERVED		2
#define MULTIBOOT_MEMORY_ACPI_RECLAIMABLE       3
#define MULTIBOOT_MEMORY_NVS                    4
#define MULTIBOOT_MEMORY_BADRAM                 5
  multiboot_uint32_t type;
  multiboot_uint32_t zero;
};

[TheLittleWho]

Could you give me a link to your multiboot 2 header? Maybe main it's out of date.

[Octocontrabass]

Here you go.

[mallard]

Some basics about memory, since this seems to come up far too often:

• Memory is not necessarily contiguous and in fact usually isn't. Common hardware has "holes" in the memory map; addresses which point nowhere (not even to memory-mapped hardware), even though there may be RAM available at both lower and higher addresses.
• Memory-mapped hardware often has "fixed" addresses (they may change to make room for other hardware, but do not alter when you add/remove RAM) so are one reason for "holes".
• Address space taken by memory-mapped hardware is not taken away from RAM. i.e. if you have 2GB ram and a video card with a 256MB "aperture", you'll have at least 2.25GB of valid addressable (obviously only 2GB of non-reserved space) memory locations. Some people get confused because a 32-bit system with 4GB of RAM will often only be able to use ~3.5GB of actual RAM (without using PAE); this only happens because there isn't enough 32-bit address space to fit in all memory-mapped hardware and all RAM. In that case some RAM gets pushed above the 32-bit "barrier" where a 32-bit non-PAE OS can't use it. (Since this is usually talked about in the context of Windows XP; Technically, Windows XP fully supported PAE, but limited address space to 32-bits for device driver compatibility reasons. Patches exist to remove this limitation.)
• Since I mentioned the video card aperture; no, a 1GB video card does not take a whole 1GB of 32-bit address space. (Modern) video cards use an "aperture" which is usually 256MB (I believe they can/do map their full memory into the >4GB space though) and use some form of "bank switching" to access the full memory through the aperture. Ancient video cards did similar things to make their full memory usable from real mode via the 128KB of sub-1MB space available to them.
• The multiboot memory map uses 64-bit value. You'll get wrong results if you treat them as 32-bit values.
• A competent physical memory manager cannot rely on all RAM being contiguous, cannot rely on being able to "classify" all address space at boot (hot-swappable memory-mapped hardware exists!) and should not rely on any particular addresses being "definitely there" above the 14MB mark; this is the end of the "classic" PC memory map. Obviously if you want to run on ancient systems with less than 16MB RAM, you can't rely on this being fully populated either.

[xenos]

Code: Select all

struct multiboot_tag_mmap

Code: Select all

struct multiboot_mmap_entry

I guess you can spot the difference.

[LtG]

I'd also like to point out a couple of things:
- Octocontrabass mentions the entries are 64-bit, he offers an explanation why you are seeing garbage. That would be a good time to read and re-read the multiboot spec, at least for the mmap part. And maybe at least test, before stating that they are 32-bit.
- On the first page there's an almost identical thread by Octacone, you should probably read that:
http://forum.osdev.org/viewtopic.php?f=1&t=32254

Octacone had a few issues, reading that thread might help you. One of the issues had to do with 32-bit vs 64-bit. There was also some stuff about PAS vs PM vs VAS vs VM, similar to what mallard said in this thread.

[Octacone]

I'd also like to point out a couple of things:
- Octocontrabass mentions the entries are 64-bit, he offers an explanation why you are seeing garbage. That would be a good time to read and re-read the multiboot spec, at least for the mmap part. And maybe at least test, before stating that they are 32-bit.
- On the first page there's an almost identical thread by Octacone, you should probably read that:
http://forum.osdev.org/viewtopic.php?f=1&t=32254

Octacone had a few issues, reading that thread might help you. One of the issues had to do with 32-bit vs 64-bit. There was also some stuff about PAS vs PM vs VAS vs VM, similar to what mallard said in this thread.

Hey @LtG, I wanted to post the same thing. It looks like you were slightly faster.
He is having almost the exact same theory related dilemmas as I've had.
Take a look at that thread, it is quite good. There are peopling explaining in detail all of your issues.
If you have any question please post them here. :=)
Quite soon enough you will realize that you can't access more than 3.5 GB on average in protected mode and you will also discover the purpose of PAE. (just wait for it).
Because of that thread, right now I am working on a totally new physical memory/address space manager and I am also writing some 64 bit printing functions. I am having to adapt/modify around 50% of my entire OS because of it.