OSDev.org

Posted: **Tue Apr 06, 2021 11:35 am**

According to documentation memory map entry looks like this:

        +-------------------+
-4      | size              |
        +-------------------+
0       | base_addr         |
8       | length            |
16      | type              |
        +-------------------+

It only specifies four values for type:

Code: Select all

..., where a value of 1 indicates available RAM, value of 3 indicates usable memory holding ACPI information, value of 4 indicates reserved memory which needs to be preserved on hibernation, value of 5 indicates a memory which is occupied by defective RAM modules and all other values currently indicated a reserved area.

In qemu, however, this field has different values, for example 20. Anyone know what this means?

Posted: **Tue Apr 06, 2021 12:39 pm**

In the case of Grub, the values for type match the ones from the E820 memory map. This is likely true here is well.

Unfortunately I cannot seem to find what memory type 20 is. Linux doesn't seem to know (or care) either:

https://elixir.bootlin.com/linux/latest ... 20/types.h

It could have something to do with SMBIOS:

https://seabios.seabios.narkive.com/KlM ... es-in-qemu

Here it says it iss indeed something to do with SMBIOS:

/* SMBIOS type 20 - Memory Device Mapped Address */

https://android.googlesource.com/platfo ... 4dc308a08/

Posted: **Tue Apr 06, 2021 4:26 pm**

How are you parsing your memory map? If you are not parsing it right, you might be getting invalid results.

Posted: **Wed Apr 07, 2021 1:04 am**

For reference (and because that offset of -4 threw me off), this is my code for that. Hope it helps:

Code: Select all

struct mbinfo {
    uint32_t flags,
             mem_lower,
             mem_upper,
             boot_device,
             cmdline,
             mods_count,
             mods_addr,
             syms[4],
             mmap_length,
             mmap_addr,
             drives_length,
             drives_addr,
             config_table,
             bootloader_name,
             apm_table,
             vbe_control_info,
             vbe_mode_info;
    uint16_t vbe_mode,
             vbe_interface_seg,
             vbe_interface_off,
             vbe_interface_len;
    uint64_t framebuffer_addr;
    uint32_t framebuffer_pitch,
             framebuffer_width,
             framebuffer_height;
    uint8_t  framebuffer_bpp,
             framebuffer_type,
             color_info[6];

};

struct mbmmap {
    uint32_t size;
    uint32_t addr[2], len[2];
    uint8_t type;
};

static void print_mem_info(unsigned len_mmap, const struct mbmmap *mmap, uint64_t *maxram)
{
    const unsigned char *const start = (void*)mmap;
    uint64_t mr = 0;
    while ((const unsigned char*)mmap - start < len_mmap)
    {
        uint64_t addr, len;
        addr = (uint64_t)mmap->addr[1] << 32 | mmap->addr[0];
        len = (uint64_t)mmap->len[1] << 32 | mmap->len[0];
        if (mmap->type == 1 && addr + len > mr)
            mr = addr + len;
        dbg_printf("%8X-%8X type %d\n", addr, addr + len, mmap->type);
        if (addr < 0x100000000)
        {
            if (addr + len > 0x100000000)
                len = 0x100000000 - addr;

            if (mmap->type == 1)
                add_avail_range(addr, len);
            else
                add_reserved_range(addr, len);
        }

        mmap = (const void*)((const unsigned char*)mmap + mmap->size + 4);
    }
    *maxram = mr;
}

void mbmain(uint32_t mbmagic, const struct mbinfo *bootinfo)
{
    [...]
        if (bootinfo->flags & 64)
            print_mem_info(bootinfo->mmap_length, (const void*)bootinfo->mmap_addr, &maxram);
}

So the "mmap_addr" in the multiboot info points immediately to a length. And afterwards all memory maps are offset by a further four bytes. And since now everything is misaligned, I have to break the 64-bit members down into two 32-bit members, else the compiler would insert padding. Yes, I am aware of __attribute__((packed)) and I choose not to use such a crutch. And so far it is working out.

Posted: **Sun Apr 18, 2021 5:50 am**

What is the purpose of GRUB's map?

It for example lists following regions as available in qemu:

Code: Select all

0x0 - 0x9fc00
0x100000 - 0x7fe0000
0x0 - 0x1b00000000

Of course, this list makes no sense if its entries are not disjoint.
Moreover, in qemu's emulated memory, definitely, there is not address 0x1b000000000000. I don't even have that much memory in my PC :).

So what's the matter?
Are those these BIOS errors that bootboot fixes?

Posted: **Sun Apr 18, 2021 8:10 am**

antoni wrote:Moreover, in qemu's emulated memory, definitely, there is not address 0x1b000000000000. I don't even have that much memory in my PC .

That's a hint that you are not parsing the entries correctly. Which is curious, because the first two look alright. But they make no sense without the actual memory type. I figured out the format of the region by hexdumping all of it; maybe that helps you out.

In general, there are no guarantees about the memory map. You might have multiple entries for the same address, they might not be sorted and you must deal with that. In general, assume the worst. If the same address is given as both RAM and something else, assume it is something else.

Posted: **Sun Apr 18, 2021 8:22 am**

antoni wrote:What is the purpose of GRUB's map?
So what's the matter?

If you are using the default amount of RAM in QEMU, the highest address will be 0x8000000. Which is suspiciously close to the second range you list (allowing for the fact that some of that address space is not useable). I'd guess that your third range is reading rubbish from beyond the end of the GRUM memory map.

It is my experience that the GRUB memory map reports the correct ranges in qemu.

Posted: **Sun Apr 18, 2021 9:39 am**

You're right,
I miscalculated length of the array because I thought one variable is (uint8_t*) but it was pointer to larger type. In correct array there are only two entries.

Posted: **Thu Apr 22, 2021 10:14 am**

Ok,
So now the highest address in memory should be 0x7fe0000, however, according to RSDP, RSDT's address is... 0x7fe1550.

This address is correct. I mapped this page in qemu and checked this:

Code: Select all

(gdb) p *(RSDT_t*) 0x7fe1550
$3 = {header = {signature = "RSDT", length = 52, revision = 1 '\001', checksum = 109 'm', OEMID = "BOCHS ", OEM_table_ID = "BXPCRSDT", OEM_revision = 1, 
    creator_ID = 1129338946, creator_revision = 1}, pointer_to_other_SDT = {134091780}}

So, how is this possible that some ACPI tables are above higher bound of memory reported by GRUB?

PS

Region 0x7fe0000 - 0x8000000 is marked as reserved, so maybe reserved regions are also usable, but in shuch case, why they are marked as reserved?

Posted: **Thu Apr 22, 2021 11:37 am**

antoni wrote:So now the highest address in memory should be 0x7fe0000, however, according to RSDP, RSDT's address is... 0x7fe1550.

I assume you mean this is the highest address of type 1, right? "Usable RAM"? Well, ACPI tables are not in usable RAM. The best you can get is "ACPI reclaimable" RAM. But some tables may be in ROM as well. "Usable RAM" would mean you can overwrite it, and you cannot overwrite the ACPI tables before reading them. "Reserved" can also mean "in use by firmware".

Posted: **Thu Apr 22, 2021 1:24 pm**

Ok, nice.
So, I need to include RESERVED entries too while parsing map. The problem with these entries is that they are not necessarily listing real, physical memory. For example in my qemu machine, where the highest address is 0x8000000, GRUB's map looks like this:

Code: Select all

base_addr = 0x0, length = 0x9fc00, available RAM                               
base_addr = 0x9fc00, length = 0x400, reserved RAM                              
base_addr = 0xf0000, length = 0x10000, reserved RAM                            
base_addr = 0x100000, length = 0x7ee0000, available RAM                        
base_addr = 0x7fe0000, length = 0x20000, reserved RAM                          
base_addr = 0xfffc0000, length = 0x40000, reserved RAM

So obviously there is no 0x100000000 nor 0xfffc0000 address in memory. Maybe it is memory mapped IO or something?
However, for my purposes, I need to know the actual ammount of RAM, even if some of its parts are "RESERVED".

Posted: **Thu Apr 22, 2021 1:45 pm**

antoni wrote:So, I need to include RESERVED entries too while parsing map.

What for? The physical memory manager needs to know about available memory, so it can give you a new page when you need one. But the Reserved memory is not that. And if you come across some physical address you need to map (like ACPI tables of MMIO windows on PCI), you just access those directly, circumventing the PMM.

antoni wrote:So obviously there is no 0x100000000 nor 0xfffc0000 address in memory.

Don't be so certain. Chipsets can be complicated.

antoni wrote:However, for my purposes, I need to know the actual ammount of RAM, even if some of its parts are "RESERVED".

What for? If you want to know how much memory is usable, then only available RAM counts (and maybe ACPI reclaimable, after you've read the tables). If you want to know how much RAM is installed, you may be better served with DMI information. That contains a string about how big the memory stick is. You will never see as much RAM usable as the sticks you installed put together. Some of it always goes to other uses.

Posted: **Thu Apr 22, 2021 2:18 pm**

If you want to know how much RAM is installed, you may be better served with DMI information.

Yes, that is exactly what I want. But I use GRUB, so obviously I can't get any BIOS information that is not passed to me by bootloader.

However, GRUB provides me with these values:

Code: Select all

multiboot_uint32_t mem_lower;
multiboot_uint32_t mem_upper;

I wonder if maybe total amount of memory is mem_lower + mem_upper? (these are values from BIOS int 0x12)

Posted: **Fri Apr 23, 2021 1:44 am**

No, its not... "The value returned for upper memory is maximally the address of the first upper memory hole minus 1 megabyte. It is not guaranteed to be this value. "
How can I detect it?

Posted: **Fri Apr 23, 2021 12:31 pm**

antoni wrote:Yes, that is exactly what I want.

I still don't know what for, but if it is just about displaying to the user how much memory they have, here you go. That's the DMI specification which contains, in rather lengthy detail, ways to read out all the strings associated with certain system components, RAM sticks included. Not exactly a bedtime read, but take some time to read it. I have no idea what you would use this information for. "You have 8GB of RAM installed" is rather useless information if you also concede "but I can make use of only 7.5GB". Isn't that second number more meaningful? That second number is the sum of usable RAM. That first one is what you can get from the DMI information.

OSDev.org

Parsing multiboot memory map.

Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.

Re: Parsing multiboot memory map.