GRUB, memory map entries.

[ExeTwezz]

Hi,

I decided to see what regions of memory I can use for free physical pages, and I know that a memory map can give me this info.

From what I know, GRUB provides a multiboot info structure, and it contains a data about the memory map: `mmap_addr' and `mmap_length' fields. In my case, the first one is addressing to 0x100A8 every time I boot the OS. And the second one is 0x90 (also every time). Also, the 6th bit in `flags' is set, this means that the memory map is correct. According to the multiboot specification, the structure of an entry of the memory map looks like this:

Code: Select all

u32   -4     size
u64    0     base_addr
u64    8     length
u32    16    type

The problem is that when I read `mmap_addr-4' (field `size'), the value is too big (0xD424). Or am I reading incorrect address? If so, then reading `mmap_addr' returns value 0x14 (20 bytes, which is perfect). But why the specification says that the `size' is at offset `-4'?

OK, I'm using offset 0 for `size' instead of `-4'. The result looks pretty, but incorrect: see the attachement.

Here is the code:

Code: Select all

    int mmap_addr = mb_info->mmap_addr;
    int mmap_length = mb_info->mmap_length;

    char str[32];

    puts ("mmap_addr = 0x");
    puts (itoa (mmap_addr, str, 16));
    puts (", mmap_length = 0x");
    puts (itoa (mmap_length, str, 16));
    puts ("\n");

    int i = mmap_addr;
    while (i < (mmap_addr + mmap_length))
    {
        int *size = (int *) i;
        int *base_addr_low = (int *) i + 4;
        int *base_addr_high = (int *) i + 8;
        int *length_low = (int *) i + 12;
        int *length_high = (int *) i + 16;
        int *type = (int *) i + 20;

        puts (" size = 0x");
        puts (itoa (*size, str, 16));
        puts (", start = 0x");
        puts (itoa (*base_addr_low, str, 16));
        puts (", end = 0x");
        puts (itoa (*base_addr_low + *length_low, str, 16));
        puts (", type = 0x");
        puts (itoa (*type, str, 16));

        puts ("\n");
        i += *size + 4; // Add 4 since the size is at offset 0, not -4.
    }

[Nable]

Code: Select all

        int *size = (int *) i;
        int *base_addr_low = (int *) i + 4;
        int *base_addr_high = (int *) i + 8;
        int *length_low = (int *) i + 12;
        int *length_high = (int *) i + 16;
        int *type = (int *) i + 20;

1. IMHO, one should never assume such things as "sizeof(int)==4". If you want types with guaranteed size - use <stdint.h> and use types from it (such as unit32_t).
2. It looks like you are misunderstanding pointer arithmetic. When you add constant to pointer, then compiler adds constant*(sizeof(*pointer)), e.g.:

Code: Select all

int *f(int x)
{
	return (int*)x + 4; // This statement will return x+sizeof(int)*4, i.e. x+16 in most cases.
	// If you really want +4 to address, then you should write at least (int*)(x+4)
}

Note that size of pointer may not be the same as size of int (btw, IMHO, using signed types for unsigned things is awful). There's uintptr_t type in <stdint.h> for your variable `i';

[ExeTwezz]

Hi,

Code: Select all

        int *size = (int *) i;
        int *base_addr_low = (int *) i + 4;
        int *base_addr_high = (int *) i + 8;
        int *length_low = (int *) i + 12;
        int *length_high = (int *) i + 16;
        int *type = (int *) i + 20;

1. IMHO, one should never assume such things as "sizeof(int)==4". If you want types with guaranteed size - use <stdint.h> and use types from it (such as unit32_t).
2. It looks like you are misunderstanding pointer arithmetic. When you add constant to pointer, then compiler adds constant*(sizeof(*pointer)), e.g.:

Code: Select all

int *f(int x)
{
	return (int*)x + 4; // This statement will return x+sizeof(int)*4, i.e. x+16 in most cases.
	// If you really want +4 to address, then you should write at least (int*)(x+4)
}

Note that size of pointer may not be the same as size of int (btw, IMHO, using signed types for unsigned things is awful). There's uintptr_t type in <stdint.h> for your variable `i';

Oh.. I really didn't know these subtleties (except, of course, difference between signed and unsigned numbers). Thank you!

I don't use the standard C library in my kernel, but I've looked the contents of `/usr/include/stdint.h' and my `types.h' has these typedefs, so I'm using `u32' now.

Code: Select all

u32 mmap_addr = mb_info->mmap_addr;
    u32 mmap_length = mb_info->mmap_length;

    char str[32];

    puts ("mmap_addr = 0x");
    puts (itoa (mmap_addr, str, 16));
    puts (", mmap_length = 0x");
    puts (itoa (mmap_length, str, 16));
    puts ("\n");

    u32 i = mmap_addr;
    while (i < (mmap_addr + mmap_length))
    {
        u32 *size = (u32 *) i;
        u32 *base_addr_low = (u32 *) (i + 4);
        u32 *base_addr_high = (u32 *) (i + 8);
        u32 *length_low = (u32 *) (i + 12);
        u32 *length_high = (u32 *) (i + 16);
        u32 *type = (u32 *) (i + 20);

        puts (" size = 0x");
        puts (itoa (*size, str, 16));
        puts (", start = 0x");
        puts (itoa (*base_addr_low, str, 16));
        puts (", end = 0x");
        puts (itoa (*base_addr_low + *length_low, str, 16));
        puts (", type = 0x");
        puts (itoa (*type, str, 16));

        puts ("\n");
        i += *size + 4;
    }

Works perfectly (see attachement).
BTW, is the last entry correct? It looks like it is incorrect...

Also, is my thought, that I shouldn't print `base_addr_high' and `length_high' since my kernel is 32-bit and they will be always 0, correct?

[Nable]

AFAIR, <stdint.h> is a freestanding header, i.e. it's not a part of C library, it's a part of compiler, so you can include it and use even without C library.
http://wiki.osdev.org/index.php?title=S ... ding&go=Go -> http://wiki.osdev.org/Bare_Bones#Freest ... vironments

Upd:

Also, is my thought, that I shouldn't print `base_addr_high' and `length_high' since my kernel is 32-bit and they will be always 0, correct?

Your kernel may be 32-bit but machine may not be, so it's possible to have some entries in >4G range. So of course you should look at high bits and never assume that they are 0.

[ExeTwezz]

AFAIR, <stdint.h> is a freestanding header, i.e. it's not a part of C library, it's a part of compiler, so you can include it and use even without C library.
http://wiki.osdev.org/index.php?title=S ... ding&go=Go -> http://wiki.osdev.org/Bare_Bones#Freest ... vironments

Oh, I think I should use it as well as other headers.

[ExeTwezz]

Your kernel may be 32-bit but machine may not be, so it's possible to have some entries in >4G range. So of course you should look at high bits and never assume that they are 0.

Ah, yes.

[max]

You'll have to read the fields from the memory map that are specified to be 64bit fields as as 64bit values. That applies to base address, length and maybe others (see the specification). Because as Nable stated, they can contain values bigger than 0xFFFFFFFF. You have to check if the memory area is in 32bit space and cut it off/exclude it from your usable space

[ExeTwezz]

Hi,

It seems like you didn't see my another question, so I'm asking it again .
The last entry is so strange (link)... Is it correct?

[no92]

I have never dealt with memory maps, but AFAIK they're not always ordered properly and may overlap (they are taken from the BIOS, which is known for being messy). That's all what I can tell.

PS: Why are you linking to SemVer on your project's GitHub page, although you already violated it?

[max]

Hi,

It seems like you didn't see my another question, so I'm asking it again .
The last entry is so strange (link)... Is it correct?

It seems like you didn't see my answer with a high chance you read the value within the memory map as a 32 bit value, but it lays within 64bit space.

It could for example be
start: 0x0000 000A 0004 0000
end: 0x0000 000B 0000 0000

If you try to output these two values as 32bit values, it will print 0x00040000 and 0x00000000 for sure.. ^^

[Brendan]

Hi

It seems like you didn't see my another question, so I'm asking it again .
The last entry is so strange (link)... Is it correct?

It seems like you didn't see my answer with a high chance you read the value within the memory map as a 32 bit value, but it lays within 64bit space.

It could for example be
start: 0x0000 000A 0004 0000
end: 0x0000 000B 0000 0000

If you try to output these two values as 32bit values, it will print 0x00040000 and 0x00000000 for sure.. ^^

Yes.

I'd expect that:

• The actual/correct information would be a "system" area from 0x00000000FC000000 to 0x00000000FFFFFFFF (describing the area used by the firmware's ROM)
• The "start" (which would be 0x00000000FC000000) is displayed wrong because the high 32-bits are ignored and the low 32-bits are treated as signed (e.g. -0x00400000) and the implementation of "atoi()" is broken for negative values
• The end is wrong, because the size (which would be 0x0000000000400000) is added to the wrong (signed, 32-bit only) value of start, giving "-0x00400000 + 0x00400000 = 0"

Cheers,

Brendan

[ExeTwezz]

Hi

It seems like you didn't see my another question, so I'm asking it again .
The last entry is so strange (link)... Is it correct?

It seems like you didn't see my answer with a high chance you read the value within the memory map as a 32 bit value, but it lays within 64bit space.

It could for example be
start: 0x0000 000A 0004 0000
end: 0x0000 000B 0000 0000

If you try to output these two values as 32bit values, it will print 0x00040000 and 0x00000000 for sure.. ^^

Yes.

I'd expect that:

• The actual/correct information would be a "system" area from 0x00000000FC000000 to 0x00000000FFFFFFFF (describing the area used by the firmware's ROM)
• The "start" (which would be 0x00000000FC000000) is displayed wrong because the high 32-bits are ignored and the low 32-bits are treated as signed (e.g. -0x00400000) and the implementation of "atoi()" is broken for negative values
• The end is wrong, because the size (which would be 0x0000000000400000) is added to the wrong (signed, 32-bit only) value of start, giving "-0x00400000 + 0x00400000 = 0"

Cheers,

Brendan

Hi,

• No, I've just printed the high 32 bits (`base_addr_high' and `length_high') and they're 0x0.
• No, my `itoa()' function prints negative numbers correctly (I've just tested it).

I've a suggestion that the `mmap_length' is incorrect, but I think it's silly because GRUB 1.99 and GRUB 2 set the same value (0x90) to `mmap_length' every time I boot.

Update: Wait.. I've just remembered that `itoa()' prints negative numbers only if their radix is 10. So, I've no suggestions.

[Brendan]

Hi,

• No, I've just printed the high 32 bits (`base_addr_high' and `length_high') and they're 0x0.
• No, my `itoa()' function prints negative numbers correctly (I've just tested it).

I've a suggestion that the `mmap_length' is incorrect, but I think it's silly because GRUB 1.99 and GRUB 2 set the same value (0x90) to `mmap_length' every time I boot.

Update: Wait.. I've just remembered that `itoa()' prints negative numbers only if their radix is 10. So, I've no suggestions.

Please note that for all modern 80x86 CPUs, at power-on the CPU begins executing at 0xFFFFFFF0; therefore there must be firmware/ROM in that area; therefore the memory map must have an entry describing that area as "system". From your screenshot you can see that the entries look correct/sane except the last, and that none of the entries that look correct/sane describe the firmware/ROM area that must exist. From this you can almost guarantee that the last entry that doesn't look correct/sane is supposed to be for the missing firmware/ROM area. Also; the fact that you're abusing 32-bit signed integer variables for 64-bit unsigned values and that the firmware/ROM area should end at 0x0000000100000000 (a value that won't fit in any 32-bit integer) is unlikely to be a coincidence.


Cheers,

Brendan