GRUB memory map - cant shift unsigned long into uint64_t

[capstrovor]

Hello,
I'm new to OS development, and currently I try to write the physical memory manager.
Therefore I want to use the memory map from GRUB.
Here's my code for looping through the memory-map structs (I'm using the struct from multiboot.h of the osdev-page):

Code: Select all

uint64_t mem_begin;
	uint64_t mem_range;
	uint64_t mem_end;

	while(mmap < (memory_map_t*) (mbt->mmap_addr + mbt->mmap_length))
	{
		mem_begin = mmap->base_addr_low;
		mem_begin |= mmap->base_addr_high << 32;
		mem_range = mmap->length_high << 32;
		mem_range |= mmap->length_low;
                //mem_end = mem_begin + mem_range;
		printf("%x%x\n", mmap->length_high, mmap->length_low);

		printf("0x%x%x to 0x%x%x -> ", mem_begin, (mem_begin >> 32), mem_range, (mem_range >> 32));
		if(mmap->type == 1)
			printf("available\n");
		else
			printf("reserved\n");

		mmap = (memory_map_t*) ((uint32_t)mmap + mmap->size + sizeof(uint32_t));
	}

Here is the output:
0x9FC00
0x00 to 0x00 ->available
0x400
0x9FC000 to 0x00 -> reserverd
... and so on
as you can see, if I use length_high and _low for the output, I get a "valid" output.
(I have to use (mem_begin >> 32) in the printf, because it has no support for 64 bit variables yet.
I'm wondering why I get a "vaild" output for the mem_begin, but not for mem_range although I use the same
code for both.
Can anyone help me please?
PS: sorry for my bad english

[Octacone]

because it has no support for 64 bit variables yet.

Hmm...

Code: Select all

typedef signed long long int int64_t
typedef unsigned long long int uint64_t

Your 32 bit CPU can indeed use those. 64 bit = 64 bit instructions, not variable types in this case.

[capstrovor]

Your 32 bit CPU can indeed use those. 64 bit = 64 bit instructions, not variable types in this case.

Yes I'm using uint64_t for mem_begin and mem_range... But I dont know why it's not working with mem_begin, but not with mem_range.
I will implement support for 64 bit variables in my printf asap

[Octocontrabass]

Code: Select all

		printf("0x%x%x to 0x%x%x -> ", mem_begin, (mem_begin >> 32), mem_range, (mem_range >> 32));

Be careful using reserved standard library function names like "printf" in code that doesn't have a C standard library. Compiler optimizations will make it behave oddly if you don't have a complete C standard library implementation. For your kernel, you may want to call it "kprintf" instead.

When you use "%x%x" as your format string, what will happen when you try to print a large number like 0x100000000?

What size are the variables you're passing to your printf()? Different sized variables are passed differently to the function. If your printf() is expecting a 32-bit integer but you're passing a 64-bit integer, it may not interpret the data correctly.

Here is the output:

The output you've provided does not match your code.

[capstrovor]

Be careful using reserved standard library function names like "printf" in code that doesn't have a C standard library. Compiler optimizations will make it behave oddly if you don't have a complete C standard library implementation. For your kernel, you may want to call it "kprintf" instead.

Ok, thanks for the advice!

When you use "%x%x" as your format string, what will happen when you try to print a large number like 0x100000000?

If I want to print numbers larger than 32 bit, I'd have to split them up into two 32 bit variables yet.

What size are the variables you're passing to your printf()? Different sized variables are passed differently to the function. If your printf() is expecting a 32-bit integer but you're passing a 64-bit integer, it may not interpret the data correctly.

printf expects unsigned int. I'll try to cast it when I pass mem_begin and mem_range to printf.

The output you've provided does not match your code.

Do you mean the "0x" before the 1st line? I just added it so it's easier to read. I'll post the original in a minute.

[capstrovor]

Ok so here is the updated code:

Code: Select all

while(mmap < (memory_map_t*) (mbt->mmap_addr + mbt->mmap_length))
	{
		mem_begin = mmap->base_addr_low;
		mem_begin |= mmap->base_addr_high << 32;
		mem_range = mmap->length_high << 32;
		mem_range |= mmap->length_low;

		kprintf("%x%x\n", mmap->length_high, mmap->length_low);

		mem_end = mem_begin + mem_range;
		kprintf("0x%x%x to 0x%x%x -> ", (uint32_t)(mem_begin & 0xFFFFFFFF), (uint32_t)(mem_begin >> 32), (uint32_t)(mem_end & 0xFFFFFFFF), (uint32_t)(mem_end >> 32));
		if(mmap->type == 1)
			kprintf("available\n");
		else
			kprintf("reserved\n");

		mmap = (memory_map_t*) ((uint32_t)mmap + mmap->size + sizeof(uint32_t));
	}

And now I get some "vaild" output for mem_range and mem_end!

Here is the original output
9FC00
0x00 to 0x9FC000 -> available
4000
0x9FC000 to 0xA00000 -> reserverd
100000
0xF00000 to 0x1000000 -> reserverd
7EE00000
0x1000000 to 0x7FE00000 -> available
200000
0x7FE00000 to 0x80000000 -> reserverd
400000
0xFFFC00000 to 0x01

I think the last line of the output is a bug in the kprintf (I'll fix that later on).
But I'm starting my kernel with qemu and I use the standard amount of ram (I start it with "qemu-system-i386 -kernel kernel.bin).
The memory map says that 0x1000000 to 0x7FE00000 is available. 0x7FE00000 is 2145386496 in decimal. That would be 2.14 GB of RAM?! But qemu is running on linux, which runs in Virtualbox and i reserverd only1.5GB RAM for the whole virtual machine... How is this possible?

[Ch4ozz]

RAM can be emulated using paging (paging out data on disk is a common way)
If I run qemu with 2gb ram it usually needs only arround 90mb on my windows machine.
It only allocates the data you really try to write to using pageguards

Also you have to remove one 0 from ALL entries because of the %X%X

[Octocontrabass]

I think the last line of the output is a bug in the kprintf (I'll fix that later on).

It's not. Your kprintf is doing exactly what you told it to do.

The memory map says that 0x1000000 to 0x7FE00000 is available.

It doesn't. You're printing the low half followed by the high half, and that's adding an extra zero to the end.

This also demonstrates the problem I was referring to earlier with using %x%x to print a 64-bit value split into two 32-bit integers. Since you aren't displaying leading zeroes, 0x100000000 will be displayed as 0x10.

[capstrovor]

RAM can be emulated using paging (paging out data on disk is a common way)
If I run qemu with 2gb ram it usually needs only arround 90mb on my windows machine.
It only allocates the data you really try to write to using pageguards

But why should it show more memory than available?
So I can use these values for my memory manager? All areas (except areas the kernel is using) can be used for the mem mng?

This also demonstrates the problem I was referring to earlier with using %x%x to print a 64-bit value split into two 32-bit integers. Since you aren't displaying leading zeroes, 0x100000000 will be displayed as 0x10.

Ok, got it! I fixed the problem by inserting another option for printf. I'm using %x for 32 bit values and %X for 64 bit values.
so here is the update code:

Code: Select all

kprintf("%X to %X -> ", mem_begin, mem_end);

And here is the new output:
0x0 to 0x9FC00 -> available
0x9FC00 to 0xA0000 -> reserved
0xF0000 to 0x100000 -> reserved
0x100000 to 0x7FE0000 -> available
0x7FE0000 to 0x8000000 -> available
0xFFFC0000 to 0x100000000 -> reserved

So it should be safe to use the areas marked as available? (Except kernel area)

[BrightLight]

So it should be safe to use the areas marked as available? (Except kernel area)

Yes. However, you might also want to take GRUB modules into consideration as they are loaded into the "available" area of the physical memory map.

[capstrovor]

Ok, thank you all for your help!
I've got one question left.
After the physical memory manager, I'll implement virtual memory support.
Should I implement a higher-half kernel immediately?
Because (I don't know if i understood it correctly) I need a higher-half kernel for user-mode only, right?
Or has it some other advantages?

[osdever]

%X is used in printf to write uppercase HEX number.