GRUB problem with modules

[max]

Hey guys!:)

I have a problem with GRUB 0.97. I'm using the stage2_eltorito to create a bootable .iso file. My menu.lst looks like this:

Code: Select all

default 0
timeout 0

title Kernel
kernel /boot/kernel
module /boot/ramdisk

So it loads my kernel binary (which contains the multiboot header) and the ramdisk module.

Then I'm creating my ISO file with:

Code: Select all

mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -o "iso/bin/image.iso" "iso/src"

This works fine so far. The actual problem is that if my ramdisk module has a size of <= 4096 bytes, GRUB does not load the kernel and instead just opens its command line.
Do GRUB modules have a minimum required size?

Also, another question: when using the GRUB memory map, can i assume that anything that lays behind the last module (and that is declared by the memory map as usable) is usable? I want to avoid overwriting the data that GRUB provides me (I'm loading my kernel to 0x100000 btw). How do you decide in your kernel where to start allocating physical pages?

Thanks
-maxdev

[zhiayang]

Hey guys!:)

Hello!

Also, another question: when using the GRUB memory map, can i assume that anything that lays behind the last module (and that is declared by the memory map as usable) is usable? I want to avoid overwriting the data that GRUB provides me (I'm loading my kernel to 0x100000 btw). How do you decide in your kernel where to start allocating physical pages?

Thanks
-maxdev

I've never used modules before and I don't remember where exactly GRUB (1) puts loaded modules. I think you should avoid using memory under 1MB (or under your kernel). GRUB will report memory from 0mb to the start of the EBDA as free – however because you can't predict where GRUB will put your module (since the size can change), you should just stay out of that area. Copy your module somewhere else if you can.


As for where to start allocating physical pages, I'd say after your kernel is good enough, although personally I have a reserved region right after that for various (read: stupid) reasons.

[jnc100]

The actual problem is that if my ramdisk module has a size of <= 4096 bytes, GRUB does not load the kernel and instead just opens its command line.

What happens if you type in the module command at the GRUB command prompt in this case? It may give you a useful error message.

Also, another question: when using the GRUB memory map, can i assume that anything that lays behind the last module (and that is declared by the memory map as usable) is usable? I want to avoid overwriting the data that GRUB provides me (I'm loading my kernel to 0x100000 btw). How do you decide in your kernel where to start allocating physical pages?

The memory map provided by GRUB is not affected by the position of either loaded modules or the kernel itself. Therefore, to determine if a page is free, it has to be marked as free in the memory map and not within a module and not within the kernel. Note also that some important areas under 1 MiB are not marked as used in the memory map, so it may make sense to mark all memory under 1 MiB as used (or at least those in Memory Map (x86)).

Regards,
John.

[max]

I've never used modules before and I don't remember where exactly GRUB (1) puts loaded modules. I think you should avoid using memory under 1MB (or under your kernel). GRUB will report memory from 0mb to the start of the EBDA as free – however because you can't predict where GRUB will put your module (since the size can change), you should just stay out of that area. Copy your module somewhere else if you can.

The memory map provided by GRUB is not affected by the position of either loaded modules or the kernel itself. Therefore, to determine if a page is free, it has to be marked as free in the memory map and not within a module and not within the kernel. Note also that some important areas under 1 MiB are not marked as used in the memory map, so it may make sense to mark all memory under 1 MiB as used (or at least those in Memory Map (x86)).

Okay, then the approach I did so far was right. GRUB always loaded the ramdisk in the memory after the kernel, so I just use everything that is after the ramdisk (so everything below for example 0x12#### is left untouched).

What happens if you type in the module command at the GRUB command prompt in this case? It may give you a useful error message.

When using "module /boot/kernel", I get an "Error 20: Multiboot kernel must be loaded before modules"
Then I tried "blocklist /boot/kernel", and that told me "Error 25: Disk read error"
So I guess that my .iso image is somehow corrupt.. I am using a mkisofs v1.14 (i586-pc-cygwin).. any idea?


@requimrar I like your coding style, mine is somewhat like that, looks imho much better than most of the kernel code out there does ^^

[max]

I just encountered another issue with my GRUB iso... when booting in Bochs, it simply ends up here:


Bochs output is:


in QEMU it works fine... im afraid something about my bootstrap code could be wrong.
Im already linking my multiboot header to the very start of the binary (http://pastebin.com/R1L4ZC9r), and and my loader should be appropriate, too (http://pastebin.com/wRJAUwZq)...
Someone please help me fix this mess

[Combuster]

The multiboot header is found as expected, so stuff boots. Which means that the code is something inside your startup code.

Most interesting here is the "Sim client size" error bochs throws as the only message after booting (and please post logs as text rather than images for searching and cellphone bandwidth reasons), which indicates you did something with graphics things and bochs nothing left to display (leaving the last image it had generated).

[zhiayang]

@requimrar I like your coding style, mine is somewhat like that, looks imho much better than most of the kernel code out there does ^^

Thanks! I try my best. Newlines and verbosity might be a bit excessive but hey, we're not in the 1980s anymore right?

[max]

please post logs as text rather

yesh sorry

Okay, so just as a test, I removed everything from my startup code except just a single write to the video memory (i tried both 0xB8000 and 0xA0000), still the same - QEMU does it, Bochs does not:

Code: Select all

extern "C" void createKernel(uint32_t endKernel, uint32_t initialStackPointer,
		MultibootInformation* grubMultibootInfo, uint32_t magicNumber) {
	if (magicNumber == MULTIBOOT_BOOTLOADER_MAGIC) {
		debug_consoleVideo_out("Hellol"); // Writes the string to the video memory
		for(;;) { asm("hlt"); }
	}
}

But, the only thing I'm doing in my loader code, is calling all global constructors, then pushing the required stuff and calling the createKernel... could this even be a problem with my version of GRUB?

Thanks! I try my best. Newlines and verbosity might be a bit excessive but hey, we're not in the 1980s anymore right?

Exactly

[Pancakes]

But, the only thing I'm doing in my loader code, is calling all global constructors, then pushing the required stuff and calling the createKernel... could this even be a problem with my version of GRUB?

The only way you can find out is to actually test this by making the loader code do nothing but an infinite loop. The execution may not even be making it to the loader code so also verify that with some type of output or displaying something to the screen.

Also, if I am not mistaken there is a way to make boches show you were it is executing and that may help you a lot by knowing what it is doing exactly. It might even be doing nothing and the actual emulator (which is kinda obvious) is having a hiccup, lol.

[max]

Okay, I just created the minimum test case - it works in Bochs:

Code: Select all

loader:
	mov ebx, 0xB8000 ; video memory
	mov eax, 4000 ; write 4000 bytes
printLoop:
	mov byte [ebx], 5
	inc ebx
	dec eax
	cmp eax, 0
	jne printLoop
hangman:
	hlt
	jmp hangman

Ill take a look at the rest of my code and then tell you whats up..

EDIT 1: Its an issue with my global constructor calling... *researching on*

[zhiayang]

EDIT 1: Its an issue with my global constructor calling... *researching on*

The wiki has some code on global constructors for C++, shouldn't be that hard to find.

[max]

EDIT 1: Its an issue with my global constructor calling... *researching on*

The wiki has some code on global constructors for C++, shouldn't be that hard to find.

Yes, but I also want to understand what I'm coding there. I just wrote a method that calls all the ctors, and this is the outcome:

In QEMU, in total 4 global constructors get called. Ive added outputs for the address of each constructor, and "called" if it was called. QEMUs output is:

Code: Select all

0x001013AD called 0x00101E4E called 0x00102B06 called 0x0010427B called

So this is right, as I am linking my ctors to the .rodata block after my .text section.
Running the same image in Bochs, it hangs at calling the first one:

Code: Select all

0x001013AD

Using objdump to find out what symbol is behind that address told me that there is this:

Code: Select all

001013ad l     F .text  0000001c _GLOBAL__sub_I_settings

What is this? oO

EDIT: SOLUTION FOUND
Okay, I created a .text-dump with objdump using:

Code: Select all

objdump -t --section .text --disassemble --demangle kernel >kernel.textdump

This revealed the following:
1. The "_GLOBAL__sub_I_settings" is a generated function, calling "__static_initialization_and_destruction_0"

Code: Select all

001013ad <_GLOBAL__sub_I_settings>:
  1013ad:	55                   	push   %ebp
  1013ae:	89 e5                	mov    %esp,%ebp
  1013b0:	83 ec 18             	sub    $0x18,%esp
  1013b3:	c7 44 24 04 ff ff 00 	movl   $0xffff,0x4(%esp)
  1013ba:	00 
  1013bb:	c7 04 24 01 00 00 00 	movl   $0x1,(%esp)
  1013c2:	e8 7f ff ff ff       	call   101346 <__static_initialization_and_destruction_0(int, int)>
  1013c7:	c9                   	leave  
  1013c8:	c3                   	ret    
  1013c9:	90                   	nop

2. The "__static_initialization_and_destruction_0" pushes some stuff, and then calls the constructors for one of my HashMaps:

Code: Select all

00101346 <__static_initialization_and_destruction_0(int, int)>:
  101346:	55                   	push   %ebp
  101347:	89 e5                	mov    %esp,%ebp
  101349:	83 ec 18             	sub    $0x18,%esp
  10134c:	83 7d 08 01          	cmpl   $0x1,0x8(%ebp)
  101350:	75 59                	jne    1013ab <__static_initialization_and_destruction_0(int, int)+0x65>
  101352:	81 7d 0c ff ff 00 00 	cmpl   $0xffff,0xc(%ebp)
  101359:	75 50                	jne    1013ab <__static_initialization_and_destruction_0(int, int)+0x65>
  10135b:	c7 04 24 10 e0 10 00 	movl   $0x10e010,(%esp)
  101362:	e8 b5 64 00 00       	call   10781c <gstl::HashMap<gstl::String, gstl::String>::HashMap()>
  101367:	c7 44 24 08 70 f8 10 	movl   $0x10f870,0x8(%esp)
  10136e:	00 
  10136f:	c7 44 24 04 10 e0 10 	movl   $0x10e010,0x4(%esp)
  101376:	00 
  101377:	c7 04 24 cc 7f 10 00 	movl   $0x107fcc,(%esp)
  10137e:	e8 c0 27 00 00       	call   103b43 <__cxa_atexit>
  101383:	c7 04 24 1c e0 10 00 	movl   $0x10e01c,(%esp)
  10138a:	e8 b1 64 00 00       	call   107840 <gstl::List<Tool*>::List()>
  10138f:	c7 44 24 08 70 f8 10 	movl   $0x10f870,0x8(%esp)
  101396:	00 
  101397:	c7 44 24 04 1c e0 10 	movl   $0x10e01c,0x4(%esp)
  10139e:	00 
  10139f:	c7 04 24 a6 80 10 00 	movl   $0x1080a6,(%esp)
  1013a6:	e8 98 27 00 00       	call   103b43 <__cxa_atexit>
  1013ab:	c9                   	leave  
  1013ac:	c3                   	ret    

3. This HashMap is laying around somewhere as a global variable.. and guess what, the constructor of my HashMap calls "new":

Code: Select all

...	/**
	 * Initializes the bucket array
	 */
	void initialize(uint32_t newBucketCount) {
		this->bucketCount = newBucketCount;
		this->bucketArray = new HashMapEntry<K, V>*[bucketCount];

		for (uint32_t i = 0; i < newBucketCount; i++) {
			this->bucketArray[i] = 0;
		}
	}
...
public:
	/**
	 * Creates the HashMap with 128 buckets
	 */
	HashMap() {
		initialize(128);
	}...

Only the devil himself knows, why my implementation of new (calling the memory allocator implicitly) did not fail in QEMU, but did so in BOCHS... I guess QEMU sets all the memory to 0, which caused some funny runtime accident allowing the allocator to do... whatever... i think I've fallen in love with objdump :'D

btw: yep, i guess i'll use bochs from now on.

also: how do you guys manage that? do you even initialize global constructors at all? disallow that somehow? or just take care to not use them accidently?

[sortie]

You should call all the global constructors (and in C, constructor functions, a gcc extension) during kernel initialization. This is how the language works (at least C++) and it's probably best to just implement it rather than dealing with trouble if you don't.

I wrote a wiki tutorial on the matter: http://wiki.osdev.org/Calling_Global_Constructors

The trick is that you have to interface with the crtbegin.o and crtend.o objects that gcc uses to magically generate a few sections used for program initialization. This is slightly bothersome as it has been designed to be more general than just global constructors. The idea is simply that you use these special objects to construct a special _init function that you call and then the work is done for you.

A special problem about global constructors is when they should be called: My recommendation is that you have a special kernel_early_main entry point that sets up the very most important aspects of the kernel that global constructors depend on. You should then call _init and thus all the global constructors. Finally, you call the actual kernel_main that can now depend on the basic kernel stuff being set up and all the global variables are properly initialized.

It's a good design principle that your global constructors, if any, do nothing more interesting than just initialize data structures, nothing that can fail or depend on other memory being initialized. Indeed, if at all possible, you should attempt to initialize such global variables statically to constants at compile-time. That is, my recommendation is that you do initialize global constructors and support them, but never actually really use them.

[max]

You should call all the global constructors (and in C, constructor functions, a gcc extension) during kernel initialization. This is how the language works (at least C++) and it's probably best to just implement it rather than dealing with trouble if you don't.

I wrote a wiki tutorial on the matter: http://wiki.osdev.org/Calling_Global_Constructors

The trick is that you have to interface with the crtbegin.o and crtend.o objects that gcc uses to magically generate a few sections used for program initialization. This is slightly bothersome as it has been designed to be more general than just global constructors. The idea is simply that you use these special objects to construct a special _init function that you call and then the work is done for you.

A special problem about global constructors is when they should be called: My recommendation is that you have a special kernel_early_main entry point that sets up the very most important aspects of the kernel that global constructors depend on. You should then call _init and thus all the global constructors. Finally, you call the actual kernel_main that can now depend on the basic kernel stuff being set up and all the global variables are properly initialized.

It's a good design principle that your global constructors, if any, do nothing more interesting than just initialize data structures, nothing that can fail or depend on other memory being initialized. Indeed, if at all possible, you should attempt to initialize such global variables statically to constants at compile-time. That is, my recommendation is that you do initialize global constructors and support them, but never actually really use them.

Okay, thanks a lot. I'll take a look at that. But: doesn't this have the exact same outcome as using the linker script to link the constructors to a location allowing them to be called by hand?
And by the way, my initial problem with GRUB not loading the module if its too small magically solved itself..

man... i love how computers are so simple, yet so complex.. <3

[sortie]

But: doesn't this have the exact same outcome as using the linker script to link the constructors to a location allowing them to be called by hand?

This is the correct way of calling global constructors (to the best of my knowledge): This is how it is done in user-space too, btw. It is worth noting you may need to do some linker script additions for it to work properly or the global constructor table can end up at a bad memory address. I'll need to investigate this further.