64-bit toolset and multiboot 1&2 headers

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
Post Reply
User avatar
eryjus
Member
Member
Posts: 286
Joined: Fri Oct 21, 2011 9:47 pm
Libera.chat IRC: eryjus
Location: Tustin, CA USA

64-bit toolset and multiboot 1&2 headers

Post by eryjus »

I recently made the choice to "improve" my kernel to 64-bits. I know I will not be able to just change my tool chain and recompile, so I started with a shiny new Linux VM. I figured I would be able to port sections of code over, but I did not want to complicate this effort with everything at once.

I started with creating an x86_64-elf-* cross compiler. I also installed QEMU x86_64 emulator. So far so good. Then I started laying in code from Higher-Half Bare Bones. When that was less than successful, I turned to Creating a 64-bit kernel, 64-bit Higher Half Kernel with GRUB 2, and even the original Bare Bones wikis. I had trouble either getting the code to link or boot with each one. Admittedly, I was pulling from several sources, but I had done this before with a 32-bit kernel. I should be able to get at least the multiboot 1 header to work!!

After lots of readelf, x86_64-elf-objdump, and hexdump commands, I finally started to see the issues (I was too close to the problem for 2 full days). I had several issues to fix:
  • My multiboot section was not the first one in the file.
  • My multiboot header was not in the required 8K.
  • My elf page size was larger than I wanted it to be.
After stripping down the code to it's essential parts, I was finally able to get the multiboot 1 header to work. Then I thought, "Why not put both headers in the code and allow a kernel to be able to be booted by either?"

The multiboot 2 documentation has a lot of holes in it, especially when you compare it to Grub's implementation. Frankly, it is my opinion that the documentation leaves enough holes that Grub has taken liberties that break the specification in its implementation. At any rate, I was finally able to get an image compiled that contained both multiboot headers and is able to be booted by Grub2 by either header.

I am considering adding my code as a wiki (without the back-story) since it took a lot of deep digging for me to pull this together. However, with the risk of creating "another bare bones how to" wiki, I thought I might ask for some feedback here first. The code is included below as the comments might help explain the issues I uncovered.

First the loader.s:

Code: Select all

;==============================================================================================
;
; loader.s
;
; This file contains the initial functions and structures required by the Multiboot loader to
; properly transfer control to the kernel after loading it in memory.  There are structures
; that are required to be 4-byte aligned and must be in the first 8K of the file for the final
; binary to be Multiboot compliant.  These structures are implemented here.
;
;    Date     Tracker  Pgmr  Description
; ----------  ------   ----  ------------------------------------------------------------------
; 2014/09/24  Initial  ADCL  Leveraged from osdev.org -- "Higher Half Bare Bones" wiki
; 2014/09/25  Initial  ADCL  Since "Higher Half Bare Bones" did not work, scrapped it all for
;                            a run at "64-bit Higher Half Kernel with GRUB2.
; 2014/09/26  Initial  ADCL  After a lot of work and a sleepless night, I have finally been
;                            able to get an elf64 image to boot with GRUB2 using the multiboot
;                            1 header.  The key to making this work was changing the page size
;                            for the linker to be 0x800 (2K).  The multiboot header is found
;                            on the third page and needs to be in the first 8K of the file.
;                            With a page size of 2K, this offset is at 4K.  If I bump the page
;                            size to the next power of 2 (0x1000 or 4K) when the multiboot
;                            header is at 8K.  You would think that is right, but it turns out
;                            that the entire header bust be contained within 8K, putting it
;                            just out of reach.  The parameter on x86_64-elf-gcc that makes
;                            this successful is "-z max-page-size=0x800".
;
;                            Due to issues in the linker.ld script which have been resolved,
;                            the multiboot header is now in the second page, not the third.
;                            Changing back to "-z max-page-size=0x1000".
;
;==============================================================================================

			bits		32
			section		.multiboot
			align		8

MBMODALIGN	equ			1<<0
MBMEMINFO	equ			1<<1
MBVIDINFO	equ			1<<2
MBAOUT		equ			1<<16		; this will not be used in our kernel

MBMAGIC		equ			0x1badb002
MBFLAGS		equ			(MBMODALIGN | MBMEMINFO | MBVIDINFO)
MBCHECK		equ			-(MBMAGIC + MBFLAGS)

MultibootHeader:					; Offset  Description
			dd			MBMAGIC		;    0    multiboot magic number
			dd			MBFLAGS		;    4    multiboot flags
			dd			MBCHECK		;    8    checksum
			dd			0			;    c    header address (flag bit 16)
			dd			0			;   10    load address (flag bit 16)
			dd			0			;   14    load end address (flag bit 16)
			dd			0			;   18    bss end address (flag bit 16)
			dd			0			;   1c    entry address (flag bit 16)
			dd			1			;   20    vidoe mode (1 == text) (flag bit 2)
			dd			80			;   24    video columns (flag bit 2)
			dd			25			;   28    video rows (flag bit 2)
			dd			8			;   2c    video color depth (flag bit 2)

;----------------------------------------------------------------------------------------------
; So, I was thinking, why would I not be able to provide BOTH the multiboot header 1 and the
; multiboot header 2 in the same file?  I could not think of any technical reason except that
; grub needs to know which signature to look for.  The command in grub.cfg is multiboot2 for
; the newer version.  The following multiboot2 header also works.  The key is going to make
; sure we get the system in a state that we can get into a consistent state for the rest of
; the kernel initialization.  The following is the multiboot2 header for this implementation.
;
; **NOTE** the docuentation does not tell you that each of tag structures must be 8-byte
; aligned.  Therefore you will see 'align 8' lines throughout this structure.
;----------------------------------------------------------------------------------------------

			bits		32
			align		16

MBMAGIC2	equ			0xe85250d6
MBLEN		equ			MultibootHeader2End - MultibootHeader2
MBCHECK2	equ			(-(MBMAGIC2 + 0 + MBLEN) & 0xffffffff)

MultibootHeader2:							; Description
			dd			MBMAGIC2			;   multiboot2 magic number
			dd			0					;   architecture: 0=32-bit protected mode
			dd			MBLEN				;   total length of the mb2 header
			dd			MBCHECK2			;   mb2 checksum

			align		8					; 8-byte aligned
Type4Start:									; Console Flags
			dw			4					;   type=4
			dw			1					;   not optional
			dd			Type4End-Type4Start	;   size = 12
			dd			1<<1				;   EGA text support
Type4End:

			align		8					; 8-byte aligned
Type6Start:									; Modue align
			dw			6					;   Type=6
			dw			1					;   Not optional
			dd			Type6End-Type6Start	;   size = 8 bytes even tho the doc says 12
Type6End:

			align		8					; 8-byte aligned
											; termination tag
			dw			0					; type=0
			dw			0					; flags=0
			dd			8					; size=8

MultibootHeader2End:

;==============================================================================================


			section		.boot
			global		EntryPoint
			align		32

EntryPoint:
			mov			ebx,0xb8000
			mov			word [ebx],0x07<<8|'A'	; put an "A" on the screen so we know we booted

.loop:
			cli
			hlt
			jmp			.loop

;==============================================================================================
The linker command script linker.ld:

Code: Select all

/*===========================================================================================*/
/*                                                                                           */
/* linker.ld                                                                                 */
/*                                                                                           */
/* This file contains the command script to feed to the linker that will determine how and   */
/* where to put the sections of the executable.  Most notably, the .multiboot section needs  */
/* to come first as it is required to be in the first 8K of the file.                        */
/*                                                                                           */
/*    Date     Tracker  Pgmr  Description                                                    */
/* ----------  ------   ----  -------------------------------------------------------------- */
/* 2014/09/24  Initial  ADCL  Leveraged from osdev.org -- "Higher Half Bare Bones" wiki      */
/* 2014/09/25  Initial  ADCL  Since "Higher Half Bare Bones" did not work, scrapped it all   */
/*                            for a run at "64-bit Higher Half Kernel with GRUB2.            */
/* 2014/09/26  Initial  ADCL  After a lot of work and a sleepless night, I have finally been */
/*                            able to get an elf64 image to boot with GRUB2 using the        */
/*                            multiboot 1 header.  The key to making this work was changing  */
/*                            the page size for the linker to be 0x800 (2K).  See loader.s   */
/*                            for more details.  In addition, I have been able to force the  */
/*                            multiboot section to be written first in the output file.      */
/*                            In order to accomplish this, I had to remove all the other     */
/*                            sections and VIRT_BASE manipulations that were taking place.   */
/*                            These will have to be added back in carefully as we go on from */
/*                            here.                                                          */
/*                                                                                           */
/*===========================================================================================*/


ENTRY(EntryPoint)

SECTIONS
{
	. = 0x100000;

	.boot :
	{
		*(.multiboot)
		*(.boot)
	}

	.text ALIGN(0x1000) :
	{
		*(.text)
	}

	/* that's all I have in this kernel.  There's no .data section */
}
Finally the makefile to pull it all together:

Code: Select all

TGT-DIR=bin
TGT-BLD=myos.bin
TGT-FILE=$(TGT-DIR)/$(TGT-BLD)
TGT-ISO=$(subst .bin,.iso,$(TGT-BLD))
TGT-CDROM=$(TGT-DIR)/$(TGT-ISO)

LINK-SCRIPT=linker.ld

ASM=nasm -felf64
LD=x86_64-elf-gcc -ffreestanding -O2 -nostdlib -z max-page-size=0x1000
LD-SCRIPT=-T $(LINK-SCRIPT)
LD-LIBS=-lgcc

ASM-SRC=$(wildcard src/*.s)
OBJ=$(sort $(subst .s,.o,$(subst src/,obj/,$(ASM-SRC))))


.PHONY: iso
iso: $(TGT-CDROM)

$(TGT-CDROM): $(TGT-FILE) bin/grub.cfg makefile
	echo Creating $@...
	mkdir -p iso/boot/grub
	cp bin/grub.cfg iso/boot/grub/
	cp $(TGT-FILE) iso/boot/
	grub2-mkrescue -o $(TGT-CDROM) iso
	rm -fR iso

.PHONY: build
build: $(TGT-FILE)

$(TGT-FILE): $(OBJ) $(LINK-SCRIPT) makefile
	echo Linking $@...
	mkdir -p bin
	$(LD) $(LD-SCRIPT) -o $@ $(OBJ) $(LD-LIBS)

obj/%.o: src/%.s makefile
	echo Assembling $<...
	mkdir -p obj
	$(ASM) $< -o $@

bin/grub.cfg: makefile
	echo Generating $@...
	mkdir -p bin
	echo set timeout 5                    >  bin/grub.cfg
	echo set default 0	                  >> bin/grub.cfg
	echo menuentry \"myos\" {       >> bin/grub.cfg
	echo   multiboot /boot/myos.bin >> bin/grub.cfg
	echo   boot							  >> bin/grub.cfg
	echo }								  >> bin/grub.cfg
	echo menuentry \"muos mb2\" {   >> bin/grub.cfg
	echo   multiboot2 /boot/myos.bin>> bin/grub.cfg
	echo   boot							  >> bin/grub.cfg
	echo }								  >> bin/grub.cfg

.PHONY: clean
clean:
	echo Cleaning...
	rm -fR obj
	rm -fR bin
	rm -fR iso
The command "make iso" will create a cdrom image that is bootable in QEMU with the following:

Code: Select all

qemu-system-x86_64 -cdrom myos.iso
Please let me know your feedback.


EDIT: of course I have nothing that is 64-bit at this point. That's coming up. With the subject a misnomer, I updated it.
Adam

The name is fitting: Century Hobby OS -- At this rate, it's gonna take me that long!
Read about my mistakes and missteps with this iteration: Journal

"Sometimes things just don't make sense until you figure them out." -- Phil Stahlheber
User avatar
Combuster
Member
Member
Posts: 9301
Joined: Wed Oct 18, 2006 3:45 am
Libera.chat IRC: [com]buster
Location: On the balcony, where I can actually keep 1½m distance
Contact:

Re: 64-bit toolset and multiboot 1&2 headers

Post by Combuster »

The multiboot 2 documentation has a lot of holes in it, especially when you compare it to Grub's implementation.
There are several versions of GRUB 2 floating around, along with correspondingly different implementations of Multiboot 2. You can off course add a tutorial that supports both versions of GRUB by their own standards, but multiboot1 should be sufficient for booting for either.

Considering you had problems with the existing tutorials it might be more wise to fix those where appropriate instead of adding more maintenance to the wiki.
"Certainly avoid yourself. He is a newbie and might not realize it. You'll hate his code deeply a few years down the road." - Sortie
[ My OS ] [ VDisk/SFS ]
Post Reply