Executing C Kernel #101

[Haroogan]

The Bare Bones tutorial is awesome. However I would like to experiment with my own Boot Loader instead of GRUB. Therefore, I have several questions. Sorry if this is going to be long, I will try my best to explain issues.

Alright, so, my 2nd Stage Boot Loader is up. The A20 and Protected Mode are enabled. Lets consider the following snippet from it (we are in PMode already):

Code: Select all

BITS 32

Code32:
	MOV AX,16
	MOV DS,AX
	MOV ES,AX
	MOV FS,AX
	MOV GS,AX
	MOV SS,AX
	MOV ESP,0x9FFFF

	;Load Kernel binary file from HD to "X" (X is some physical address in RAM)
	;JMP 8:X

Here, within comments, I've just wrote my own vision of how I should start executing Kernel. Am I right?

The kernel:

Code: Select all

void kmain()
{
   unsigned char *videoram = (unsigned char *) 0xb8000;
   videoram[0] = 65;
   videoram[1] = 0x07;
}

Now, to start executing this "8:X" address there must be this kmain function. I guess that I can't rely on the compiler/linker to place it right in the beginning of Kernel binary file, can I? So, there are 2 possible scenarios:

1. If I CAN'T rely on it, then as far as I understand here comes "loader.s" in Bare Bones tutorial to solve this problem, because assembly's binary output is much more tunable, right? Therefore, by linking assembly "loader.o" and "kernel.o" I will get Kernel binary file with the first instruction on label "loader:" so jumping to 8:X will 100% execute it?

2. If I CAN rely on it, then it is probably linker, who can put "kmain" at the beginning of Kernel binary file. Since linker script contains "ENTRY(...)" directive. What if I just put "kmain" there? Then after jumping to 8:X will I 100% execute "kmain"?

Now I have several questions about linker. Consider the following linker script:

Code: Select all

SECTIONS{
    . = 0x00100000;

    .text :{
        *(.text)
    }

    .rodata ALIGN (0x1000) : {
        *(.rodata)
    }

    .data ALIGN (0x1000) : {
        *(.data)
    }

    .bss : {
        sbss = .;
        *(COMMON)
        *(.bss)
        ebss = .;
    }
}

Code: Select all

. = 0x00100000;

1. AFAIK it means that program is going to be located at 1 MB in RAM. Remember "X"? My Stage 2 Boot Loader is responsible for placing Kernel program to X address, then why on earth do I have to specify this "0x00100000" or any other "X" for linker?

Code: Select all

ALIGN (0x1000)

2. AFAIK this is used to place different sections within 4 KB blocks in RAM (I guess it is related with paging somehow). OK, lets imagine that all my sections (text, rodata, data and bss) are 1 KB long each. Does it mean that Kernel binary file (produced by linker) will look like this:

1 KB of text
3 KB of some garbage (maybe zeroes)
1 KB of rodata
3 KB of some garbage (maybe zeroes)
1 KB of data
3 KB of some garbage (maybe zeroes)
1 KB of bss
3 KB of some garbage (maybe zeroes)

I believe, this is the only way to align. Am I right?

Here goes one more similar question about the .bss section in general. Look:

Code: Select all

section .bss
   resb STACKSIZE 

I've read somewhere that .bss section is VIRTUAL. Like "When you assemble - the output binary file will not contain those STACKSIZE bytes. But when you execute it this section will appear..." - I'm like "Whatta?!" Who on earth is going to create this section for me in runtime?

I think that's all for now. Thanks in advance guys and sorry for long post again.

[xenos]

Let me try to answer at least some of your questions:

Now, to start executing this "8:X" address there must be this kmain function. I guess that I can't rely on the compiler/linker to place it right in the beginning of Kernel binary file, can I? So, there are 2 possible scenarios:

In general, you can't. There may be some nasty tricks (make sure that kernel.o is the first object file on the linker command line, make sure that kmain is the first function in kernel.c...) which may or may not work, but this is implementation defined. It is not guaranteed that any particular linker will place kmain at the first byte of the resulting (flat) binary.

1. If I CAN'T rely on it, then as far as I understand here comes "loader.s" in Bare Bones tutorial to solve this problem, because assembly's binary output is much more tunable, right? Therefore, by linking assembly "loader.o" and "kernel.o" I will get Kernel binary file with the first instruction on label "loader:" so jumping to 8:X will 100% execute it?

Although assembly is much more tunable, I wouldn't rely on this either.

2. If I CAN rely on it, then it is probably linker, who can put "kmain" at the beginning of Kernel binary file. Since linker script contains "ENTRY(...)" directive. What if I just put "kmain" there? Then after jumping to 8:X will I 100% execute "kmain"?

The ENTRY(...) directive is useful mainly for executable formats such as ELF which allow the specification of an "entry point" at which execution should begin, and which (in general) differs from the first byte of the binary. Those executable formats are not just flat binaries, but contain such kind of information in a header at the beginning of the file.

If you use ENTRY(kmain) and link to an ELF file instead of a flat binary, your kmain will not be loaded at "X", i.e., at the beginning of your binary file. Instead, "X" will point to the ELF file header. You could use the information stored in the header to find out where kmain is (remember: ENTRY(...) enters this information as the "entry point" into the file header). Alternatively, you could try something like this to write the address of kmain to the first dword of the binary file, so you could jump to 8:[X] instead:

Code: Select all

SECTIONS{
    . = 0x00100000;

    .text :{
        LONG(kmain);
        *(.text)
    }

    .rodata ALIGN (0x1000) : {
        *(.rodata)
    }

    .data ALIGN (0x1000) : {
        *(.data)
    }

    .bss : {
        sbss = .;
        *(COMMON)
        *(.bss)
        ebss = .;
    }
}

. = 0x00100000;

1. AFAIK it means that program is going to be located at 1 MB in RAM. Remember "X"? My Stage 2 Boot Loader is responsible for placing Kernel program to X address, then why on earth do I have to specify this "0x00100000" or any other "X" for linker?

Assume your program contains some variable Y. During program execution, you have some code line in which you need the address of Y, Z = &Y. The variable Y is placed somewhere in the binary. But to get the absolute address of Y, your code needs to know where it will be loaded at runtime. This address resolution is done by the linker, so you need to tell it where you will load the binary image.

Code: Select all

ALIGN (0x1000)

2. AFAIK this is used to place different sections within 4 KB blocks in RAM (I guess it is related with paging somehow). OK, lets imagine that all my sections (text, rodata, data and bss) are 1 KB long each. Does it mean that Kernel binary file (produced by linker) will look like this:

Almost correct. But since the bss section does not occupy any space, it will be omitted from the binary. The alignment bytes after the last included section are also omitted. The output should look like this:

1 KB of text
3 KB of some garbage (maybe zeroes)
1 KB of rodata
3 KB of some garbage (maybe zeroes)
1 KB of data

I believe, this is the only way to align. Am I right?

If you stick with a flat binary - yes. If you use an executable format such as ELF, the placement of the sections in the file and in memory may differ.

I've read somewhere that .bss section is VIRTUAL. Like "When you assemble - the output binary file will not contain those STACKSIZE bytes. But when you execute it this section will appear..." - I'm like "Whatta?!" Who on earth is going to create this section for me in runtime?

Your bootloader is responsible for that. It (somehow) needs to know where to load your binary file, how many bytes to load and how many bytes to reserve for things like bss. Again, it is useful to use a format such as ELF because it contains all this information in the file header.

[Haroogan]

Awesome mate, you've just cleared things up. So this is why in Bare Bones tutorial they use ELF. Seems like GRUB will do all dirty work on parsing ELF header to allocate stack memory and jump to "loader:" entry point right?

[xenos]

Yes, that's one reason why the combination of GRUB and ELF kernels is rather popular

[Haroogan]

Got 2 more questions here:

1. I can't find anywhere: how to make LD to produce ELF output? Do I have to explicitly set it somehow in the script or is it automatic (by default)?
2. Why in Bare Bones Tutorial they "align 4" both sections? Is it crucial?

[garob]

1. If your host OS uses ELF as its executable format it should work straight from the tutorial, if it doesn't(Windows,Mac) then build a cross-compiler.
Actually just build a cross-compiler.

2. From the Multiboot spec

The Multiboot header must be contained completely within the first 8192 bytes of the OS image, and must be longword (32-bit) aligned.

On a 32 bit architecture the stack should be 32-bit aligned.

[saptrap]

I think a great example of what you were originally thinking is Pure64. It's a nice minimal 2nd stage bootloader. Included is a do-nothing c kernel that is a flat binary. If you try compiling, keep in mind it's 64-bit.

-mike

[saptrap]

Code: Select all

OUTPUT_FORMAT("binary")
ENTRY(main)
SECTIONS
{
	.text 0x0000000000100000 :
	{
		*(.text)
	}
	.data :
	{
		*(.data)
	}
	.bss :
	{
		*(.bss)
	}
}

I went ahead and looked it up. Pure64 uses the above linker script and then just jumps to it via:

jmp 0x0000000000100000 ; Jump to the kernel

-mike

[xenos]

1. I can't find anywhere: how to make LD to produce ELF output? Do I have to explicitly set it somehow in the script or is it automatic (by default)?

I also recommend using a "plain elf cross compiler". See GCC Cross-Compiler in the wiki. For example, I use a cross compiler with --target=i686-pc-elf for my 32 bit kernel.

To explicitly set the output format to ELF, you can specify something like OUTPUT_FORMAT(elf32-i386) in your linker script. (Yes, it must be i386, not i686 or anything else here, because elf32-i386 is the name of the ELF format for the IA32 architecture. There are others, though.)

[Haroogan]

Strange... This script links with no problem:

Code: Select all

OUTPUT(Kernel.bin)

ENTRY(start)

SECTIONS
{
  .text 0x100000 :
  {
    code = .; _code = .; __code = .;
    *(.text)
    . = ALIGN(4096);
  }

  .data :
  {
     data = .; _data = .; __data = .;
     *(.data)
     *(.rodata)
     . = ALIGN(4096);
  }

  .bss :
  {
    bss = .; _bss = .; __bss = .;
    *(.bss)
    . = ALIGN(4096);
  }

  end = .; _end = .; __end = .;
} 

But when I start bochs, GRUB shows its command line and when I type "boot" nothing happens...

These two scripts do not even link:

Code: Select all

OUTPUT(Kernel.bin)
OUTPUT_FORMAT(elf32-i386)

ENTRY(start)

SECTIONS
{
  .text 0x100000 :
  {
    code = .; _code = .; __code = .;
    *(.text)
    . = ALIGN(4096);
  }

  .data :
  {
     data = .; _data = .; __data = .;
     *(.data)
     *(.rodata)
     . = ALIGN(4096);
  }

  .bss :
  {
    bss = .; _bss = .; __bss = .;
    *(.bss)
    . = ALIGN(4096);
  }

  end = .; _end = .; __end = .;
} 

and

Code: Select all

OUTPUT(Kernel.bin)
OUTPUT_FORMAT(binary)

ENTRY(start)

SECTIONS
{
  .text 0x100000 :
  {
    code = .; _code = .; __code = .;
    *(.text)
    . = ALIGN(4096);
  }

  .data :
  {
     data = .; _data = .; __data = .;
     *(.data)
     *(.rodata)
     . = ALIGN(4096);
  }

  .bss :
  {
    bss = .; _bss = .; __bss = .;
    *(.bss)
    . = ALIGN(4096);
  }

  end = .; _end = .; __end = .;
} 

The following message I get: C:\MinGW\bin\ld.exe: cannot perform PE operations on non PE output file 'Kernel/Kernel.bin'.

Edit: I'm currently reading Cross Compiler on wiki, but I'm using Eclipse and I've already created project with "Cross GCC Compiler Toolchain"... - isn't it the same?

[xenos]

The following message I get: C:\MinGW\bin\ld.exe: cannot perform PE operations on non PE output file 'Kernel/Kernel.bin'.

This indicates that you are not using a cross compiler / linker, but the linker that is shipped with MinGW. Probably you're also using the compiler that is shipped with MinGW. Both operate on PE object files ("portable executable", typically used by Windows), which are rather different from ELF files. It therefore cannot produce ELF output. If you wish to create ELF files on Windows, you really need to compile your own cross compiler toolchain. But never mind, it's not that difficult.

Edit: I'm currently reading Cross Compiler on wiki, but I'm using Eclipse and I've already created project with "Cross GCC Compiler Toolchain"... - isn't it the same?

I don't know what this setting does, since I never used Eclipse. But I'm pretty sure it will not compile your project with a cross compiler toolchain, unless you compile and install such a toolchain by hand. That's where the wiki tutorial comes into the game.

[Haroogan]

The only thing I found is MSYS Basher (msys.bat):



I give up with the 1st step, lol:

What to download? Binutils? Or gcc-core? Or Binutils and gcc-core together? BTW there is no gcc-core, there are only gcc-x.x.x...

Code: Select all

export PREFIX=/usr/local/cross
export TARGET=i586-elf
cd /usr/src
mkdir build-binutils build-gcc

Export what? Why i586? Where should I create /usr/local/cross? Should I put downloaded Binutils into usr/src?

This is nightmare. - I'm like sitting here reading this wiki and like - "What's happening? "

[Combuster]

I give up with the 1st step

That tutorial is pretty much the entry test for OS development. If you can't get through it, it either means you can't read or lack the basic prerequisites altogether.

[Haroogan]

Mhm... Another subjective stereotype, Mr. Obvious?

[xenos]

The only thing I found is MSYS Basher (msys.bat):

Actually I don't know which shell MinGW provides, I never used it... I could recommend using Linux instead of Windows for OS development, but that's a personal preference, and I know that there are a lot of Windows guys here in this forum who might be able to help you with this issue

What to download? Binutils? Or gcc-core? Or Binutils and gcc-core together? BTW there is no gcc-core, there are only gcc-x.x.x...

You need binutils (it contains things such as linker, assembler, objdump...) and gcc-core (the C compiler), and maybe gcc-g++ if you wish to use C++. Both gcc-core and gcc-g++ can be found in the gcc-x.x.x directories. Once you downloaded them, unpack them wherever you would like to place these source files, it doesn't have to be /usr/src. Do not use something like Winzip for unpacking, as it might play around with the source files, change Unix linebreaks to Windows linebreaks... Better use the "tar" command that comes with MinGW. Unpacking binutils should create a directory names binutils-x.x.x with the source files, while both gcc-core and gcc-g++ should go into the gcc-x.x.x directory.

Code: Select all

export PREFIX=/usr/local/cross
export TARGET=i586-elf
cd /usr/src
mkdir build-binutils build-gcc

Export what? Why i586? Where should I create /usr/local/cross? Should I put downloaded Binutils into usr/src?

"export" defines an environment variable, so whenever you use $PREFIX on the command line, it will be replaced by /usr/local/cross. Try echo $PREFIX and see what happens.

I guess i586 (which means Intel Pentium CPU) was the newest gcc target at the time the tutorial was written. I use i686-pc-elf instead for my 32 bit cross compiler.

/usr/local/cross is a typical Unix directory path which is used in this tutorial to place the cross compiler toolchain. In Unix, /usr is one of the system directories in the "root path" (similar to C:\ in Windows). MinGW should define its own root path, which contains a directory named usr. You can place you cross compiler there, but you can also place it somewhere else. You only need to make sure that your cross compiler can be found (i.e., by setting the Windows $PATH variable) when you compile your kernel.

This is nightmare. - I'm like sitting here reading this wiki and like - "What's happening? "

I guess most of the language used on that wiki page is very common to Linux users, but rather confusing to Windows users...