Issues with building a higher-half kernel

[Techflash]

I am trying to build a higher-half kernel so that I can use BOOTBOOT for 64-bit booting. But I'm running into some major linker failures and I simply have no clue where I would start. I am using the x86_64-elf cross compilation toolchain, although I do also have i686-elf available. From what I can see, it seems like the compiler is for some reason generating 32-bit code??? I invoke the linker with the following command line:

Code: Select all

x86_64-elf-gcc --sysroot=/TFOS/sysroot -isystem=/usr/include -T arch/x86_64/linker.ld -mno-red-zone -o tfos.elf -Ofast -g -ffreestanding -Wall -Wextra -Wpedantic -Werror -Wshadow -Wformat=2 -Wconversion -Wunused-parameter -fstack-protector-all -DCONFIG_KERN_VERSION_MAJOR=0 -DCONFIG_KERN_VERSION_MINOR=0 -DCONFIG_KERN_VERSION_PATCH=6 -DCONFIG_KERN_MAXARGS=65535 -DCONFIG_BITS=64  arch/x86_64/crti.o arch/x86_64/crtbegin.o arch/x86_64/boot.o arch/x86_64/pushpopregs.o arch/x86_64/cursor.o arch/x86_64/tty.o arch/x86_64/testing.o arch/x86_64/io/in.o arch/x86_64/io/out.o arch/x86_64/io/misc.o arch/x86_64/hardware/cpu/lidt.o arch/x86_64/hardware/pcspk/pcspeaker.o arch/x86_64/hardware/cpu/GDT.o arch/x86_64/hardware/pit/pit.o arch/x86_64/messages.o kernel/kernel.o kernel/stack.o kernel/panic.o kernel/bootDisplay.o arch/x86_64/crtend.o arch/x86_64/crtn.o -nostdlib -lk -lgcc

And here is the linker log:

Code: Select all

arch/x86_64/crtbegin.o: in function `deregister_tm_clones':
crtstuff.c:(.text+0x1): relocation truncated to fit: R_X86_64_32 against symbol `__TMC_END__' defined in .eh_frame section in tfos.elf
crtstuff.c:(.text+0x18): relocation truncated to fit: R_X86_64_32 against `.tm_clone_table'
arch/x86_64/crtbegin.o: in function `register_tm_clones':
crtstuff.c:(.text+0x31): relocation truncated to fit: R_X86_64_32 against symbol `__TMC_END__' defined in .eh_frame section in tfos.elf
crtstuff.c:(.text+0x5a): relocation truncated to fit: R_X86_64_32 against `.tm_clone_table'
arch/x86_64/crtbegin.o: in function `__do_global_dtors_aux':
crtstuff.c:(.text+0x88): relocation truncated to fit: R_X86_64_32 against `.dtors'
crtstuff.c:(.text+0x8e): relocation truncated to fit: R_X86_64_32 against symbol `__DTOR_END__' defined in .dtors section in arch/x86_64/crtend.o
crtstuff.c:(.text+0xdb): relocation truncated to fit: R_X86_64_32 against `.eh_frame'
arch/x86_64/crtbegin.o: in function `frame_dummy':
crtstuff.c:(.text+0x10c): relocation truncated to fit: R_X86_64_32 against `.bss'
crtstuff.c:(.text+0x111): relocation truncated to fit: R_X86_64_32 against `.eh_frame'
kernel/kernel.o: in function `kernelMain':
/TFOS/kernel/kernel/kernel.c:33:(.text+0x84): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.8'
/TFOS/kernel/kernel/kernel.c:37:(.text+0xad): additional relocation overflows omitted from the output
collect2: error: ld returned 1 exit status

I am very confused about specifically the `crtstuff.c` errors, as that file does not exist anywhere in my project, and actually, not even anywhere on my local filesystem:

Code: Select all

(prompt)$ sudo find / -name "crtstuff.c"
[sudo] password for techflash: 

(prompt)$

Any help would be appreciated. Willing to do any debugging necessary to get this to work correctly.

[iansjack]

Why do you think 32-bit code is being generated?

The errors are telling you that there are references in your code to relative addresses that are larger than 32-bits. ( https://www.technovelty.org/c/relocatio ... t-wtf.html ) It's not really possible to tell where the error lies without seeing the commands and source that created your object files. Can you give a link to a repository containing your code and Makefile?

"crtstuff.c" is part of the gcc source code.

[Techflash]

Sure, give me a moment to commit all of my recent changes. The link is https://github.com/techflashYT/Techflash-OS
UPDATE: alright the changes are pushed to the repo.

[nullplan]

My guess is you are not compiling with "-mcmodel=kernel". The errors in linking libgcc tell me that your version of that library was not compiled with that switch either. My advice would be to drop "-lgcc" from the linker command line and fix any linker errors that occur by writing the function yourself, preferably in assembler so GCC can't turn it into anything weird. Remember that you will likely only need simple implementations, not fast ones.

From what I could find out, the functions giving you trouble here will always be linked in with libgcc, and are entirely useless unless you use the rather esoteric "Transactional Memory" extension to C++. Zero cost abstraction, my behind.

[Techflash]

My guess is you are not compiling with "-mcmodel=kernel". The errors in linking libgcc tell me that your version of that library was not compiled with that switch either. My advice would be to drop "-lgcc" from the linker command line and fix any linker errors that occur by writing the function yourself, preferably in assembler so GCC can't turn it into anything weird. Remember that you will likely only need simple implementations, not fast ones.

From what I could find out, the functions giving you trouble here will always be linked in with libgcc, and are entirely useless unless you use the rather esoteric "Transactional Memory" extension to C++. Zero cost abstraction, my behind.

Hmm, I followed the standard "Cross-Compiler" tutorial from the OSDev Wiki. I'll try it later, as my build machine is a Core 2 Duo PC from 2007, and rebuilding GCC is basically a 1 hour journey, so I'll probably wait until I get off for lunch in a few hours to start that.

[Octocontrabass]

Sounds like we need to add "--disable-tm-clone-registry" to the cross-compiler guide wiki, along with a link to the guide for building an appropriate libgcc with no red zone and your choice of code model. Or maybe this guide using multilib instead.

[Techflash]

Sounds like we need to add "--disable-tm-clone-registry" to the cross-compiler guide wiki, along with a link to the guide for building an appropriate libgcc with no red zone and your choice of code model. Or maybe this guide using multilib instead.

I already built it with the appropriate stuff for -mno-red-zone, and compiled both libgcc, and my kernel with it. but I did not yet build libgcc with the previously mentioned -mcmode=kernel flag. My lunch is going to be soon, so I will now start that process. It should be done at some point within the next hour or so (again, the build PC is just a Linux box with a Core 2 Duo E8400, so it's not exactly very fast, the PC hasn't really had any upgrades since it was built in 2007)

UPDATE 1: Configured GCC again, using no-red-zone changes, and now compiling `all-gcc`
UPDATE 2: Even after the sed command in the tutorial, it still seems to give the no PIC support error. I don't know what's wrong with it. Might need help.
UPDATE 3: Just tried doing it all again from scratch without even -j2, still didn't work, just took even longer

[davmac314]

UPDATE 1: Configured GCC again, using no-red-zone changes, and now compiling `all-gcc`
UPDATE 2: Even after the sed command in the tutorial, it still seems to give the no PIC support error. I don't know what's wrong with it. Might need help.

You say no-red-zone changes, do you mean mcmodel changes? You need to use -mcmodel=kernel both for when building your compiler, and when building your kernel. (You need both -mno-red-zone and -mcmodel=kernel, in both cases).

[Octocontrabass]

UPDATE 2: Even after the sed command in the tutorial, it still seems to give the no PIC support error. I don't know what's wrong with it. Might need help.

Probably it was written assuming an older version of GCC and doesn't work with the version you're building.

What version of GCC and binutils are you working with?

[Techflash]

UPDATE 2: Even after the sed command in the tutorial, it still seems to give the no PIC support error. I don't know what's wrong with it. Might need help.

Probably it was written assuming an older version of GCC and doesn't work with the version you're building.

What version of GCC and binutils are you working with?

I just git cloned the latest source for each. One sec while I try to find an exact version number.

[Techflash]

UPDATE 1: Configured GCC again, using no-red-zone changes, and now compiling `all-gcc`
UPDATE 2: Even after the sed command in the tutorial, it still seems to give the no PIC support error. I don't know what's wrong with it. Might need help.

You say no-red-zone changes, do you mean mcmodel changes? You need to use -mcmodel=kernel both for when building your compiler, and when building your kernel. (You need both -mno-red-zone and -mcmodel=kernel, in both cases).

No, I mean for no-red-zone changes, I mean the t-x86_64-elf file and the slight change to config.gcc

[davmac314]

No, I mean for no-red-zone changes, I mean the t-x86_64-elf file and the slight change to config.gcc

Well then, that's why you still have the problem.

The "building a cross compiler" page doesn't say it, but you need "-mcmodel=kernel" for the libgcc build as well, assuming your kernel is going to build with that model. See the link from Octonobrass on building libgcc with the appropriate memory model.

Or, follow the advice of nullplan above:

My advice would be to drop "-lgcc" from the linker command line and fix any linker errors that occur by writing the function yourself, preferably in assembler so GCC can't turn it into anything weird. Remember that you will likely only need simple implementations, not fast ones.

[Techflash]

No, I mean for no-red-zone changes, I mean the t-x86_64-elf file and the slight change to config.gcc

Well then, that's why you still have the problem.

The "building a cross compiler" page doesn't say it, but you need "-mcmodel=kernel" for the libgcc build as well, assuming your kernel is going to build with that model. See the link from Octonobrass on building libgcc with the appropriate memory model.

Or, follow the advice of nullplan above:

My advice would be to drop "-lgcc" from the linker command line and fix any linker errors that occur by writing the function yourself, preferably in assembler so GCC can't turn it into anything weird. Remember that you will likely only need simple implementations, not fast ones.

I had both, one second, I can show you the command line I was using & the logs:

Code: Select all

[prompt] $ make all-target-libgcc CFLAGS_FOR_TARGET='-O2 -mcmodel=kernel -mno-red-zone'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/libiberty'
make[2]: Entering directory '/programsBuild/crosscomp/build-gcc/libiberty/testsuite'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/programsBuild/crosscomp/build-gcc/libiberty/testsuite'
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/libiberty'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/lto-plugin'
make  all-am
make[2]: Entering directory '/programsBuild/crosscomp/build-gcc/lto-plugin'
make[2]: Leaving directory '/programsBuild/crosscomp/build-gcc/lto-plugin'
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/lto-plugin'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/intl'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/intl'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/libiberty'
make[2]: Entering directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/libiberty/testsuite'
make[2]: Nothing to be done for 'all'.
make[2]: Leaving directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/libiberty/testsuite'
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/libiberty'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/fixincludes'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/fixincludes'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/libcpp'
test -f config.h || (rm -f stamp-h1 && make stamp-h1)
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/build-x86_64-pc-linux-gnu/libcpp'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/zlib'
true "AR_FLAGS=rc" "CC_FOR_BUILD=gcc" "CFLAGS=-g -O2  " "CXXFLAGS=-g -O2  " "CFLAGS_FOR_BUILD=-g -O2" "CFLAGS_FOR_TARGET=-O2 -mcmodel=kernel -mno-red-zone" "INSTALL=/usr/bin/install -c" "INSTALL_DATA=/usr/bin/install -c -m 644" "INSTALL_PROGRAM=/usr/bin/install -c" "INSTALL_SCRIPT=/usr/bin/install -c" "LDFLAGS=-static-libstdc++ -static-libgcc " "LIBCFLAGS=-g -O2  " "LIBCFLAGS_FOR_TARGET=-O2 -mcmodel=kernel -mno-red-zone" "MAKE=make" "MAKEINFO=makeinfo --split-size=5000000 " "PICFLAG=" "PICFLAG_FOR_TARGET=" "SHELL=/bin/sh" "EXPECT=expect" "RUNTEST=runtest" "RUNTESTFLAGS=" "exec_prefix=/home/techflash/opt/cross" "infodir=/home/techflash/opt/cross/share/info" "libdir=/home/techflash/opt/cross/lib" "prefix=/home/techflash/opt/cross" "tooldir=/home/techflash/opt/cross/x86_64-elf" "AR=ar" "AS=as" "CC=gcc" "CXX=g++" "LD=/usr/lib/gcc/x86_64-pc-linux-gnu/12.1.0/../../../../x86_64-pc-linux-gnu/bin/ld" "LIBCFLAGS=-g -O2  " "NM=nm" "PICFLAG=" "RANLIB=ranlib" "DESTDIR=" DO=all multi-do # make
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/zlib'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/libbacktrace'
make  all-am
make[2]: Entering directory '/programsBuild/crosscomp/build-gcc/libbacktrace'
true  DO=all multi-do # make
make[2]: Leaving directory '/programsBuild/crosscomp/build-gcc/libbacktrace'
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/libbacktrace'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/libcpp'
test -f config.h || (rm -f stamp-h1 && make stamp-h1)
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/libcpp'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/libcody'
# --enable-maintainer-mode to rebuild ../../gcc/libcody/config.h.in, or make MAINTAINER=touch
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/libcody'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/libdecnumber'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/libdecnumber'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/fixincludes'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/fixincludes'
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/gcc'
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/gcc'
Checking multilib configuration for libgcc...
make[1]: Entering directory '/programsBuild/crosscomp/build-gcc/x86_64-elf/libgcc'
# If this is the top-level multilib, build all the other
# multilibs.
make[2]: Entering directory '/programsBuild/crosscomp/build-gcc/x86_64-elf/libgcc'
make[3]: Entering directory '/programsBuild/crosscomp/build-gcc/x86_64-elf/no-red-zone/libgcc'
# If this is the top-level multilib, build all the other
# multilibs.
/programsBuild/crosscomp/build-gcc/./gcc/xgcc -B/programsBuild/crosscomp/build-gcc/./gcc/ -B/home/techflash/opt/cross/x86_64-elf/bin/ -B/home/techflash/opt/cross/x86_64-elf/lib/ -isystem /home/techflash/opt/cross/x86_64-elf/include -isystem /home/techflash/opt/cross/x86_64-elf/sys-include    -O2 -mcmodel=kernel -mno-red-zone -mno-red-zone -O2  -g -O2 -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE  -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wstrict-prototypes -Wmissing-prototypes -Wold-style-definition  -isystem ./include  -fpic -g -DIN_LIBGCC2 -fbuilding-libgcc -fno-stack-protector -Dinhibit_libc -fpic -I. -I. -I../../.././gcc -I../../../../gcc/libgcc -I../../../../gcc/libgcc/. -I../../../../gcc/libgcc/../gcc -I../../../../gcc/libgcc/../include  -DHAVE_CC_TLS   -o _muldi3.o -MT _muldi3.o -MD -MP -MF _muldi3.dep -DL_muldi3 -c ../../../../gcc/libgcc/libgcc2.c -fvisibility=hidden -DHIDE_EXPORTS
cc1: error: code model kernel does not support PIC mode
make[3]: *** [Makefile:501: _muldi3.o] Error 1
make[3]: Leaving directory '/programsBuild/crosscomp/build-gcc/x86_64-elf/no-red-zone/libgcc'
make[2]: *** [Makefile:1212: multi-do] Error 1
make[2]: Leaving directory '/programsBuild/crosscomp/build-gcc/x86_64-elf/libgcc'
make[1]: *** [Makefile:127: all-multi] Error 2
make[1]: Leaving directory '/programsBuild/crosscomp/build-gcc/x86_64-elf/libgcc'
make: *** [Makefile:14024: all-target-libgcc] Error 2

[davmac314]

Code: Select all

cc1: error: code model kernel does not support PIC mode

Well that's the error (and it's due to the compilation command line having both -fpic and -mcmodel=kernel).

From the wiki page that Octonobrass linked:

This may require a workaround. First compile binutils as usual, then add the binaries to the PATH, and start to compile gcc. When compiling libgcc with -mcmodel=kernel, it will fail. Then patch the Makefile to disable PIC, repeat and continue:

You need to follow those instructions (or find some other way to prevent libgcc being built as PIC; you could try configuring with --disable-shared, if not doing that already). It's not clear to me whether you are currently building libgcc separately or as part of GCC as a whole, but It may be necessary to build it separately as per that page.

[Techflash]

Code: Select all

cc1: error: code model kernel does not support PIC mode

Well that's the error (and it's due to the compilation command line having both -fpic and -mcmodel=kernel).

From the wiki page that Octonobrass linked:

This may require a workaround. First compile binutils as usual, then add the binaries to the PATH, and start to compile gcc. When compiling libgcc with -mcmodel=kernel, it will fail. Then patch the Makefile to disable PIC, repeat and continue:

You need to follow those instructions (or find some other way to prevent libgcc being built as PIC; you could try configuring with --disable-shared, if not doing that already). It's not clear to me whether you are currently building libgcc separately or as part of GCC as a whole, but It may be necessary to build it separately as per that page.

That's exactly what I did, I ran that command, then I ran the sed command to modify the makefile for libgcc, then I ran it again and it still gave the error.

UPDATE: Ran it again again with a clean install (backed up my current opt/cross folder) and started from scratch, building binutils according to "GCC Cross Compiler", and GCC according to "Building GCC for mcmodel=kernel", and now it compiled!
UPDATE 2: Welp, closer, now I have a bunch or relocation errors from kernel.c???

Code: Select all

/opt/cross/lib/gcc/x86_64-elf/13.0.0/../../../../x86_64-elf/bin/ld: warning: tfos.elf has a LOAD segment with RWX permissions
kernel/kernel.o: in function `kernelMain':
/TFOS/kernel/kernel/kernel.c:33:(.text+0x84): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.8'
/TFOS/kernel/kernel/kernel.c:37:(.text+0xad): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.8'
/TFOS/kernel/kernel/kernel.c:38:(.text+0xbc): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.1'
/TFOS/kernel/kernel/kernel.c:55:(.text+0xf7): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.1'
/TFOS/kernel/kernel/kernel.c:55:(.text+0xfc): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.1'
/TFOS/kernel/kernel/kernel.c:57:(.text+0x11c): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.1'
/TFOS/kernel/kernel/kernel.c:95:(.text+0x205): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.8'
/TFOS/kernel/kernel/kernel.c:102:(.text+0x23e): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.8'
/TFOS/kernel/kernel/kernel.c:109:(.text+0x268): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.8'
/TFOS/kernel/kernel/kernel.c:118:(.text+0x29b): relocation truncated to fit: R_X86_64_32 against `.rodata.str1.8'
/TFOS/kernel/kernel/kernel.c:122:(.text+0x2b6): additional relocation overflows omitted from the output

Basically every function call for some reason. I'm gonna try shifting some files around in the linking order and see what I can get.

UPDATE 3: Fixed it, now I'm just left with:

Code: Select all

/opt/cross/lib/gcc/x86_64-elf/13.0.0/../../../../x86_64-elf/bin/ld: warning: tfos.elf has a LOAD segment with RWX permissions

And I have no idea how to fix it. I know what it means, it means the segment LOAD has Read Write and Execute permissions (which is bad because it leaves the kernel open to memory corruption), but I'm not really sure what to do about it. I'm guessing probably something in the linkerscript?