Page 3 of 4
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Fri Jan 08, 2021 6:41 pm
by Schol-R-LEA
nexos wrote:How come you don't just just install the mingw packages from APT? That would be a lot easier. I have had many problems with Clang, so I would stay away from it.
I am on Manjaro, so it would be Pacman (or Pamac) rather than APT, but I see what you mean. I was hoping to simply have my own clean builds of multiple cross-compilers (including some which don't have pre-existing versions AFAICT), but at this point using the pre-existing ones is the best approach I suppose.
As it happens, when I tried to use a local build of the host compiler (after fixing another issue where I'd forgotten I would actually need the headers and local shared libs for the local host compiler) I got the following problem building said compiler:
Code: Select all
In file included from /home/schol-r-lea/Deployments/cross-dev-utils/gcc/libcpp/charset.c:21:
/home/schol-r-lea/Deployments/cross-dev-utils/gcc/libcpp/system.h:41:10: fatal error: new: No such file or directory
41 | #include <new>
| ^~~~~
compilation terminated.
make[1]: *** [Makefile:224: charset.o] Error 1
make[1]: Leaving directory '/home/schol-r-lea/Deployments/common/gcc/build/x86_64-pc-linux-gnu/libcpp'
make: *** [Makefile:9104: all-libcpp] Error 2
So yeah, right now I'll just use the repo versions of MINGW.
EDIT: When I used the
/bin/x86_64-w54-mingw32-gcc, I got the following error:
Code: Select all
/usr/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: data.o:data.c:(.data+0x20): undefined reference to `LibStubStriCmp'
/usr/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: data.o:data.c:(.data+0x28): undefined reference to `LibStubMetaiMatch'
/usr/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: data.o:data.c:(.data+0x30): undefined reference to `LibStubStrLwrUpr'
/usr/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: data.o:data.c:(.data+0x38): undefined reference to `LibStubStrLwrUpr'
collect2: error: ld returned 1 exit status
However, this looks to be an actual issue with linking to GNU-EFI, rather than a compiler issue, so I am going to call that progress of a sort. When I NULLed the values as recommended in
UEFI App Bare Bones, it compiled and linked correctly. I'll move on towards building the disk image next.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Sat Jan 09, 2021 3:03 am
by kzinti
If all you want is to build a UEFI bootloader, you don't need mingw or gnu-efi. I simply use a plain x86_64-none-elf cross-compiler and do relocations myself (not unlike gnu-efi, but without gnu-efi). It is not very complicated and solves a lot of problem, including the need for mingw.
I am more than happy to point you at working code if you are interested... basically you just need some very simple relocation code at startup and to use x86_64-none-objcopy to build a PE file from the ELF executable.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Sat Jan 09, 2021 2:11 pm
by Schol-R-LEA
kzinti wrote:If all you want is to build a UEFI bootloader, you don't need mingw or gnu-efi. I simply use a plain x86_64-none-elf cross-compiler and do relocations myself (not unlike gnu-efi, but without gnu-efi). It is not very complicated and solves a lot of problem, including the need for mingw.
I am more than happy to point you at working code if you are interested... basically you just need some very simple relocation code at startup and to use x86_64-none-objcopy to build a PE file from the ELF executable.
Yes, please, I would like to see that very much. I would need to see that sort of code eventually anyway, as I still intend to implement my own toolchain in the future. Many thanks to you, oh 3 meter tall feline engine of destruction.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Sat Jan 09, 2021 5:22 pm
by kzinti
You can do this with any cross compiler or even your machine's hosted compiler. The approach is to create a relocatable ELF file, convert it to a PE file and run some relocation code when starting the program.
Flags for compilation and linking:
Code: Select all
gcc -fpic -mno-red-zone -ffreestanding -fshort-wchar
ld -nostdlib -shared -Bsymbolic
I use a linker script to put the bootloader at address 0, this makes the relocation code trivial and easier to debug. The important part is to make sure you include .reloc, .got.plt and .got. Here is what I use (I can't remember if the ".hash" section is important for this or not, it might be). Also any .bss should be put in the .data section as UEFI loaders don't seem to understand / handle .bss properly. Finally the "ImageBase" symbol in the linker script is important as it is used by the startup code to calculate where the program was relocated by the UEFI loader.
https://github.com/kiznit/rainbow-os/bl ... 64/efi.lds
Once you have your ELF file, you use objcopy to convert it to a UEFI PE file (I include .init / .fini / .ctors and .dtors, you might or might not care about these). Make sure to include both ".rela" and ".reloc" as this is where the relocations are stored.
Code: Select all
objcopy -j .text -j .init -j .fini -j .rodata -j .data -j .ctors -j .dtors -j .dynamic -j .dynsym -j .rel.* -j .rela.* -j .reloc --target efi-app-x86_64 bootloader.elf bootloader.efi
Finally you need to do relocations at the beginning of your startup code. Here is the relocation code for x86_64:
https://github.com/kiznit/rainbow-os/bl ... /reloc.cpp
The startup code:
https://github.com/kiznit/rainbow-os/bl ... .S#L35-L49
This bit is important too:
https://github.com/kiznit/rainbow-os/bl ... #L100-L114
There is also code for ia32 if you are interested, it works pretty much the same way.
Also make sure you use the MS ABI when declaring UEFI protocol functions. I use this before including my UEFI headers:
Code: Select all
#define EFIAPI __attribute__((ms_abi))
Let me know if you have any questions or comments...
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Mon Feb 08, 2021 2:06 pm
by Schol-R-LEA
Just as a quick update (after a bit of a hiatus): I am going to try building Rainbow-OS and review the code you've written, in order to see how your approach works in practice. I'll let you know if I have any questions about it.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Mon Feb 08, 2021 9:39 pm
by kzinti
Sounds good... One thing to note is that you will need to build a custom cross-compiler to compile Rainbow OS.
It should be (I have not tried this on a different machine) as simple as:
This will build and install the cross-compiler at ~/opt/cross, so you'll need to add ~/opt/cross/bin to your path before you can compile rainbow-os and/or the bootloader.
If you want to change the location, change this line:
https://github.com/kiznit/rainbow-os/bl ... gcc.mk#L26
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 2:03 pm
by Schol-R-LEA
kzinti wrote:Sounds good... One thing to note is that you will need to build a custom cross-compiler to compile Rainbow OS.
It should be (I have not tried this on a different machine) as simple as:
This will build and install the cross-compiler at ~/opt/cross, so you'll need to add ~/opt/cross/bin to your path before you can compile rainbow-os and/or the bootloader.
If you want to change the location, change this line:
https://github.com/kiznit/rainbow-os/bl ... gcc.mk#L26
I couldn't get the buil;d to work, as it happens. the error I got was:
Code: Select all
tar: OSes/rainbow-os/toolchain/build/newlib-4.1.0.tar.gz: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
make[1]: *** [gcc.mk:158: OSes/rainbow-os/toolchain/build/newlib-4.1.0] Error 2
make[1]: Leaving directory '/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other OSDev Members OSes/rainbow-os/toolchain'
make: *** [Makefile:40: x86_64-elf] Error 2
The log shows that newlib was downloaded, so I don't know why it couldn't build it.
EDIT: It turns out it is due to a peculiarity of the path I was building it in. I'll fix that now.
EDIT 2: I did get an error message at the end of the toolchain build, but I can't tell if it is actually relevant or not.
Code: Select all
make[2]: Leaving directory '/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/x86_64-elf/gcc-10.2.0'
make[1]: *** [gcc.mk:87: build-gcc] Error 2
make[1]: Leaving directory '/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain'
make: *** [Makefile:40: x86_64-elf] Error 2
In hopes that it wasn't, I tried running the default build for the OS image, and got the following errors:
Code: Select all
x86_64-elf-gcc -mno-red-zone -O2 -Wall -Wextra -Werror -ffreestanding -fbuiltin -fno-pic -std=gnu++20 -fpic -D__rainbow__ -DKERNEL_ARCH=x86_64 -DKERNEL_X86_64 -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/ -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/ -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//include -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//../user/include -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//third_party -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//third_party/uefi -MMD -MP -c /home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/elfloader.cpp -o elfloader.cpp.o
In file included from /home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/display.hpp:31,
from /home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/display.cpp:27:
/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//include/rainbow/boot.hpp:30:10: fatal error: cstdint: No such file or directory
30 | #include <cstdint>
| ^~~~~~~~~
In file included from /home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/crt.cpp:27:
/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/boot.hpp:30:10: fatal error: cstddef: No such file or directory
30 | #include <cstddef>
| ^~~~~~~~~
compilation terminated.
/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/boot.cpp:27:10: fatal error: cstring: No such file or directory
27 | #include <cstring>
| ^~~~~~~~~
compilation terminated.
compilation terminated.
make[1]: *** [/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//mk/rules.mk:31: boot.cpp.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//mk/rules.mk:31: crt.cpp.o] Error 1
make[1]: *** [/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//mk/rules.mk:31: display.cpp.o] Error 1
/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src/boot/elfloader.cpp:28:10: fatal error: cstring: No such file or directory
28 | #include <cstring>
| ^~~~~~~~~
compilation terminated.
make[1]: *** [/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/src//mk/rules.mk:31: elfloader.cpp.o] Error 1
make[1]: Leaving directory '/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/build/x86_64/boot/efi'
make: *** [Makefile:68: boot] Error 2
These error messages seem to indicate that newlib wasn't installed, but I am not certain.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 4:11 pm
by kzinti
make[2]: Leaving directory '/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/x86_64-elf/gcc-10.2.0'
make[1]: *** [gcc.mk:87: build-gcc] Error 2
make[1]: Leaving directory '/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain'
make: *** [Makefile:40: x86_64-elf] Error 2
Looks like it failed to build GCC on line 87. There shouldn't be any error there. You would have to scroll up your terminal to see what the actual error is.
Since gcc failed to build, so did newlib and libstdc++. This explains the missing C++ headers.
Sorry this isn't something that I get to test on different environments... But I do rebuild GCC multiple times per week these days with no issues. So I'd be curious to know why it failed so that it can be fixed.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 5:53 pm
by Schol-R-LEA
OK, looking again at the Makefile for the toolchain, this is the error which comes up first:
Code: Select all
g++ -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp -I. -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp/../include -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp/include -g -O2 -W -Wall -Wno-narrowing -Wwrite-strings -Wmissing-format-attribute -pedantic -Wno-long-long -fno-exceptions -fno-rtti -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp -I. -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp/../include -I/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp/include -c -o directives.o -MT directives.o -MMD -MP -MF .deps/directives.Tpo /home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp/directives.c
In file included from /home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp/charset.c:21:
/home/schol-r-lea/Documents/Programming/Projects/OS-Experiments/Other-OSDev-Members-OSes/rainbow-os/toolchain/build/gcc-10.2.0/libcpp/system.h:41:10: fatal error: new: No such file or directory
41 | #include <new>
| ^~~~~
compilation terminated.
So yes, the errors reported there earlier are definitely relevant. Is there some part of the build for either newlib or GCC which I am missing?
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 5:57 pm
by kzinti
I am not sure what is going on here... <new> is either part of GCC itself or part of libstdc++v3 (I think it is the later). It is not part of newlib.
This header (and many other C++ headers) should be at /opt/cross/x86_64-elf/include/c++/10.2.0
Your error is happening while building libcpp which is part of libstdc++v3. It's like it cannot find its own header?!?
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 5:59 pm
by Schol-R-LEA
I added a log of the error output to my last post as an attachment. I will try to trim the whole build log so I can do the same with that.
EDIT: I attached the two parts of that log to this post.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 6:04 pm
by kzinti
I checked and it looks like <new> is part of libsupc++, which is part of libstdc++v3. So I am not sure what is going on here...
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 6:32 pm
by kzinti
Some people having the same issue:
https://github.com/dslm4515/Musl-LFS/issues/9
How is your path setup?
In my case I have "/home/kzinti/opt/cross/bin" at the end of $PATH.
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Tue Feb 09, 2021 11:21 pm
by Schol-R-LEA
My default PATH is pretty long, actually. The first two sections are
Code: Select all
PATH=/home/schol-r-lea/opt/cross/bin:/home/schol-r-lea/opt/bin:
Re: [UEFI bare bones/x-compiler config] - LD not finding li
Posted: Wed Feb 10, 2021 1:11 am
by kzinti
After some digging I found some dependency issues between the different part of the builds. This is now fixed.
Sorry for the trouble, I suppose I never saw this as I always build on top of an existing cross-compiler... Mmm.
You will have to delete the /opt/cross directory and the toolchain/build directory before attempting to build again.
Schol-R-LEA wrote:My default PATH is pretty long, actually. The first two sections are
According to the link I found above, this could explain the problem with not finding <new>. I would move the cross-compiler to the end of the path (that's what I have) or remove it entirely from the path while building the toolchain.