Building NewLib

[IanSeyler]

I am trying to create a port of NewLib for BareMetal OS so applications will be able to use standard C calls (like printf()). I have followed the Porting Newlib doc (http://wiki.osdev.org/Porting_Newlib) but ran into a few issues.

First of all it looks like Newlib 1.15.0 does not support x86-64 so I am using the latest (1.19.0). I am doing the compile in an Ubuntu 64-bit 11.04 VM and have the extracted Newlib and a "build" folder along side it.

I run this to generate the makefile:
../newlib-1.19.0/configure --target=x86_64-pc-none
I then had to adjust the makefile to tell it that the target programs are the same as the host programs (as I am on the same architecture, I don't need a cross-compiler setup?)

Running make looks good other than it is compiling with the "-m32" argument instead of "-m64". Eventually the build will fail with "Error: bad register name" because it does not recognize the 64-bit register names (probably because it looks to be compiling a 32-bit binary).

How can I force it to compile to 64-bit binaries?

Thanks in advance,
-Ian

Output when it fails:

Code: Select all

make[8]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/libc/misc'
Making all in machine
make[8]: Entering directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/libc/machine'
Making all in x86_64
make[9]: Entering directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/libc/machine/x86_64'
cc -B/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/ -isystem /home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/targ-include -isystem /home/ian/Downloads/newlib/newlib-1.19.0/newlib/libc/include -B/home/ian/Downloads/newlib/build/x86_64-pc-none/32/libgloss/x86_64 -L/home/ian/Downloads/newlib/build/x86_64-pc-none/32/libgloss/libnosys -L/home/ian/Downloads/newlib/newlib-1.19.0/libgloss/x86_64  -m32 -DPACKAGE_NAME=\"newlib\" -DPACKAGE_TARNAME=\"newlib\" -DPACKAGE_VERSION=\"1.19.0\" -DPACKAGE_STRING=\"newlib\ 1.19.0\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I. -I../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64 -DMISSING_SYSCALL_NAMES -fno-builtin     -DMISSING_SYSCALL_NAMES -fno-builtin   -g -O2  -m32 -c -o lib_a-setjmp.o `test -f 'setjmp.S' || echo '../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/'`setjmp.S
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S: Assembler messages:
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:25: Error: bad register name `%rbx'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:26: Error: bad register name `%rbp'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:27: Error: bad register name `%r12'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:28: Error: bad register name `%r13'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:29: Error: bad register name `%r14'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:30: Error: bad register name `%r15'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:31: Error: bad register name `%rsp)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:32: Error: bad register name `%rax'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:33: Error: bad register name `%rsp)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:34: Error: bad register name `%rax'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:35: Error: bad register name `%rax'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:39: Error: bad register name `%rsi'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:41: Error: bad register name `%rdi)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:44: Error: bad register name `%rdi)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:45: Error: bad register name `%rdi)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:46: Error: bad register name `%rdi)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:47: Error: bad register name `%rdi)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:48: Error: bad register name `%rdi)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:49: Error: bad register name `%rdi)'
../../../../../../../newlib-1.19.0/newlib/libc/machine/x86_64/setjmp.S:50: Error: bad register name `%rdi)'
make[9]: *** [lib_a-setjmp.o] Error 1
make[9]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/libc/machine/x86_64'
make[8]: *** [all-recursive] Error 1
make[8]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/libc/machine'
make[7]: *** [all-recursive] Error 1
make[7]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib/libc'
make[6]: *** [all-recursive] Error 1
make[6]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib'
make[5]: *** [all] Error 2
make[5]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/32/newlib'
make[4]: *** [multi-do] Error 1
make[4]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/newlib'
make[3]: *** [all-multi] Error 2
make[3]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/newlib'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/ian/Downloads/newlib/build/x86_64-pc-none/newlib'
make[1]: *** [all-target-newlib] Error 2
make[1]: Leaving directory `/home/ian/Downloads/newlib/build'
make: *** [all] Error 2

[Cognition]

Dunno how much this will help, but newlib also answers some of the porting requirements in the FAQ on their site here: http://sourceware.org/newlib/faq.html#q6

From the sounds of it changing some of the configuration information as described in that section should allow you to tweak the build flags.

[gerryg400]

Adding newlib (or some c library) to your project will change your life. It's the biggest productivity gain I made in my OS.

I tried building newlib like you're doing on x86_64 for a few hours and gave up. I'm can't say whether it works or not, just that I could not get it working.

You definitely need to build a cross-compiler ! So, in the end I decided to invest the time in making my own tool-chain. It took a day or two but was well worth the effort. I followed this tutorial. Since then I've made some changes to my custom target and written some patches and scripts to automate the process.

[Edit] The main reason I say you need a c compiler is that it is your best way to control the tools that your users use. You need everyone who is making software for your platform to be using exactly the same assembler, compiler, library etc. so that you can support them properly.

[IanSeyler]

I tried the exact same steps I was using before on Arch Linux (another 64-bit VM) and it worked correctly.

During the compile it was not inserting "-m32" into the commands. Any ideas why this would be happening or where I can remove it from? I have no idea where it is pulling that arguement from.

Thanks,
-Ian

[IanSeyler]

After some further tinkering I have what appears to be a working Newlib for BareMetal OS. I ended up following the build script of Boomstick (http://wiki.osdev.org/Boomstick) to get things going. No cross compiler required. I'm surprised that it is so large though. 5MiB for libc.a alone but it is functional!

[IanSeyler]

"strip -d" gets rid of the debugging symbols so libc.a gets down to just over 1MiB now. Compiled a "Hello World" program using the C printf() function and it executed correctly (Binary was 44KiB though, Almost 3X the size of the OS kernel!).

The question I have now is what do I do with "CRT0"? Just compile it and link it as the first object?

[JamesM]

"strip -d" gets rid of the debugging symbols so libc.a gets down to just over 1MiB now. Compiled a "Hello World" program using the C printf() function and it executed correctly (Binary was 44KiB though, Almost 3X the size of the OS kernel!).

The question I have now is what do I do with "CRT0"? Just compile it and link it as the first object?

Strip the binary afterwards too. There are symbols only required for linking that linger.

Yeah, just compile it and either put it on the link line every time or recompile your cross compiler, baking in the crt0 location at configure time as in the wiki tutorial.

[IanSeyler]

strip doesn't work on flat binaries (which is what I am using).

Here is the latest issue while trying to use fgets()...
helloc.c

Code: Select all

// Hello World C (Newlib) Test Program (v1.0, June 17 2011)
// Written by Ian Seyler
//
// BareMetal compile:
//
// GCC (Tested with 4.6.0)
// gcc -c helloc.c -o helloc.o
// ld -T app.ld -o helloc.app helloc.o libc.a

#include <stdio.h>
#include <stdlib.h>

char userinput[160];

int main(void)
{
        unsigned long tint = 0xFFFFFFFFFFFFFFFE;
        tint += 1;
        printf("Please type your name: ");
        fgets(userinput, 100, stdin);
        printf("Hello %s, from C! %llu\n", userinput, tint);
        return 0;
}

Code: Select all

[root@arch64 test]# gcc -c helloc.c -o helloc.o
[root@arch64 test]# ld -T app.ld -o helloc.app helloc.o libc.a
helloc.o: In function `main':
helloc.c:(.text+0x2a): undefined reference to `stdin'

Did I miss something in the glue? It should recognize stdin...

app.ld

Code: Select all

OUTPUT_FORMAT("binary")
OUTPUT_ARCH("i386:x86-64")
ENTRY(main)
SECTIONS
{
        . = 0x0000000000200000;
        .text : { *(.text) }
        .data : { *(.data .rodata) QUAD(ADDR(.bss)+SIZEOF(.bss)) }
        .bss : { *(.bss) }
        end = .; _end = .; __end = .;
}

[gerryg400]

stdin is defined in <stdio.h>. Is gcc including the newlib <stdio.h> or your native platform <stdio.h> ?

[Solar]

The error message is a linker error, not a compiler error. I.e., the compiler sees the definition of stdin allright, but at link time the object isn't there. Does Newlib set up stdin and stdout in crt0.o, by any chance?

[gerryg400]

In newlib stdin is a macro so you should get a message saying that something other than stdin is missing. This is from Newlib <stdio.h>

Code: Select all

#ifndef _REENT_ONLY
#define	stdin	(_REENT->_stdin)
#define	stdout	(_REENT->_stdout)
#define	stderr	(_REENT->_stderr)
#else /* _REENT_ONLY */
#define	stdin	(_impure_ptr->_stdin)
#define	stdout	(_impure_ptr->_stdout)
#define	stderr	(_impure_ptr->_stderr)
#endif /* _REENT_ONLY */

IIRC, in Linux stdin is an extern of type FILE *, so that linker message would only be expected if the Linux header were being included.

[IanSeyler]

gerryg400, that was exactly the issue. The initial C compile was looking at the Linux headers.. not the Newlib ones.

The quick fix:

Code: Select all

gcc -I ../../newlib-1.19.0/newlib/libc/include/ -c helloc.c -o helloc.o

It compiles and links normally now.

Now just to fix the bugs with the system calls.

For anyone else looking at porting Newlib, all of my current notes are here: http://code.google.com/p/baremetal/wiki/BuildingNewlib

Thanks for the help everyone.

Best regards,
-Ian

[gerryg400]

The quick fix:

Code: Select all

gcc -I ../../newlib-1.19.0/newlib/libc/include/ -c helloc.c -o helloc.o

I assume you will make the real fix now (which in the medium to long term is considerably quicker and far less heart-breaking than the quick fix).

[Combuster]

hint: it is not a fix because gcc is still allowed to use the linux headers; you just give it two stdio.h's to choose from and it happens to pick the right one. GCC Cross-Compiler to the rescue.