porting newlib issues

[stdcall]

Hi.
I successfully ported newlib according to the manual in the wiki, and successfully linked a user-space application against it.
However whenever I call any function in libc they program crash as it access some corrupt pointer in newlib's code.
The application is compiled as an executable ELF and it's loaded by the kernel through the program headers.

My speculation is that perhaps the bss is not initialized.

Any thoughts ?

The code for crt0.c is pasted below.

Code: Select all

#include <fcntl.h>

extern void exit(int code);
extern int main (int argc, int argv);

void _start(int argc, int argv) {

    int ex = main(argc, argv);
    exit(ex);
}

[davidv1992]

The newlib codebase depends on initialization code to be run before calling main. This is done through the special linker sections .init and .fini. This requires some runtime support, which is mostly provided through crt{i,n,begin,end}.o, but the functions these define still have to be called by the c library. Assuming a gcc resulting from the instructions in the wiki, your crt0 should call _init before doing anything using newlib, and call atexit with _fini as argument before passing control to main to make sure that gets called on program exit.

[stdcall]

That makes perfect sense. I will update the wiki once I'll have it working properly.
Thanks !

[stdcall]

The newlib codebase depends on initialization code to be run before calling main. This is done through the special linker sections .init and .fini. This requires some runtime support, which is mostly provided through crt{i,n,begin,end}.o, but the functions these define still have to be called by the c library. Assuming a gcc resulting from the instructions in the wiki, your crt0 should call _init before doing anything using newlib, and call atexit with _fini as argument before passing control to main to make sure that gets called on program exit.

Well, it doesn't work.
I've searched the newlib codebase for _init function and didn't find any implementation for that, the function exists though, so I guess GCC put it there.
are you sure this is how it's done ? do you have perhaps reference code for what you suggest ?

[davidv1992]

_init and _fini are provided by gcc, in fact that is the primary job of the crt{i,n,begin,end}.o object files i referred to (they also seem to do some other things, but I haven't yet taken the time to figure that out completely). The only reference for what I do in crt0 are several other implementations in newlib doing the same thing (i.e. calling _init and registering _fini with atexit), and it fixing some problems for me where I initially used it.

Beyond that, assuming you use ELF, are you loading your software correctly, in particular, do you zero the portion of the sections that aren't backed by file data. This is a requirement from the elf standard, and could be required for variables in the bss to be initialized correctly (as gcc's code assumes that anything it puts in the bss is initialized to 0 before it gets control). And just to make sure we aren't missing something obvious, does your loader (or pre-crt0 code) provide a proper stack for the software to use?

If those aren't the causes for your problems, then which libc functions are giving you trouble. Does it already start with functionality like strlen and memcpy (which should be self-contained functions with no dependencies on global state beyond the stack), or does it come from functions with more complex inner workings, e.g. printf, fopen and the like?

[simeonz]

Double check what I am about to say, but here are my 5 cents.

First, regarding your specific issue, there is a function in newlib\libc\misc\init.c - __libc_init_array, that you probably need to call before main. Also, there are several implementations of the entry point in crt0.c in several locations (for different platforms). If nothing else you can use them as reference in creating yours. I am just sharing observations. I don't know if it is relevant.

Generally speaking, gcc generates elfs according to the System V ABI. This depends on the output format, compilation/linking options, the toolchain build options, but this is the default behavior, and even the alternative behaviors emulate it to some extent. There are some linux specific conventions (such as those in the Linux Standard Base), which gcc also tries to conform to and even ports them to non-conforming/generic platforms.

Ignoring any conformance, the minimum you need to provide is a _start routine in order to call the .init section before calling main, and register the .fini section with atexit. Furthermore, for .init and .fini to work, you must provide their prologues and epilogues in crti.o and crtn.o. Those files are ordered on the linker command line by gcc to properly sandwitch the rest of the _init and _fini instructions and form function bodies (assuming the prologues and epilogues are correct). Your implementation of _start will be in crt1.o, again, in order to have the linker pick it up automatically on request by the gcc front-end. This is all in the wiki, I believe.

However, if your compiler is built with --enable-initfini-array (check -v), you also need to iterate the pointers inside the sections .init_array and .fini_array, right after calling .init and right before calling .fini correspondingly. If you want to support dynamic loading, you need to use tags from the PT_DYNAMIC segment (i.e. use DT_PREINIT_ARRAY, DT_INIT_ARRAY, DT_INIT, DT_FINI_ARRAY, DT_FINI inside your loader). If you want to be System V ABI compliant, you need to initialize the pre-loaded shared objects of the executable using those tags in specific dependency-based order, then skip the initialization and termination routines of the executable itself and call _start with rdx argument that points to finalization code.

If you want to support certain language facilities - .ctors and .dtors (deprecated and may never be used by your compiler), exeption handling, or transactional memory, you need to add either .init (and .fini) or .init_array (and .fini_array) entry and perform the necessary setup and cleanup. You can do that using crtbegin.o and crtend.o (although whether you use atexit or crtend.o depends on the facility).

In other words, there are platform and compiler specific issues. You need to inspect your elf output and compiler build configuration to decide what you are or will be dealing with.

Some interesting references (some duplicate the wiki):
System V Application Binary Interface
Linux Standard Base (LSB), etc.
Linux x86 Program Start Up
Acronyms relevant to Executable and Linkable Format (ELF) (Search for "What do _start and __libc_start_main do")
The ELF format - how programs look from the inside
How Initialization Functions Are Handled
Mini FAQ about the misc libc/gcc crt files

[dchapiesky]

your speculation that bss is not initalized is correct...

check your elf loader code...

you may also have to initalize the REENT pointer.... (if you support threading in userland)

as well as setup the environment... (and how do you pass the environment/command line args to your application)

The _start() function provided in the wiki is not terribly helpful....

look at the various crt0.c in the newlib source distribution for ideas (don't get too confused by the linux version as linux does alot of what crt0.c would do in an embedded system and then you start looking at linux code and well that is just pain and suffering)....

worst case look at magenta project newlib crt0.c

and if you used gcc to cross compile and link your app for your os - are you forcing the use of your ld script or accidently using the host system's ld script....

cheers

[stdcall]

worst case look at magenta project newlib crt0.c

Google Magenta ? which project are you referring ?

[stdcall]

simeonz, Thanks for the answer

First, regarding your specific issue, there is a function in newlib\libc\misc\init.c - __libc_init_array, that you probably need to call before main. Also, there are several implementations of the entry point in crt0.c in several locations (for different platforms). If nothing else you can use them as reference in creating yours. I am just sharing observations. I don't know if it is relevant.

I actually called that, didn't help.

[davidv1992]

I think it is time to look beyond crt0 for the problems you are describing.

Two questions:
- Does your ELF loader properly initialize bss to zero?
- Which newlib functions fail?