OSDev.org

Posted: **Thu May 25, 2017 8:20 pm**

So I just completed http://wiki.osdev.org/Porting_Newlib. I realize that page does not have the best advice and glosses over libgloss (sorry) but it seems to work just fine for my purposes. I'm interested to see if what I did seems fine for basic purposes. Note that I only cared about getting newlib working and I didn't do anything in the way of OS specific toolchain.

- Compile binutils and GCC with --with-sysroot pointing to some directory
- Follow the above link
- Have a implementation-defined list of indices for system calls and have each stub call an implementation-defined software interrupt with arguments, i.e:

Code: Select all

  #define sys_write 4
  int write(int file, char *ptr, int len) {
      int res; 
      asm volatile("int $0x80" : "=a" (res) : "0" (sys_write), "b" ((int)file), "c" ((long)ptr), "d" ((int)len));  
      return res;
  }

Real code would wrap this in a macro
- Compile and install newlib files to the sysroot
- Treat 0x80 like any other interrupt and add an assembly stub/to the IDT
- In the ISR handler, if it's interrupt 0x80, pass the regs to the system call handler
- Check regs->eax for the system call type; call the corresponding function with the rest of the GPRs as arguments
- Link against -lc -lm (I have the feeling this is not necessary, so I screwed up somewhere)

Code: Select all

write(1, "Test", sizeof("Test"));

seems to work just fine.

What I'm looking to do next is to figure out how to replace newlib's malloc with liballoc (I already have liballoc working kernel-wise) as it seems sbrk is not very popular nowadays. This would ostensibly make testing parts of the library easier before I get userspace working.

Sidenote:

From my search it seems other people are having problems with sbrk and need to zero out the BSS. I'm guessing this only becomes a problem when you enter userspace. For now my stub looks like this:

Code: Select all

 caddr_t sbrk(int incr) {
      errno = ENOMEM;
      return (caddr_t) -1;
  }

which I mirror'd after the linux man page.

Posted: **Fri May 26, 2017 2:50 am**

goku420 wrote: What I'm looking to do next is to figure out how to replace newlib's malloc with liballoc (I already have liballoc working kernel-wise) as it seems sbrk is not very popular nowadays. This would ostensibly make testing parts of the library easier before I get userspace working.

Not sure exactly how it's done in Newlib's build system (since I replaced it with a simple makefile after struggling to convince it to build me a shared library for over a month), but there's a macro called "MALLOC_PROVIDED" macro that will disable Newlib's internal malloc/free implementation. Looking at the wiki, you likely define it in the same way as "SIGNAL_PROVIDED" in "configure.host".

Once that's done, you should be able to just add the liballoc source file to your "EXTRA_lib_a_SOURCES".

Posted: **Fri May 26, 2017 3:36 am**

mallard wrote:Not sure exactly how it's done in Newlib's build system (since I replaced it with a simple makefile after struggling to convince it to build me a shared library for over a month), but there's a macro called "MALLOC_PROVIDED" macro that will disable Newlib's internal malloc/free implementation. Looking at the wiki, you likely define it in the same way as "SIGNAL_PROVIDED" in "configure.host".

Once that's done, you should be able to just add the liballoc source file to your "EXTRA_lib_a_SOURCES".

Thanks. I guess it would be the perfect time to actually write mmap. Since I don't have a filesystem or userspace, my implementation should be relatively simple:

- Flags: MAP_ANONYMOUS, PROT_WRITE, PROT_READ, MAP_PRIVATE
- As a result, the fd and offset arguments are irrelevant
- The start address will be a nearby page boundary (round down?)
- It needs to be mapped in multiples of page sizes

Posted: **Sat May 27, 2017 11:35 am**

mallard wrote:
goku420 wrote: What I'm looking to do next is to figure out how to replace newlib's malloc with liballoc (I already have liballoc working kernel-wise) as it seems sbrk is not very popular nowadays. This would ostensibly make testing parts of the library easier before I get userspace working.
Not sure exactly how it's done in Newlib's build system (since I replaced it with a simple makefile after struggling to convince it to build me a shared library for over a month), but there's a macro called "MALLOC_PROVIDED" macro that will disable Newlib's internal malloc/free implementation. Looking at the wiki, you likely define it in the same way as "SIGNAL_PROVIDED" in "configure.host".

Once that's done, you should be able to just add the liballoc source file to your "EXTRA_lib_a_SOURCES".

I struggled too convincing the Newlib create a shared library for a bare metal system. The bare metal configuration for ARM just makes statically linked archived that is supposed to be used in statically linked embedded system like Cortex-M3 systems with simple operating systems. Shared library configurations only exists for existing OSes like Linux together with ARM. My solution was the following:

First force -fPIC into the build that creates an archive. Something like this for the configuration script (here I use -fpie but -fPIC can be used just as well).

Code: Select all

./configure --prefix=/home/johndoe/newlib-pie --target=arm-none-eabi --disable-newlib-supplied-syscalls --enable-newlib-io-long-long --enable-newlib-io-long-double \
--enable-newlib-io-c99-formats --disable-multilib --with-cpu=armv7-a --with-mode=thumb --with-float=soft --disable-nls CFLAGS_FOR_TARGET="-g -O2 -fpie -fdata-sections -ffunction-sections"

Then you have an .a file but built with -fPIC. Now you want to link this archive into an actual share library and this can be done with the following command line for the GCC linker.

Code: Select all

-shared -T SharedLibrary.lds -z common-page-size=4096 -z max-page-size=4096 \
-Bsymbolic -no-allow-shlib-undefined -nostdlib -whole-archive ${CLIB_LIBRARY_PATH}/libc.a -no-whole-archive

SharedLibrary.lds is a linker script customized for your OS for shared libraries. Also you need extra source files that implements some of the required POSIX calls. If you know you will not be using a POSIX call you can just put something like "_wait = 0;" in the linker script in order to "stub" _wait (it will crash if ever used).

Obviously the Newlib configuration script is for existing systems like Linux and so on. Also it is using libtool to make shared libraries which is another dependency which is OS dependent. Another way is to create "customized toolchain" but I haven't bothered doing that. Instead you can "fool" Newlib like this instead.

The same strategy will also work for libstdc++ together with newlib (--with-newlib). I hope they keep the possibility to compile for newlib.

Posted: **Sat May 27, 2017 3:33 pm**

So that was actually relatively painless. I only made 3 changes:

- Made the change to configure.host like mallard suggested
- Add _malloc_r and friends
- Replace all instances of #include <liballoc.h> with #include <stdlib.h>

newlib requires _malloc_r and friends to be defined even if you don't use reentrancy. You can just discard the _reent parameter and return a call to malloc. Since I'm linking in liballoc anyway, it will be found by newlib.

In the future I suspect that I will have to fix this and actually add the liballoc hooks as a source as suggested by mallard; the problem is my kernel is in C++ so I'm not sure at the moment how I'll do that.

EDIT:

Success. The solution was to not to distribute liballoc_hooks with the newlib glue, only liballoc itself. To confirm that it's working I called strdup (something I knew would guarantee newlib to call malloc) and set a breakpoint in gdb:

Code: Select all

(gdb) break _malloc_r
Breakpoint 1 at 0xc0104d80: file ../../../../../../newlib-2.2.0-1/newlib/libc/sys/myos/malloc_wrappers.c, line 6.
(gdb) continue
Continuing.

Breakpoint 1, _malloc_r (ptr=0xc0120380 <impure_data>, size=14) at ../../../../../../newlib-2.2.0-1/newlib/libc/sys/myos/malloc_wrappers.c:6
6		return PREFIX(malloc) (size);
(gdb) s
malloc (req_size=14) at ../../../../../../newlib-2.2.0-1/newlib/libc/sys/myos/liballoc.c:242
242	{

OSDev.org

Porting newlib, messy first steps

Porting newlib, messy first steps

Re: Porting newlib, messy first steps

Re: Porting newlib, messy first steps

Re: Porting newlib, messy first steps

Re: Porting newlib, messy first steps