OSDev.org

File structure:
/NyanOS:
->Makefile
->README.md
->fs - some data files for making the filesystem, currently I didn't write any utilities for formatting.
->images
->Pure64-NyanOS-mod - contains a bit modified Pure64 bootloader for my OS.
->boot - contains different bootloader stages.
->kernel - contains only one file (kernel_main.xnc), which is three lines long: ->vm - contains the XNC virtual machine source files.

"make" builds everything and writes all to images/NyanOS.img
"make qemu" boot QEMU from images/NyanOS.img

My build process is divided into three parts:

• Toolchain
• Userspace
• Kernel (and modules)

The toolchain is a mess of bash scripts that facilitate downloading tarballs, extracting them, patching their source trees, and then building them. Part of this is the native toolchain - gcc, binutils - but the standard build also includes cross-compiling a number of libraries - newlib, zlib, libpng, Cairo, Mesa. Ideally, this gets run once when setting up a development machine, but updating external libraries and making patches will necessitate re-running at least parts of it - newlib in particular gets rebuilt so often I bothered to write a script just for rebuilding it alone.

The userspace build process is managed mostly by a Makefile, but is also aided by a Python script that automatically detects dependences between applications and libraries. That Python script used to also manage the build process until I moved it into the core Makefile. The resulting binaries are placed within a template directory (that also includes a bunch of static content like default configs, resources for the UI, etc.).

The kernel is also built by the same Makefile, but without any additional magic. I use a single Makefile for everything, mostly with automatic targets (%.o from %.c). The same process also builds kernel modules, which end up in the template directory under a "mod" subdirectory.

After building the userspace and the kernel, a disk image is generated from the template directory. The standard setup doesn't bother installing a bootloader as in a development environment I just run qemu directly with its built-in multiboot support. For distribution purposes, I have a cobbled-together script that uses mkfs.ext2, some crazy Linux disk magic, and grub-install.

You can take a look at a sample run of the userspace and kernel steps of the build process through Travis CI, or curl a log into your terminal. Travis also performs a simple test over a serial console.

Hey klange,

klange wrote:newlib in particular gets rebuilt so often I bothered to write a script just for rebuilding it alone.

why do you have to rebuild it that often?

max wrote:Hey klange,

klange wrote:newlib in particular gets rebuilt so often I bothered to write a script just for rebuilding it alone.

why do you have to rebuild it that often?

Well In newlib you have your system calls implemented and your crt0 stuff, and these things are generally often changed. I am going to write a libnewlibsupc and let my newlib stubs call the routines from there to solve that problem and gain more flexibility.

max wrote:Hey klange,

klange wrote:newlib in particular gets rebuilt so often I bothered to write a script just for rebuilding it alone.

why do you have to rebuild it that often?

Martin's got the right idea above - unlike the rest of the toolchain, there's a lot of change happening in my extensions to newlib as I add support for new things that belong in the C library. While I might build GCC once and be done with it, every time I add something new the C library - some new system call, or a new piece of userspace functionality (like when I added getpwent and friends), I rebuild newlib. I don't use a glue library, I have my own sys directory in my newlib "fork", which means I do need to rebuild all of newlib - but it's not particularly slow, so it works out.

martinFTW wrote:Well In newlib you have your system calls implemented and your crt0 stuff, and these things are generally often changed.

Not necessarily - I compiled my GCC so that it requires a Ghost-specific library (libghostapi, which also contains system call wrappers and stuff), and made the build process of it so that it copies the CRTs & the compiled API to my toolchain directory. Basically what you plan to do with libnewlibsupc

klange wrote:every time I add something new the C library ... I do need to rebuild all of newlib

Okay. I do actually use a glue library, I was snoopy how you do it