Page 1 of 2
Setting up build environment with Meson
Posted: Wed Jun 21, 2017 9:47 pm
by treyzania
Greetings!
This is my second swing at trying OS development. The first time wasn't much more than a Hello World, so this time I want to try to do it right. I'm trying to set up Meson (
http://mesonbuild.com/) as the build system for my kernel and am looking for some suggestions.
Here's how I'd like to structure the project:
Code: Select all
/
|--- arch/
| |--- amd64/
| | |--- boot.S
| | |--- syscall.S
| | |--- init.c
| | \--- [etc...]
| |--- arm/
| \--- [etc...]
|--- kernel/
| |--- mm/
| |--- mod/
| |--- fs/
| \--- [etc...]
\--- [etc...]
What I've tried to set up so far is having a main meson.build script in the root of the project and another in the root of each arch that handles building object data for each of those components (that gets included with `subdir` or `subproject` depending on which arch you're building for). But what I'm not sure about is how to link all of the different components into a single ELF binary that I can give to GRUB, and how to actually package a GRUB ISO that I can boot off. I'd like to do all of this with Meson, but it seems like its cross-compilation and special linking behavior support seems to be a little bit lacking, or at least have lacking documentation.
Does anybody have any tips for dealing with this in Meson?
Thanks!
Re: Setting up build environment with Meson
Posted: Thu Jun 22, 2017 12:08 am
by iansjack
I'm not familiar with Meson but if, as you say, it lacks support for cross-compiler and custom linking then I'd suggest it is not a good environment for OS development. Is there a reason that you are not happy to use "make"?
Re: Setting up build environment with Meson
Posted: Thu Jun 22, 2017 12:28 am
by treyzania
It does have support for custom link settings (as listed here:
http://mesonbuild.com/Reference-manual. ... _arguments) and cross compilation, but it seems like the documentation for builds with settings spread out in separate chunks is rather lacking. I was wondering if anybody had any experience with it.
Re: Setting up build environment with Meson
Posted: Thu Jun 22, 2017 7:28 am
by rupertsteel
I have gotten meson working for a very simple os, but I do have several problems with my system.
My meson build file currently cannot do the correct link order for c++ global constructors and it cannot build a iso image in the meson build file.
Here is my current tree
Code: Select all
|-- meson-build/
| \-- [build output and meson generated files]
|-- boot.asm
|-- build-iso.make
|-- cross_file.txt
|-- crti.asm
|-- crto.asm
|-- grub.cfg
|-- kernel.cpp
|-- linker.ld
\-- meson.build
meson.build
Code: Select all
# Specify the project
project('myos', 'c', 'cpp')
# Find where the crtbegin and crtend object files are so we can link to them
find_crtbegin_result = run_command('i686-elf-gcc', ['$CFLAGS', '-print-file-name=crtbegin.o'])
find_crtend_result = run_command('i686-elf-gcc', ['$CFLAGS', '-print-file-name=crtend.o'])
crtbegin_obj = find_crtbegin_result.stdout().strip()
crtend_obj = find_crtend_result.stdout().strip()
# I use nasm for my assembly files, so this is how to use it
# First of all we find nasm
nasm = find_program('nasm')
# We then define a generator that will make object files, if you need different command line
# options to the assembler, then make a new generator
asm_gen = generator(nasm,
output: '@[email protected]',
arguments: ['-felf32', '@INPUT@', '-o', '@OUTPUT@'])
# Generators actually take in a array of files to generate
bootstrapobjs = asm_gen.process(['boot.asm'])
# crti and crtn need to be handled by themselves, because the link order is important to them
crtiobj = asm_gen.process(['crti.asm'])
crtnobj = asm_gen.process(['crtn.asm'])
# note, I currently haven't worked out a way to specify the correct link order, i will do a bug report soon
# to try and figure out if there is a way to do this.
# To verify that the constructors actually worked, I edited the ninja build file
# generated by meson to make sure the constructors worked, but since my current kernel
# doesn't use any, I don't need the constructors to work.
linker_file = files('linker.ld')
# Because crtbegin and crtend are already generated object files, we specify them with the
# objects setting, but because of this, they are last on the command line, breaking global constructors
# We use our linker script by depending on it at link time, and specifying our linker args for our kernel
kernel_exe = executable('myos.elf', bootstrapobjs, crtiobj, 'kernel.cpp', crtnobj,
cpp_args: ['-O2', '-Wall', '-Wextra', '-fno-exceptions', '-fno-rtti', '-nostdlib'],
link_args: ['-O2', '-lgcc', '-nostdlib', '-T', '../linker.ld'],
link_depends: 'linker.ld',
objects: [crtbegin_obj, crtend_obj])
# currently it is easer having a seperate make file just for making the iso image, I am planing
# on finding a better way to do this
grub_file = files('grub.cfg')
iso_builder = find_program('build-iso.make')
iso_gen = custom_target('build-iso',
output: ['myos.iso'],
command: ['make', '-f', iso_builder],
build_by_default: true,
depends: [kernel_exe],
depend_files: 'grub.cfg')
cross_file.txt
Code: Select all
[binaries]
c = 'i686-elf-gcc'
cpp = 'i686-elf-g++'
ar = 'i686-elf-ar'
strip = 'i686-elf-strip'
[properties]
has_function_printf = false
c_args = ['-ffreestanding']
cpp_args = ['-ffreestanding']
c_link_args = ['-ffreestanding', '-nostdlib', '-lgcc']
cpp_link_args = ['-ffreestanding', '-nostdlib', '-lgcc']
[host_machine]
system = 'myos-kernel'
cpu_family = 'x86'
cpu = 'i686'
endian = 'little'
This build setup still needs more work and probably needs changing when I start using a libc.
Re: Setting up build environment with Meson
Posted: Mon Mar 08, 2021 10:29 pm
by kzinti
I've decided to give Meson a try and am running into the same problems as above. The googles are coming up dry.
Basically crtbegin.o and crtend.o are provided by GCC. I provide my own crti.o and crtn.o. libgcc is also linked in. All these happy campers need to be linked in the right order or things don't work.
With Makefile (my current build system), it roughly looks like this:
Code: Select all
CRTBEGIN = $(shell $(CC) $(CFLAGS) -print-file-name=crtbegin.o)
CRTEND = $(shell $(CC) $(CFLAGS) -print-file-name=crtend.o)
CRTI = $(filter %/crti.S.o, $(OBJECTS))
CRTN = $(filter %/crtn.S.o, $(OBJECTS))
OBJECTS2 = $(filter-out $(CRTI) $(CRTN), $(OBJECTS))
bootloader: $(OBJECTS) $(LDSCRIPT)
$(LD) $(LDFLAGS) $(CRTI) $(CRTBEGIN) $(OBJECTS2) -lgcc $(CRTEND) $(CRTN) -o $@
Now with Meson, I can't find a way to specify the link order for the CRTxxx files.
I see that the Meson project has a few open issues about supporting this, but it doesn't seem to be going anywhere.
Example:
https://github.com/mesonbuild/meson/issues/4467
Does anyone know a way to make this work? Otherwise it's probably going to mean getting back to the hell that is CMake.
My meson.build so far:
Code: Select all
crtbegin = run_command('i686-rainbow-elf-gcc', ['$CFLAGS', '-print-file-name=crtbegin.o']).stdout().strip()
crtend = run_command('i686-rainbow-elf-gcc', ['$CFLAGS', '-print-file-name=crtend.o']).stdout().strip()
libgcc = meson.get_compiler('cpp').find_library('libgcc')
link_deps = [
libgcc,
]
exe = executable('boot',
sources: sources,
include_directories: includes,
dependencies: link_deps,
objects: [crtbegin, crtend], # That's wrong! They end up after all the other objects!
)
Re: Setting up build environment with Meson
Posted: Mon Mar 08, 2021 11:12 pm
by kzinti
I suppose I don't really need crtbegin.o and crtend.o for my bootloader and kernel since I am using init arrays (and not __CTOR_LIST__). Still I am curious if there is a solution to the above that doesn't involve writing a wrapper script.
Re: Setting up build environment with Meson
Posted: Tue Mar 09, 2021 7:34 am
by thomtl
You can solve the link order problem in another way, basically instead of dumping everything into the .init and .fini sections you can have a .init_start and .init_end section where respectively crti and crtn go in, and then do thesame with .fini, then you can put these next to each other using a linker script.
To illustrate this is how i did it with my OS,
Code: Select all
.text : AT(ADDR(.text) - KERNEL_VBASE) {
*(.text*)
*(.init_start)
*(.init)
*(.init_end)
*(.fini_start)
*(.fini)
*(.fini_end)
}
And then crti.s being
Code: Select all
.section .init_start
.global _init
.type _init, @function
_init:
push %rbp
movq %rsp, %rbp
.section .fini_start
.global _fini
.type _fini, @function
_fini:
push %rbp
movq %rsp, %rbp
Re: Setting up build environment with Meson
Posted: Tue Mar 09, 2021 10:16 am
by kzinti
That works, thanks for the idea. I was really hoping for a workaround that involved changing the build scripts, but I am starting to think that I should not hold my breath.
Re: Setting up build environment with Meson
Posted: Tue Mar 09, 2021 4:19 pm
by Octocontrabass
Couldn't you adjust the linker script to order the sections according to which file they came from?
Something along these lines:
Code: Select all
.ctors :
{
/* gcc uses crtbegin.o to find the start of
the constructors, so we make sure it is
first. Because this is a wildcard, it
doesn't matter if the user does not
actually link against crtbegin.o; the
linker won't look for a file to match a
wildcard. The wildcard also means that it
doesn't matter which directory crtbegin.o
is in. */
KEEP (*crtbegin.o(.ctors))
KEEP (*crtbegin?.o(.ctors))
/* We don't want to include the .ctor section from
the crtend.o file until after the sorted ctors.
The .ctor section from the crtend file contains the
end of ctors marker and it must be last */
KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .ctors))
KEEP (*(SORT(.ctors.*)))
KEEP (*(.ctors))
}
.dtors :
{
KEEP (*crtbegin.o(.dtors))
KEEP (*crtbegin?.o(.dtors))
KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .dtors))
KEEP (*(SORT(.dtors.*)))
KEEP (*(.dtors))
}
Re: Setting up build environment with Meson
Posted: Tue Mar 09, 2021 5:32 pm
by kzinti
Octocontrabass wrote:Couldn't you adjust the linker script to order the sections according to which file they came from?
I didn't know you could do this, very cool.
Re: Setting up build environment with Meson
Posted: Tue Mar 09, 2021 5:36 pm
by kzinti
thomtl wrote:You can solve the link order problem in another way (...) then you can put these next to each other using a linker script.
Do you have an example of how to properly setup a linker script with Meson? I can't seem to figure it out.
Right now I do this (big ugly hard coded path and Meson doesn't understand that this is a file, meaning that changing the build directory location breaks things):
Code: Select all
project('boot',
['c', 'cpp'],
version : '1.0',
default_options : [
'cpp_link_args=-T ../../../../src/boot/machine/bios/bios.lds',
...
This should be doable as I can point Meson to the link script and it is recognized:
Code: Select all
link_script = files('machine/bios/bios.lds')
bootloader = executable('bootloader',
sources: sources,
include_directories: includes,
link_depends: link_script,
)
Basically, how do I get Meson to use "-T {link_script}" instead of having to hard code its location in "cpp_link_args"?
Re: Setting up build environment with Meson
Posted: Tue Mar 09, 2021 6:01 pm
by kzinti
Got it working, but I find it a bit unsatisfying:
Code: Select all
link_script = 'machine/bios/bios.lds'
bootloader = executable('bootloader',
sources: sources,
include_directories: includes,
link_args: [ '-T' + meson.source_root() / link_script ],
link_depends: link_script,
)
Re: Setting up build environment with Meson
Posted: Wed Mar 10, 2021 6:19 am
by thomtl
kzinti wrote:Basically, how do I get Meson to use "-T {link_script}" instead of having to hard code its location in "cpp_link_args"?
I don't think there is sadly, I do something similar but with "add_global_link_arguments", I did find
this article, but it also doesn't come with a real solution, just moving the problem to the cross file I guess.
Re: Setting up build environment with Meson
Posted: Wed Mar 10, 2021 11:15 am
by kzinti
I can now build my bootloader using Meson. I must say that was started as an exciting journey ended up with a pile of hacks and workarounds.
Meson looks very good to build user space, but when it comes to bare metal (or custom functionality), it doesn't seem to work so well. It also has strong opinion on how I should structure my project (i.e. the subproject functionality) and the behaviour of subdir is inconsistent.
I know CMake is more flexible and can do everything I want (and also has a ninja generator), but its scripting language irks me the wrong way. Maybe I'll give it a try as I haven't looked at it in some time.
I'm looking for a new build system because what I have right now is slow and not scalable. I'd like to stop having to maintain my own solution and use something that just works. I am not sure that I have found that yet.
Meson:
- No understanding of link scripts
- No way to specify link order
- compiler.find_library('libgcc') works, but there is no builtin-mechanism to find crtbegin.o / crtend.o
- using run_command(compiler, '$CFLAGS', '-print-file-name=crtbegin.o'): kinda works, but only uses the c_args of the project (and ignores any args added after the project declaration). I ended up having to put my args in an array, add them with add_project_arguments() and then re-using the array inside run_command(). Effectively bypassing Meson's support for C args entirely to make it work.
- Can't specify where I want my subprojects subdirs (I want them out of source), so I ended up having to use a symlink to access them from what Meson considers a legal path.
There is probably more that I forgot, but these were the main things leaving me unsatisfied.
Re: Setting up build environment with Meson
Posted: Wed Mar 10, 2021 1:35 pm
by kzinti
Right now I do this:
Code: Select all
extra_flags = []
if (machine == 'efi')
extra_flags += '-fpic'
if (arch == 'x86_64')
extra_flags += '-mno-red-zone'
endif
endif
add_project_arguments(extra_flags, language: ['c', 'cpp'])
cpp = meson.get_compiler('cpp')
libgcc = cpp.find_library('libgcc')
crtbegin = run_command(cpp, [extra_flags, '-print-file-name=crtbegin.o']).stdout().strip()
crtend = run_command(cpp, [extra_flags, '-print-file-name=crtend.o']).stdout().strip()
It turns all that all three files are in the same folder. So if there was some way to extract the path of "libgcc" from the "libgcc" variable, I could compose the two other paths and get rid of this extra_flags hack. But I can't seem to find a way.