Setting up build environment with Meson

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
treyzania
Posts: 2
Joined: Wed Jun 21, 2017 9:28 pm
Libera.chat IRC: treyzania
Location: Boston, MA
Contact:

Setting up build environment with Meson

Post by treyzania »

Greetings!

This is my second swing at trying OS development. The first time wasn't much more than a Hello World, so this time I want to try to do it right. I'm trying to set up Meson (http://mesonbuild.com/) as the build system for my kernel and am looking for some suggestions.

Here's how I'd like to structure the project:

Code: Select all

/
|--- arch/
|  |--- amd64/
|  |  |--- boot.S
|  |  |--- syscall.S
|  |  |--- init.c
|  |  \--- [etc...]
|  |--- arm/
|  \--- [etc...]
|--- kernel/
|  |--- mm/
|  |--- mod/
|  |--- fs/
|  \--- [etc...]
\--- [etc...]
What I've tried to set up so far is having a main meson.build script in the root of the project and another in the root of each arch that handles building object data for each of those components (that gets included with `subdir` or `subproject` depending on which arch you're building for). But what I'm not sure about is how to link all of the different components into a single ELF binary that I can give to GRUB, and how to actually package a GRUB ISO that I can boot off. I'd like to do all of this with Meson, but it seems like its cross-compilation and special linking behavior support seems to be a little bit lacking, or at least have lacking documentation.

Does anybody have any tips for dealing with this in Meson?

Thanks!
User avatar
iansjack
Member
Member
Posts: 4703
Joined: Sat Mar 31, 2012 3:07 am
Location: Chichester, UK

Re: Setting up build environment with Meson

Post by iansjack »

I'm not familiar with Meson but if, as you say, it lacks support for cross-compiler and custom linking then I'd suggest it is not a good environment for OS development. Is there a reason that you are not happy to use "make"?
treyzania
Posts: 2
Joined: Wed Jun 21, 2017 9:28 pm
Libera.chat IRC: treyzania
Location: Boston, MA
Contact:

Re: Setting up build environment with Meson

Post by treyzania »

It does have support for custom link settings (as listed here: http://mesonbuild.com/Reference-manual. ... _arguments) and cross compilation, but it seems like the documentation for builds with settings spread out in separate chunks is rather lacking. I was wondering if anybody had any experience with it.
rupertsteel
Posts: 1
Joined: Sat May 27, 2017 11:56 pm
Libera.chat IRC: rupertsteel

Re: Setting up build environment with Meson

Post by rupertsteel »

I have gotten meson working for a very simple os, but I do have several problems with my system.

My meson build file currently cannot do the correct link order for c++ global constructors and it cannot build a iso image in the meson build file.

Here is my current tree

Code: Select all

|-- meson-build/
| \-- [build output and meson generated files]
|-- boot.asm
|-- build-iso.make
|-- cross_file.txt
|-- crti.asm
|-- crto.asm
|-- grub.cfg
|-- kernel.cpp
|-- linker.ld
\-- meson.build
meson.build

Code: Select all

# Specify the project
project('myos', 'c', 'cpp')

# Find where the crtbegin and crtend object files are so we can link to them
find_crtbegin_result = run_command('i686-elf-gcc', ['$CFLAGS', '-print-file-name=crtbegin.o'])
find_crtend_result = run_command('i686-elf-gcc', ['$CFLAGS', '-print-file-name=crtend.o'])
crtbegin_obj = find_crtbegin_result.stdout().strip()
crtend_obj = find_crtend_result.stdout().strip()

# I use nasm for my assembly files, so this is how to use it

# First of all we find nasm
nasm = find_program('nasm')

# We then define a generator that will make object files, if you need different command line
# options to the assembler, then make a new generator
asm_gen = generator(nasm,
    output: '@[email protected]',
    arguments: ['-felf32', '@INPUT@', '-o', '@OUTPUT@'])

# Generators actually take in a array of files to generate
bootstrapobjs = asm_gen.process(['boot.asm'])

# crti and crtn need to be handled by themselves, because the link order is important to them
crtiobj = asm_gen.process(['crti.asm'])
crtnobj = asm_gen.process(['crtn.asm'])

# note, I currently haven't worked out a way to specify the correct link order, i will do a bug report soon
# to try and figure out if there is a way to do this.
# To verify that the constructors actually worked, I edited the ninja build file
# generated by meson to make sure the constructors worked, but since my current kernel
# doesn't use any, I don't need the constructors to work.

linker_file = files('linker.ld')

# Because crtbegin and crtend are already generated object files, we specify them with the
# objects setting, but because of this, they are last on the command line, breaking global constructors
# We use our linker script by depending on it at link time, and specifying our linker args for our kernel
kernel_exe = executable('myos.elf', bootstrapobjs, crtiobj, 'kernel.cpp', crtnobj,
    cpp_args: ['-O2', '-Wall', '-Wextra', '-fno-exceptions', '-fno-rtti', '-nostdlib'],
    link_args: ['-O2', '-lgcc', '-nostdlib', '-T', '../linker.ld'],
    link_depends: 'linker.ld',
    objects: [crtbegin_obj, crtend_obj])

# currently it is easer having a seperate make file just for making the iso image, I am planing
# on finding a better way to do this	

grub_file = files('grub.cfg')
iso_builder = find_program('build-iso.make')
iso_gen = custom_target('build-iso',
    output: ['myos.iso'],
    command: ['make', '-f', iso_builder],
    build_by_default: true,
    depends: [kernel_exe],
    depend_files: 'grub.cfg')
cross_file.txt

Code: Select all

[binaries]
c = 'i686-elf-gcc'
cpp = 'i686-elf-g++'
ar = 'i686-elf-ar'
strip = 'i686-elf-strip'

[properties]
has_function_printf = false

c_args = ['-ffreestanding']
cpp_args = ['-ffreestanding']
c_link_args = ['-ffreestanding', '-nostdlib', '-lgcc']
cpp_link_args = ['-ffreestanding', '-nostdlib', '-lgcc']

[host_machine]
system = 'myos-kernel'
cpu_family = 'x86'
cpu = 'i686'
endian = 'little'
This build setup still needs more work and probably needs changing when I start using a libc.
kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

I've decided to give Meson a try and am running into the same problems as above. The googles are coming up dry.

Basically crtbegin.o and crtend.o are provided by GCC. I provide my own crti.o and crtn.o. libgcc is also linked in. All these happy campers need to be linked in the right order or things don't work.

With Makefile (my current build system), it roughly looks like this:

Code: Select all

CRTBEGIN = $(shell $(CC) $(CFLAGS) -print-file-name=crtbegin.o)
CRTEND   = $(shell $(CC) $(CFLAGS) -print-file-name=crtend.o)
CRTI = $(filter %/crti.S.o, $(OBJECTS))
CRTN = $(filter %/crtn.S.o, $(OBJECTS))
OBJECTS2 = $(filter-out $(CRTI) $(CRTN), $(OBJECTS))

bootloader: $(OBJECTS) $(LDSCRIPT)
	$(LD) $(LDFLAGS) $(CRTI) $(CRTBEGIN) $(OBJECTS2) -lgcc $(CRTEND) $(CRTN) -o $@
Now with Meson, I can't find a way to specify the link order for the CRTxxx files.

I see that the Meson project has a few open issues about supporting this, but it doesn't seem to be going anywhere.
Example: https://github.com/mesonbuild/meson/issues/4467

Does anyone know a way to make this work? Otherwise it's probably going to mean getting back to the hell that is CMake.

My meson.build so far:

Code: Select all

crtbegin = run_command('i686-rainbow-elf-gcc', ['$CFLAGS', '-print-file-name=crtbegin.o']).stdout().strip()
crtend = run_command('i686-rainbow-elf-gcc', ['$CFLAGS', '-print-file-name=crtend.o']).stdout().strip()
libgcc = meson.get_compiler('cpp').find_library('libgcc')

link_deps = [
    libgcc,
]

exe = executable('boot',
    sources: sources,
    include_directories: includes,
    dependencies: link_deps,
    objects: [crtbegin, crtend],     # That's wrong! They end up after all the other objects!
)

kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

I suppose I don't really need crtbegin.o and crtend.o for my bootloader and kernel since I am using init arrays (and not __CTOR_LIST__). Still I am curious if there is a solution to the above that doesn't involve writing a wrapper script.
User avatar
thomtl
Member
Member
Posts: 66
Joined: Mon Sep 03, 2018 2:25 am

Re: Setting up build environment with Meson

Post by thomtl »

You can solve the link order problem in another way, basically instead of dumping everything into the .init and .fini sections you can have a .init_start and .init_end section where respectively crti and crtn go in, and then do thesame with .fini, then you can put these next to each other using a linker script.
To illustrate this is how i did it with my OS,

Code: Select all

.text : AT(ADDR(.text) - KERNEL_VBASE) {
		*(.text*)

		*(.init_start)
		*(.init)
		*(.init_end)

		*(.fini_start)
		*(.fini)
		*(.fini_end)
	}
And then crti.s being

Code: Select all

.section .init_start
.global _init
.type _init, @function
_init:
    push %rbp
    movq %rsp, %rbp

.section .fini_start
.global _fini
.type _fini, @function
_fini:
    push %rbp
    movq %rsp, %rbp
kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

That works, thanks for the idea. I was really hoping for a workaround that involved changing the build scripts, but I am starting to think that I should not hold my breath.
Octocontrabass
Member
Member
Posts: 5568
Joined: Mon Mar 25, 2013 7:01 pm

Re: Setting up build environment with Meson

Post by Octocontrabass »

Couldn't you adjust the linker script to order the sections according to which file they came from?

Something along these lines:

Code: Select all

  .ctors          :
  {
    /* gcc uses crtbegin.o to find the start of
       the constructors, so we make sure it is
       first.  Because this is a wildcard, it
       doesn't matter if the user does not
       actually link against crtbegin.o; the
       linker won't look for a file to match a
       wildcard.  The wildcard also means that it
       doesn't matter which directory crtbegin.o
       is in.  */
    KEEP (*crtbegin.o(.ctors))
    KEEP (*crtbegin?.o(.ctors))
    /* We don't want to include the .ctor section from
       the crtend.o file until after the sorted ctors.
       The .ctor section from the crtend file contains the
       end of ctors marker and it must be last */
    KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .ctors))
    KEEP (*(SORT(.ctors.*)))
    KEEP (*(.ctors))
  }
  .dtors          :
  {
    KEEP (*crtbegin.o(.dtors))
    KEEP (*crtbegin?.o(.dtors))
    KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .dtors))
    KEEP (*(SORT(.dtors.*)))
    KEEP (*(.dtors))
  }
kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

Octocontrabass wrote:Couldn't you adjust the linker script to order the sections according to which file they came from?
I didn't know you could do this, very cool.
Last edited by kzinti on Tue Mar 09, 2021 5:37 pm, edited 1 time in total.
kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

thomtl wrote:You can solve the link order problem in another way (...) then you can put these next to each other using a linker script.
Do you have an example of how to properly setup a linker script with Meson? I can't seem to figure it out.

Right now I do this (big ugly hard coded path and Meson doesn't understand that this is a file, meaning that changing the build directory location breaks things):

Code: Select all

project('boot',
  ['c', 'cpp'],
  version : '1.0',
  default_options : [
    'cpp_link_args=-T ../../../../src/boot/machine/bios/bios.lds',
...
This should be doable as I can point Meson to the link script and it is recognized:

Code: Select all

link_script = files('machine/bios/bios.lds')

bootloader = executable('bootloader',
    sources: sources,
    include_directories: includes,
    link_depends: link_script,
)
Basically, how do I get Meson to use "-T {link_script}" instead of having to hard code its location in "cpp_link_args"?
kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

Got it working, but I find it a bit unsatisfying:

Code: Select all

link_script = 'machine/bios/bios.lds'

bootloader = executable('bootloader',
    sources: sources,
    include_directories: includes,
    link_args: [ '-T' + meson.source_root() / link_script ],
    link_depends: link_script,
)
User avatar
thomtl
Member
Member
Posts: 66
Joined: Mon Sep 03, 2018 2:25 am

Re: Setting up build environment with Meson

Post by thomtl »

kzinti wrote:Basically, how do I get Meson to use "-T {link_script}" instead of having to hard code its location in "cpp_link_args"?
I don't think there is sadly, I do something similar but with "add_global_link_arguments", I did find this article, but it also doesn't come with a real solution, just moving the problem to the cross file I guess.
kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

I can now build my bootloader using Meson. I must say that was started as an exciting journey ended up with a pile of hacks and workarounds.

Meson looks very good to build user space, but when it comes to bare metal (or custom functionality), it doesn't seem to work so well. It also has strong opinion on how I should structure my project (i.e. the subproject functionality) and the behaviour of subdir is inconsistent.

I know CMake is more flexible and can do everything I want (and also has a ninja generator), but its scripting language irks me the wrong way. Maybe I'll give it a try as I haven't looked at it in some time.

I'm looking for a new build system because what I have right now is slow and not scalable. I'd like to stop having to maintain my own solution and use something that just works. I am not sure that I have found that yet.

Meson:
- No understanding of link scripts
- No way to specify link order
- compiler.find_library('libgcc') works, but there is no builtin-mechanism to find crtbegin.o / crtend.o
- using run_command(compiler, '$CFLAGS', '-print-file-name=crtbegin.o'): kinda works, but only uses the c_args of the project (and ignores any args added after the project declaration). I ended up having to put my args in an array, add them with add_project_arguments() and then re-using the array inside run_command(). Effectively bypassing Meson's support for C args entirely to make it work.
- Can't specify where I want my subprojects subdirs (I want them out of source), so I ended up having to use a symlink to access them from what Meson considers a legal path.

There is probably more that I forgot, but these were the main things leaving me unsatisfied.
Last edited by kzinti on Wed Mar 10, 2021 8:21 pm, edited 1 time in total.
kzinti
Member
Member
Posts: 898
Joined: Mon Feb 02, 2015 7:11 pm

Re: Setting up build environment with Meson

Post by kzinti »

Right now I do this:

Code: Select all

extra_flags = []

if (machine == 'efi')
    extra_flags += '-fpic'
    if (arch == 'x86_64')
        extra_flags += '-mno-red-zone'
    endif
endif

add_project_arguments(extra_flags, language: ['c', 'cpp'])

cpp = meson.get_compiler('cpp')
libgcc = cpp.find_library('libgcc')
crtbegin = run_command(cpp, [extra_flags, '-print-file-name=crtbegin.o']).stdout().strip()
crtend = run_command(cpp, [extra_flags, '-print-file-name=crtend.o']).stdout().strip()
It turns all that all three files are in the same folder. So if there was some way to extract the path of "libgcc" from the "libgcc" variable, I could compose the two other paths and get rid of this extra_flags hack. But I can't seem to find a way.
Post Reply