I'd like to share my experience on bootstraping my hobby os with cmake & clang, which is, generally speaking, kind of unusual.
os info: i386-elf & linux abi
os repo link: https://github.com/neko-para/nyan
---
## Selecting c library for kernel
I was using llvm-libc and libc++, but now I've switched to musl and libc++. It's quite a disaster if there's differences between your kernel libc and user libc. I would have to carefully hide conflicting symbols and types (e.g. errnos, time_t).
If you still want to use llvm-libc, check out https://github.com/llvm/llvm-project/issues/140863. You'll need to apply some patchs to build llvm-libc for i386.
## Steps to setup toolchains
1. Build clang for i686-elf or i686-unknown-elf
This step is optional, but I recommend to do so, especially on MacOS. The apple clang is quite different. Clang from brew would be a choice. I've met a problem that clang on macos cannot choose the proper linker, thus it always introduces some macos stuffs. Here's the issue link https://github.com/llvm/llvm-project/issues/140683. I had to set CMAKE_CXX_LINK_EXECUTABLE to use lld directly.
2. Install musl headers
configure musl and run `make install-headers`
3. Build compiler-rt
compiler-rt would automatically enable x86_64 if you're targetting i386. But this would cause problem, as our compiler isn't configured for multilib (aka. not support -m32 -m64). `-DLLVM_DEFAULT_TARGET_TRIPLE=i686-unknown-linux -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON` would help.
Although there's an option to disable shared library, it still seems to generate two targets anyway. This would break Ninja, so if you met sth. like duplicate rules, try to use `Unix Makefiles`. This problem also applys to libc++.
Compiler-rt itself needs to be installed besides the compiler. You could also just put it somewhere and link it manually later..
4. Build full musl
Use LIBCC env to points to the compiler-rt library.
5. Build libc++
You'll have to disable all the features (those LIBCXX_ENABLE_XXX) to make it compile. `LIBCXX_HAS_MUSL_LIBC=ON` and `LIBCXX_CXX_ABI=none` is required here.
Libc++abi requires rtti to work.
6. Build libc++ for user
Enable necessary features and build. You may also want to build libc++abi and libunwind.
N. Actually, maybe we don't ever needs to build musl and libc++ for kernel. We just needs extract those headers, and we won't link to neither libc nor libc++. But I havn't try it.
## Linker script traps
Use `*(.text*)`, `*(.rodata*)`, etc. Clang would generate some sections like .text.abc, .rodata.foo. You'll need the wildcard to let lld put them into the right place.
## C++ STL features
STL requires a fatal routine for exception (__libcpp_verbose_abort) and some libc functions (memset, memcpy and so on).
__libcpp_verbose_abort is a printf-like function, but its formats only contain %s, %i and %d. Thus it is easy to roll it on your own, dispatching it into your kernel debug output.
Those libc functions from string.h are also quite straight-forward and easy to write.
After implementing your own memory allocation, you can implement new and delete operators. Then almost all of those containers and algorithms become available. Note that although you can write an allocator for your allocation methods instead, I'd recommend you to replace (fulfill) global new and delete. Some tools don't support custom allocator (e.g. std::function), and writing an allocator with correct typedefs is quite complex.
## Global constructor
You'll need __libc_init_array for those complex assignments outside functions. You may just search this inside musl and copy its implementation. Only four external symbols and one function. Those external symbols (e.g. __init_array_start) will be handled by linker, and you don't need to deal with crti and crtn. Just remember to call it right after your paging setup (if you're writting higher half kernel).
A write-up on building os written in c++ with cmake & clang
Re: A write-up on building os written in c++ with cmake & clang
## About cmake toolchain file
You can use `set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)` to bypass the compiler check from cmake.
About building libc++, you'll need to add _GNU_SOURCE definition into toolchain. Libc++ needs some macros that need _GNU_SOURCE or some similar feature macros to be defined.
You can use `set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY)` to bypass the compiler check from cmake.
About building libc++, you'll need to add _GNU_SOURCE definition into toolchain. Libc++ needs some macros that need _GNU_SOURCE or some similar feature macros to be defined.