What does your Makefile look like? (or, how to build kernel)

[tkahn6]

So I just got my kernel to boot for the first time.

'Cool!', I thought. 'Now I'll write a function to write a string to the screen!'

I did that, and I even imposed some structure in the way my files are organized.

Code: Select all

- src
   |
   + --- kernel
   |         |
   |         + --- kernel.c
   |
   + --- lib
   |       |
   |       + --- frontio.h
   |          |
   |           -- frontio.c

1) Does this look right? I'm not highly aware of the standards for source/file structure in kernel development
2) I'm using qemu to boot my OS.
- I followed http://wiki.osdev.org/Bare_bones#Booting_the_kernel
- Created loader.s, kernel.c, and linker.ld
- assembled, compiled, and linked
- ran cat stage1 stage2 pad kernel.bin > floppy.img
- ran qemu to boot floppy.img and loaded my OS from GRUB issuing it the commands kernel 200+18 and boot

It says in the tutorial that 'The file kernel.bin is now your kernel (all other files are no longer needed).' but I'm assuming that is just meant within the scope of the article - I have to re-link loader.o with linker.ld (along with the other files besides kernel.o) to get kernel.bin, correct?

So this is where a Makefile comes in I think. The Makefile would automate the process of compiling the source and linking it into kernel.bin (and bash scripting would take care of the rest).

Does this sound right? Do you even use a Makefile? How should I go about creating one. I read http://wiki.osdev.org/Makefile but to be honest, it was not very helpful.

Thanks!!

[manonthemoon]

1) Does this look right? I'm not highly aware of the standards for source/file structure in kernel development

It looks ok. I don't think there are "standards", but often there are separate folders for source and include files. For example, "src", "include", "src/lib/", "include/lib/". Typically, the "lib" folder is used for stuff like your own versions of string.h and printf/scanf stuff. But basically you can do whatever you want.

2) I have to re-link loader.o with linker.ld (along with the other files besides kernel.o) to get kernel.bin, correct?

Yes, correct. Link with all object files that you compile from your source files. (But technically you don't link with linker.ld. That's just a script)

Does this sound right? Do you even use a Makefile?

Yup, sounds right. I use a makefile. They are extremely helpful. Without one, you have to manually recompile and relink your files. Also, the make utility automatically checks the timestamps of your files and recompiles only what's necessary to update kernel.bin. It'll save you time when you have dozens of files but you only change one. Then only that one file gets recompiled and linked to a fresh kernel.bin.

I'll post my makefile in a few minutes so you can see what a good one looks like

[manonthemoon]

Here's a shortened version of my makefile. Note: you must use actual tabs (not four spaces), or else make gives an error message. (For my version, at least).

I added some explanatory comments:

Code: Select all

# CC is the name of your C compiler and CFLAGS are the options you wish to affect every file.
CC = gcc -I ./include
CFLAGS += -ansi -Wall -Wextra -W -pedantic -nostdlib -nostartfiles -D DEBUG
CFLAGS += -nodefaultlibs -fno-builtin -fno-exceptions -fno-rtti

# Here, I'm listing every file that I want to compile.
# Each file has a corresponding subsection below
KERNELO += bin/loader.o bin/main.o bin/kiostream.o bin/kio.o bin/cpu.o
KERNELO += bin/cputables.o bin/language.o bin/multiboot.o bin/memphys.o
KERNELO += bin/mutex.o bin/mempage.o

# This is the overall project.  At the shell, you would type "make osdev"
osdev:              kernel.bin boot.img

# I'm using GRUB for my bootloader.  That's what all that mcopy stuff is for,
# putting the files onto a floppy image for Bochs.
# Do whatever you need to do for your bootloader and emulator
boot.img: kernel.bin idbg.bin bootfloppy.img menu.cfg ksymbols
                    cp bootfloppy.img $@
                    chmod +w $@
                    mcopy -o -i $@ kernel.bin ::/boot
                    mcopy -o -i $@ menu.cfg ::/boot
                    mcopy -o -i $@ ksymbols ::/boot

# Here we go.  Notice the use of KERNELO, as defined above, instead of manually listing every object
# file in different places. Also, I use objdump and the -M option of ld to get symbol tables.
# Don't copy those parts if you don't want them, it just makes large, mostly useless files that I need
# for my debugger.  I commented out those lines.

kernel.bin: linker.ld $(KERNELO)
#                   ld -M -T linker.ld -o $@ $(KERNELO) > kernel.map
                    ld -T linker.ld -o $@ $(KERNELO)
#                   objdump kernel.bin -x -C > kernel.dmp
#                   objdump kernel.bin -d >> kernel.dmp
                    rm -f idbg.bin bin/idbg.o

# an assembly code file
bin/loader.o: src/loader.asm
                    nasm -f elf $< -o $@

# a source code file.  You need one of these for each file in your project
bin/main.o: src/main.cpp
                    $(CC) $(CFLAGS) -o $@ -c $<

bin/kio.o: src/kio.cpp
                    $(CC) $(CFLAGS) -o $@ -c $<

# .... it goes on and on, but each entry is the same except the file names
# I won't include my entire makefile because it gets repetitive and this is already getting too long!


# There is no such file as "cleanosdev" but if you type "make cleanosdev" it will
# delete the intermediate object files.
.PHONY: cleanosdev

cleanosdev:
                    rm -f bin/*.o
                    rm -f *.bin

Hope that helps.

[tkahn6]

Thank you for your help

[Brendan]

Hi,

Do you even use a Makefile?

I do use a small makefile for each little utility I write, but these utilities are to help me manage the OS (and web site) and aren't considered part of the OS itself.

For anything important (anything that will become part of the OS) I write it in assembly, where the first file is always called "0index.asm" and includes all the other files that make up the binary. I've got a utility that searches my project directory looking sub-directories that have a file called "0index.asm" (which indicates a set of source files that create a binary). It parses these "0index.asm" files to finds all dependencies (all "%include" and "incbin" statements), then works out which dependencies have been changed and what order the binaries need to be assembled in. Basically for the majority of the OS I never need to touch scripts or makefiles - with one "./build" command it all just works.

Note: This same build utility is also responsible for converting the assembly source file into the cross-linked HTML pages you see on my web site and generating the other web pages (and is written as a multi-threaded process to speed things up).

However, it's not quite that simple. There's one script that runs make for the utilities, then runs my "./build" command. If the "./build" command needed to assemble anything it runs a second script; and this second script does things like building a FAT/GRUB boot floppy image, copying files into my TFTP directory, copying files into my "/boot" directory, and running my utilities to create native boot floppy images and CD-ROM images.

I use a keyboard shortcut in KDE to run the first script - I press F12 and the web site and everything else is updated (and I can boot test machines from network, etc). To rebuild everything it currently takes about 6 seconds, but it's extremely rare for everything to change at once, and usually it takes less than half a second.


Cheers,

Brendan

[tkahn6]

@Brendan

Thats really cool. I was considering writing a bash script to parse my files and do that exact thing (xcept they would be <include>s not %includes).
It's also really cool that you've got it all hooked up to F12. I'm using gedit and I have my build/load qemu routine hooked up to F8

@manonthemoon

Thank you so much for your help. It's not often someone really takes the time to help you understand something new. With a little tweaking I got it to work perfectly. Your comments were really helpful.

Can you explain how you construct your bootloader floppy image file? I notice its done differently than mine and your way looks better.

Thanks!

[manonthemoon]

You're welcome, no problem!

As far as the boot floppy goes, I can't take credit. I'm not sure where I saw it, but I think I got it from some tutorial somewhere.

Here's how it works

1. Copy "bootfloppy.img" to "boot.img". The source file is like a blank template that comes with GRUB, and the destination is the actual boot image.
2. Set write privileges (that might not be necessary)
3. Use the mtools to put kernel.bin, menu.cfg (for GRUB, check the docs), and any other files that you want.

mtools includes mcopy, a utility that is probably already installed on your computer. If not, it shouldn't be hard to find. Its purpose is to copy files between Unix-style and DOS-style file systems. It works with disk images, obviously.

[yemista]

You can also do it by hand if you are on linux.

Code: Select all

 
dd bs=512 count=1 if=boot.o of=$(OUTPUT)
dd bs=512 count=120 seek=1 if=kernel.img of=$(OUTPUT)

Where output is either a mounted floppy, or the name of your floppy image file that you will use for an emulator

[tkahn6]

@yemista

That's how I was doing it before. Now I just cat > bootfloppy.img

@manonthemoon

When GRUB comes up after you power on bochs, do you have to type kernel 200+18 (or something similar) and then boot?

[alethiophile]

The way I do it is, first I use a makefile to build the actual kernel.bin (not hard, just google 'how to write a makefile'), I keep a single disk image set up as a loop device, then I run a script that mounts it, copies relevant files and unmounts it. (This need be run as root, but that's fixable.)

[manonthemoon]

When GRUB comes up after you power on bochs, do you have to type kernel 200+18 (or something similar) and then boot?

I use "kernel /boot/kernel.bin".

[Solar]

I read http://wiki.osdev.org/Makefile but to be honest, it was not very helpful.

...and if you would elaborate on why you didn't consider it helpful, we might explain, and/or improve the tutorial...

[stealther]

I read http://wiki.osdev.org/Makefile but to be honest, it was not very helpful.

As for me it was very helpful, but it's describing an advanced technique.
Try making some basic makefiles to understand the basics.
http://www.opussoftware.com/tutorial/TutMakefile.htm
http://www.cs.umd.edu/class/spring2002/ ... efile.html
http://mrbook.org/blog/tutorials/make/
This is my makefile at this moment:
(It's not well-tested anyway)

Code: Select all

TARGET	= 32

ECHO	= @echo
LD		= @ld
CC		= @gcc
AS		= @nasm
EM		= @qemu-system-x86_64
RM		= @rm

DEST	= build/
IMAGE	= $(DEST)floppy.img

#---------------------------------------------------------
# Platform
#---------------------------------------------------------
ifeq ($(TARGET),32)
ASFLAGS		= -felf
CCFLAGS		= -m32
LDFLAGS		= -melf_i386
endif

ifeq ($(TARGET),64)
ASFLAGS		= -felf64
CCFLAGS		= -m64
LDFLAGS		= -melf_x86_64
endif

#---------------------------------------------------------
# Debug
#---------------------------------------------------------
ifdef DEBUG
CCFLAGS		+= -g -D DEBUG
endif

#---------------------------------------------------------
# General
#---------------------------------------------------------
LDFLAGS		+= -nodefaultlibs --strip-all
CCFLAGS		+= -ffreestanding -nostartfiles -nostdinc -nodefaultlibs -fno-builtin -fno-exceptions -fno-rtti
EMFLAGS		+= -m 128 -smp 4 -boot a
#---------------------------------------------------------
# Warnings
#---------------------------------------------------------
CCFLAGS		+= -pedantic -Wall -Wextra -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings \
			   -Wmissing-declarations -Wredundant-decls -Winline -Wno-long-long -Wconversion

#---------------------------------------------------------
# File enums
#---------------------------------------------------------
INCLDIRS = include

AUXFILES = Makefile Readme.txt menu.lst grubfd.conf linker.ld
GRUBFILES = grub/stage1 grub/stage2 grub/fat_stage1_5

ASMFILES = $(patsubst ./%, %, $(shell find -name "*.asm"))
CPPFILES = $(patsubst ./%, %, $(shell find -name "*.cpp"))
HDRFILES = $(patsubst ./%, %, $(shell find -name "*.h"))
ALLFILES = $(ASMFILES) $(CPPFILES) $(HDRFILES) $(AUXFILES)

#OUTFILES = $(patsubst %.asm$(TARGET),$(DEST)$(TARGET)/%.out,$(ASMFILES))
OBJFILES = $(patsubst %.cpp,$(DEST)$(TARGET)/%.obj,$(CPPFILES))
DEPFILES = $(patsubst %.cpp,$(DEST)$(TARGET)/%.d,$(CPPFILES))

TEMPDIRS = $(sort $(dir $(OUTFILES) $(OBJFILES) $(DEPFILES)))
LINKFILES = $(DEST)$(TARGET)/loader.out $(OBJFILES)
#---------------------------------------------------------
# Rules
#---------------------------------------------------------
.PHONY: all kernel clean tar todo floppy run

all:
	$(MAKE) kernel TARGET=32
	$(MAKE) kernel TARGET=64

structure:
	@for dir in $(TEMPDIRS); do mkdir -p $$dir; done; true

kernel: structure $(LINKFILES)
	$(ECHO) "Linking..."
	$(LD) $(LDFLAGS) -M -T linker.ld -o $(DEST)kernel$(TARGET).bin $(LINKFILES) > $(DEST)kernel$(TARGET).map

$(DEST)$(TARGET)/%.obj: %.cpp
	$(ECHO) "Compiling $<"
	$(CC) $(CCFLAGS) -I $(INCLDIRS) -MMD -MP -MT $*.d -o $@ -c $<

$(DEST)$(TARGET)/loader.out: loader/loader$(TARGET).asm
	$(ECHO) "Assembling $<"
	$(AS) $(ASFLAGS) -o $@ $<

#---------------------------------------------------------
# Floppy
#---------------------------------------------------------
floppy: $(IMAGE)

$(IMAGE): $(GRUBFILES) $(AUXFILES) all
	@echo Creating floppy image...
	@dd if=/dev/zero of=$(IMAGE) bs=512 count=2880 > /dev/null
	@mkfs -V -t msdos $(IMAGE) > /dev/null
	@echo Installing GRUB...
	@mmd -o -i $(IMAGE) ::/boot
	@mcopy -s -o -i $(IMAGE) grub ::/boot
	@grep -v ^# grubfd.conf | grub --batch > /dev/null
	@echo Installing TurtlOS kernel...
	@mcopy -o -i $(IMAGE) menu.lst ::/boot/grub
	@mcopy -o -i $(IMAGE) $(DEST)kernel32.bin ::/
	@mcopy -o -i $(IMAGE) $(DEST)kernel64.bin ::/

#---------------------------------------------------------
# Utilities
#---------------------------------------------------------
clean:
	-$(RM) -rf $(DEST)
tar:
	-@tar -cjf TurtlOS.tar.bz2 $(ALLFILES) $(GRUBFILES)
todo:
	-@for file in $(ALLFILES); do fgrep -H -e TODO -e FIXME $$file; done; true
run: $(IMAGE)
	$(EM) $(EMFLAGS) -fda $(IMAGE)

-include $(DEPFILES)

Any comments or suggestions are really appreciated!

[AndreaOrru]

Aren't you linking all the library to your kernel, in that way?

[Solar]

I read http://wiki.osdev.org/Makefile but to be honest, it was not very helpful.

As for me it was very helpful, but it's describing an advanced technique.

Correct; personally, I consider Makefiles that require you to add new source / header files / dependencies manually, or that rely on recursive make invocations for subdirectories, to be buggy (i.e., incorrect). It's like buggy C code: It might do what it was intended to do, but the problems are there, waiting to bite you.