The little book about OS development

[willedwards]

A really nice beginners course:

http://littleosbook.github.io

[Octocontrabass]

The tutorial says nothing about a cross-compiler. You need a cross-compiler for OS development.

I haven't read through all of it, but I wouldn't be surprised if there are other mistakes.

[ExeTwezz]

Cool design.
Now in bookmarks.

[Combuster]

Look, it doesn't assemble!

Code: Select all

out 0x3D4, 14

[Icee]

Look, it doesn't assemble!

Not just that. The linker script (first one) is wrong, struct gdt is wrong, and that is just what immediately draws attention. Overall, the little book seems to be written by people who have done little to no testing of their code snippets.

UPD: oh, the IDT loading code is wrong, too.

[willedwards]

I posted this thinking it a good, useful resource for everyone especially beginners.

I'm a bit taken aback by how quick people are to dismiss it and think they've found some critical fault. I didn't post this so the immature could feel superior!

Anyone who has found a genuine fault with the guide can submit issues or, better yet, pull-request corrections at https://github.com/littleosbook/littleosbook

The source-code for the actual OS is available online https://github.com/helino/aenix and it does - obviously - build ok.

[Icee]

I posted this thinking it a good, useful resource for everyone especially beginners.

Posting a beginners guide that contains errors that are hard for beginners to identify and debug does more harm than good.

Anyone who has found a genuine fault with the guide can submit issues or, better yet, pull-request corrections at https://github.com/littleosbook/littleosbook

Could you please define what a genuine fault is so that I don't bother you with my immaturity anymore?

The source-code for the actual OS is available online https://github.com/helino/aenix and it does - obviously - build ok.

Which is a good thing. But irrelevant to the guide in question because the code excerpts in the two locations don't match.

[helino]

Hi guys,

one of the authors here. First of all, huge thanks to everyone in this community! Without the wiki and these forums, I would never have gotten into OS development. So why did we write this book if the wiki here and the forums already contains everything you need? Our goal was to try to create an easy to follow guide that also contained a lot of references if the reader wanted to dig deeper.

The tutorial says nothing about a cross-compiler. You need a cross-compiler for OS development.
I haven't read through all of it, but I wouldn't be surprised if there are other mistakes.

There are definitely other mistakes in the book! I have created an issue for adding information about using a cross-compiler: https://github.com/littleosbook/littleosbook/issues/12. Just a question though, since the OS created in the guide is for x86, and the user is most likely using an x86-64 CPU, isn't passing -m32 to gcc enough (as well as other flags to not use standard headers, libc etc)?

Look, it doesn't assemble!

Not just that. The linker script (first one) is wrong,

An issue has been filed, see https://github.com/littleosbook/littleosbook/issues/10

struct gdt is wrong

Issue filed: https://github.com/littleosbook/littleosbook/issues/13

and that is just what immediately draws attention. Overall, the little book seems to be written by people who have done little to no testing of their code snippets.

Hmm, yes, we should definetely try add tests for the snippets in the book to avoid (embarassing) issues like these. Filed an issue for this as well:
https://github.com/littleosbook/littleosbook/issues/14

UPD: oh, the IDT loading code is wrong, too.

Filed https://github.com/littleosbook/littleosbook/issues/15. Thanks for all the feedback, much appreciated!

I posted this thinking it a good, useful resource for everyone especially beginners.

Posting a beginners guide that contains errors that are hard for beginners to identify and debug does more harm than good.

Agree, but thanks to feedback from everyone in here (as well as from reddit and github), this can hopefully become a correct beginner friendly guide. The other alternative would of course have been to not publish this at all, hopefully people can get some value out of it (even in its current state).

Anyone who has found a genuine fault with the guide can submit issues or, better yet, pull-request corrections at https://github.com/littleosbook/littleosbook

Could you please define what a genuine fault is so that I don't bother you with my immaturity anymore?

I don't think your replies are immature, again, the feedback is much appreciated! Feel free to post any errors in this thread, or open an issue on github.

The source-code for the actual OS is available online https://github.com/helino/aenix and it does - obviously - build ok.

Which is a good thing. But irrelevant to the guide in question because the code excerpts in the two locations don't match.

Agreed, this needs to be fixed.

Again, thanks to everyone that read through parts of this text! I'm grateful for any comments, issues etc.

[Combuster]

I'm a bit taken aback by how quick people are to dismiss it and think they've found some critical fault. I didn't post this so the immature could feel superior!

That depends on what is considered a critical fault. It starts with "ubuntu" while basically it requires a specifically 64-bit linux (i.e. 32-bit ubuntu would break as well). It also seems to suggest flat binaries for applications and relies on undefined behaviour to start them where they are supposed to be started. If you add to that the syntax errors present in the document, it quickly comes to the point of another schoolkid having done something and next thing we know is that we have to support yet another broken tutorial and likely remind them of the "Is there a tutorial on" beginner mistake.

It might not have fallen to the rock bottom of disastrous tutorials at first glance - and considering the number of noob errors not present it's unlikely it will actually hit rock bottom - but time is certain to make more bugs known, and it's merely a matter of time until history repeats itself and people show up here being the only place for discussions on the matter, and we have to unlearn things from someone. Simply advertising it as "really nice" will obviously result in replies "no it isn't, please get it away before someone actually believes it".

------

And while I was just adding a rant on how mr. The Mill probably wasn't the original author, I see that the person in question decided to show up after not updating it since 2012.

[sortie]

There are definitely other mistakes in the book! I have created an issue for adding information about using a cross-compiler: https://github.com/littleosbook/littleosbook/issues/12. Just a question though, since the OS created in the guide is for x86, and the user is most likely using an x86-64 CPU, isn't passing -m32 to gcc enough (as well as other flags to not use standard headers, libc etc)?

No, it's not enough. The root issue is that the gcc that comes with your Linux is a compiler for Linux, and your OS is not Linux. It's that simple. Any issues that arise and you have to work around comes from this fact. You're lying to the compiler, rather than working with it.

Code: Select all

-m32 -nostdlib -nostdinc -fno-builtin -fno-stack-protector -nostartfile -nodefaultlibs

This list of compiler options contains harmful options for osdev. -m32 should not be passed, that's implied when you use a proper cross-compiler. -nostdlib is a link time option (should not be passed when just compiling) (and implies -nostartfiles and -nodefaultlibs - the fact you're passing all three means you don't know what the option does.) -fno-builtin is implied by -ffreestanding, which you must pass, but don't. -fno-stack-protector is to work around that some distro compilers are hardened, which is good, the better solution is to implement the stack smashing protector, although it's not enabled by default with a cross-compiler.

It's fine you're writing a book, but you really should be responsible for keeping your resources up to date with current osdev wiki recommendations, which evolve over time. We really don't need another supposedly nice, but not community edited and full of pitfalls resource. You should compile like bare bones does.

I suggest you try to embrace serious osdev, like how someone very experienced would do it, rather than trying simplify it down for beginners. There's value in teaching people how to do things in a way that lasts, is correct, and will serve them in the future. For instance, stuff like flat binaries is cute, but isn't good enough in any way. Teach them how a proper OS would do it.

[Combuster]

There are definitely other mistakes in the book! I have created an issue for adding information about using a cross-compiler: https://github.com/littleosbook/littleosbook/issues/12. Just a question though, since the OS created in the guide is for x86, and the user is most likely using an x86-64 CPU, isn't passing -m32 to gcc enough (as well as other flags to not use standard headers, libc etc)?

Many distro's these days have patched GCCs - you'll already see that happen with the necessity of -fno-stack-protector which isn't needed on a vanilla gcc for linux, nor the cross-compiler. As far as the documentation goes, the ubuntu compiler also hard-depends on a patched version of glibc and will potentially break again the moment you start adding libc-named functions. You would also need to forcibly disable PIC to support the default hardened compiler settings on some non-ubuntu linuxes.

Bottom line, the trend is to bundle distros with a gcc that contains more and more host-dependent patches that will break OS development code.

not passing the stack protector flag would be safer and put people face-first in the linker error message, hopefully making them use the proper toolchain instead of falling into the second line of Ubuntu pranks.

[iansjack]

@helino - The very best of luck with this project, and it's very mature of you to take somewhat harsh criticism so well. I'm sure it will take a little while, and a lot of feedback, to come up with a polished (I was going to say "finished", but I'm not sure you will ever finish) product.

It's easy to criticize, particularly if we can't be bothered to take the time to produce such a guide ourselves and I have ever admiration for you for making the attempt. If we were all willing, and capable, to do this sort of work you wouldn't need to!

[Chandra]

Teach them how a proper OS would do it.

In fact, there actually is no such thing as a proper way to write an OS.

The point is, everybody starts small. Software development is an iterative process, unless you like to stick with a model that will take you straight to a pit. If I'm not wrong, almost all software development projects need major re-write at some point in the future. Take for instance Grub2, written totally from the scratch. A fair user would find nothing wrong with Grub legacy, but the developers did, probably tired of adding patches to incorporate new features. Same applies to OS Dev. You start it small. You do all the wrong things and end up with a wrong OS. Eventually, you realize somewhat correct way to do certain things and if you have enough energy and motivation left within you to rewrite everything, you'll do it fairly well. This goes on towards next iteration, where you enhance several other aspects. Those who try to follow a perfect setup are probably just in their design phase, and it is highly unlikely that they will get anywhere. Maybe in a decade or two, they'll manage a fairly well OS, but by then, it would already have been outdated due to technological change. So what's the point?

Practice is always different from theory. There was once DOS and then came series of Windows If only Microsoft had followed a proper way to develop an OS, we'd not need so many upgrades. All they had to do is to predict the technology change so that their original OS could adapt to its changing environment. Same follows for almost any development projects.

Anyway, let the writers have their moment. Your criticism is valid but hey, encourage them for improvement.

[Avidanborisov]

I could be wrong, but isn't setting the stack inside the .bss section going to cause problems?

[sortie]

Teach them how a proper OS would do it.

In fact, there actually is no such thing as a proper way to write an OS.

Don't fall victim to that kind of thinking. When implementating operating systems features, you often have the option between doing it well or doing it in a hacky way. I'm not talking perfection, but simple, good correctness. I've seen a lot of systems take the quick and dirty route, and they've paid for it later on. Instead of implementing a qsort that uses insertion sort and does lots of memory allocations that fail, you should do a proper O(n log n) implementation. It doesn't need to be fast, but it's decent and correct. Stuff like not using a cross-compiler and other dirty hacks done early on just gets some pleasure now in exchange for trouble later. I've spent a lot of time rewriting and making my OS better, when I should just have done things properly early on. If you can see issues with your implementation and can imagine a considerably better implementation, implement that instead.

Avidanborisov: Stack in the .bss is technically allowed.