Newbie here. Should I use GRUB as my bootloader?

[einzeptor]

Title. Or should I write my own?

Also, is it a manageable task if I use GRUB for now, and after my kernel gets going (or gets completed), write my own bootloader?

[kzinti]

Title. Or should I write my own?

That depends if you are more interested in writing a boot loader or a kernel. If it is, as I suspect, a kernel, then yes use GRUB (or something else, really). It can always be replaced later.

Also, is it a manageable task if I use GRUB for now, and after my kernel gets going (or gets completed), write my own bootloader?

Yes.

[einzeptor]

Title. Or should I write my own?

That depends if you are more interested in writing a boot loader or a kernel. If it is, as I suspect, a kernel, then yes use GRUB (or something else, really). It can always be replaced later.

Also, is it a manageable task if I use GRUB for now, and after my kernel gets going (or gets completed), write my own bootloader?

Yes.

Thanks for your answer

[Brendan]

Hi,

Or should I write my own?

Yes, then no, then yes.

When you're just starting out, you should spend a little time (e.g. maybe 3 days) writing your own boot loader just so you have some idea of what it does and how it works.

Then you should switch to GRUB and get experience with writing a kernel.

Finally, after you've got experience with "kernel things" (memory management SMP, scheduling, etc) and decide to write a higher quality OS you'll need your own boot code. The reason for this is that multi-boot severely limits the design of the OS (e.g. prevents you from doing memory management in a clean way, prevents you from auto-selecting a default video mode to use during boot in a way that works reliably, prevents you from having 2 or more kernels and auto-selecting which kernel most suits the computer, prevents you from having any security at all, prevents you from using your own file system, etc).

Note that you wouldn't write a boot loader. You'd write:

• a boot loader for UEFI
• a boot loader for BIOS for PXE/network boot
• a boot loader for BIOS for "no emulation El Torito" CD-ROM
• a boot loader for BIOS for floppy disks (no partitions, no "int 0x13 extensions")
• a boot loader for BIOS for MBR partitioned hard disk
• a boot loader for BIOS for GPT partitioned hard disk
• possibly, some sort of "second stage" that is started by all boot loaders and has common code that prepares for the kernel

Also, is it a manageable task if I use GRUB for now, and after my kernel gets going (or gets completed), write my own bootloader?

In theory it's not hard. In practice the main reason to switch from GRUB to your own boot loader is because multi-boot has crippled the design of your OS; and to fix that you'll need to redesign the OS, which isn't as simple as just replacing the boot loader.

Mostly; by the time you've gained the experience needed to make the switch from "initial/learning OS" to "final/high quality OS" (if you ever do - a lot of people only want to learn and have no intention of building a high quality OS), you'll be wanting to replace both the boot loader and the kernel anyway.


Cheers,

Brendan

[kzinti]

Given the question the OP is asking, I really believe there is no right answer at this point. He needs to get experience. He can do that using GRUB or by writing a boot loader. No matter what he does next, he will end up rewriting both.

The right answer in my mind is work on what is most interesting to you. This is the best way to learn and stay motivated.

[Nutterts]

Or should I write my own?

I'm also very new here but in the past I already had done a "Hello World!" real mode thingie booting of a floppy. So I thought I was already a genius so why waste time writing my own bootloader? So I started out with Grub and the multiboot specification. Basicly followed the barebones Pascal tutorial and played with it.

Oh how bad an ego can hurt when it's hit with reality at the speed of light!

There's stuff I understand and there's stuff I don't get yet. I could swear there's a wizard behind a curtain somewhere when booting off Grub. Been trying to lure him out with cheese puffs. But it's fun ending up in protected mode when you barely know how to get there from real mode yourself. It's something to play with, experiment, learn. But it's not even in that simplistic state anywhere of what I want.

The right answer in my mind is work on what is more interesting to you. This is the best way to learn and stay motivated.

Thats basicly the best advice you can get! It's always the journey that really counts.

So for me it's back to real mode assembler, writing a multi stage chain loading 16bit bootloader/mini-kernel which eventually will boot my 32/64bit kernel. With "eventually" I mean I'm aiming for a release somewhere in the year 2042.

[Kevin]

I could swear there's a wizard behind a curtain somewhere when booting off Grub. Been trying to lure him out with cheese puffs.

How comes that you don't suspect a wizard behind a curtain when using BIOS functions?

The truth is that you always start working at some level on top of software that other people have written (the hidden wizard), and you just get to choose which level it is. It may be the processor microcode, it may be the firmware, it may be the bootloader, it may be the OS, it may be an interpreter for a script language.

Usually, if you write one thing, you build on top of the next lower level. If you write a script, you build on top of the interpreter. If you write an interpreter, you build on top of the OS. If you write a bootloader, you build on top of the firmware. Each of these is a project of its own.

As long as you're a newbie, I would advise against attacking multiple projects for which you have little or no experience. This means, either do an OS and leave the bootloader alone for now, or do a bootloader and forget about the OS for now (where "for now" doesn't mean two days, but at least half a year or so).

[iansjack]

I would say that to write a (useful) bootloader you first have to understand how a kernel works and what information it needs passed to it by the boot process. So concentrate on producing a simple, or even moderately complex, kernel first and then, if you want to, at a later stage roll your own bootloader. I suspect that most "I have written a bootloader" exercises consist of a large amount of cut-and-pasted code, often poorly understood.

The one advantage I see for a beginner in writing your own bootloader (as an intellectual exercise) is that it means you have to understand how to switch the processor from real to protected mode. But you could always use Grub and then write a 64-bit kernel; then you would learn the corresponding technique of switching from protected mode to long mode. (I strongly recommend that anyone writing a kernel nowadays looks at using 64-bits; the processor offers so much more in this mode. Also it means that you will - like it or not - have to learn how to use paging; I think that any half-serious OS will use paging.)

The big disadvantage (IMO) of writing your own bootloader is that you get stuck with real mode and/or using BIOS calls. We see this so often here, with people going to ridiculous lengths to be able to still use BIOS calls from protected mode. I would say just forget the BIOS (and so forget bootloaders) until you have learnt how to do for yourself, in protected or long mode, what the BIOS does for you in real mode.

[Nutterts]

How comes that you don't suspect a wizard behind a curtain when using BIOS functions?

Because I'm not ignorant about them. I know how to use software interrupt or replace one. I understand the IVT and if I really needed too I could decompile a BIOS function. It's a solid foundation for me to build on.

And yes it's all a progression of abstraction. What I have in mind right now basically comes down to [bootloader] -> [16bit kernel] -> [32bit kernel] -> [64bit kernel]. I also want the kernels, servers and boot settings to be loaded of a SFS boot device/partition.

I could revert back to rmode from Grub. But would that be the same machine state as if I just used my own bootsector? Next to that. Is it smart for someone who can't get to pmode from rmode himself to skip that part? Especially if in the end he wants to get to long mode?

The big disadvantage (IMO) of writing your own bootloader is that you get stuck with real mode and/or using BIOS calls. We see this so often here, with people going to ridiculous lengths to be able to still use BIOS calls from protected mode.

Yeah. Just want to make sure nobody thinks I'm saying someone should do that. Although I own some 8088/86 machines, so it makes sense for me to have a 16bit kernel that's self sufficient and can boot off a floppy. But on anything even slightly newer it's just for setting up a nice and informed protected mode and loading a kernel above 1MB.

[Kevin]

Because I'm not ignorant about them. I know how to use software interrupt or replace one. I understand the IVT and if I really needed too I could decompile a BIOS function. It's a solid foundation for me to build on.

You understand the interface (and I think only part of it), but probably not the implementation. It's the same with the bootloader: Yes, you definitely should understand its interface in order to use it. The Multiboot specification (as one example) is short and easy enough to be understood quickly. Understanding its implementation isn't necessary - but if you really needed it, you could just read the source code, just like you could in theory decompile the BIOS, but probably won't actually do it.

I also want the kernels, servers and boot settings to be loaded of a SFS boot device/partition.

SFS? Seriously? But okay, let's not disgress...

I could revert back to rmode from Grub. But would that be the same machine state as if I just used my own bootsector? Next to that. Is it smart for someone who can't get to pmode from rmode himself to skip that part? Especially if in the end he wants to get to long mode?

Why would you go back to RM? More or less the only thing it's useful for is switching to PM. And when you're already in PM, you obviously don't need to do that. The machine state at the time that GRUB jumps to your kernel is well documented in the Multiboot specification.

It is smart to skip things that aren't required for what you want to achieve. If I want to write a shell script, I don't start with writing a new OS. I just use the OS that's already there. You want to write a bootloader and you seem to be okay with the fact that you didn't initialise the memory controller yourself, but let the firmware do that. Why would you not be okay with letting the bootloader switch to PM instead of doing it on your own?

I mean, if you want to mess with real mode because you think it's fun or something, by all means, go ahead. But you seem to imply that after BIOS, but still in RM is the one right place to start, and that's simply wrong. The experience from messing with RM is mostly worthless for your kernel (or even higher levels), similar to how the experience from initialising the memory controller would be worthless for anything higher level. It's more or less unrelated knowledge.

[Nutterts]

It is smart to skip things that aren't required for what you want to achieve. If I want to write a shell script, I don't start with writing a new OS. I just use the OS that's already there. You want to write a bootloader and you seem to be okay with the fact that you didn't initialise the memory controller yourself, but let the firmware do that. Why would you not be okay with letting the bootloader switch to PM instead of doing it on your own?

Because I want A at address X and B at address Y. I want the first 1MB to be organized in such and such way.

If It wasn't a hobby but a commercial thing I'd not even go there. But it is a hobby and I want to learn how to get from real mode to long mode myself. Not just copy/paste some assembler code, not relying on a magic piece of software to it for me. Getting there on my own merrit with nothing but an assembler and the technical manual.

It's like food. I can make a killer lasagna, all fresh ingredients and perfect in every way. But generally for myself I just buy one at the store and trow it in the microwave for a few minutes. Why go thru all the effort to do something I already know how to do just to feed myself. I feel like that's you basic argument and imho it's hard to disagree with either way of looking at it.

I mean, if you want to mess with real mode because you think it's fun or something, by all means, go ahead. But you seem to imply that after BIOS, but still in RM is the one right place to start, and that's simply wrong.

I'm not implying that RM is the one right place to start because it's a fact that's it's the only right place to start. That's how the processor works. Even if you switch right to protected mode or long mode you start out with a at-least a few real mode instructions.

So I'd argue knowing how to switch to protected mode (or long mode) yourself is just as important to know as how to cook your own lasagna. I can't make this kind of lasagna and I feel it's important to learn to cook one before I buy one.

But hey, if someone doesn't want to learn to cook there own lasagna you won't see me judging them. I'm also not saying it's the only way, or the best way and if you've , or does but doesn't want too. No probs! I'm not judging. done it so many times that it's boring then you probably shouldn't.

The experience from messing with RM is mostly worthless for your kernel (or even higher levels), similar to how the experience from initialising the memory controller would be worthless for anything higher level. It's more or less unrelated knowledge.

The experience isn't worthless, in the end it becomes outdated. That's a big difference and highly subjective. Although knowing how the initialize the memory controller is worthless for anything higher level. If it doesn't get initialized, your nowhere.

In my case I also collect some vintage computers that don't even have a protected mode. So I have some extra incentives to do more in real mode then any sane person might do.

[iansjack]

Well, really you need to write your own BIOS. The BIOS in your computer is not a given with the processor, it's extra software that the manufacturer has installed. So you need to write your own BIOS to really handle the processor from switch on.

You have to compromise at some stage and use an existing foundation. A lot of us would consider Protected Mode to be a good starting point and to accept any foundation that gets us there. But you don't have to - your choice.

[Kevin]

So I'd argue knowing how to switch to protected mode (or long mode) yourself is just as important to know as how to cook your own lasagna. I can't make this kind of lasagna and I feel it's important to learn to cook one before I buy one.

Yeah, I'm just saying that as a beginner, most people would be better off cooking a simple lasagna with bought lasagna sheets than taking up a second project on the side and worrying about the best way to make the dough for it. You can always try if you can make a better dough when you already know how to make a lasagna.

The experience isn't worthless, in the end it becomes outdated. That's a big difference and highly subjective. Although knowing how the initialize the memory controller is worthless for anything higher level. If it doesn't get initialized, your nowhere.

I qualified this with "for your kernel". You cannot reuse much of the knowledge for your kernel because RM is a quite different environment. Of course it's important that someone loaded the kernel, someone wrote the firmware to initialise the memory controller and load the bootloader, someone designed the CPU to run the firmware on, someone dug up the copper used in the CPU, etc. But it's not important that it was you who did it, you wouldn't have much advantage from knowing the implementation details (rather than just the interface) while writing the kernel.

In my case I also collect some vintage computers that don't even have a protected mode. So I have some extra incentives to do more in real mode then any sane person might do.

That's a good reason for doing things in RM. Just be aware that you are the exception with this and be careful with recommending others to do the same without checking that their situation is the same, because very likely it's not.

[Nutterts]

be careful with recommending others to do the same without checking that their situation is the same, because very likely it's not.

Afaik I didn't recommend anyone to do it that way. If I gave that impression then I apologize. What I was trying to say is that it depends on many things and that your own opinion might change. Maybe in a few weeks or months I'll hate every word I wrote in this thread because I decided it was a dead end.

You have to compromise at some stage and use an existing foundation.

Exactly. I'm not arguing with that. I really think I either explained myself wrong due to English not being my first language. Or that I some how touched some sensitive subject without realizing it.

I just have a creazy dream that I can have a floppy that'll load a real mode kernel. Which if it sees it's not running on a mere 8088 automatically finds and loads the 32bit (or 64bit) kernel and servers that best fit the hardware. If I didn't have my collection of 8088 and the joy I have in using them I might still be expanding on that barebone.

But like I said... maybe I'll eat my own words in some weeks or months.

[Brendan]

Hi,

I think that (in general) there's a massive difference between micro-kernels and monolithic kernels.

For monolithic kernels you've basically got a massive blob of security vulnerabilities that does "everything" and the boot code just does a few trivial things (get memory map, load kernel, setup default video mode, pass control to kernel). In this case "boot code" and "boot loader" mostly mean the same thing.

For micro-kernels the kernel itself does very little (e.g. some memory management, scheduling, IPC and not much else). It requires careful planning of the boot code because the boot code has to take care of everything from file IO to updating the screen up until native drivers and services are started; and you also have to worry about "hand off" for each thing (e.g. when the boot code starts a native video driver you need some way of transferring the responsibility of displaying video from the boot code to the video driver, when the VFS is started you need some way of transferring the responsibility of file IO from the boot code to the VFS service, etc).

In this case "boot code" is all the code used during boot - it often consists of multiple pieces where boot loader is just one of those pieces. For example (my OS design) there's a boot loader, a "boot manager", modules for detecting CPU features, modules for handling video/serial output, "kernel setup" modules, an "init" process, etc. Typically (based on multiple previous implementations) the total amount of boot code involved is many times larger than the kernel; and is also harder to design and harder to implement than the kernel; and has much more impact on the OS as a whole than the kernel. The boot loader itself is the part that all the later pieces of boot code depend on - the beginning of an intricately crafted plan. It's not something that you design using "auto-pilot".


Cheers,

Brendan