Generic bootloader

[shikhin]

Hi,

Recent discussions on the IRC channel (#osdev) prompted me to write this post.

For a long time, I've had some gripes with GRUB (or multiboot, to some extent) (this isn't a post about GRUB, but I got to put my reasons):

• It isn't exactly user-friendly, IMHO. Some might debate this - but, to me, it looks like creating a simple bootable disk image could be simpler than it currently is, with GRUB.
• It isn't up-to-date. It doesn't properly support UEFI. The PXE implementation is buggy. GRUB 2 might attempt to fix some of these problems, but we don't know till when would it get out of beta, and what the state of the bootloader might be, till then.
• Multiboot fails at several levels. The memory map could be better. VBE support could be much better - wouldn't have required any real mode emulator or an equivalent hack, then.

Obviously, I could wait for GRUB 2 to fix these problems, or as froggey suggested, take the help of some members, and write up a better replacement.

What I propose to do is to start a project for a bootloader (let's call it Generic bootloader, for a lack of any better name). The bootloader would obviously aim at beginners entering OSDev, who don't want to write any bootloaders themselves, but aren't happy with GRUB/multiboot.

I do severely lack with any clear plans on my bootloader. My main motive, via this post, is to gather some members, who are interested to help me with this bootloader (or perhaps provide any reason why I shouldn't write one). Once (if) sufficient people are drawn to this project, I'll start drawing up the specifications with them. The implementation would follow soon.

Regards,
Shikhin

NOTE: It probably isn't correct to post at midnight, with just some vague ideas. However, I'm afraid if I don't take any steps, my enthusiasm might die down till tomorrow. If you have any questions, please free to ask. If you are interested in joining me, you can reply to this post, or PM me.

[zhiayang]

Splendid!

I enrol.


EDIT: EFI Refuses to boot my loader! RGH.

[Brendan]

Hi,

In my opinion; the first step would be to create a specification that clearly defines how the boot loader gets any information it needs from the "thing being booted", how the boot loader passes information to the "thing being booted", and how the boot loader passes control to the "thing being booted" (including the state of various pieces of hardware when this happens - e.g. if the "thing being booted" can assume A20 is enabled, if PCI devices are pre-configured, if interrupts and interrupt controllers are setup or not, etc).

Without some sort of specification that developers can use to determine what they'd need to implement, developers can't know what they'd need to implement..


Cheers,

Brendan

[sortie]

Naturally, such a Generic Bootloader would have to co-exist with GRUB 2, at least to some extend. For instance, I like to run my own Multiboot-compatible operating system on my own machines, and those happen to have GRUB 2 installed. Perhaps this could be accomplished using chain loading.

[bluemoon]

I also not quite comfortable with GRUB and I had write my own boot loader (with limited features just enough to load my kernel).
Eventually I will need a better, more complete loader, in terms of firmware (BIOS, UEFI) and supported media and formats (CD, usb, network, or even boot image from internet#);
I'm interested in GRUB alternatives.


#) Some ASUS broad has embed a mini-linux and able to access internet and provide a mini text-based browser as a special boot option, I was inspired by this and hope my future boot loader may boot using internet image just like normal network boot

[rdos]

IMHO, GRUB and GRUB 2 attempt to do too much. A bootloader should just load some binary image-file, and possibly some additional modules, and then pass some pointer to a structure in 32-bit flat mode with paging disabled. Everything else is overkill. It should definitely not relocate or fixup some types of executable formats, it should not enable paging, and it should not leave control in long mode (as that means paging must be enabled). It should be able to boot from as many file systems as possible, and this should be transparent to users (and not module-based!). There should be no "multiboot" header in the OS image, but rather the pointer passed at startup should be general enough to be able to be extended in future releases without breaking backwards compability.

[shikhin]

Hi,

Before replying to the previous posts, I want to list down some of the basic goals of this project:

• Replace multiboot, i.e., several OS projects can use this bootloader.
• Be easy to use. Has a good set of tools and documentation.
• Be easy to configure.
• Support almost everything a beginner (or even advanced OSdev'er, for that matter) might require from a bootloader (nothing extra, though).

The languages I'd use would be assembly (intel style, using NASM - could be changed if several contributors demand so), and C (gcc or clang - can't change, sorry).

The planned bootloader should be able to boot from PXE, CDs, floppies and harddisks. They'd boot off from BIOS & UEFI, and would only be intended for x86 and AMD64 (x86_64). They should roughly do what GRUB does, right now. They would also support enabling paging (and long mode), picking up any video mode (based on some preferences), passing the pointers to ACPI & MPS tables (since UEFI requires you to get those pointers from the firmware), etc. Anymore, and we'd be getting into the specifications (more on that, later).

To be honest, I never wrote any bootloader for a hard disk. On the other hand, I've written one for PXE, floppies and CDs.

Splendid! I enrol.

Code: Select all

contributors++;

In my opinion; the first step would be to create a specification that clearly defines how the boot loader gets any information it needs from the "thing being booted", how the boot loader passes information to the "thing being booted", and how the boot loader passes control to the "thing being booted" (including the state of various pieces of hardware when this happens - e.g. if the "thing being booted" can assume A20 is enabled, if PCI devices are pre-configured, if interrupts and interrupt controllers are setup or not, etc).

In my opinion; the first step would be to list down some of the goals the project aims to fulfill. These are nonvolatile, and remain fixed throughout the project; rest all things are secondary (including the one you just said) and volatile. I would prefer drawing up the specifications with constant advice (and guidance) from whoever decides to participate.

Without some sort of specification that developers can use to determine what they'd need to implement, developers can't know what they'd need to implement..

Developers start with goals, then draw up specifications, then implement what they'd need to implement. However, since you're perhaps one of those on these forums without whom this project would lack severe momentum, if you really require them, I'd be willing to draw up a rough draft of the specifications.

Shall I count you in, then?

Naturally, such a Generic Bootloader would have to co-exist with GRUB 2, at least to some extend. For instance, I like to run my own Multiboot-compatible operating system on my own machines, and those happen to have GRUB 2 installed. Perhaps this could be accomplished using chain loading.

For my own bootloaders, I had some sort of a stage planned which would boot from GRUB, switch back to real mode, gather the rest of the information needed, and pass on to the next stage. Or, we could chain load. Naturally, these all quirks would be planned with the specifications.

Shall I count you in?

I also not quite comfortable with GRUB and I had write my own boot loader (with limited features just enough to load my kernel). Eventually I will need a better, more complete loader, in terms of firmware (BIOS, UEFI) and supported media and formats (CD, usb, network, or even boot image from internet#); I'm interested in GRUB alternatives.

Other than the mini text-based browser, I think we can do everything you also plan with your more complete bootloader; shall I count you in?

IMHO, GRUB and GRUB 2 attempt to do too much. A bootloader should just load some binary image-file, and possibly some additional modules, and then pass some pointer to a structure in 32-bit flat mode with paging disabled. Everything else is overkill. It should definitely not relocate or fixup some types of executable formats, it should not enable paging, and it should not leave control in long mode (as that means paging must be enabled). It should be able to boot from as many file systems as possible, and this should be transparent to users (and not module-based!). There should be no "multiboot" header in the OS image, but rather the pointer passed at startup should be general enough to be able to be extended in future releases without breaking backwards compability.

Hrm. Our definitions of bootloaders seem to be contradictory. For me, a bootloader should take control from the firmware, load some binary modules, do ANYTHING firmware related, and do every basic thing needed to be done once during boot. I guess I shouldn't count you in, then?

Regards,
Shikhin

[bluemoon]

I think we can do everything you also plan with your more complete bootloader; shall I count you in?

Sure, I can offer design opinion (if you need) and some nasm-capable code monkey effort.

[shikhin]

Hi,

I think we can do everything you also plan with your more complete bootloader; shall I count you in?

Sure, I can offer design opinion (if you need) and some nasm-capable code monkey effort.

I've started drawing out a rough draft of the design. I shall share here, when it's reached a respectable stage. However, since I've never written a hard disk bootloader, I'm curious as to how people support multiple FSs? You have support for all in one binary, or anything more modular?

Regards,
Shikhin

[bluemoon]

The design I use (and probably quite common) have multiple stages for boot loader:

Code: Select all

0. standard MBR
  - Load 512 bytes VBR, this is independent to OS, and can be replaced by any other standard MBR
1. VBR
  - This is specific for each type of file system, ie. I have FAT16.asm for FAT16, FAT32.asm for FAT32
  - This VBR provide the file loading API (at pre-defined entry point) to subsequent stages
  - This VBB then load /boot.bin from the file system
2. BOOT.BIN
  - Memory Map
  - Enable A20
  - Detect CPU
3. I also built this into boot.bin
  - Load 32/64bit kernel and initrd
  - Setup consistent environment (CPU registers state, 32-bit protected mode without paging, or compatibility mode with identity mapped region, etc)
  - pass control to kernel

So, to support more FS you basically write different VBRs with the file loading API. You may also need to extend the stages to support different media source (BIOS HDD service, or PXE, or EFI) in similar manner.

[rdos]

My boot-loader design is a little different.

For native booting:
The first stage is a 512 byte boot sector that loads 14 sectors directly after the MBR. Then control is passed to the first byte of this area.

The second stage is a 14 sector loader (corresponding to VBR) that will look in a FAT file system for bootable images in the root, and let the user select to boot one of them. The selected image will then be loaded to physical address 0x120000. Relevant OS information will be filled into the 0x1000 byte system_data_selector (this information include memory maps and positions of RDOS images).

The second stage is linked to a common loading object module. This part will know about the layout of RDOS images, and would locate the kernel module and jump to it in protected mode with paging disabled.

For GRUB booting:
Grub would load a pseudo-image that contains the multiboot header. This image will setup the needed info in system_data_selector, put the processor in the expected mode, and call the common part that boots RDOS. The RDOS binaries are loaded as modules.

As can be seen, this boot scenario cannot be supported by a generic boot-loader. If the boot-loader sets up paging, my interface will need to revert back to no paging and redo it from scratch. If it sets up a GDT or IDT, same thing, I'd have to replace those with my own GDT and IDT.

[zhiayang]

One thing I would like this 'Generic Bootloader' to do is load ELF64 files, into long mode.

And sure, we need to set up paging. So: we should have a way of telling the boot loader (maybe something akin to a menu.lst..?) how much space to Identity Map.

For every boot loader that loads in PMode, we pretty much need to re-do the GDT and IDT anyway; Since Identity Mapping would already be set up, it would save some extra steps on the OS's part.

Although I foresee a problem: We wouldn't know where the paging structures are, so perhaps we include it's address in the structure passed to the kernel.

[rdos]

Hrm. Our definitions of bootloaders seem to be contradictory. For me, a bootloader should take control from the firmware, load some binary modules, do ANYTHING firmware related, and do every basic thing needed to be done once during boot. I guess I shouldn't count you in, then?

Depends. If the plan is to provide most of the initialization parts of an OS, which in the end will be a highly comforming OS implementation since it is impossible to do this in a common way for widely divergent OSes, then, count my out. OTOH, if the project aims at supporting more than GCC cross-compiled OS-images that initialize some *nix environment, then maybe.

[rdos]

One thing I would like this 'Generic Bootloader' to do is load ELF64 files, into long mode.

Can't your tool-chain output relocated binaries?

And sure, we need to set up paging. So: we should have a way of telling the boot loader (maybe something akin to a menu.lst..?) how much space to Identity Map.

Most OSes don't have memory identity-mapped after boot, so why do you want that setup? The place of the kernel also is very different between OSes and operating modes.

[zhiayang]

Can't your tool-chain output relocated binaries?

It can. It would just save a step. Since it would be nice to have long mode set up, it a by-product of such would be to be able to load ELF64 files.

Most OSes don't have memory identity-mapped after boot, so why do you want that setup? The place of the kernel also is very different between OSes and operating modes.

Well... It depends on the specification we're going for. If there exists a config file, it would not be impossible to implement a series of options for such cases. Just giving ideas.