OSDev.org

bluemoon wrote:The design I use (and probably quite common) have multiple stages for boot loader:

Code: Select all

0. standard MBR
  - Load 512 bytes VBR, this is independent to OS, and can be replaced by any other standard MBR
1. VBR
  - This is specific for each type of file system, ie. I have FAT16.asm for FAT16, FAT32.asm for FAT32
  - This VBR provide the file loading API (at pre-defined entry point) to subsequent stages
  - This VBB then load /boot.bin from the file system
2. BOOT.BIN
  - Memory Map
  - Enable A20
  - Detect CPU
3. I also built this into boot.bin
  - Load 32/64bit kernel and initrd
  - Setup consistent environment (CPU registers state, 32-bit protected mode without paging, or compatibility mode with identity mapped region, etc)
  - pass control to kernel

So, to support more FS you basically write different VBRs with the file loading API. You may also need to extend the stages to support different media source (BIOS HDD service, or PXE, or EFI) in similar manner.

rdos wrote:My boot-loader design is a little different.

rdos wrote:...loads 14 sectors directly after the MBR...

rdos wrote:...a FAT file system...

rdos wrote:As can be seen, this boot scenario cannot be supported by a generic boot-loader. If the boot-loader sets up paging, my interface will need to revert back to no paging and redo it from scratch. If it sets up a GDT or IDT, same thing, I'd have to replace those with my own GDT and IDT.

rdos wrote:If the plan is to provide most of the initialization parts of an OS, which in the end will be a highly comforming OS implementation since it is impossible to do this in a common way for widely divergent OSes, then, count my out. OTOH, if the project aims at supporting more than GCC cross-compiled OS-images that initialize some *nix environment, then maybe.

rdos wrote:Most OSes don't have memory identity-mapped after boot, so why do you want that setup? The place of the kernel also is very different between OSes and operating modes.

Griwes wrote:After thinking for a while, count me in.

Griwes wrote:First, the bootloader should be file system agnostic, but not filesystem unaware (not that I'm arguing with that "no filesystem needed" from Brendan) - user (= operating system being booted, or it's developer; I'm going to keep using that word for them) should be able to put a filesystem driver, in format specified in specification (for example, with array of function addresses in first X bytes or something; it MUST be simple) in first (starting from Yth; we gotta get 2nd stage somewhere) Z sectors of disk/partition, and keep kernel (or application; who said that it must be kernel? again, let's stick with term "application" here; for more not really rational rationale look at naming paragraph) in some hardcoded location, like `boot/boot.gand` (why `.gand`? See naming paragraph). If those Z sectors aren't in filesystem driver format, they should be treated as application that is to be booted; that would be simple solution for both people wanting to be able to easily booted application, as well as to those wanting to recreate the partition every time they recompile their kernel.

Griwes wrote:Second, the boot sector should be filesystem aware; that is, I think that should be able to create per-filesystem (M/V)BR template file, that would allow the same code work (after, well, reassembling it) for different filesystems, that use different parts of (M/V)BR, as well as the legacy partition table. Just create template (putting some zeroes at places that shouldn't be overwritten to allow assembler to produce binary, but also creating some kind of 512 byte "mask", so HDD installer - assuming there is one - will be able to write boot sector without overwriting old data). Note: I have no idea how this is done in different OS installers, but I assume they use special filesystem drivers and preassembled boot sectors for supported filesystem; this solution *could* make it simpler and more generic.

Griwes wrote:Third, least important from those here - I talked about this on IRC before kinda like about a joke, but I think it's worth considering - name for the bootloader (or rather "the specification" and "the official specification implementation"). Following ELF and DWARF, I would like to propose GANDALF as name for the bootloader - `Generic, AdvaNceD Application Loader` (F could mean something like `(Filesystem aware)`). Here we get to that strange `.gand` extension - well, I hope it's self explanatory. We could even go with naming boot manager (to be written as another application for the bootloader!) like BALROG, where `LROG` at the end could mean `LoadeR Of Gandalf`; I still have to figure out how to use BA at the beginning. But again, that's not that important.

Wouldn't you like to write about yourselves "member of Gandalf Development Team"?

Finally, other features it would have to have, or could have - more important first:
- memory map sanitization - probably not necessary for UEFI (*probably*), but definitely recommended for ol' BIOS
- VBE mode setting (configurable at install)
- getting IOAPICs and LAPICs in known state (it's not much work, after all; by known state, I mean "disable everything you can")
- if it would already mess with APICs, maybe some basic ACPI parsing (like combining various entries of MADT, creating table that doesn't have variable-length entries, maybe - not much people would use it, but having that data parsed by write-once-use-all-the-time tool wouldn't hurt, especially when there is no SRAT and SLIT - applying SRAT/SLIT to that MADT structure and to memory map) - I mean, one table with variable length entries less to parse = profit!
- (?) AP booting (?)

• Memory map sanitation is a must. I do it in my project too (though I'm not sure of the code quality; I wrote it a time ago, and never looked at it again).
• VBE mode setting should also be there (as an option). For this, I planned to have some set of controls in the config file, which would allow the OS to specify which types of modes it allows. For example, an OS could do something like the following in the config file:

  Code: Select all

  filter-vbe-ratio: 4,3; filter-vbe-max-resolution: 1024,786; filter-vbe-max-bpp: 32;

  That way, everyone can decide which modes they want to switch to, or not. It could perhaps also give a *specific* list of which modes it wants. The idea is open.
• No, I don't think I want to do anything with IOAPIC or LAPIC. The most I'd do is to get the PIT into a clean state (no "hangover" interrupts, all interrupts masked, cli). You may convince me otherwise.
• Again, same for ACPI parsing. What I plan to do, however, is to give the address of the RSDP & MPS main table to the kernel. That's necessarily... since UEFI does table-probing in a different firmware-dependent manner. You may convince me otherwise (for parsing ACPI).
• Again, I don't want to do AP booting. Each OS might take that differently, and having all those options in the configuration file... bah.

rdos wrote:Why not search the root on all available partitions for valid boot-images.

rdos wrote:I don't like the idea of having configuration files for boot-loaders. Why not search the root on all available partitions for valid boot-images (if you use some type of signature this would work)? That's the method I use for RDOS boot-loader. It searches the partitions for valid RDOS images, and gives the user a list of possible boot choices. That way there is no need for configuration files.

About putting the image at fixed disc positions. If the boot-loader is installed in the MBR, a possible solution is to put the VBR directly after the MBR. That should not be used by any filesystem. Again, that is what RDOS bootloader does. It always installs itself in the MBR, and then uses 14 sectors directly after the MBR for second stage.

bluemoon wrote:

rdos wrote:Why not search the root on all available partitions for valid boot-images.

As discussed above, this feature is provided by the boot manager and not the objective of this (these) boot loaders.

And I don't think it's good to put custom MBR (and there is no need anyway); a custome MBR introduce one more incompatible factor for other OS.

Shikhin wrote:I don't think you can use your method of having 14 sectors after the MBR for an already partitioned disk, can you?

bluemoon wrote:

Shikhin wrote:I don't think you can use your method of having 14 sectors after the MBR for an already partitioned disk, can you?

It depends on how far you want to compile to standard.
The MBR itself works on any LBA, however modern tools allocate partition on head boundary - so sector 2~62 is usually unused.

Having said that, if one in a zillion chance the user uses an ancient tool, or a custom OS tool that utilize all sectors, will you take the blame?

jnc100 wrote:Surely some file system support is a must. Nearly all modern kernels require the boot loader to supply some sort of extra module that is loaded from the file system (e.g. an initial ramdisk), and therefore you need to be able to parse the file system to locate these and load them, or do you plan on hard coding the LBAs of these modules at install time and hoping the system doesn't move them?

jnc100 wrote:In addition, if going down the direct-to-long mode route I'd suggest you simply load the kernel to whatever virtual address it is supposed to be loaded at, not identity map. Then provide the kernel with a list of used and free physical memory areas and also a list of used virtual memory areas. Unfortunately if we keep expanding the information to pass to the kernel in this way then eventually you will end up with something that looks a lot like GRUB 2.

Shikhin wrote:Hi,

rdos wrote:I don't like the idea of having configuration files for boot-loaders. Why not search the root on all available partitions for valid boot-images (if you use some type of signature this would work)? That's the method I use for RDOS boot-loader. It searches the partitions for valid RDOS images, and gives the user a list of possible boot choices. That way there is no need for configuration files.

That might work. However, for that, you'd be assuming that all the different kinds of kernels are same. They support all video modes, they want to be in 32-bit flat mode, etc. Without a configuration file, I don't see how generic this could be...

Shikhin wrote:

jnc100 wrote:Surely some file system support is a must. Nearly all modern kernels require the boot loader to supply some sort of extra module that is loaded from the file system (e.g. an initial ramdisk), and therefore you need to be able to parse the file system to locate these and load them, or do you plan on hard coding the LBAs of these modules at install time and hoping the system doesn't move them?

I think most people do that by having all those at a "reserved" area.

Shikhin wrote:For physical memory - that's easy. I just add another type of entry to the memory map called "system area" which is what the kernel + modules use. For the virtual memory areas, I'd put the virtual addresses along the list of modules I pass to the kernel. That doesn't look overly convoluted to me.

rdos wrote:That information is put in the image itself. It knows that it wants to be loaded in protected mode with paging disabled, or that it wants to be loaded in long mode with paging enabled. It is better to let the OS image supply this data rather than a configuration file, which eventually would be edited by an user that might not know the details of the OS image.

jnc100 wrote:The initial ram disk on my linux box (stock ubuntu) is 14 MB in size - that's quite a large 'reserved area'.

Shikhin wrote:You're right, its not. Unfortunately when you try and support lots of different kernels, e.g. real mode, protected mode, long mode you realise you need to pass different information to each, then when you consider you boot from different mediums e.g. floppy/HD/CD/network etc you may want to pass information about those too (e.g. the bootp address and the tftp server in the case of network boot) and then when you add in 'optional' stuff like vbe modes your kernel information structure either becomes very large or you have a list of tagged items like Multiboot 2.

Shikhin wrote:

Griwes wrote:First, the bootloader should be file system agnostic, but not filesystem unaware (not that I'm arguing with that "no filesystem needed" from Brendan) - user (= operating system being booted, or it's developer; I'm going to keep using that word for them) should be able to put a filesystem driver, in format specified in specification (for example, with array of function addresses in first X bytes or something; it MUST be simple) in first (starting from Yth; we gotta get 2nd stage somewhere) Z sectors of disk/partition, and keep kernel (or application; who said that it must be kernel? again, let's stick with term "application" here; for more not really rational rationale look at naming paragraph) in some hardcoded location, like `boot/boot.gand` (why `.gand`? See naming paragraph). If those Z sectors aren't in filesystem driver format, they should be treated as application that is to be booted; that would be simple solution for both people wanting to be able to easily booted application, as well as to those wanting to recreate the partition every time they recompile their kernel.

At first looks, it seems to be a nice solution. However, I still don't like the idea of putting anything in the first N sectors. As far as I know (though I may be wrong), ext2/3/4 don't support having any such reserved sectors at the start of the disk/partition. Moreover, entrusting the user to write a filesystem driver... is bad. Of course, we could have some pre-built drivers for each fs, and then put them in the first N sectors. But that'd be almost similar to bluemoon's VBR per filesystem design.

Shikhin wrote:

Griwes wrote:Second, the boot sector should be filesystem aware; that is, I think that should be able to create per-filesystem (M/V)BR template file, that would allow the same code work (after, well, reassembling it) for different filesystems, that use different parts of (M/V)BR, as well as the legacy partition table. Just create template (putting some zeroes at places that shouldn't be overwritten to allow assembler to produce binary, but also creating some kind of 512 byte "mask", so HDD installer - assuming there is one - will be able to write boot sector without overwriting old data). Note: I have no idea how this is done in different OS installers, but I assume they use special filesystem drivers and preassembled boot sectors for supported filesystem; this solution *could* make it simpler and more generic.

I'm sorry, I lost you here. Could you please rephrase the above paragraph, or something?

bits 16
org 0x7c00

%macro begin 0
begin:
    jmp 0x0:start
%endmacro

%macro code 0
start:
    mov ax, 0
    mov ds, ax
    mov es, ax 
    ; ...all the bootloader code
%endmacro

%macro fill 0
    times 510 - ($-$$) db 0
    dw 0xaa55
%endmacro

%include "gandboot.inc"
begin
db "Example Label   "
; ...whatever is required to be here
code
fill

Shikhin wrote:

Griwes wrote:Finally, other features it would have to have, or could have - more important first:
- memory map sanitization - probably not necessary for UEFI (*probably*), but definitely recommended for ol' BIOS
- VBE mode setting (configurable at install)
- getting IOAPICs and LAPICs in known state (it's not much work, after all; by known state, I mean "disable everything you can")
- if it would already mess with APICs, maybe some basic ACPI parsing (like combining various entries of MADT, creating table that doesn't have variable-length entries, maybe - not much people would use it, but having that data parsed by write-once-use-all-the-time tool wouldn't hurt, especially when there is no SRAT and SLIT - applying SRAT/SLIT to that MADT structure and to memory map) - I mean, one table with variable length entries less to parse = profit!
- (?) AP booting (?)

• Memory map sanitation is a must. I do it in my project too (though I'm not sure of the code quality; I wrote it a time ago, and never looked at it again).
• VBE mode setting should also be there (as an option). For this, I planned to have some set of controls in the config file, which would allow the OS to specify which types of modes it allows. For example, an OS could do something like the following in the config file:

  Code: Select all

  filter-vbe-ratio: 4,3; filter-vbe-max-resolution: 1024,786; filter-vbe-max-bpp: 32;

  That way, everyone can decide which modes they want to switch to, or not. It could perhaps also give a *specific* list of which modes it wants. The idea is open.
• No, I don't think I want to do anything with IOAPIC or LAPIC. The most I'd do is to get the PIT into a clean state (no "hangover" interrupts, all interrupts masked, cli). You may convince me otherwise.
• Again, same for ACPI parsing. What I plan to do, however, is to give the address of the RSDP & MPS main table to the kernel. That's necessarily... since UEFI does table-probing in a different firmware-dependent manner. You may convince me otherwise (for parsing ACPI).
• Again, I don't want to do AP booting. Each OS might take that differently, and having all those options in the configuration file... bah.

%include "gandboot.inc"
begin
db "Example Label "
; ...whatever is required to be here
code
fill