OSDev.org

The Place to Start for Operating System Developers
It is currently Sun Dec 17, 2017 10:47 pm

All times are UTC - 6 hours




Post new topic Reply to topic  [ 145 posts ]  Go to page 1, 2, 3, 4, 5 ... 10  Next
Author Message
 Post subject: Generic bootloader
PostPosted: Sun Dec 02, 2012 12:13 pm 
Offline
Member
Member

Joined: Sat Oct 09, 2010 3:35 am
Posts: 272
Hi,

Recent discussions on the IRC channel (#osdev) prompted me to write this post.

For a long time, I've had some gripes with GRUB (or multiboot, to some extent) (this isn't a post about GRUB, but I got to put my reasons):

  • It isn't exactly user-friendly, IMHO. Some might debate this - but, to me, it looks like creating a simple bootable disk image could be simpler than it currently is, with GRUB.
  • It isn't up-to-date. It doesn't properly support UEFI. The PXE implementation is buggy. GRUB 2 might attempt to fix some of these problems, but we don't know till when would it get out of beta, and what the state of the bootloader might be, till then.
  • Multiboot fails at several levels. The memory map could be better. VBE support could be much better - wouldn't have required any real mode emulator or an equivalent hack, then.

Obviously, I could wait for GRUB 2 to fix these problems, or as froggey suggested, take the help of some members, and write up a better replacement.

What I propose to do is to start a project for a bootloader (let's call it Generic bootloader, for a lack of any better name). The bootloader would obviously aim at beginners entering OSDev, who don't want to write any bootloaders themselves, but aren't happy with GRUB/multiboot.

I do severely lack with any clear plans on my bootloader. My main motive, via this post, is to gather some members, who are interested to help me with this bootloader (or perhaps provide any reason why I shouldn't write one). Once (if) sufficient people are drawn to this project, I'll start drawing up the specifications with them. The implementation would follow soon.

Regards,
Shikhin

NOTE: It probably isn't correct to post at midnight, with just some vague ideas. However, I'm afraid if I don't take any steps, my enthusiasm might die down till tomorrow. If you have any questions, please free to ask. If you are interested in joining me, you can reply to this post, or PM me.

_________________
http://shikhin.in/

Current status: Gandr.


Last edited by shikhin on Sun Dec 02, 2012 12:17 pm, edited 2 times in total.

Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 12:16 pm 
Offline
Member
Member
User avatar

Joined: Tue Dec 27, 2011 7:57 am
Posts: 343
Splendid!

I enrol.


EDIT: EFI Refuses to boot my loader! RGH.

_________________
[mx] kernel: http://github.com/requimrar/mx


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 12:23 pm 
Offline
Member
Member
User avatar

Joined: Sat Jan 15, 2005 12:00 am
Posts: 8172
Location: At his keyboard!
Hi,

In my opinion; the first step would be to create a specification that clearly defines how the boot loader gets any information it needs from the "thing being booted", how the boot loader passes information to the "thing being booted", and how the boot loader passes control to the "thing being booted" (including the state of various pieces of hardware when this happens - e.g. if the "thing being booted" can assume A20 is enabled, if PCI devices are pre-configured, if interrupts and interrupt controllers are setup or not, etc).

Without some sort of specification that developers can use to determine what they'd need to implement, developers can't know what they'd need to implement.. ;)


Cheers,

Brendan

_________________
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 12:36 pm 
Offline
Member
Member
User avatar

Joined: Wed Mar 21, 2012 3:01 pm
Posts: 920
Naturally, such a Generic Bootloader would have to co-exist with GRUB 2, at least to some extend. For instance, I like to run my own Multiboot-compatible operating system on my own machines, and those happen to have GRUB 2 installed. Perhaps this could be accomplished using chain loading.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 12:53 pm 
Offline
Member
Member
User avatar

Joined: Wed Dec 01, 2010 3:41 am
Posts: 1723
Location: Hong Kong
I also not quite comfortable with GRUB and I had write my own boot loader (with limited features just enough to load my kernel).
Eventually I will need a better, more complete loader, in terms of firmware (BIOS, UEFI) and supported media and formats (CD, usb, network, or even boot image from internet#);
I'm interested in GRUB alternatives.


#) Some ASUS broad has embed a mini-linux and able to access internet and provide a mini text-based browser as a special boot option, I was inspired by this and hope my future boot loader may boot using internet image just like normal network boot :)


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 1:20 pm 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2180
IMHO, GRUB and GRUB 2 attempt to do too much. A bootloader should just load some binary image-file, and possibly some additional modules, and then pass some pointer to a structure in 32-bit flat mode with paging disabled. Everything else is overkill. It should definitely not relocate or fixup some types of executable formats, it should not enable paging, and it should not leave control in long mode (as that means paging must be enabled). It should be able to boot from as many file systems as possible, and this should be transparent to users (and not module-based!). There should be no "multiboot" header in the OS image, but rather the pointer passed at startup should be general enough to be able to be extended in future releases without breaking backwards compability.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 9:40 pm 
Offline
Member
Member

Joined: Sat Oct 09, 2010 3:35 am
Posts: 272
Hi,

Before replying to the previous posts, I want to list down some of the basic goals of this project:

  • Replace multiboot, i.e., several OS projects can use this bootloader.
  • Be easy to use. Has a good set of tools and documentation.
  • Be easy to configure.
  • Support almost everything a beginner (or even advanced OSdev'er, for that matter) might require from a bootloader (nothing extra, though).

The languages I'd use would be assembly (intel style, using NASM - could be changed if several contributors demand so), and C (gcc or clang - can't change, sorry).

The planned bootloader should be able to boot from PXE, CDs, floppies and harddisks. They'd boot off from BIOS & UEFI, and would only be intended for x86 and AMD64 (x86_64). They should roughly do what GRUB does, right now. They would also support enabling paging (and long mode), picking up any video mode (based on some preferences), passing the pointers to ACPI & MPS tables (since UEFI requires you to get those pointers from the firmware), etc. Anymore, and we'd be getting into the specifications (more on that, later). :)

To be honest, I never wrote any bootloader for a hard disk. On the other hand, I've written one for PXE, floppies and CDs.

requimrar wrote:
Splendid! I enrol.


Code:
contributors++;


Brendan wrote:
In my opinion; the first step would be to create a specification that clearly defines how the boot loader gets any information it needs from the "thing being booted", how the boot loader passes information to the "thing being booted", and how the boot loader passes control to the "thing being booted" (including the state of various pieces of hardware when this happens - e.g. if the "thing being booted" can assume A20 is enabled, if PCI devices are pre-configured, if interrupts and interrupt controllers are setup or not, etc).


In my opinion; the first step would be to list down some of the goals the project aims to fulfill. These are nonvolatile, and remain fixed throughout the project; rest all things are secondary (including the one you just said) and volatile. I would prefer drawing up the specifications with constant advice (and guidance) from whoever decides to participate.

Brendan wrote:
Without some sort of specification that developers can use to determine what they'd need to implement, developers can't know what they'd need to implement.. ;)


Developers start with goals, then draw up specifications, then implement what they'd need to implement. However, since you're perhaps one of those on these forums without whom this project would lack severe momentum, if you really require them, I'd be willing to draw up a rough draft of the specifications.

Shall I count you in, then? :)

sortie wrote:
Naturally, such a Generic Bootloader would have to co-exist with GRUB 2, at least to some extend. For instance, I like to run my own Multiboot-compatible operating system on my own machines, and those happen to have GRUB 2 installed. Perhaps this could be accomplished using chain loading.


For my own bootloaders, I had some sort of a stage planned which would boot from GRUB, switch back to real mode, gather the rest of the information needed, and pass on to the next stage. Or, we could chain load. Naturally, these all quirks would be planned with the specifications.

Shall I count you in? :)

bluemoon wrote:
I also not quite comfortable with GRUB and I had write my own boot loader (with limited features just enough to load my kernel). Eventually I will need a better, more complete loader, in terms of firmware (BIOS, UEFI) and supported media and formats (CD, usb, network, or even boot image from internet#); I'm interested in GRUB alternatives.


Other than the mini text-based browser, I think we can do everything you also plan with your more complete bootloader; shall I count you in? :)

rdos wrote:
IMHO, GRUB and GRUB 2 attempt to do too much. A bootloader should just load some binary image-file, and possibly some additional modules, and then pass some pointer to a structure in 32-bit flat mode with paging disabled. Everything else is overkill. It should definitely not relocate or fixup some types of executable formats, it should not enable paging, and it should not leave control in long mode (as that means paging must be enabled). It should be able to boot from as many file systems as possible, and this should be transparent to users (and not module-based!). There should be no "multiboot" header in the OS image, but rather the pointer passed at startup should be general enough to be able to be extended in future releases without breaking backwards compability.


Hrm. Our definitions of bootloaders seem to be contradictory. For me, a bootloader should take control from the firmware, load some binary modules, do ANYTHING firmware related, and do every basic thing needed to be done once during boot. I guess I shouldn't count you in, then? :)

Regards,
Shikhin

_________________
http://shikhin.in/

Current status: Gandr.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 10:10 pm 
Offline
Member
Member
User avatar

Joined: Wed Dec 01, 2010 3:41 am
Posts: 1723
Location: Hong Kong
Shikhin wrote:
I think we can do everything you also plan with your more complete bootloader; shall I count you in? :)

Sure, I can offer design opinion (if you need) and some nasm-capable code monkey effort.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 10:42 pm 
Offline
Member
Member

Joined: Sat Oct 09, 2010 3:35 am
Posts: 272
Hi,

bluemoon wrote:
Shikhin wrote:
I think we can do everything you also plan with your more complete bootloader; shall I count you in? :)

Sure, I can offer design opinion (if you need) and some nasm-capable code monkey effort.


I've started drawing out a rough draft of the design. I shall share here, when it's reached a respectable stage. However, since I've never written a hard disk bootloader, I'm curious as to how people support multiple FSs? You have support for all in one binary, or anything more modular?

Regards,
Shikhin

_________________
http://shikhin.in/

Current status: Gandr.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Sun Dec 02, 2012 10:58 pm 
Offline
Member
Member
User avatar

Joined: Wed Dec 01, 2010 3:41 am
Posts: 1723
Location: Hong Kong
The design I use (and probably quite common) have multiple stages for boot loader:

Code:
0. standard MBR
  - Load 512 bytes VBR, this is independent to OS, and can be replaced by any other standard MBR
1. VBR
  - This is specific for each type of file system, ie. I have FAT16.asm for FAT16, FAT32.asm for FAT32
  - This VBR provide the file loading API (at pre-defined entry point) to subsequent stages
  - This VBB then load /boot.bin from the file system
2. BOOT.BIN
  - Memory Map
  - Enable A20
  - Detect CPU
3. I also built this into boot.bin
  - Load 32/64bit kernel and initrd
  - Setup consistent environment (CPU registers state, 32-bit protected mode without paging, or compatibility mode with identity mapped region, etc)
  - pass control to kernel


So, to support more FS you basically write different VBRs with the file loading API. You may also need to extend the stages to support different media source (BIOS HDD service, or PXE, or EFI) in similar manner.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Mon Dec 03, 2012 2:03 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2180
My boot-loader design is a little different.

For native booting:
The first stage is a 512 byte boot sector that loads 14 sectors directly after the MBR. Then control is passed to the first byte of this area.

The second stage is a 14 sector loader (corresponding to VBR) that will look in a FAT file system for bootable images in the root, and let the user select to boot one of them. The selected image will then be loaded to physical address 0x120000. Relevant OS information will be filled into the 0x1000 byte system_data_selector (this information include memory maps and positions of RDOS images).

The second stage is linked to a common loading object module. This part will know about the layout of RDOS images, and would locate the kernel module and jump to it in protected mode with paging disabled.

For GRUB booting:
Grub would load a pseudo-image that contains the multiboot header. This image will setup the needed info in system_data_selector, put the processor in the expected mode, and call the common part that boots RDOS. The RDOS binaries are loaded as modules.

As can be seen, this boot scenario cannot be supported by a generic boot-loader. If the boot-loader sets up paging, my interface will need to revert back to no paging and redo it from scratch. If it sets up a GDT or IDT, same thing, I'd have to replace those with my own GDT and IDT.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Mon Dec 03, 2012 2:06 am 
Offline
Member
Member
User avatar

Joined: Tue Dec 27, 2011 7:57 am
Posts: 343
One thing I would like this 'Generic Bootloader' to do is load ELF64 files, into long mode.

And sure, we need to set up paging. So: we should have a way of telling the boot loader (maybe something akin to a menu.lst..?) how much space to Identity Map.

For every boot loader that loads in PMode, we pretty much need to re-do the GDT and IDT anyway; Since Identity Mapping would already be set up, it would save some extra steps on the OS's part.

Although I foresee a problem: We wouldn't know where the paging structures are, so perhaps we include it's address in the structure passed to the kernel.

_________________
[mx] kernel: http://github.com/requimrar/mx


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Mon Dec 03, 2012 2:12 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2180
Shikhin wrote:
Hrm. Our definitions of bootloaders seem to be contradictory. For me, a bootloader should take control from the firmware, load some binary modules, do ANYTHING firmware related, and do every basic thing needed to be done once during boot. I guess I shouldn't count you in, then? :)


Depends. If the plan is to provide most of the initialization parts of an OS, which in the end will be a highly comforming OS implementation since it is impossible to do this in a common way for widely divergent OSes, then, count my out. OTOH, if the project aims at supporting more than GCC cross-compiled OS-images that initialize some *nix environment, then maybe.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Mon Dec 03, 2012 2:15 am 
Offline
Member
Member

Joined: Wed Oct 01, 2008 1:55 pm
Posts: 2180
requimrar wrote:
One thing I would like this 'Generic Bootloader' to do is load ELF64 files, into long mode.


Can't your tool-chain output relocated binaries?

requimrar wrote:
And sure, we need to set up paging. So: we should have a way of telling the boot loader (maybe something akin to a menu.lst..?) how much space to Identity Map.


Most OSes don't have memory identity-mapped after boot, so why do you want that setup? The place of the kernel also is very different between OSes and operating modes.


Top
 Profile  
 
 Post subject: Re: Generic bootloader
PostPosted: Mon Dec 03, 2012 2:26 am 
Offline
Member
Member
User avatar

Joined: Tue Dec 27, 2011 7:57 am
Posts: 343
rdos wrote:
Can't your tool-chain output relocated binaries?


It can. It would just save a step. Since it would be nice to have long mode set up, it a by-product of such would be to be able to load ELF64 files.

rods wrote:
Most OSes don't have memory identity-mapped after boot, so why do you want that setup? The place of the kernel also is very different between OSes and operating modes.


Well... It depends on the specification we're going for. If there exists a config file, it would not be impossible to implement a series of options for such cases. Just giving ideas.

_________________
[mx] kernel: http://github.com/requimrar/mx


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 145 posts ]  Go to page 1, 2, 3, 4, 5 ... 10  Next

All times are UTC - 6 hours


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group