OSDev.org

Posted: **Sat Oct 22, 2011 5:22 pm**

Hi OsDev forum! First, thanks for all the amazing information found on this site, and I look forward to contributing to the community knowledge base in the future.

I've recently started getting into OS dev and currently have a working single stage bootloader that simply prints some characters, loads a kernel (3 sectors following bootsector) that implements some VGA functions, and a c utility that packages all of that up into a flat binary floppy image. It's at the point where I have a working knowledge of what's going on, and it's time to start expanding, starting with a proper bootloader. (Also, I'm fully aware that GRUB will do everything and more, but I'm entering OS development for educational value, and I feel writing a bootloader is valuable).

Also, please know that I've spent a few weeks reading the wiki/searching the forum/googling about bootloader theory/reading the GRUB src before coming to you with questions, so I apologize if there's something obvious out there I missed. I did my best!

So my goal is to move away from a flat binary image, and get some sort of file system going. I understand that the classic approach is a FAT12 floppy with a simple bootsector that can parse the filesystem (or else a utility that hard-codes the sector/size of the second stage on disk, ala grub-setup) because of the simplicity of FAT12. However, I do have problems with this approach:
1) FAT12 is old and deprecated. This will never be my OS's main file system, so why spend time on it now if there are other options that aren't terribly more complicated and are more useful (ISO)
2) Floppies are old and deprecated; people boot off CDs (which are becomming also-deprecated in favor of USB sticks now...)
3) Tool support is deprecated. I'm devving on Win7x64 through cygwin. There don't seem to be any virtual floppy emulators that work on Win7x64, and I'd rather not deal with a linux VM just to mount/write to a floppy
4) Tool support is less convenient. Even if I ignore #3 and go the VM route (or use linux natively even) it doesn't seem like the floppy image will ever be as easy as a CD image. You can just throw a mkisofs command into a makefile to generate your bootable image ready for VM testing, whereas you'd need to actually mount a virtual floppy on a virtual floppy drive, copy files, unmount it, and then fire up the VM. I suppose you could put all those commands into a makefile, but it'd be very environment-specific compared to the mkisofs approach.

In researching my options based on these criteria, it seems like El-Torrito is probably what I'm after because:
1) easy to create ISOs without dealing with drive virtualization
2) cross platform (mkisofs works in cygwin)
3) Booting from CD is something all modern OSes need to do anyway at install time, so it won't be 'wasted energy' long term

However, I do have a few questions based on what I've been read:
1) First and foremost, is any of the information/understanding I've articulated above inaccurate?

2) Is el-torrito the best route to take until my OS is ready to implement real filesystems/be installed on a hard drive (and if I'm not scared of taking on a slightly more advanced FS than fat12)?

3) Using el-torrito, is there any size limit on the bootloader anymore (other than the limitations of 0x7C00 up until the end of free low-memory itself)? Does the El-Torrito method load the entire file specified by the -b flag, or does it only load one sector?

4) Is the concept of a multi-stage bootloader still needed in el-torrito? If so, why?

5) I'm a little bit confused as to how the boot information table works in El-Torrito. The wiki seems to suggest that the mkisofs utility takes "-b /path/to/loader" and installs that file into the boot sector, and if passed proper parameters fills in the info like bi_PrimaryVolumeDescriptor. My question is...why do we care? If this file is already loaded into memory, this info doesn't really do anything for us unless we want to self-relocate (and why do that?)? (Afterthought: I guess bi_PrimaryVolumeDescriptor is quite useful, but what about the boot file location and length)

6) There's example source code on the el-torrito wiki that shows a simple bootloader with [ORG 0x7C00] but there's a post on the forum that says "the boot record on the CD can ask the BIOS to load 400 KiB at 0x00006000, instead of just one sector that is always loaded at 0x007C00". Is this a discrepancy, or am I getting confused between emulation and no emulation?

7) Speaking of emulation, is there any value to using emulation instead of no-emulation? The impression I get is emulation is an older practice, but at the same time I could see emulation being useful because the bootloader architecture would be roughly the same across CD, floppy, and HD (other than specifics, like which file system is implemented): a 512byte sector that knows where to find a second stage that knows how to read filesystems/set up for pmode. Seems like no-emulation would make it so you'd need a completely different CD specific bootloader that does all of that?

Thanks so much for your help!
Sean

Posted: **Sat Oct 22, 2011 5:46 pm**

A floppy is too small to contain all the files of even a hobby operating system. On the other hand, to use a hard disk you have got to have a working file system, and a file system is unlikely to be the first thing anybody develops, unless they use an already existing one, like ext3.

So that leaves CDs/DVDs, and who says they are deprecated? I still use a lot of them, and only occasionally have recourse to a USB stick.

There isn't much need to worry about the internal workings of El Torito, the utility which puts the boot loader on the CD (or in the ISO image) for you will take care of the technicalities. Personally I do not use emulation. After writing an operating system, going back to write a different loader for hard disks will seem a fairly trivial task.

The entire bootloader is loaded; not just the first sector. Whether or not you need more than one stage in your loader will depend upon the design of the loader.

You will need to know about the Primary Volume Descriptor and directory structure when you start loading files other than the boot loader from the CD.

Posted: **Sun Oct 23, 2011 4:15 am**

Casm wrote:A floppy is too small to contain all the files of even a hobby operating system.

Well, I wouldn't say that. Look at MenuetOS:

http://www.menuetos.net

A graphical desktop operating system with USB, TCP/IP and various programs included, all on a floppy disk. Sure, you can't fit many media files on there, but when it comes to raw code, there's a LOT you can do in 1.44MB.

M

Posted: **Sun Oct 23, 2011 5:53 am**

theseankelly wrote:1) First and foremost, is any of the information/understanding I've articulated above inaccurate?

Yes,

3) Tool support is deprecated.

Not sure about the Win7-x64 but we do have lots of tools than we'd a decade ago.

2) Is el-torrito the best route to take until my OS is ready to implement real filesystems/be installed on a hard drive (and if I'm not scared of taking on a slightly more advanced FS than fat12)?

We already had several arguments over the best method of booting an OS. Some smart people think that 'Network booting' is the best way to go, while other prefer 'Booting from CD'. Evaluate your requirements and make your own decision.

3) Using el-torrito, is there any size limit on the boot-loader anymore (other than the limitations of 0x7C00 up until the end of free low-memory itself)? Does the El-Torrito method load the entire file specified by the -b flag, or does it only load one sector?

El-Torrito is just a standard for 'Booting from CD'. It includes things like Floppy Emulation, Hard Disk Emulation or No Emulation. Under No Emulation mode, the rest of the process involves parsing the ISO 9660 filesystem.

A typical CD bootsector is 2KB is size. Moreover, the limitation of having to load the bootsector at address 0x7c00 is theoretically dropped. You can specify the load address while you're burning the image off to the disk(MagicISO allows you to supply the load address).

4) Is the concept of a multi-stage bootloader still needed in el-torrito? If so, why?

That depends upon your implementation. What stuffs would you like to get done before passing control to the kernel? Can you code it under the 2 KB limit? If you can't, multi-stage bootloader comes handy.

5) I'm a little bit confused as to how the boot information table works in El-Torrito. The wiki seems to suggest that the mkisofs utility takes "-b /path/to/loader" and installs that file into the boot sector, and if passed proper parameters fills in the info like bi_PrimaryVolumeDescriptor. My question is...why do we care? If this file is already loaded into memory, this info doesn't really do anything for us unless we want to self-relocate (and why do that?)?

The Primary Volume Descriptor is an important component of the ISO9660 filesystem. It's where you can find the block size of your media. However, most of the info can be simply ignored.

6) There's example source code on the El-torrito wiki that shows a simple boot-loader with [ORG 0x7C00] but there's a post on the forum that says "the boot record on the CD can ask the BIOS to load 400 KiB at 0x00006000, instead of just one sector that is always loaded at 0x007C00".

That is true, as defined by the standard, though I haven't tried it myself. However, 2 KB boot-sector is commonly used. And yes, you can ask the BIOS to load your boot-sector to any lower memory.

Is this a discrepancy, or am I getting confused between emulation and no emulation?

Under emulation, the boot-sector complies with the sector size of the emulated device, generally, 512 bytes per sector.

7) Speaking of emulation, is there any value to using emulation instead of no-emulation? The impression I get is emulation is an older practice, but at the same time I could see emulation being useful because the bootloader architecture would be roughly the same across CD, floppy, and HD (other than specifics, like which file system is implemented): a 512byte sector that knows where to find a second stage that knows how to read filesystems/set up for pmode. Seems like no-emulation would make it so you'd need a completely different CD specific bootloader that does all of that?

I'd recommend No emulation.It is quite good that you've same standard followed all over the disk. You can even write a piece of code that can perform emulation, if desired, at the run time, while the disk is actually burned as No Emulation.

Posted: **Sun Oct 23, 2011 7:16 am**

Hi,

theseankelly wrote:1) FAT12 is old and deprecated.

Agreed. Not necessarily because FAT (including FAT16 and FAT32) is old and deprecated; but because there's no sane file permissions, the file size limits are too limiting, the timestamps lack precision, etc.

theseankelly wrote:2) Floppies are old and deprecated; people boot off CDs (which are becomming also-deprecated in favor of USB sticks now...)

Floppies are old and deprecated, but they're relatively simple and can be emulated by most things (including bootable CDs that emulate floppies).

theseankelly wrote:3) Tool support is deprecated. I'm devving on Win7x64 through cygwin. There don't seem to be any virtual floppy emulators that work on Win7x64, and I'd rather not deal with a linux VM just to mount/write to a floppy

Why do you need a virtual floppy emulator? Typically you only need to be able to generate a file (that can be used as a floppy image, and copied to a floppy with something like the Rawrite utility or used "as is" by emulators).

theseankelly wrote:4) Tool support is less convenient.

For floppy images you typically don't need any tools because there's no required structure. I've seen people create floppy images using the (*nix) "dd" command, with the concatenation feature of the DOS "copy" command, with simple assembly (e.g. "incbin" the binary files at the right places), etc. For CD images you're required to use (a variation of) ISO9660, which too complex/messy to be done with simple tools, so you have need to find a tool (like mkisofs) that's specially designed to handle the mess.

theseankelly wrote:2) Is el-torrito the best route to take until my OS is ready to implement real filesystems/be installed on a hard drive (and if I'm not scared of taking on a slightly more advanced FS than fat12)?

No. You should be designing the OS so that you can easily use any one of many boot loaders. Basically you shouldn't be asking "floppy or hard disk or CD", but should be thinking "floppy and hard disk and CD (and PXE and anything else anyone can think of)".

For booting from CD, "no-emulation" El-Torito is probably the best way of doing it (but it is a little more work than just re-using a floppy boot loader with "floppy emulation" El-Torito, or just re-using a hard disk boot loader with "hard disk emulation" El-Torito).

For hard disks, you do not need a file system at all. You can boot from sequential raw/unformatted sectors. Later on, you can design your file system so that the first "n" sectors of the partition are reserved for boot code and continue using the same "boot from sequential sectors" hard disk boot loader. This is actually the way I prefer to do it, as it means you don't need to worry about designing the file system until later, and can also have several completely different file systems using the same hard disk boot loader.

theseankelly wrote:3) Using el-torrito, is there any size limit on the bootloader anymore (other than the limitations of 0x7C00 up until the end of free low-memory itself)? Does the El-Torrito method load the entire file specified by the -b flag, or does it only load one sector?

For El-Torito, the boot file size limit is 65536 sectors (using virtual 512-byte sectors and *not* 2048-byte sectors), which is 32 MiB. You can't actually use a file that size for "no-emulation" on 80x86 (due to the "end of free low memory" problem that you already know about). For floppy and hard disk emulation you get the same limit (e.g. for "hard disk emulation" you're limited to an emulated hard disk that is 32 MiB or less).

theseankelly wrote:4) Is the concept of a multi-stage bootloader still needed in el-torrito? If so, why?

Depends. In my opinion, for a well designed OS, the first stage depends on the type of boot device while the second stage contains code that doesn't depend on the boot device. For example, you'd have a "floppy boot loader" and a "CD boot loader", and both of these would start the exact same "second stage".

theseankelly wrote:5) I'm a little bit confused as to how the boot information table works in El-Torrito.

It doesn't. The boot information table is something that mkisofs made up, and isn't part of any standard (and isn't part of El-Torito). To be honest, I think mkisofs slapped in an ugly hack (their "boot information table") for a specific boot loader (that was written by someone who was too lazy to do things right).

theseankelly wrote:6) There's example source code on the el-torrito wiki that shows a simple bootloader with [ORG 0x7C00] but there's a post on the forum that says "the boot record on the CD can ask the BIOS to load 400 KiB at 0x00006000, instead of just one sector that is always loaded at 0x007C00". Is this a discrepancy, or am I getting confused between emulation and no emulation?

I don't see any discrepancy. The boot loader can ask to be loaded at any "real mode starting segment", including both 0x07C0:0x0000 (or 0x00007C00) and 0x0600:0x0000 (or 0x00006000) and anything else. It is recommended that you use the segment 0x07C0 though, because some BIOSs have bugs and don't correctly support alternative load addresses.

Cheers,

Brendan

Posted: **Sun Oct 23, 2011 12:37 pm**

theseankelly wrote:1) FAT12 is old and deprecated. This will never be my OS's main file system, so why spend time on it now if there are other options that aren't terribly more complicated and are more useful (ISO)

FAT1x can be useful to training FS manipulations that change its structure. FAT16 is still actual for USB sticks and small disk partitions. Now "my OS's main file system" is FAT32. It's not progressive too but I have on-device FS independent kernel therefore I not so worry about it.

2) Floppies are old and deprecated; people boot off CDs (which are becomming also-deprecated in favor of USB sticks now...)

CD/DVD is more suitable for me because I still have no USB drivers. I use USB drives only to load kernel stubs that don't work with disks in PM.

3) Tool support is deprecated. I'm devving on Win7x64 through cygwin. There don't seem to be any virtual floppy emulators that work on Win7x64, and I'd rather not deal with a linux VM just to mount/write to a floppy
4) Tool support is less convenient. Even if I ignore #3 and go the VM route (or use linux natively even) it doesn't seem like the floppy image will ever be as easy as a CD image. You can just throw a mkisofs command into a makefile to generate your bootable image ready for VM testing, whereas you'd need to actually mount a virtual floppy on a virtual floppy drive, copy files, unmount it, and then fire up the VM. I suppose you could put all those commands into a makefile, but it'd be very environment-specific compared to the mkisofs approach.

Try my mkfloppy sources.

2) Is el-torrito the best route to take until my OS is ready to implement real filesystems/be installed on a hard drive (and if I'm not scared of taking on a slightly more advanced FS than fat12)?

CDFS (including El Torito) support is useful but you should have some additional r/w FS. With only CDFS support you even can't make good Live CD or OS distribution CD.

4) Is the concept of a multi-stage bootloader still needed in el-torrito? If so, why?

Modern systems use stage2 boot loader not only because boot block can have small size in some FS. Using stage2 makes boot process more flexible. You can transfer specific parameter string to the kernel, you can load so much modules as you need, you can load specific version of kernel and so on. My kernel can be loaded directly by stage1, or by native stage2, or by Multiboot-compatible stage2 (additional PM entry point is used).

5) I'm a little bit confused as to how the boot information table works in El-Torrito. The wiki seems to suggest that the mkisofs utility takes "-b /path/to/loader" and installs that file into the boot sector, and if passed proper parameters fills in the info like bi_PrimaryVolumeDescriptor. My question is...why do we care? If this file is already loaded into memory, this info doesn't really do anything for us unless we want to self-relocate (and why do that?)? (Afterthought: I guess bi_PrimaryVolumeDescriptor is quite useful, but what about the boot file location and length)

El Torito doesn't define fixed location of "boot sector". It defines the link where "boot sector" is located.

6) There's example source code on the el-torrito wiki that shows a simple bootloader with [ORG 0x7C00] but there's a post on the forum that says "the boot record on the CD can ask the BIOS to load 400 KiB at 0x00006000, instead of just one sector that is always loaded at 0x007C00". Is this a discrepancy, or am I getting confused between emulation and no emulation?

El Torito uses same fields to describe floppy/hard disk images for emulation modes and boot loader image for "No emulation" mode. In first case you should specify 1 as Sector Count and 0/7C0h as Load Segment, in second case you can specify any suitable values but to be sure in success you should specify 4 and 0/7C0h. Value 8 as Sector Count is used in Windows Vista/Seven. It's reliable too.

7) Speaking of emulation, is there any value to using emulation instead of no-emulation? The impression I get is emulation is an older practice, but at the same time I could see emulation being useful because the bootloader architecture would be roughly the same across CD, floppy, and HD (other than specifics, like which file system is implemented): a 512byte sector that knows where to find a second stage that knows how to read filesystems/set up for pmode. Seems like no-emulation would make it so you'd need a completely different CD specific bootloader that does all of that?

No. I use only "No emulation" mode to boot my OS. Emulation modes are bad suitable for PM OSes.

Posted: **Sun Oct 23, 2011 4:07 pm**

Hey all, thanks for a lot of very helpful insight! Some follow up questions:

From Brendan's post:

Why do you need a virtual floppy emulator? Typically you only need to be able to generate a file (that can be used as a floppy image, and copied to a floppy with something like the Rawrite utility or used "as is" by emulators).

Well I've been getting by with just dd'ing raw files to a floppy image, but I thought that if I wanted to have a filesystem I'd need to mount/format the floppy (or write my own C program that mimics formatting -- is that what your mkfloppy code does, egos?) Or at least mount it to copy files over to the disk. I guess that's not true?

No. You should be designing the OS so that you can easily use any one of many boot loaders. Basically you shouldn't be asking "floppy or hard disk or CD", but should be thinking "floppy and hard disk and CD (and PXE and anything else anyone can think of)".

I suppose I should clarify -- I'm not designing my OS around my boot method. Long term, yes the OS will support many different booting options. However in the short term I figured I'd get one of them (floppy or CD) working and working well, and then focus on the kernel. Then when I'm ready to expand, go back and write a HD bootsector and add FS support as needed. Until then, why write the HD bootsector now if I won't be testing it for many months?

Depends. In my opinion, for a well designed OS, the first stage depends on the type of boot device while the second stage contains code that doesn't depend on the boot device. For example, you'd have a "floppy boot loader" and a "CD boot loader", and both of these would start the exact same "second stage".

That's a goal I'm shooting towards so I'm glad it's doable -- so how would you handle different filesystems in that second stage? My best guess is to have an install utility that modifies some values in stage2 so you can find the kernel based on sector instead of parsing a FS?

It doesn't. The boot information table is something that mkisofs made up, and isn't part of any standard (and isn't part of El-Torito). To be honest, I think mkisofs slapped in an ugly hack (their "boot information table") for a specific boot loader (that was written by someone who was too lazy to do things right).

So to obey standards, I'd be better off just ignoring this "feature" of mkisofs entirely?

And from Egos' post:

CDFS (including El Torito) support is useful but you should have some additional r/w FS. With only CDFS support you even can't make good Live CD or OS distribution CD.

That brings up a question that's been lingering in the back of my mind -- how DO Live CDs work when it comes to writing? Like with linux, I can write files to the desktop but of course they aren't actually writing to the disc. Are they just using ramdisks?

So, understanding that there's no correct answer and based on your feedback, here's the approach I think I'd like to take in my short term development -- could you let me know if there are any glaring design flaws?
1) I'm going to choose to boot from CD rather than floppy images -- just a choice of preference based on your feedback
2) I'll have a two stage bootloader, the first stage being traditional 512bytes. Even though I'm NOT going to use emulation, this will keep bootloader design a bit more consistent across boot media in the future
3) Since I'm booting from CD, I'll implement the ISO filesystem and then work on memory/process management, etc. Once my core kernel is more solid, I'll begin thinking about installation to a HD, and the associated filesystems that go with that, and will revisit the bootloader appropriately

Thanks very much, once again -- I haven't seen too many forums with people as quick to help as you all are!
Sean

Posted: **Sun Oct 23, 2011 5:11 pm**

theseankelly wrote:That brings up a question that's been lingering in the back of my mind -- how DO Live CDs work when it comes to writing? Like with linux, I can write files to the desktop but of course they aren't actually writing to the disc. Are they just using ramdisks?

They can detect known r/w FSes on some disks and can use them for storing data. In my OS I use RAM disks only as option to store temperal files.

2) I'll have a two stage bootloader, the first stage being traditional 512bytes. Even though I'm NOT going to use emulation, this will keep bootloader design a bit more consistent across boot media in the future

Stage1 is device and FS specific boot loader. You can use 2 kb boot loader for CD/DVD. I have stage1 with size 512 bytes for FAT1x (they are two different for floppies/FAT12 and disks/FAT16), with size 1 kb for FAT32, with size 2 kb for CDFS.

3) Since I'm booting from CD, I'll implement the ISO filesystem and then work on memory/process management, etc. Once my core kernel is more solid, I'll begin thinking about installation to a HD, and the associated filesystems that go with that, and will revisit the bootloader appropriately

That's good. Just make unified boot spec. for your OS now to simplify your work in the future. I brought out my Boot Spec. more than ten years ago and now it's still actual and powerful.

Posted: **Sun Oct 23, 2011 6:11 pm**

is that what your mkfloppy code does, egos?

Not exactly. My mk* sources allow to generate disk images with FS structure (files and directories) by compilling sources by fasm. For example (don't take attention to bulky MBR markup, it's experimental; look at FAT16 partition markup):

Code: Select all

PART_STATE = 80h
PART_TYPE = 4
HEADS = 16
SPT = 63

file "../bootcode/disk/mbr/alter-2/alter-2.bin",440
db ?,?,?,?,0,0
db PART_STATE,1,1,0
db PART_TYPE,HEADS-1,(CYLINDERS-1) and 300h shr 2 + SPT,(CYLINDERS-1) and 0FFh
dd SPT,(CYLINDERS*HEADS-1)*SPT
db 3*16 dup 0
dw 0AA55h

rb SPT*512-$

include "mkfat16.inc"

IP_MEDIA = 0F8h
IP_SPT = SPT
IP_HEADS = HEADS

orgimage "../bootcode/disk/fat16/boot.bin"

orgdir root
dent linkzimage,"LINK    PK",FA_ARC
findir root

stof linkzimage,"../tools/sysinfo/sysinfo.bin"

finimage MINSIZE,0

db HEADS*SPT*512-1 - ($+HEADS*SPT*512-1) mod (HEADS*SPT*512) dup 0

CYLINDERS = $/(HEADS*SPT*512)

Markup style provided by mkfloppy is not so good (as in mkfat16) but I will improve it in the near future.

Posted: **Mon Oct 24, 2011 9:23 am**

Stage1 is device and FS specific boot loader. You can use 2 kb boot loader for CD/DVD. I have stage1 with size 512 bytes for FAT1x (they are two different for floppies/FAT12 and disks/FAT16), with size 1 kb for FAT32, with size 2 kb for CDFS.

Do you parse a FAT12/16 filesystem in 512 bytes, or calculate where stage two is on disk at install time (or boot media creation time)?

This statement implies that Stage2 is device and FS non-specific -- is that true, generally? If so, how does stage2 find the Kernel? I guess that could be solved in two ways -- 1) stage1 is responsible for finding the kernel in the same way it finds stage2, or 2) stage2 is FS non-specific in that it has support for ALL possible filesystems and can detect which it's dealing with, and behave appropriately?

That's good. Just make unified boot spec. for your OS now to simplify your work in the future. I brought out my Boot Spec. more than ten years ago and now it's still actual and powerful.

Unified boot spec meaning...standardized place for the bootloader to put system information that the kernel can find?

Also, should I consider editing the wiki in some minor ways to add clarification on the points discussed here? If I had these questions and couldn't find the answer, I'm sure others will struggle through this in the future. Or is wiki-editing (understandably) reserved for the more senior contributors?

Posted: **Mon Oct 24, 2011 11:39 am**

theseankelly wrote:Do you parse a FAT12/16 filesystem in 512 bytes, or calculate where stage two is on disk at install time (or boot media creation time)?

I parse FAT in my stage1 (when GRUB is not used). For FAT1x I use special methods to put the code in such small space (FAT12: using short BPB introduced in some earlier version of DOS, using hard-coded constants; FAT16: using one *.pk file which contains concatenated both kernel file and boot device and FS driver file). I use the structure stored within stage1 as alternative for FS parsing only for CDFS because stage1 that parses CDFS structure is still crude.

This statement implies that Stage2 is device and FS non-specific -- is that true, generally? If so, how does stage2 find the Kernel? I guess that could be solved in two ways -- 1) stage1 is responsible for finding the kernel in the same way it finds stage2, or 2) stage2 is FS non-specific in that it has support for ALL possible filesystems and can detect which it's dealing with, and behave appropriately?

Stage2 can use external device and FS specific driver(s). Stage1 can help stage2 to load the driver(s). Stage2 can use basic driver(s) to load additional driver(s). My stage1 loads native stage2 in same way as it loads the kernel, just names are different. The second variant is fully right.

Unified boot spec meaning...standardized place for the bootloader to put system information that the kernel can find?

Generally, yes. It means that boot spec. should describe only device and FS independent generalized interfaces.

Also, should I consider editing the wiki in some minor ways to add clarification on the points discussed here? If I had these questions and couldn't find the answer, I'm sure others will struggle through this in the future. Or is wiki-editing (understandably) reserved for the more senior contributors?

I think you can. I don't do it just because my english is bad.

Posted: **Tue Oct 25, 2011 11:48 am**

Hi,

theseankelly wrote:From Brendan's post:
Why do you need a virtual floppy emulator? Typically you only need to be able to generate a file (that can be used as a floppy image, and copied to a floppy with something like the Rawrite utility or used "as is" by emulators).
Well I've been getting by with just dd'ing raw files to a floppy image, but I thought that if I wanted to have a filesystem I'd need to mount/format the floppy (or write my own C program that mimics formatting -- is that what your mkfloppy code does, egos?) Or at least mount it to copy files over to the disk. I guess that's not true?

For booting from floppy (just like booting from hard drives) you don't need a file system at all, and you can boot from sequential sectors. Unlike hard drives, you'll probably never want a file system on the boot floppy (as the OS's boot files will consume almost all the space anyway).

For a real-life example; my OS needs about 4 files to boot. The boot loader loads the next stage into memory (the first file), then the next stage asks the first stage to load the other files a little later (when they're needed). The important thing is that the next stage doesn't know or care how the boot loader finds/loads those other files.

For my floppy boot loader, there's a table containing the "starting LBA and length" for each of the files at the beginning of the second sector of the boot loader. I have a utility (slapped together in C) to create a floppy image, which concatenates the files (and adds padding between them) and sets the "starting LBA and length" for them in the boot loader's table. The "make floppy" utility also supports different floppy formats (e.g. 1680 KiB floppies, 1440 KiB floppies, 1200 KiB floppies, 720 KiB floppies, etc) and also sets the correct values in the BPB (which the floppy boot loader uses to determine how many sectors per track, how many heads, etc).

Note 1: My "no emulation" CD boot loader is very different - it understands ISO9660 and expects to find the files in the "/boot/" directory. My PXE boot loader just knows the file names and downloads them via. TFTP. My "multiboot" boot loader is different again - the files are already in RAM (loaded as "modules") so when my boot loader is asked to load one of the other files it just says "here it is".

Note 2: The hard disk boot loader (which includes USB flash) would be similar to the floppy boot loader, except it'd probably use "offset within this partition" rather than "starting LBA" in the table. I rarely bother with hard disk boot loaders though, as it involves creating partitions to suit the device (and potentially installing a default MBR to load the OS's boot loader) and nobody wants to trash their hard drive just to test an OS. Eventually I'll create disk partitioning tools for my OS and have some sort of semi-automated (menu driven?) installer - people would download the "OS installation" CD image, burn it to CD, boot from the CD, then use my installer, partitioning tools, etc to install the OS on their hard drive.

Note 3: Don't forget about UEFI. What I described above is "boot loader" and "stage 1.5" for PC BIOS systems. There's also a "stage 2" after this, which runs in protected mode and doesn't know/care what the firmware was (and relies on the previous stage to do anything firmware specific, like getting a memory map and setting up a default video mode). The "stage 2" is responsible for deciding which kernel to start and starting it.

theseankelly wrote:That's a goal I'm shooting towards so I'm glad it's doable -- so how would you handle different filesystems in that second stage? My best guess is to have an install utility that modifies some values in stage2 so you can find the kernel based on sector instead of parsing a FS?

Yes - a utility that sets/modifies some values in the boot loader. However, I still wouldn't allow the OS's boot files to be stored in the file system itself - they'd be stored sequentially in a "reserved" area; and not stored like normal (potentially fragmented) files in the file system itself, where they can be seen by the end user (and possibly trashed accidentally by "root"). Basically a 500 MiB bootable partition might have a 50 MiB reserved area at the start of the partition with a 450 MiB file system in the remainder. Of course you could also easily support a special "boot only" partition type, where there's the reserved area for booting and no file system at all; and unbootable partitions (which don't have the reserved area).

theseankelly wrote:
It doesn't. The boot information table is something that mkisofs made up, and isn't part of any standard (and isn't part of El-Torito). To be honest, I think mkisofs slapped in an ugly hack (their "boot information table") for a specific boot loader (that was written by someone who was too lazy to do things right).
So to obey standards, I'd be better off just ignoring this "feature" of mkisofs entirely?

The standards don't say you can't have mkisofs' "boot information table" (e.g. you can have the table and still be 100% compatible with ISO9660 and El Torito). I personally wouldn't use it though, because I doubt many other CD image utilities support it (and don't see why I'd want to make it hard for someone using Nero or something else) and have never needed it anyway.

Cheers,

Brendan

Posted: **Tue Oct 25, 2011 1:50 pm**

Thanks again.

So then...I'm still not seeing any real advantage to having a bootable floppy image, other than it being easier to implement. In fact, I have implemented this (flat binary floppy) and am wishing to move beyond it to a CD now. I was planning on just deprecating my floppy stuff and not fixing it up, unless there is an advantage to keeping it going that I'm not seeing?

Posted: **Tue Oct 25, 2011 2:28 pm**

Hi,

theseankelly wrote:So then...I'm still not seeing any real advantage to having a bootable floppy image, other than it being easier to implement.

If you support old computers (e.g. early 1990's or older) then you'll come across systems that are only able to boot from floppy or hard disk, and to get an OS onto the hard disk you have to install it from floppy. Note: This can include booting a minimal OS from floppy that includes suitable drivers for the CD or network (and then loading the remaining drivers, etc from elsewhere).

If you don't support old computers; then other reasons include:

Widely supported (in a standardised way) by emulators and real computers that aren't too new (people only really started the "no floppy" thing about 5 years ago, so there's still plenty of "not too old" systems out there)
Floppy images tend to be a bit smaller, which can make it easier for people to download and test
They make a good testing ground for other things. For example, it's not too hard to write a floppy driver and use floppies to test things like partitioning tools and file systems and OS installers without worrying about messing up your hard drive
You get more experience with a wider variety of hardware
Marketing. If you can show that your OS is lean enough to be useful when booted from floppy, then people who don't like "bloated" OSs (Windows, Linux) may be more likely to download the CD and try it.

If you weight up the advantages of supporting floppy (nothing too significant, other than old computers) and disadvantages of supporting floppy (nothing too significant - it's not like a floppy boot loader requires lots of code maintenance or anything), you end up with no clear reason to support floppy and no clear reason not to.

For me, it's more about being thorough - it's like there's a list of hardware that an OS could support, and floppy is an easy one to cross off the "not supported" list.

Cheers,

Brendan

Posted: **Tue Oct 25, 2011 3:44 pm**

theseankelly, floppy is a legacy storage device. My floppy boot loader was written about ten years ago in my pupilage. But it can be used now because it has same boot interface which my modern boot loaders use. You can forget about floppy, or use your flat binary floppy when it needs, or take my floppy boot loader if you need bootable formatted floppy and its interface is suitable for you. Your wish is right. As Brendan said the good way is to realise OS distribution CD (CDFS support is welcomed). In first time it even could be a Demo CD without partitioning tool, r/w storage device and FS support. Floppy just for fun

OSDev.org

Bootloader: Floppy vs HD vs CD

Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD

Re: Bootloader: Floppy vs HD vs CD