OSDev.org

Kevin wrote:

kerravon wrote:It seems neat to me. You can standardize on iso format and burn it to either cdrom or hard disk/USB stick.

Are you sure that you will never want to write data to the hard disk after creating the image?

You can write data so long as the fake mbr actually gets control, as the bios thinks this is a normal hard disk, albeit not a fat.

Well, as long as never need to append to an existing file. If you bump into another file, you will then have to move the entire damn thing. Doesn't sound very useful.

Gigasoft wrote:Well, as long as never need to append to an existing file. If you bump into another file, you will then have to move the entire damn thing. Doesn't sound very useful.

I don't think we're on the same page.

I'm not sure what you're visualizing because I'm not familiar with standard CDs.

My proposal is an El torito CD. With a normal hard disk image.

So it is a completely normal fat. The os will be able to extend files normally.

However, forget iso for a moment. The disk will start, lba 0, with a fake mbr. It doesn't matter what the id is, noone is meant to recognize or use it.

The 8086 code in the fake mbr knows where the El torito hard disk image, thus the real mbr, starts, and loads that.

It needs to mess around with a fake dl so that the real mbr knows that it has been invoked from a fake mbr, not the bios booting a CD, and as such, the extra hidden sectors need to be tracked during boot and then by the os.

The vbr normally contains the number of hidden sectors from the real mbr, but in my case, when booting from hard disk, not cd, that number will be incorrect, because there is an extra 40960 bytes, starting with the fake mbr, which is part of the iso reserved area which is normally NULs.

The 8086 code in the vbr will need to be aware of this, again via a fake dl.

And the os, which already has knowledge of hidden sectors, will have that count increased if it was invoked via the fake mbr chain.

Let me know if that still doesn't make sense.

kerravon wrote: Let me know if that still doesn't make sense.

It just seems to be a terrific amount of complication with no gain. What’s wrong with the conventional hard disk layout?

iansjack wrote:

kerravon wrote: Let me know if that still doesn't make sense.

It just seems to be a terrific amount of complication with no gain. What’s wrong with the conventional hard disk layout?

Why do you think it is complicated? It's probably 100 lines of code.

The gain is that you only need to ship an iso.

Other people only ship an iso too, but they have a lot of code to achieve that. Grub emulating a CD or something.

I don't know why iso didn't standardize CDs on conventional hard disk layout, which is what I would have preferred, but given that they didn't, this is the neatest image to ship, in my eyes.

kerravon wrote:Other people only ship an iso too, but they have a lot of code to achieve that. Grub emulating a CD or something.

GRUB doesn't emulate anything. It uses the CD's ISO 9660 filesystem regardless of boot medium.

It's actually pretty similar to what you're doing, but without any hard disk images.

kerravon wrote:I don't know why iso didn't standardize CDs on conventional hard disk layout, which is what I would have preferred, but given that they didn't, this is the neatest image to ship, in my eyes.

There was no standard hard disk layout when CDs were standardized, and even if there had been, it wouldn't make sense for a CD since CDs could be written only once.

Octocontrabass wrote:

kerravon wrote:Other people only ship an iso too, but they have a lot of code to achieve that. Grub emulating a CD or something.

GRUB doesn't emulate anything. It uses the CD's ISO 9660 filesystem regardless of boot medium.

Then it presents that as a read-only hard disk?

It's actually pretty similar to what you're doing, but without any hard disk images.

Grub is a lot of code compared to the minimal code I need.

kerravon wrote:I don't know why iso didn't standardize CDs on conventional hard disk layout, which is what I would have preferred, but given that they didn't, this is the neatest image to ship, in my eyes.

There was no standard hard disk layout when CDs were standardized, and even if there had been, it wouldn't make sense for a CD since CDs could be written only once.

I don't follow. First of all, hard disks were available in the early 1980s. I believe CDs were standardized in the late 1980s.

I don't know why they couldn't just add whatever extra stuff they wanted after the MBR.

And what's wrong with a read only hard disk?

kerravon wrote:Then it presents that as a read-only hard disk?

It does not present anything as anything. A CD is a CD, a hard disk is a hard disk, a USB stick is whatever the firmware chooses to present it as (which is very rarely a CD, historically often a floppy disk, and in the waning days of BIOS systems would have been a hard disk) up until the OS prods the magic handover bits to say it knows what the hell USB even is.

ISO9660 doesn't use the first CD sector of a medium: That's a whole 2048 bytes you can just do whatever you want with. What GRUB chooses to do is put a traditional boot sector in the first 512 bytes. This is a very common strategy and is called an isohybrid. El Torito specifies how BIOS may locate a boot payload on a CD - actually, it specifies how firmware in general can locate several boot payloads, as it was designed with the thought of supporting multiple architectures. With an El Torito isohybrid, you could have entirely different codepaths between your MBR and your El Torito boot payload - one taken by hard disks and floppy disks, the other by actual CDs - but there's no reason to do so: The BIOS disk read functions are generic - ah=42h will read a CD, a hard disk, a floppy disk - it doesn't care. You do need to query the sector size, though, as CDs use 2048 byte sectors and hard disks use 512 (or at least pretend to), but with a bit of division or multiplication this difference is irrelevant. So, GRUB, and most other isohybrid bootloaders, have a common second stage and both the MBR and the El Torito boot payloads load the same thing and continue on their merry way. GRUB is big and complicated, and after loading the second stage it will have filesystem drivers so it can read ISO9660 or FAT or EXT or various other things, but it does not care much about the medium they are on: it continues to jump back to the BIOS to perform disk operations until/unless a native driver is loaded for a storage mechanism.

One neat thing about El Torito payloads is that they can be more than 512 bytes. How much more is uncertain - a BIOS may not support loading hundreds of sectors, for example, but it may be enough that your El Torito payload can be the second stage, and your MBR boot code can load your El Torito payload from a fixed offset. This is how I do things - my El Torito boot payload is a whopping 25K, containing my whole generic BIOS loader.

Another neat thing that comes from El Torito's support for multiple boot payloads is that EFI specifies an El Torito payload to make EFI-bootable CDs. Unfortunately, it specifies that payload be a tiny FAT partition image, which can be a bit of a pain. Another trick I pull is to set up that FAT image for 2048 byte sectors, put all of the files I would normally put on my CD in it (which used to be several but is now just a kernel and ramdisk), ensuring they are contiguous, give the CD empty files with the right names, and then patch up the ISO filesystem afterwards with the offsets and sizes of the files in the FAT - EFI presents the FAT filesystem through its file-oriented disk APIs, and my BIOS loader sees the ISO9660 filesystem, but they both get the same files! As a special case of this, one particular build of my OS also has a file in the ISO that spans the whole disk (something ISO9660 is perfectly happy to do!), that just so happens to have "PDF" as its file extension, but that is somewhat off-topic for this thread.

Octocontrabass wrote:Syslinux uses 0x17. [...] Some BIOSes will examine the partition table and misbehave or refuse to boot if a partition starts at LBA 0.

For isohybrids, a partition ID of 0xCD is used with a claimed start LBA of 1. I think the original isohybrid implementation would throw in a fake partition with ID 0x17, which is supposedly assigned as a Windows "internal filesystem". GRUB ISOs are readable when written to a hard disk, at least under Linux, so either some devious probing is happening or the fake start LBA is understood.

My own hand-crafted isohybrids do not present any partitions, but seem to work on most hardware I've tried them on when written to USB sticks and booted as hard disks - but I'm probably just lucky!

kerravon wrote:Grub is a lot of code compared to the minimal code I need.

GRUB is a lot of code compared to the minimal code anyone needs. If you want to compare against a simple bootloader, look at Syslinux instead of GRUB. (Syslinux can also boot hybrid ISOs, and it works pretty much the same way GRUB does.)

kerravon wrote:I don't follow. First of all, hard disks were available in the early 1980s. I believe CDs were standardized in the late 1980s.

Yes, and in the late 80s there were still multiple types of computers, each with their own hard disk format. The DOS MBR was only standard for IBM-compatible PCs.

kerravon wrote:And what's wrong with a read only hard disk?

Hard disk filesystems keep track of free space. Keeping track of free space uses some of the disk's storage capacity, and storage is expensive. A read-only disk has no free space.

ISO is just a partition on that hard disk, the hard disk remains a hard disk. You can always write to it.
it may be read only in some (inexistent) BIOSes if you use the BIOS to access it, but as soon as you create a driver you will have full access.

I never heard of a hard disk (HDD, USB...) that is read-only

You can try the FAT32 File System, it is easy and universal. I make my testing USB and HDD Image with my own driver, and it works perfectly with Windows.

Devc1,

devc1 wrote:ISO is just a partition on that hard disk, the hard disk remains a hard disk. You can always write to it.
it may be read only in some (inexistent) BIOSes if you use the BIOS to access it, but as soon as you create a driver you will have full access.

I never heard of a hard disk (HDD, USB...) that is read-only

You can try the FAT32 File System, it is easy and universal. I make my testing USB and HDD Image with my own driver, and it works perfectly with Windows.

Devc1,

I do use fat32.

I just wrap it in iso so that I can put it on a cdrom.

The proposal is then adding minimal code to the iso to enable the iso to be written to a real hard disk and still go through legacy boot.

And did you succeed in it or you need some help ?

devc1 wrote:And did you succeed in it or you need some help ?

I was after the iso partition number which I now have.

I wasn't given any suggestions for fake dl and hidden sectors, so I'll just make that up myself when I start development.

Making my fdisk create a vhd is the first step along the path and I am currently thinking about that.

I have some vhd created on windows 7. One of them is 470 mb in size suitable for the cd.

I need to fix some aspects of my CD creation program too.