BIOS INT 0x13 partial reads

[Octocontrabass]

I think what is more important to this conversation is what existing BIOS implementations do, not what our opinion is for future implementations.

Existing BIOS implementations (at least the ones I've looked at) blindly send the command to the drive and return an error when the drive rejects it.

I've also came across a BIOS (tested in PCem) that would return the 20 bytes of the first entry and 4 bytes of the second one when requesting 24 bytes.

Yikes. I'd like to know which BIOS that is.

So now the question is, should all read errors on these devices be considered fatal and not retry any failed read operation?

Ancient hard drives were as dumb as floppy drives, so retrying after a read failure may have been useful back then. Modern drives automatically retry failed reads, so there's no point in retrying the BIOS call.

For drives that misalign the start of the logical-to-physical mapping, wouldn't both partitions and filesystem blocks be misaligned?

Only if the partitioning software is unable to detect the characteristics of the disk.

Second, is there a reliable way to determine the correct alignment for a disk?

It depends on the type of disk. For IDE/SATA disks that speak the ATA Command Set, the alignment is reported in the IDENTIFY DEVICE output in word 209.

[zerodivision]

Yikes. I'd like to know which BIOS that is.

AMIBIOS, at least versions 1.00.09.AF2 and 1.00.18.CS1 (quite ironically, the latter is the one used by the aforementioned Intel VS440FX). This breaks the general advice to request 24 bytes to be ready for ACPI 3.0 entries. I think the correct solution is to first check for ACPI 3.0 and only then request 24 bytes. Otherwise request 20 bytes. However, detecting ACPI is not a beginner-friendly solution to put on the wiki.

Ancient hard drives were as dumb as floppy drives, so retrying after a read failure may have been useful back then. Modern drives automatically retry failed reads, so there's no point in retrying the BIOS call.

How much ancient?

For IDE/SATA disks that speak the ATA Command Set, the alignment is reported in the IDENTIFY DEVICE output in word 209.

Noted. However, the bootloader may reside also on SCSI drives, USB flash drives, and others. The bootloader would have to establish which device it resides on (apart the DL number), then implement some minimal driver functionality to communicate with the respective controller to determine alignment. Thus I don't know how much practical is it.

[Octocontrabass]

How much ancient?

MFM, RLL, ESDI, and pre-ATA IDE (1993-ish).

bootloader

A bootloader has no reason to use multisector reads in the first place.

[Octocontrabass]

AMIBIOS, at least versions 1.00.09.AF2

I disassembled the INT 0x15 handler in this BIOS, and... yeah, I see the problem.

I think the correct solution is to first check for ACPI 3.0 and only then request 24 bytes. Otherwise request 20 bytes. However, detecting ACPI is not a beginner-friendly solution to put on the wiki.

An easier solution is to request 24 bytes for the first entry and examine bit 0 of the extended attributes. Thanks to the backwards compatibility problems mentioned on the wiki, bit 0 of the extended attributes must always be set (and ACPI 4.0 redefined it to always be set anyway), so if the BIOS writes a value with bit 0 clear or doesn't write anything at all, the BIOS only supports 20-byte entries, and you need to request the first entry again.

[BenLunt]

Existing BIOS implementations (at least the ones I've looked at) blindly send the command to the drive and return an error when the drive rejects it.

Then, in my opinion, that is a faulty BIOS. Doing anything blindly defeats the purpose of a robust BIOS interface. A well written BIOS will first check to see if the request is within range of the media, as well as possibly making other checks before requesting sectors from the hardware.

A bootloader has no reason to use multisector reads in the first place.

Please expand on this. If the bootloader is loading from a FAT volume, the FAT itself, the root, and then the 2nd (or 3rd) stage loader will most likely be more than a single sector in size, making multisector reads a must.

Maybe you are thinking of the MBR code, which in most cases will need to load a single sector. However, some FAT volumes have more than one sector for the Volume Boot Record (VBR), the ext2 has two sectors, the LeanFS has up to 33 sectors, as well as others may and usually will have multisector VBRs.

Ben
- https://www.fysnet.net/osdesign_book_series.htm

[zerodivision]

MFM, RLL, ESDI, and pre-ATA IDE (1993-ish).

I don't intend to support MFM, RLL and ESDI drives as they are too obsolete. I'm undecided on pre-ATA-standardization IDE drives (if you refer to these). If they don't have too many differences from post-ATA-standardization IDE drives and their reliability isn't too bad, I'd probably want to support them because it wouldn't be too much additional effort.

A bootloader has no reason to use multisector reads in the first place.

It's not necessary, but can be more efficient. I have already implemented splitting the read requests on track boundaries for CHS (undecided for LBA, either at 127 sectors, or at 120 sectors, or at LBA % 8 == 0), so the slight added complexity isn't an issue. Besides, the case where read requests can fail applies to both singlesector and multisector reads and as such there isn't much if any better reliability achieved by reading sector-by-sector. Unless I'm missing something?

Maybe it's not unreasonable to make the assumption that on 512e drives the filesystem has been formatted in 4K blocks that are properly aligned. Thus reading at most 15 such consecutive blocks (120 consecutive 512-byte logical sectors) will be a properly aligned request.

An easier solution is to request 24 bytes for the first entry and examine bit 0 of the extended attributes. Thanks to the backwards compatibility problems mentioned on the wiki, bit 0 of the extended attributes must always be set (and ACPI 4.0 redefined it to always be set anyway), so if the BIOS writes a value with bit 0 clear or doesn't write anything at all, the BIOS only supports 20-byte entries, and you need to request the first entry again.

If I understand your method correctly, I suppose it will work in practice. I think it's very unlikely that a memory region would start on an odd byte, thus bit 0 is expected to always be clear if the BIOS returns 24 bytes despite an entry consisting of 20 bytes. Except that now bit 0 being clear doesn't mean that the entry ought to be ignored, but that 20-byte entries should be requested instead.

However, for my OS I still intend to check the presence of ACPI and its version because that information will be passed to the kernel anyway.

Then, in my opinion, that is a faulty BIOS. Doing anything blindly defeats the purpose of a robust BIOS interface. A well written BIOS will first check to see if the request is within range of the media, as well as possibly making other checks before requesting sectors from the hardware.

I agree with this, however it often doesn't happen in practice. In fact, no BIOS among those I tested with PCem seems to be doing it, otherwise they wouldn't return success with unmodified PCem. There is an argument to be made that these BIOSes were written over 30 years ago and that modern BIOSes are likely more robust, but one must not rely on that.

Would you please tell me whether your BIOS supports the configuration of the Intel VS440FX machine in PCem? If not, would you please suggest another configuration to try? Also, have you tested your BIOS with QEMU and, if so, how do you run it? I'm also curious to see the results.

[BenLunt]

Would you please tell me whether your BIOS supports the configuration of the Intel VS440FX machine in PCem? If not, would you please suggest another configuration to try?

As I have stated before, it is for the i440fx.

Also, have you tested your BIOS with QEMU and, if so, how do you run it? I'm also curious to see the results.

At the bottom of the git page, it has a link to the documentation, which shows exactly how to use it in Bochs and QEMU.

So you don't have to build it, I have placed current binaries for both Bochs and Qemu at the git page linked to above.

[BenLunt]

I just created a test myself, trying to read eight sectors from four sectors away from the end of the media. Wouldn't you know it, my BIOS returns Carry clear and AL = 8...... Ouch. I thought for sure I had watched for this. (Anyone got any crow I can borrow?)

SeaBIOS and the BIOS that comes with Bochs returns the same result, however Bochs itself properly reported the error.

Code: Select all

00162303317i[FLOPPY] increment_sector: clamping cylinder to max
00162303317e[FLOPPY] LBA 2880 passed end of disk.
00163903322e[FLOPPY] LBA 2881 passed end of disk.
00165503325e[FLOPPY] LBA 2882 passed end of disk.

I will have to look into this, make a fix, and return and report.

[Octocontrabass]

A well written BIOS will first check to see if the request is within range of the media, as well as possibly making other checks before requesting sectors from the hardware.

Why? If there's a problem with the request, the drive will return an error.

If the bootloader is loading from a FAT volume, the FAT itself, the root, and then the 2nd (or 3rd) stage loader will most likely be more than a single sector in size, making multisector reads a must.

The bootloader can use multiple single-sector reads, so it doesn't need multisector reads.

If they don't have too many differences from post-ATA-standardization IDE drives and their reliability isn't too bad, I'd probably want to support them because it wouldn't be too much additional effort.

The differences are mostly in parts that were considered optional in the first ATA standard. If you're relying on IDENTIFY DEVICE to tell you which optional functions the drive supports, most pre-ATA IDE drives will be fine. Some require parking the heads before powering them off, but that's easy enough to add if you want to support it (though unfortunately it requires someone to manually set the landing zone). Beyond that is the weird niche stuff you probably won't want to bother with.

Some ESDI controllers/drives are also similar enough to ATA-standard IDE that they'll work without doing anything else. (Speaking of ESDI, it is possible to make the STANDBY IMMEDIATE command work on pre-ATA IDE drives...)

It's not necessary, but can be more efficient.

How much faster is it, though? Have you measured? Is the difference really worth writing and maintaining all the extra code? Is there any difference at all?

Maybe it's not unreasonable to make the assumption that on 512e drives the filesystem has been formatted in 4K blocks that are properly aligned.

It doesn't matter, every 512e drive has a read cache that's perfectly capable of servicing sequential read requests of any alignment (or lack thereof).

However, for my OS I still intend to check the presence of ACPI and its version because that information will be passed to the kernel anyway.

Firmware vendors have this annoying habit of mixing tables from different versions of the ACPI specification, so it's hard to get more specific than "ACPI 2.0 or later" or "before ACPI 2.0" without actually parsing all of the tables.

There is an argument to be made that these BIOSes were written over 30 years ago and that modern BIOSes are likely more robust, but one must not rely on that.

I would expect the opposite. The software of 30 years ago relied more on the BIOS than anything today.

Would you please tell me whether your BIOS supports the configuration of the Intel VS440FX machine in PCem?

PCem expects you to provide the update files extracted from one of Intel's BIOS flash programs instead of an actual BIOS ROM, so an actual BIOS ROM isn't going to work.

[zerodivision]

As I have stated before, it is for the i440fx.

I get that, but does it support that configuration of PCI slots? Or the use of that specific BIOS flash memory device?

At the bottom of the git page, it has a link to the documentation, which shows exactly how to use it in Bochs and QEMU.

I finally got it to run with QEMU. But doesn't having to specifically use "-machine q35" make the BIOS not suitable for the i440fx chipset?

I finally tested with your BIOS and it returned CF set and AL = 0, but I don't know whether you had fixed in the meanwhile.

(though unfortunately it requires someone to manually set the landing zone)

That's set in the BIOS, so I assume that the OS can read it from the CMOS. Although not every pre-ATA-standardization IDE drive actually needs it, according to this Vogons forum post.

How much faster is it, though? Have you measured? Is the difference really worth writing and maintaining all the extra code? Is there any difference at all?

I haven't measured it yet, but I'll probably do it at some point when I'll be reading more files. For now, I'll keep multisector reads because I've already implemented them and it's not an enormous amount of extra code anyway.

It doesn't matter, every 512e drive has a read cache that's perfectly capable of servicing sequential read requests of any alignment (or lack thereof).

It doesn't matter as far as functionality is concerned. However, older OSes suffered performance degradation on 512e drives and especially if the filesystem blocks weren't properly aligned.

Well, maybe then it is a reasonable assumption that the filesystem blocks are properly aligned. If they aren't, performance will be degraded the entire time the OS runs and uses the partition. Even if reading in the bootloader is made marginally faster when there are more than 127 consecutive sectors to read (e.g. start from unaligned, end at aligned, start at aligned, end at unaligned), it still won't improve performance after boot.

PCem expects you to provide the update files extracted from one of Intel's BIOS flash programs instead of an actual BIOS ROM, so an actual BIOS ROM isn't going to work.

Several machines use files that are actual BIOS ROMs. The VS440FX is initialized to create a 240K flash memory device but I haven't figured out what to make out of it.

[BenLunt]

A well written BIOS will first check to see if the request is within range of the media, as well as possibly making other checks before requesting sectors from the hardware.

Why? If there's a problem with the request, the drive will return an error.

I don't disagree with you, though as zerodivision and I have found, not all current emulated BIOS implementations return error when trying to read past the end of the media, and in my opinion, it just makes for a more robust firmware.

If the bootloader is loading from a FAT volume, the FAT itself, the root, and then the 2nd (or 3rd) stage loader will most likely be more than a single sector in size, making multisector reads a must.

The bootloader can use multiple single-sector reads, so it doesn't need multisector reads.

That's what I was thinking you meant, just wasn't sure. There was a possibility you meant something I was missing.

As I have stated before, it is for the i440fx.

I get that, but does it support that configuration of PCI slots? Or the use of that specific BIOS flash memory device?

I'm sorry, I don't understand what you are asking. What do you mean by PCI slots? The count of or specific hardware plugged in to them? This shouldn't matter to the firmware, as long as the firmware knows how to parse them, correct? Or do you mean, the specific PCI bridges?

At the bottom of the git page, it has a link to the documentation, which shows exactly how to use it in Bochs and QEMU.

I finally got it to run with QEMU. But doesn't having to specifically use "-machine q35" make the BIOS not suitable for the i440fx chipset?

I noticed this last night. It use to run just fine without it, but a recent change changed that. I am investigating now.

I finally tested with your BIOS and it returned CF set and AL = 0, but I don't know whether you had fixed in the meanwhile.

Is your test the 13h/42h service of the extended services, or is it the 13h/02h CHS services?

Thanks,
Ben

[Octocontrabass]

That's set in the BIOS, so I assume that the OS can read it from the CMOS.

Not CMOS, FDPT.

Although not every pre-ATA-standardization IDE drive actually needs it, according to this Vogons forum post.

Most don't need it. Auto-park was already common before IDE existed.

However, older OSes suffered performance degradation on 512e drives and especially if the filesystem blocks weren't properly aligned.

Yes, because OSes access multiple files at once and perform a mixture of reads and writes. A bootloader that only performs sequential reads of a single file at a time will see no difference.

The VS440FX is initialized to create a 240K flash memory device but I haven't figured out what to make out of it.

The VS440FX has a 256K Intel 28F002BV-T flash ROM. You have to replace all 256K of that ROM to make it work.

I don't disagree with you, though as zerodivision and I have found, not all current emulated BIOS implementations return error when trying to read past the end of the media, and in my opinion, it just makes for a more robust firmware.

Why aren't they returning errors? Are they ignoring an error from the emulated drive? Or is the emulated drive accepting an invalid command?

[zerodivision]

I'm sorry, I don't understand what you are asking. What do you mean by PCI slots? The count of or specific hardware plugged in to them? This shouldn't matter to the firmware, as long as the firmware knows how to parse them, correct? Or do you mean, the specific PCI bridges?

I meant the PCI slots 0x0B, 0x0F, 0x11, 0x13 as defined in the PCem source code, but that shouldn't be a problem if the firmware doesn't use hardcoded values.

Is your test the 13h/42h service of the extended services, or is it the 13h/02h CHS services?

I tested with INT 0x13 AH=0x02.

The VS440FX has a 256K Intel 28F002BV-T flash ROM. You have to replace all 256K of that ROM to make it work.

Defining a new machine based on the VS440FX doesn't work for me, because I'm very likely missing something in the machine definition, initialization or runtime. Modifying the VS440FX machine itself to not use the flash ROM altogether and to load another BIOS works with BenLunt's BIOS up to the point where I select to boot from the hard disk but it says "No boot device specified", although the disk is detected and listed. Using SeaBIOS causes PCem to crash though. Interestingly, using the original BIOS ROMs for the machine continues to work even without the flash ROM.

BenLunt, if you want to test your BIOS with PCem, look for "case ROM_VS440FX:" in mem_bios.c and replace the block that follows with this (make sure that i440fx.bin is in the roms/vs440fx directory where PCem expects it after installation):

Code: Select all

f = romfopen("vs440fx/i440fx.bin", "rb");
if (!f) break;
romfread(rom + 0x20000, 0x20000, 1, f);
fclose(f);
biosmask = 0x3ffff;
return 1;

Then look for the "at_vs440fx_init()" function definition and remove or comment out the following line to disable the flash ROM device:

Code: Select all

device_add(&intel_flash_28fb200bxt_device);

I hope I haven't forgot anything.