Check if device is bootable

[Stamerlan]

Hi, guys!
I'm writing setup and I need to check if BIOS able to boot from some devices (especially NVME), is there any possibilities to get list of bootable devices or smth like that?

Have a nice day!
Thank you!

[Brendan]

Hi,

I'm writing setup and I need to check if BIOS able to boot from some devices (especially NVME), is there any possibilities to get list of bootable devices or smth like that?

Sadly, no (not that I'm aware of).


Cheers,

Brendan

[Stamerlan]

Hi, Brendan,

Thank you for your reply.
If you try to install windows 10 to NVME device on qemu 2.4.0.1 it will show an error message:

Windows cannot be installed to this disk. This computer's hardware may not support booting to this disk. Ensure that the disk's controller is enable in the computer's BIOS menu.

So, I guess, Windows has a mechanism to check it.

P.S. SeaBIOS (1.8.2) has no support for booting from NVME device

[Brendan]

Hi,

If you try to install windows 10 to NVME device on qemu 2.4.0.1 it will show an error message:

Windows cannot be installed to this disk. This computer's hardware may not support booting to this disk. Ensure that the disk's controller is enable in the computer's BIOS menu.

So, I guess, Windows has a mechanism to check it.

P.S. SeaBIOS (1.8.2) has no support for booting from NVME device

It's possible that the Windows gets a list of disk devices from the BIOS using int 0x13 Extensions - Get Drive Parameters and a list of disks/disk controllers from auto-detection (e.g. starting with PCI bus enumeration), and then cross-checks both lists to determine that the device isn't supported by the BIOS. In this case, if the BIOS did support the device Windows wouldn't know if the BIOS is actually able to boot from it or not (although that might be a reasonably safe assumption).

Note that I've had a look at an Intel document (Booting from an NVMe PCIe Intel Solid State Drive) that suggests that booting from NVMe requires 64-bit UEFI. I can't think of a sane reason why UEFI is needed (or why a BIOS couldn't support booting from NVMe); and assume this only applies to existing/older motherboards and may not apply to future motherboards.


Cheers,

Brendan

[Stamerlan]

Hi,

It's possible that the Windows gets a list of disk devices from the BIOS using int 0x13 Extensions - Get Drive Parameters and a list of disks/disk controllers from auto-detection (e.g. starting with PCI bus enumeration), and then cross-checks both lists to determine that the device isn't supported by the BIOS. In this case, if the BIOS did support the device Windows wouldn't know if the BIOS is actually able to boot from it or not (although that might be a reasonably safe assumption).

Yep, you are right windows setup uses int 0x13 ah=48 bios function to get information about device and reads (3 times?) the 1st sector of the disk. Then it uses windows nvme driver:
IDENTIFY
SET FEATURES
creates i/o queues (I guess, submission queue number equal to smp-cores and only 1 completion queue)
GET FEATURES
and then it reads the 1st sector of the disk several times from different submission queues (several times?).

It compares results of IDENTIFY and int 0x13 results and contents of 1st sector read by bios and nvme-driver.

I need to know more details, I think on the next week I will have an answer to "what exactly windows does"

Note that I've had a look at an Intel document (Booting from an NVMe PCIe Intel Solid State Drive) that suggests that booting from NVMe requires 64-bit UEFI. I can't think of a sane reason why UEFI is needed (or why a BIOS couldn't support booting from NVMe); and assume this only applies to existing/older motherboards and may not apply to future motherboards.

I think it because nvme-device uses 64-bit addressing to locate queues and registers (but as far as I remember there was a mechanism to map registers to 4GB address space).

Have a nice day!
Thank you, Brendan, you really helped me!

[Brendan]

Hi,

It's possible that the Windows gets a list of disk devices from the BIOS using int 0x13 Extensions - Get Drive Parameters and a list of disks/disk controllers from auto-detection (e.g. starting with PCI bus enumeration), and then cross-checks both lists to determine that the device isn't supported by the BIOS. In this case, if the BIOS did support the device Windows wouldn't know if the BIOS is actually able to boot from it or not (although that might be a reasonably safe assumption).

Yep, you are right windows setup uses int 0x13 ah=48 bios function to get information about device and reads (3 times?) the 1st sector of the disk. Then it uses windows nvme driver:
IDENTIFY
SET FEATURES
creates i/o queues (I guess, submission queue number equal to smp-cores and only 1 completion queue)
GET FEATURES
and then it reads the 1st sector of the disk several times from different submission queues (several times?).

It compares results of IDENTIFY and int 0x13 results and contents of 1st sector read by bios and nvme-driver.

That's surprisingly thorough!

I need to know more details, I think on the next week I will have an answer to "what exactly windows does"

Note that I've had a look at an Intel document (Booting from an NVMe PCIe Intel Solid State Drive) that suggests that booting from NVMe requires 64-bit UEFI. I can't think of a sane reason why UEFI is needed (or why a BIOS couldn't support booting from NVMe); and assume this only applies to existing/older motherboards and may not apply to future motherboards.

I think it because nvme-device uses 64-bit addressing to locate queues and registers (but as far as I remember there was a mechanism to map registers to 4GB address space).

You'd just set the highest 32 bits of the 64-bit addresses to zero...

I think the main (only?) problem is likely to be firmware support. Maybe the NVMe device comes with a boot ROM that only contains a 64-bit UEFI driver (and doesn't have a BIOS compatible "option ROM" like SCSI controllers had).

This makes some sense to me; as NVMe is/was mostly a server part; and only recently became sane/affordable (compared to SATA SSD) for normal/high-end consumer use. For both of those markets it doesn't really make that much sense worrying about legacy BIOS systems.

Of course none of that helps with Qemu at the moment; but Qemu is open source and I'd assume someone will add "bootable NVMe" to SeaBIOS sooner or later.


Cheers,

Brendan

[Stamerlan]

Hello,

May be it will be interesting for smb: In addition windows setup writes 2 (random?) bytes at sector 0, offset: 0x1B8 and 0x1BA with bios write function (int 0x13) and tries to read it by nvme-driver to check if everything ok.

So windows setup checks if bios supports booting from nvme in the next way:
1. int 0x13 ah=0x48 to get disk size from EDD (only sector count and sector size may be valid)
2. reads sector 0 using bios
3. using bios writes 2 (random?) bytes at sector 0 offset 0x1B8 and 0x1BA (For example: 0x01B8: b4 6e b4 6e)
4. using windows nvme-driver sends IDENTIFY (checks disk size match with EDD)
5. reads sector 0 (using windows nvme-driver) to check is signature at sector 0 offset 0x1B8 and 0x1BA valid

Happy hacking!

[Brendan]

Hi,

May be it will be interesting for smb: In addition windows setup writes 2 (random?) bytes at sector 0, offset: 0x1B8 and 0x1BA with bios write function (int 0x13) and tries to read it by nvme-driver to check if everything ok.

So windows setup checks if bios supports booting from nvme in the next way:
1. int 0x13 ah=0x48 to get disk size from EDD (only sector count and sector size may be valid)
2. reads sector 0 using bios
3. using bios writes 2 (random?) bytes at sector 0 offset 0x1B8 and 0x1BA (For example: 0x01B8: b4 6e b4 6e)
4. using windows nvme-driver sends IDENTIFY (checks disk size match with EDD)
5. reads sector 0 (using windows nvme-driver) to check is signature at sector 0 offset 0x1B8 and 0x1BA valid

Well, that's scary.

Apparently Microsoft have decided that it's OK to trash the MBR of every disk (even when the disk and/or MBR doesn't belong to Windows at all) by storing a 32-bit "disk signature" at offset 0x01B8 the first time Windows sees the disk.


Cheers,

Brendan

[Stamerlan]

Hello,

Look at wiki: https://en.wikipedia.org/wiki/Master_bo ... tor_layout section Structure of a modern standard MBR
At offset 0x1B8 there is 32-bit disk signature.

My disk has no any data, I hope if windows finds MBR of another type it will restore it, but I didn't check it.

Have a nice day!

[Brendan]

Hi,

Look at wiki: https://en.wikipedia.org/wiki/Master_bo ... tor_layout section Structure of a modern standard MBR
At offset 0x1B8 there is 32-bit disk signature.

Yes.

Traditionally (for MBR partitioning) the disk space before the beginning of the first partition "belongs" to the boot manager (which may or may not have anything to do with any specific OS); and no OS vendor has a right to mandate anything in this area because it's not part of any specific OS.

To put it another way, how do you think Microsoft would react if Linux developers decided to slap one of their ugly hacks in the middle of Microsoft's default MBR code?


Cheers,

Brendan

[Antti]

My disk has no any data, I hope if windows finds MBR of another type it will restore it, but I didn't check it.

I think it is unacceptable if the disk gets modified. In this case "has no any data" is not a defence. If the disk contains no data, i.e. is a perfectly empty disk, it makes a HUGE difference to the original state. Almost empty is not empty and it may be very relevant.

[mallard]

The whole MBR scheme was invented by Microsoft/IBM back in the 1980s. If their standard says that offset 0x1B8 is "reserved" for a signature, then that's the standard that any software that makes use of the MBR needs to comply with it. It might be a pain to do so, but it's at least 30 years too late to change it.

The signature requirement is also included in the UEFI specification (s. 5.2.1 and s. 5.2.3); of course the entire MBR is optional there, but if you have an MBR and use those 4 bytes for something else, you're out-of-spec and shouldn't be surprised if whatever you put there gets trashed.

[Brendan]

Hi,

The whole MBR scheme was invented by Microsoft/IBM back in the 1980s. If their standard says that offset 0x1B8 is "reserved" for a signature, then that's the standard that any software that makes use of the MBR needs to comply with it. It might be a pain to do so, but it's at least 30 years too late to change it.

There never was an actual standard - it's a "de facto standard" that is accepted by all OS vendors to ensure compatibility between different OSs (especially in dual-boot scenarios) and people writing/providing "boot managers" (used to choose which OS/partition to boot).

The signature requirement is also included in the UEFI specification (s. 5.2.1 and s. 5.2.3); of course the entire MBR is optional there, but if you have an MBR and use those 4 bytes for something else, you're out-of-spec and shouldn't be surprised if whatever you put there gets trashed.

It's not a requirement of UEFI; which says (emphasis mine):

• "Unique Disk Signature, this is an optional feature and not on all hard drives. This value is always written by the OS and is never written by EFI firmware."

If the primary OS installed on the computer is (e.g.) Linux (and Windows is just a second OS in a dual-boot system); this does not give Windows the right to tamper with another OS's disk.

The real problem is that parts of the disk (and parts of the computer) do not belong to any specific OS in the first place. No OS owns the partition table (all OSs share it). No OS owns the MBR (all OSs share it). No OS owns the UEFI system partition (all OSs share it). No OS owns UEFI variables (all OSs share them).

Now...

My OS is intended as a distributed OS - multiple computers on a network work together to form a cluster. If an attacker can put a computer on the network that pretends to be part of a cluster (but isn't) then it's a massive security problem (the attacker could get full access to all data on all computers). To guard against this I use remote attestation.

For BIOS systems; how it works is that the firmware (using TPM) keeps a checksum of each piece of code that's used during boot. After boot this checksum is used to ensure other computers know that the computer is who it says it is and can be trusted. This checksum includes the firmware itself, the MBR, the OS's boot code, the OS's kernel, etc. If any one piece is tampered with then remote attestation fails and all of the other computers refuse to talk to the computer that was maliciously violated by Microsoft.


Cheers,

Brendan

[mallard]

There never was an actual standard - it's a "de facto standard" that is accepted by all OS vendors to ensure compatibility between different OSs (especially in dual-boot scenarios) and people writing/providing "boot managers" (used to choose which OS/partition to boot).

The entire "x86 PC" is a "de facto standard" invented/defined primarily by IBM, Intel and Microsoft. Whether it's a standard defined by the likes of ISO or it's just "de facto" that's the standard and you should follow it

It's not a requirement of UEFI; which says (emphasis mine):

• "Unique Disk Signature, this is an optional feature and not on all hard drives. This value is always written by the OS and is never written by EFI firmware."

The current version (2.5) of the UEFI standard doesn't say that. It says "Unique Disk Signature This may be used by the OS to identify the disk from other disks in the system. This value is always written by the OS and is never written by EFI firmware." for the Legacy MBR or "Unused. Set to zero" for the Protective MBR. Nowhere does it say that it's optional (except for the fact that the entire MBR is optional). Your quote is from an extremely out-of-date version of the spec, prior to 2010's version 2.2. If you're basing your code on out-of-date specs, you're going to have problems...

The real problem is that parts of the disk (and parts of the computer) do not belong to any specific OS in the first place. No OS owns the partition table (all OSs share it). No OS owns the MBR (all OSs share it). No OS owns the UEFI system partition (all OSs share it). No OS owns UEFI variables (all OSs share them).

That's not really a "problem" if all the OSs on a particular computer follow a consistent standard. All current standards for the MBR include the "signature", so no OS/bootloader should use those 4 bytes for anything else.

For BIOS systems; how it works is that the firmware (using TPM) keeps a checksum of each piece of code that's used during boot. After boot this checksum is used to ensure other computers know that the computer is who it says it is and can be trusted. This checksum includes the firmware itself, the MBR, the OS's boot code, the OS's kernel, etc. If any one piece is tampered with then remote attestation fails and all of the other computers refuse to talk to the computer that was maliciously violated by Microsoft.

So basically, you're reinventing UEFI secure boot, along with the "boot sector virus protection" feature of 1990s BIOSs. Since the 4 bytes at offset 0x1B8 don't form part of the boot code you've got no reason to store their checksum. Also, it's a massive stretch to call using a field how it's documented and intended "malicious violation".

[Brendan]

Hi,

There never was an actual standard - it's a "de facto standard" that is accepted by all OS vendors to ensure compatibility between different OSs (especially in dual-boot scenarios) and people writing/providing "boot managers" (used to choose which OS/partition to boot).

The entire "x86 PC" is a "de facto standard" invented/defined primarily by IBM, Intel and Microsoft. Whether it's a standard defined by the likes of ISO or it's just "de facto" that's the standard and you should follow it.

We're saying the same thing from opposite perspectives - it is a de facto standard and UEFI and Microsoft should follow it.

It's not a requirement of UEFI; which says (emphasis mine):

• "Unique Disk Signature, this is an optional feature and not on all hard drives. This value is always written by the OS and is never written by EFI firmware."

The current version (2.5) of the UEFI standard doesn't say that. It says "Unique Disk Signature This may be used by the OS to identify the disk from other disks in the system. This value is always written by the OS and is never written by EFI firmware." for the Legacy MBR or "Unused. Set to zero" for the Protective MBR. Nowhere does it say that it's optional (except for the fact that the entire MBR is optional).

Do you understand the difference between "may" and "must"?

This may be used by the OS to identify the disk (but it's still perfectly legal for the OS to use it for executable MBR code instead).

Your quote is from an extremely out-of-date version of the spec, prior to 2010's version 2.2. If you're basing your code on out-of-date specs, you're going to have problems...

"PC compatible" computers are full of backward compatibility for very good reasons. Breaking backward compatibility (and breaking all older OSs that complied with previous versions of a specification) is almost never acceptable. Don't forget that most computers that come with UEFI also have a BIOS compatibility layer and that hybrid "MBR + GPT" is entirely possible; and nothing prevents you from running (e.g.) DOS 1.0 from 1981 on a UEFI system purchased last week.

The real problem is that parts of the disk (and parts of the computer) do not belong to any specific OS in the first place. No OS owns the partition table (all OSs share it). No OS owns the MBR (all OSs share it). No OS owns the UEFI system partition (all OSs share it). No OS owns UEFI variables (all OSs share them).

That's not really a "problem" if all the OSs on a particular computer follow a consistent standard. All current standards for the MBR include the "signature", so no OS/bootloader should use those 4 bytes for anything else.

It's very much a problem if you install 2 different versions of Windows (e.g. Win7 and Win10) in separate partitions and they both fail to keep track of which disk is what because they're constantly trashing each other's disk signatures.

For BIOS systems; how it works is that the firmware (using TPM) keeps a checksum of each piece of code that's used during boot. After boot this checksum is used to ensure other computers know that the computer is who it says it is and can be trusted. This checksum includes the firmware itself, the MBR, the OS's boot code, the OS's kernel, etc. If any one piece is tampered with then remote attestation fails and all of the other computers refuse to talk to the computer that was maliciously violated by Microsoft.

So basically, you're reinventing UEFI secure boot, along with the "boot sector virus protection" feature of 1990s BIOSs.

This was designed by the Trusted Computing Group not me (I'm just using it for its intended purpose). Secure boot is completely different (based on digital signatures and not measurement). It has nothing to do with the virus protection features of 1990s BIOSs which did none of this any only prevented writes to the MBR.

Since the 4 bytes at offset 0x1B8 don't form part of the boot code you've got no reason to store their checksum.

The firmware (not me) "measures" the entire sector. It makes no difference if it's executable or not. Please note that other data (e.g. ACPI tables, etc) is also measured (by firmware, not me).

Also, it's a massive stretch to call using a field how it's documented and intended "malicious violation".

UEFI specs say that "the OS" (my OS) may use those bytes. If any other OS that is not "the OS" (e.g. Windows) modifies those bytes then it's a malicious violation.

Of course (as I've already said) the MBR shouldn't belong to any specific OS. An OS only "owns" data in its partitions and has no right to modify any part of any disk outside of its own partitions unless it has the computer owner's consent. If UEFI specs says otherwise then the UEFI specs are wrong (but they don't, they only say "optional" or "may" and never say "must").


Cheers,

Brendan