uefi real pc bios doesn't recognize test boot drive

Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
User avatar
austanss
Member
Member
Posts: 377
Joined: Sun Oct 11, 2020 9:46 pm
Location: United States

uefi real pc bios doesn't recognize test boot drive

Post by austanss »

I've been working on a bootloader from my OS because I really, really badly want my OS to work on real hardware.

I've gotten it to a mature point. I can load my kernel, in long mode, using operable graphics, on a UEFI-based QEMU virtual machine (without CSM support). I have a colorful screen in my bootloader so I wanted to boot it on real hardware to test my beautiful creation.

I plugged in my 64GB USB 3.0 drive, dd'ed the ISO to my USB stick. Rebooted, went into boot menu, and the only available UEFI option was my daily driver OS. I attempted to boot by manually booting the drive, however it still didn't work. I looked over and over and still didn't find anything.

Anyone know why?

source: https://github.com/microNET-OS/microCOR ... rizet/UEFI

_--computer--_

brand: HP
year: 2012
bios: American Megatrends K01 v3.03 (HP version)

Also I don't have Secure Boot or Fast Boot on.
Skylight: https://github.com/austanss/skylight

I make stupid mistakes and my vision is terrible. Not a good combination.

NOTE: Never respond to my posts with "it's too hard".
Octocontrabass
Member
Member
Posts: 5568
Joined: Mon Mar 25, 2013 7:01 pm

Re: uefi real pc bios doesn't recognize test boot drive

Post by Octocontrabass »

It doesn't work because you're writing an image that is not partitioned and is not formatted as FAT32.

Partition your USB drive and format a partition as FAT32, then copy your files to it.
nexos
Member
Member
Posts: 1081
Joined: Tue Feb 18, 2020 3:29 pm
Libera.chat IRC: nexos

Re: uefi real pc bios doesn't recognize test boot drive

Post by nexos »

Burn it to a CDROM and put it in your machine. It should work then.
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg
User avatar
austanss
Member
Member
Posts: 377
Joined: Sun Oct 11, 2020 9:46 pm
Location: United States

Re: uefi real pc bios doesn't recognize test boot drive

Post by austanss »

nexos wrote:Burn it to a CDROM and put it in your machine. It should work then.
Yea I don't have any CDs. I have an optical drive, not any CDs.
Skylight: https://github.com/austanss/skylight

I make stupid mistakes and my vision is terrible. Not a good combination.

NOTE: Never respond to my posts with "it's too hard".
User avatar
austanss
Member
Member
Posts: 377
Joined: Sun Oct 11, 2020 9:46 pm
Location: United States

Re: uefi real pc bios doesn't recognize test boot drive

Post by austanss »

Octocontrabass wrote:It doesn't work because you're writing an image that is not partitioned and is not formatted as FAT32.

Partition your USB drive and format a partition as FAT32, then copy your files to it.
I tried extracting the ISO and flashing that FAT32 image to the USB drive, didn't work.
I tried leaving the FAT32 image on the USB drive, didn't work.
I tried dd'ing the FAT32 image to the USB drive, didn't work.
I also tried forgoing the FAT32 image, didn't work.
I also tried manually running the UEFI application in my BIOS's "Run UEFI Application..." menu. (drive didn't show up)

I've dd'ed my ISO images before, many, in fact, including my own OS and Linux installers, and they've all gone off without a hitch. I don't understand what the difference is here.
Last edited by austanss on Tue Nov 10, 2020 1:27 pm, edited 1 time in total.
Skylight: https://github.com/austanss/skylight

I make stupid mistakes and my vision is terrible. Not a good combination.

NOTE: Never respond to my posts with "it's too hard".
Octocontrabass
Member
Member
Posts: 5568
Joined: Mon Mar 25, 2013 7:01 pm

Re: uefi real pc bios doesn't recognize test boot drive

Post by Octocontrabass »

The UEFI-compatible Linux installers include a partition table and a partition formatted as FAT32.

You're using mformat to create a FAT12 volume, but UEFI requires FAT32.
User avatar
austanss
Member
Member
Posts: 377
Joined: Sun Oct 11, 2020 9:46 pm
Location: United States

Re: uefi real pc bios doesn't recognize test boot drive

Post by austanss »

Thank you Octacontrabass. My problem was solved by formatting as FAT32 and replacing the files back in (which I thought I had already did, lol)
Skylight: https://github.com/austanss/skylight

I make stupid mistakes and my vision is terrible. Not a good combination.

NOTE: Never respond to my posts with "it's too hard".
User avatar
bzt
Member
Member
Posts: 1584
Joined: Thu Oct 13, 2016 4:55 pm
Contact:

Re: uefi real pc bios doesn't recognize test boot drive

Post by bzt »

In case someone reads this topic later:

ISO - typically images containing ISO9660 file system, specifically designed for CDROMs (or UDF in case of DVDs). See Bootable CD.
USB stick - does not work with ISO images, for an UEFI compatible, dd'able image see wiki page Bootable Disk.

Special images that comply with both standards are called "hybrid images".
Octocontrabass wrote:You're using mformat to create a FAT12 volume, but UEFI requires FAT32.
No, the UEFI spec allows FAT12, however very few real-life firmware implementation supports that. So FAT32 is not required, but recommended.

Cheers,
bzt
User avatar
zaval
Member
Member
Posts: 656
Joined: Fri Feb 17, 2017 4:01 pm
Location: Ukraine, Bachmut
Contact:

Re: uefi real pc bios doesn't recognize test boot drive

Post by zaval »

UEFI specification mandates using FAT32 for ESP and also UEFI spec says ESP can reside on every medium, supported by Boot Services protocols - meaning, on removable devices as well. So, if it's marked as ESP, it must be FAT32, no matter where it resides. It may be of other type otherwise. the relevant quotes:
EFI encompasses the use of FAT32 for a system partition, and FAT12 or FAT16 for removable media.
A System Partition can reside on any media that is supported by EFI Boot Services.
Also, dd'ing a GPT image would be a very bad idea. you screw up GUID uniqueness. post processing is needed. But personally, in the case of pure UEFI, I'd go with just a folder on a FAT formatted (any type) USB stick, where I'd put my whole installation set, with an OSL incl. and just pointed to it manually during FW BDS (boot device selection) phase. and then, it would start a "live" session for the OS for letting users have a look or just run a setup/install process immediately. no need to prepare any "images", just throw your install set at your USB stick and you are ready to conquer the world. :)
Last edited by zaval on Tue Nov 10, 2020 6:28 pm, edited 1 time in total.
ANT - NT-like OS for x64 and arm64.
efify - UEFI for a couple of boards (mips and arm). suspended due to lost of all the target park boards (russians destroyed our town).
User avatar
bzt
Member
Member
Posts: 1584
Joined: Thu Oct 13, 2016 4:55 pm
Contact:

Re: uefi real pc bios doesn't recognize test boot drive

Post by bzt »

.....aaaaand our beloved troll who knows always better but actually wasn't able to produce anything yet is here! Yay! :-)
zaval wrote:UEFI specification mandates using FAT32 for ESP
Absolutely not. From the UEFI spec:
The file system supported by the Extensible Firmware Interface is based on the FAT file system. EFI defines a specific version of FAT that is explicitly documented and testable. Conformance to the EFI specification and its associate reference documents is the only definition of FAT that needs to be implemented to support EFI.
Meaning it doesn't matter what's in the M$ FAT specification, EFI specifies it's own file system format.
zaval wrote:Also, dd'ing a GPT image would be a very bad idea.
How would you write an image file to a disk otherwise? Who said that you can't generate the image dynamically before dd'ing it?
zaval wrote:But personally, in the case of pure UEFI, I'd go with just a folder on a FAT formatted (any type) USB stick, where I'd put my whole installation set, with an OSL incl. and just pointed to it manually during FW BDS (boot device selection) phase.
Could we please see your installer (app, script whatever) that does all of this in your build environment? Do you even have an automated workflow for creating a bootable image from your source code?

Cheers,
bzt
nexos
Member
Member
Posts: 1081
Joined: Tue Feb 18, 2020 3:29 pm
Libera.chat IRC: nexos

Re: uefi real pc bios doesn't recognize test boot drive

Post by nexos »

:roll: :roll:
Please don't derail this thread.
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg
User avatar
zaval
Member
Member
Posts: 656
Joined: Fri Feb 17, 2017 4:01 pm
Location: Ukraine, Bachmut
Contact:

Re: uefi real pc bios doesn't recognize test boot drive

Post by zaval »

well, well. see, how "I" start "bickering". the only thing missing, is that fan girl, following me and telling "he thinks, I am wrong"... but anyway, just a couple of remarks.
bzt wrote: Meaning it doesn't matter what's in the M$ FAT specification, EFI specifies it's own file system format.
"non" troll, either show me UEFI FAT specification or you are laughingstock. :)

for other people, not as "advanced" in parallel reality observing as bzt, and just for the sake of truth - UEFI doesn't specify one and instead clearly refers to the MS spec. :)
The file system supported by the Extensible Firmware Interface is based on the FAT file system. EFI defines a specific version of FAT that is explicitly documented and testable. Conformance to the EFI specification and its associate reference documents is the only definition of FAT that needs to be implemented to support EFI. To differentiate the EFI file system from pure FAT, a new partition file system type has been defined.
the EFI specification doesn't contain FAT specification per se and "associate reference documents" in this respect is this one:
Appendix Q References, Q.1 Related Information wrote: Microsoft Extensible Firmware Initiative FAT32 File System Specification, Version 1.03,
Microsoft Corporation, December 6, 2000, http://www.microsoft.com/whdc/system/platform/
firmware/fatgen.mspx
the whole "newness" and "difference" of EFI FAT is this one:
To differentiate the EFI file system from pure FAT, a new partition file system type has been defined.
summarizing, UEFI FAT is one defined by the MS FAT specification, with the additional requirements regarding its typing and emphasis on using FAT32 for ESP, laid out in the UEFI spec. that's it. there are also notes about using only UCS-2 subset opposed to all UTF-16 and always supporting long names. it's a subset of a thing, specified in A and rules for subsetting, specified in B. where A - is MS FAT spec, B - UEFI spec.

and this one again, I quoted already, but it's you, so, here is yet once:
EFI encompasses the use of FAT32 for a system partition, and FAT12 or FAT16 for removable media.
if you cannot grasp this, then sorry.
bzt wrote: Could we please see your installer (app, script whatever) that does all of this in your build environment? Do you even have an automated workflow for creating a bootable image from your source code?
I don't use "bootable image" for UEFI.
Last edited by zaval on Tue Nov 10, 2020 7:22 pm, edited 6 times in total.
ANT - NT-like OS for x64 and arm64.
efify - UEFI for a couple of boards (mips and arm). suspended due to lost of all the target park boards (russians destroyed our town).
nexos
Member
Member
Posts: 1081
Joined: Tue Feb 18, 2020 3:29 pm
Libera.chat IRC: nexos

Re: uefi real pc bios doesn't recognize test boot drive

Post by nexos »

Zaval, you aren't the one bickering here. You posted an answer designed to helpful. There is nothing wrong with that. And now you called a troll? I see something wrong with that.
"How did you do this?"
"It's very simple — you read the protocol and write the code." - Bill Joy
Projects: NexNix | libnex | nnpkg
User avatar
bzt
Member
Member
Posts: 1584
Joined: Thu Oct 13, 2016 4:55 pm
Contact:

Re: uefi real pc bios doesn't recognize test boot drive

Post by bzt »

nexos wrote:Please don't derail this thread.
Have no worries, I won't respond to zaval.

Have a nice day!
bzt
User avatar
zaval
Member
Member
Posts: 656
Joined: Fri Feb 17, 2017 4:01 pm
Location: Ukraine, Bachmut
Contact:

Re: uefi real pc bios doesn't recognize test boot drive

Post by zaval »

^ but before vamoosing, - don't forget to show UEFI FAT specification, "genius".
ANT - NT-like OS for x64 and arm64.
efify - UEFI for a couple of boards (mips and arm). suspended due to lost of all the target park boards (russians destroyed our town).
Post Reply