Page 1 of 2
UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 9:03 am
by jamesread
Hi,
I am working my way through the
https://wiki.osdev.org/UEFI_Bare_Bones tutorial. I haven't been able to install the OVFM firmware to test on Qemu so I wanted to skip to test directly on my spare hardware.
To this end how do you make a bootable USB. I tried sudo dd if=cdimage.iso of=/dev/sdb && sync but the resulting USB stick is unbootable. In the file browser I get two files on the USB stick boot.catalog and fat.img
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 9:40 am
by bzt
jamesread wrote:I haven't been able to install the OVFM firmware to test on Qemu so I wanted to skip to test directly on my spare hardware.
Bad idea. Always try your code in an emulator first. What was the problem? Do you have any error messages? Those could be helpful.
jamesread wrote:To this end how do you make a bootable USB.
Read
https://wiki.osdev.org/Bootable_Disk, that may help. The
https://wiki.osdev.org/UEFI#Creating_disk_images page also has examples on how to create such image. For creating a bootable image programatically, take a look at my
mkbootimg tool.
jamesread wrote:I tried sudo dd if=cdimage.iso of=/dev/sdb && sync but the resulting USB stick is unbootable.
Of course. The ISO is an image of an ISO9660 file system, but for booting from an USB stick under UEFI you'll need an image partitioned with GPT, having an ESP partition formatted with FAT file system.
For the records, qemu is capable of creating such an image on-the-fly. You just specify a directory on the command line, and the guest VM will see that as a perfectly valid ESP.
Code: Select all
qemu-system-x86_64 -bios $(OMVF) -m 64 -hda fat:$(ESPdir) ...
Note that OVMF requires at least 64Mb of RAM, otherwise it won't work. You only need the ".fd" file, nothing else.
If you want to use EFI NVRAM variables too, then read
https://wiki.osdev.org/UEFI#Emulation_w ... U_and_OVMF.
The quickest and dirtiest, but also simplest solution, dd my
ready-to-use image to your USB stick, mount it, then replace EFI/BOOT/BOOTX64.EFI with your own loader.
Cheers,
bzt
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 11:06 am
by jamesread
Regarding OVMF I ran
Code: Select all
git clone https://github.com/tianocore/edk2.git
Then in the absence of any installation instructions I ran:
This gave me the following output:
Code: Select all
WORKSPACE: /home/yaakov/osdev002/edk2
EDK_TOOLS_PATH: /home/yaakov/osdev002/edk2/BaseTools
CONF_PATH: /home/yaakov/osdev002/edk2/Conf
Copying $EDK_TOOLS_PATH/Conf/build_rule.template
to /home/yaakov/osdev002/edk2/Conf/build_rule.txt
Copying $EDK_TOOLS_PATH/Conf/tools_def.template
to /home/yaakov/osdev002/edk2/Conf/tools_def.txt
Copying $EDK_TOOLS_PATH/Conf/target.template
to /home/yaakov/osdev002/edk2/Conf/target.txt
./edksetup.sh: line 214: return: can only `return' from a function or sourced script
I then tried:
This gave me the following error:
Code: Select all
make -C BrotliCompress
make[2]: Entering directory '/home/yaakov/osdev002/edk2/BaseTools/Source/C/BrotliCompress'
gcc -c -I ./brotli/c/include -I .. -I ../Include/Common -I ../Include/ -I ../Include/IndustryStandard -I ../Common/ -I .. -I . -I ../Include/X64/ -MD -fshort-wchar -fno-strict-aliasing -fwrapv -fno-delete-null-pointer-checks -Wall -Werror -Wno-deprecated-declarations -Wno-stringop-truncation -Wno-restrict -Wno-unused-result -nostdlib -g -O2 BrotliCompress.c -o BrotliCompress.o
BrotliCompress.c:20:10: fatal error: ./brotli/c/common/constants.h: No such file or directory
20 | #include "./brotli/c/common/constants.h"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [../Makefiles/footer.makefile:21: BrotliCompress.o] Error 1
make[2]: Leaving directory '/home/yaakov/osdev002/edk2/BaseTools/Source/C/BrotliCompress'
make[1]: *** [GNUmakefile:77: BrotliCompress] Error 2
make[1]: Leaving directory '/home/yaakov/osdev002/edk2/BaseTools/Source/C'
make: *** [GNUmakefile:19: Source/C] Error 2
make: Leaving directory '/home/yaakov/osdev002/edk2/BaseTools'
Given that the information in the tutorial seems outdated and that OVMF seems to lack installation instructions I gave up. I do agree though. It was a bad idea to skip testing in Qemu.
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 12:09 pm
by bzt
jamesread wrote:Regarding OVMF I ran
Just download a pre-compiled OVMF.fd from the official site:
https://github.com/tianocore/tianocore. ... /wiki/OVMF
or try one of these:
https://github.com/BlankOn/ovmf-blobs
Cheers,
bzt
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 12:29 pm
by jamesread
Success. Thanks. OVMF is now working. Still working on getting a bootable USB working though.
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 1:23 pm
by bzt
jamesread wrote:Success. Thanks. OVMF is now working. Still working on getting a bootable USB working though.
Well done! Try to dd
my known-to-work image to an USB stick, mount it and replace EFI/BOOT/BOOTX64.EFI, that's the simplest way. Or you can create a
loop device on the image directly and mount that.
Code: Select all
# gzip -d disk-x86.img.gz
# losetup -o65536 /dev/loop0 disk-x86.img
# mount /dev/loop0 somedir
# cp yourloader.efi somedir/EFI/BOOT/BOOTX64.EFI
# umount somedir
# losetup -d /dev/loop0
and you can boot that disk-x86.img with qemu too. Otherwise follow the instructions on
Bootable Disk wiki page if you want to create everything from scratch.
Cheers,
bzt
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 1:49 pm
by jamesread
What's the correct command to test with Qemu and the new image I have created? I tried
Code: Select all
qemu-system-x86_64 -L OVMF_dir/ -bios osdev002/OVMF-pure-efi.fd -cdrom osdev005/disk-x86.img
I also dd'd the image to my USB but no joy trying to boot from it.
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 3:14 pm
by isaacwoods
This gave me the following error:
You need to fetch the submodules. Try:
Re: UEFI bare bones bootable usb stick
Posted: Thu Dec 10, 2020 3:42 pm
by zaval
bzt wrote:
jamesread wrote:
I tried sudo dd if=cdimage.iso of=/dev/sdb && sync but the resulting USB stick is unbootable.
Of course. The ISO is an image of an ISO9660 file system, but for booting from an USB stick under UEFI you'll need an image partitioned with GPT, having an ESP partition formatted with FAT file system.
here we go again... despite I've shown even the screenshots, proving it's a wrong and misleading statement, bzt keeps repeating it, further confusing new people. This is not true! Not for OVMF nor for real hardware.
james, you say, you dd'd image, now try this. during FW init, break into it - either Boot Manager or UEFI shell and if the first, look for "boot from the file" option in the FW Boot Manager, if the second, launch your .efi file from the shell. In the former case, you might locate it, knowing it sits on the USB and is CD file system, and, then move through the file system tree until you find your .efi file, and launch it, in the second case - just type it's path in the shell, the common way. Assuming, that, of course, that tutorial recommends using the "last attempt" option, the file system part of the path would be something like:
\efi\boot\bootia32.efi
if you really made the image properly, it will work. unlike what bzt says, it doesn't have to be GPT nor ESP. even better, you just can test your loader without burning the whole USB stick with an iso image, just make an "efi" folder in the root of the stick, then make there your folder, say "james" (\efi\james), and then put there your files and launch them the way I described. later, you'll create a Load Option and FW will start it automatically.
Re: UEFI bare bones bootable usb stick
Posted: Fri Dec 11, 2020 7:58 am
by bzt
jamesread wrote:What's the correct command to test with Qemu and the new image I have created?
Take a look at the Makefile on the link I've provided. It has several qemu invocations. For example "make efi" will boot it from disk with EFI, and "make eficdrom" will boot from cdrom using EFI.
Code: Select all
efi:
qemu-system-x86_64 -bios $(OVMF) -m 64 -drive file=disk-x86.img,format=raw -serial stdio
eficdrom:
qemu-system-x86_64 -bios $(OVMF) -m 64 -cdrom disk-x86.img -serial stdio
The image has been tested with these commands and known to work.
jamesread wrote:I tried
Code: Select all
qemu-system-x86_64 -L OVMF_dir/ -bios osdev002/OVMF-pure-efi.fd -cdrom osdev005/disk-x86.img
With "-cdrom" you'll need an ISO9660 image with a Boot Catalog. My mkbootimg tool created a special image that has such catalog with both BIOS boot and EFI boot entries. It shouldn't matter if you load it from disk or a cdrom. But for that to work, you need a special hybrid layout and an ISO9660 El Torito Boot Catalog in the image.
jamesread wrote:I also dd'd the image to my USB but no joy trying to boot from it.
You must have used a bad image. Can you boot the original, unmodified image? Have you made any other modifications to it other than overwriting BOOTX64.EFI? Have you unmounted and sync'd properly? Do you have any error messages?
But first of all, can you boot the unmodified image?
zaval wrote:here we go again... despite I've shown even the screenshots, proving it's a wrong and misleading statement, bzt keeps repeating it, further confusing new people. This is not true! Not for OVMF nor for real hardware.
Nobody cares what screenshots you have shown (proves nothing, plus you've already represented us photoshoped "screenshots"), and it is not me who is saying dd'ing a cdrom image to an USB stick doesn't work. It is the OP who has boot problems with that configuration! Just saying I'm wrong when they experience the opposite won't help them, not even a bit! Why don't you try to make yourself useful and figure out why "dd if=iso of=stick" is NOT working for them?
Cheers,
bzt
Re: UEFI bare bones bootable usb stick
Posted: Fri Dec 11, 2020 4:58 pm
by zaval
bzt, I wasn't and am not going to argue with you, you are wrong and I've proven that, I just wanted to help the OP. that's all. I never posted here "photoshopped" screenshots. we don't know, why his stick isn't working.
your conclusion, that his problem is because of it's not GPT and ESP is just as hilarious as many things from you when it comes to understanding UEFI (somehow
).
but it's really getting tiresome. for being purely on topic. james, let's do it easier - just create a folder on a FAT USB stick, as I outlined in the above post, put there your OSL and its stuff and be sure - it will work. I did this a million times, both on real hardware and emulators and it always worked. it's the easiest for the beginning. why would you need to burn a USB stick as a "CDROM" anyway? maybe for bzt, creating an iso and burning it everytime is easier, than updating the folder, but for most of people, I'd guess, the latter is easier.
Re: UEFI bare bones bootable usb stick
Posted: Fri Dec 11, 2020 5:48 pm
by Doctor5555
I'm going to assume that you have a valid 64-bit PE executable built and linked using clang or a gcc cross compiler (not the .img file).
In order to run this in QEMU in the simplest way possible, start with an empty folder. Make sure the executable is named "bootx64.efi", and place it:
Code: Select all
<your folder>/efi/boot/bootx64.efi
Next, replace the 'path/to' s with the correct paths and then run the following command:
Code: Select all
qemu-system-x86_64 -drive if=pflash,format=raw,file=path/to/OVMF.fd -drive format=raw,file=fat:rw:path/to/<your folder> -M accel=kvm:tcg -serial stdio
This tells QEMU to emulate the folder as a fat-formatted drive and allows it to be booted from, without needing to make an image every time.
Re: UEFI bare bones bootable usb stick
Posted: Sat Dec 12, 2020 7:17 am
by bzt
zaval wrote:bzt, I wasn't and am not going to argue with you, you are wrong and I've proven that
Yes, you're arguing, you're saying I'm wrong while you've proven nothing.
Writing an ISO9660 image on an USB stick
does not work for the OP. Period.
ps: it does not work on my testbeds either. Neither in UEFI, nor in legacy CSM mode. The
UEFI spec does not mention that ISO9660 should work on non-CDROM media, where did you get that?
- Page 320 section 10.3.5.1 there's no El Torito option in the Hard Disk Media Device Path
- Page 321 section 10.3.5.2 says that for CDROM Media Device Path implicitly El Torito ISO-9660 file system assumed (it does not say other paths should recognize that file system)
- Page 502 section 13.3.2.1 says that partition discovery for CDROMs and DVDROMs use El Torito ISO-9660 (it does not say this file system should be recognized on non-CDROM paths)
Conclusion: whether ISO9660 can be used on an USB stick or not is entirely firmware implementation specific. You have a firmware that can, while me and the OP have such a firmware which can't.
Cheers,
bzt
Re: UEFI bare bones bootable usb stick
Posted: Sat Dec 12, 2020 11:43 am
by jamesread
bzt wrote:
Code: Select all
efi:
qemu-system-x86_64 -bios $(OVMF) -m 64 -drive file=disk-x86.img,format=raw -serial stdio
I tried with this command and got the following output
Code: Select all
BdsDxe: failed to load Boot0001 "UEFI QEMU DVD-ROM QM00003 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Secondary,Master,0x0): Not Found
BdsDxe: loading Boot0002 "UEFI QEMU HARDDISK QM00001 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Primary,Master,0x0)
BdsDxe: starting Boot0002 "UEFI QEMU HARDDISK QM00001 " from PciRoot(0x0)/Pci(0x1,0x1)/Ata(Primary,Master,0x0)
Hello, world!
BdsDxe: loading Boot0000 "UiApp" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(462CAA21-7614-4503-836E-8AB6F4662331)
BdsDxe: starting Boot0000 "UiApp" from Fv(7CB8BDC9-F8EB-4F34-AAEA-3EE4AF6516A1)/FvFile(462CAA21-7614-4503-836E-8AB6F4662331)
So, I guess it's working. It doesn't seem to work when I boot a machine from USB though.
Re: UEFI bare bones bootable usb stick
Posted: Sat Dec 12, 2020 11:54 am
by jamesread
OK, I just tried on another device (lenovo ideapad 320). It seems to be stuck in a never ending loop of booting and rebooting. It prints some stuff to screen but no Hello World message. The output doesn't stay on screen long enough for me to be able to copy it and post it here.