TUTORIAL - Boot USB Drive using 64-Bit UEFI CLASS 3 - x86_64

[DiggDug]

Just to say up front first, I love OSDev.org. Great work !!
I am writing this because I found some spotty info at best about how to get a UEFI USB Thumb Drive to boot on a UEFI Class 3 computer. So I wanted to share what I found out through a lot of trial and error.

THE HISTORY BEHIND THIS
I recently had bought a laptop that had windows 10 on it. I found out the hard way that I couldn't install any OS that was not 64-Bit. I also found out that I couldn't install Windows 7, since it was not a modern UEFI CLASS 3 based OS. What I mean is, my laptop was not UEFI-CSM class 2 or older. ( No legacy BIOS compatibility. )

This link is just one of many sources that say this. No more 16/32 Bit Booting. ( Although the article doesn't come out and say no more 16-Bit, but this tutorial I am providing will show you that 16-Bit is not required to boot anymore with UEFI Class 3, which is where Intel is heading. )
https://www.anandtech.com/show/12068/in ... fi-by-2020

This doesn't apply to everyone currently who have a computer that can run windows 7 64-Bit ( or even a 32-Bit OS for that matter ). EFI has been around for over a decade, but the BIOS in general has allowed legacy boot ( 16-Bit BOOT.BIN loading kind of files ) to load the OS. And in some cases, if your computer has CSM as an option, you most likely have a UEFI Class 2 Firmware or older.

I found out, my laptop is a UEFI Class 3, which doesn't allow anything other then 64-Bit UEFI boot loading. No options for legacy booting. So the 2020 date that Intel has set just shows that they already begun making hardware that is strictly 64-Bit, and in a little over a year from the time of this post, that is all they will be making. ( NOTE : This doesn't mean they are removing 16/32 Bit Registers )

I am coming from a Windows point of view, so if I get something wrong with regards to Linux, I do apologize ahead of time. I also know that parts of this tutorial might be totally wrong, but this is what I know from trial and error and what works. I am open to input.

PREREQUISITES
1.) Windows that has diskpart on it. I believe most windows versions ( Starting with VISTA ) have this.
2.) Linux, to compile the BOOTx64.EFI file.
EDIT UPDATE (4-10-2019) : I was successful in getting this to compile under Windows 8.1. I No longer need Linux. Make sure to use the 64-Bit version of GCC 8.1 or newer.
I used gcc 8.2 from this link ----> https://nuwen.net/mingw.html.
3.) A Bootable USB Thumb Drive ( minimum of 256 Megs, although in my tests I have an 8 gig Thumb Drive ).
4.) GNU-EFI.zip file that has all the UEFI code in it.
5.) A UEFI Class 3 computer ( Desktop or Laptop ------ I tested this on a laptop )

I followed this tutorial on how to get the GNU-EFI.zip file as well as how to compile it on a linux host. I wish there was a tutorial to get this to compile on a windows host that would actually work.
https://wiki.osdev.org/UEFI_Bare_Bones

CREATING THE UEFI CLASS 3 USB BOOT DRIVE
In windows, Plug in your USB Thumb Drive and format it as FAT32.

Open a command prompt in windows.

Type: diskpart ( at this point you will be asked about security in windows, just click yes and it will open another console window that uses the diskpart environment. )

In diskpart console type: list disk ( this will show you the list of all of the relevent drives, including the USB stick )

Type: select disk 1 ( 1 was the thumb drive for me, it might be a different number for you )

Type: clean

Type: convert gpt

Type: create partition efi

Type: list disk

NOTE : At this point you should see an asterisk next to the USB drive showing that it is a GPT partition

Type: exit

Now your USB Thumb drive is set for GPT. Now format the USB Drive as FAT32 under windows. After that, open the windows file explorer and at the root of the Thumb Drive create this folder structure ---> EFI\Boot\

Once that is done, your USB Drive is bootable as a UEFI GPT Boot drive. All that is missing is the EFI file that is required by the BIOS. Notice, the lack of anything related to 16-Bit booting.

At this point you could literally copy the windows BOOTX64.EFI file into this thumb drive and into the EFI\Boot\ folder. Just make sure the NAME is correct. It MUST BE BOOTX64.EFI. This goes for your own OS that you create from this. ( NOTE : I tried to rename the file to other names, but it wouldn't boot. Only the file named BOOTX64.EFI worked. )

If you want to know more about why the EFI\BOOT\ folder structure is important, read this page.
http://www.uefi.org/registry

And if you want to know why BOOTX64.EFI is important, you need to get the PDF files from that same site. This tells you everything you need to know about UEFI.
http://www.uefi.org/specifications

On that page you will need the latest UEFI Specification PDF.

Now to compile the EFI, this is where Linux comes in, because honestly, I still do not understand why I couldn't get this to work under windows, even when compiled successfully. It simply wouldn't work. So on to your linux machine.

Go to this page and read it. It has all the info you need to compile the EFI file. https://wiki.osdev.org/UEFI_Bare_Bones

Grab the GNU-EFI.zip file, extract it on linux.

Then make sure you read the directions on how to install the MinGWx64 cross compiler.

Then create the hello.c file that is suggested on that same page.

Make sure your extracted GNU-EFI.zip file is extracted into the same folder where your C file is at. In my case the folder name was gnu-efi-3.0.8, but I renamed it to gnu-efi.

Now, one change I had to make, and this is according to the tutorial here at OSDev, go into gnu-efi/lib/ folder and look for data.c.

CHANGE THIS

Code: Select all

EFI_UNICODE_COLLATION_INTERFACE   LibStubUnicodeInterface = {
    LibStubStriCmp,
    LibStubMetaiMatch,
    LibStubStrLwrUpr,
    LibStubStrLwrUpr,
    NULL,   // FatToStr
    NULL,   // StrToFat
    NULL    // SupportedLanguages
};

TO THIS

Code: Select all

EFI_UNICODE_COLLATION_INTERFACE   LibStubUnicodeInterface = {
    NULL,
    NULL,
    NULL,
    NULL,
    NULL,   // FatToStr
    NULL,   // StrToFat
    NULL    // SupportedLanguages
};

Save the file.

After all of that, open a console window where your hello.c file is at and enter in these commands one at a time.

Code: Select all

x86_64-w64-mingw32-gcc -ffreestanding -Ignu-efi/inc -Ignu-efi/inc/x86_64 -Ignu-efi/inc/protocol -c -o hello.o hello.c

Code: Select all

x86_64-w64-mingw32-gcc -ffreestanding -Ignu-efi/inc -Ignu-efi/inc/x86_64 -Ignu-efi/inc/protocol -c -o data.o gnu-efi/lib/data.c

Code: Select all

x86_64-w64-mingw32-gcc -nostdlib -Wl,-dll -shared -Wl,--subsystem,10 -e efi_main -o BOOTX64.EFI hello.o data.o -lgcc

Remember, those commands only work if you have the correct path to the GNU-EFI folder inside the folder where you are compiling the hello.c file.

If successful, just copy the resulting BOOTX64.EFI file onto the thumb drive EFI\Boot\ folder and you now have a bootable UEFI Class 3 USB Thumb Drive.

FINAL NOTES
This is pretty much all there is to it. Now you can use DD to make an IMG file from the USB drive. Or what ever you choose from here.
However if you use a different file system, other then FAT32, you must seperate the FAT32 GPT 100Meg partition from your OS, and then have the BOOTX64.EFI point to your kernal that resides on your OS partition. How to load and call that kernel is up to your OS design and is outside the scope of this tutorial. Your OS uses the rest of the space on your thumb drive.

Another note is, this is exactly how Microsoft does it for their Windows 8.x and Windows 10 systems. The UEFI resides on a 100Meg partitition. This is also where the multi-Boot takes place. For your OS you need a minimum of 48 Megs for this partition in case you have more then 1 OS to boot as a choice. You will notice that a lot of tutorials will tell you that you need 48 Megs, and even in the specs it looks like their minimum is also 48 Megs. ( Although I might have misread that, it is how it comes across to me. )

Thanks for reading, hope this helps some of you out there.

[Octacone]

Nice tutorial and all, but this class 3 thing is a first class BS.
Why are they making it so hard for us OS developers and limiting our abilities. They're basically forcing us to switch.

Want to use a 32-bit OS, well no, screw you user/developer you'll play by our rules
Want to install DOS for nostalgic reasons, well screw you
Want to use a good old BIOS, nope get used to UEFI
Want to use a custom filesystem, nope, you have to use FAT 32 because M$ paid us
Want to use Virtual 8086 Mode, hell no boy, you have to go 64 bits
Want to use a simple and small embedded RTOS, nope

Why are they forcing my OS to be 64-bit. What if I don't want my OS to be 64-bit?
What if I want my OS to run on newer hardware but it's 32-bit only?
This translates into me wasting my time rewriting my entire freaking bootloader and using some bloated definitions and code from some UEFI (Intel and M$) monopoly.
Also I would have to find a workaround in order to boot my OS since it's not 64-bit...

I understand that they want to phase out BIOS and do something newer, but removing all the features I've gotten used to over the years on which I've laid my OS's foundation is just ridiculous.
Those are just my 0.2$, hope to get some constructive criticism and ways to override this default behavior.

[DiggDug]

Nice tutorial and all, but this class 3 thing is a first class BS.
Why are they making it so hard for us OS developers and limiting our abilities. They're basically forcing us to switch.

Want to use a 32-bit OS, well no, screw you user/developer you'll play by our rules
Want to install DOS for nostalgic reasons, well screw you
Want to use a good old BIOS, nope get used to UEFI
Want to use a custom filesystem, nope, you have to use FAT 32 because M$ paid us
Want to use Virtual 8086 Mode, hell no boy, you have to go 64 bits
Want to use a simple and small embedded RTOS, nope

Why are they forcing my OS to be 64-bit. What if I don't want my OS to be 64-bit?
What if I want my OS to run on newer hardware but it's 32-bit only?
This translates into me wasting my time rewriting my entire freaking bootloader and using some bloated definitions and code from some UEFI (Intel and M$) monopoly.
Also I would have to find a workaround in order to boot my OS since it's not 64-bit...

I understand that they want to phase out BIOS and do something newer, but removing all the features I've gotten used to over the years on which I've laid my OS's foundation is just ridiculous.
Those are just my 0.2$, hope to get some constructive criticism and ways to override this default behavior.

I totally feel your pain. I literally said all those things and then some.

Also, I literally thought MS was behind it too. But, as I dug further, I found out, MS did not have the final say in this either. This was all brought about by the UEFI group. Intel, Apple, ARM as well as others, all came together and decided this together. And since Apple, who switched to Intel Hardware, uses 64-Bit for their OS, 64-bit was the deciding factor in that.

Believe me, I was upset at first too. But I didn't give up, and I wanted to see for myself why all this was taking place. Now I know, all of this has to do with making a standard so that all Hardware manufactures can work together toward certain other goals they all have in mind.

I read this article, it might help you understand more. Although I may not agree with everything he says, he does put some info into perspective I had never thought about.

https://www.happyassassin.net/2014/01/2 ... work-then/

Remember, that if you have not set your BIOS to UEFI Class 3, OR you do not have a UEFI Class 3 computer, then this tutorial most likely will not help you. And you would need to figure out how to code into your boot loader a "fallback to legacy boot" ( I.E. use UEFI Class 2 ( CSM Mode) or older ) if UEFI Class 3 is not present.

https://wiki.osdev.org/UEFI

Notice this line on that page ---> "A class 3 machine is a UEFI system that does not support CSM. UEFI class 3 machines only run UEFI applications and do not implement CSM for backwards compatibility with legacy bootloaders."

Hope this helps.

EDIT UPDATE : I forgot to mention, another reason for the UEFI 64-Bit move on the part of Intel and all other Hardware name brands, is because of the limits of the Legacy Booting. The limit is the inability to boot on Hard Drives larger then 2.1 Terrabyte. So to get around that, UEFI was the solution.

Source Here:
http://www.uefi.org/sites/default/files ... _Sheet.pdf

You can also look further into other info about UEFI Shell Scripting that you might need in several books, but one I recommend is this one :
Harnessing the UEFI Shell: Moving the Platform Beyond Dos ( You might need to hunt down the 2nd edition )
https://www.amazon.com/Harnessing-UEFI- ... 501514806/

and

Embedded Firmware Solutions: Development Best Practices for the Internet of Things
https://www.amazon.com/Embedded-Firmwar ... 484200713/

EDIT UPDATE 2 :
I just found this......
I am posting this link, to further show another way to write your main file. ( the Hello.c ---> BOOTX64.EFI file )
http://x86asm.net/articles/uefi-program ... index.html
Specifically, the "Building an UEFI application" section.

EDIT UPDATE 3 :
I would like to thank BenLunt ( a member of OSDev ). With his books and help he has given me, I have learned a lot about Operating Systems.
I highly recommend his books.
Thank you.

[Brendan]

Hi,

Nice tutorial and all, but this class 3 thing is a first class BS.
Why are they making it so hard for us OS developers and limiting our abilities. They're basically forcing us to switch.

Want to use a 32-bit OS, well no, screw you user/developer you'll play by our rules
Want to install DOS for nostalgic reasons, well screw you
Want to use a good old BIOS, nope get used to UEFI
Want to use a custom filesystem, nope, you have to use FAT 32 because M$ paid us
Want to use Virtual 8086 Mode, hell no boy, you have to go 64 bits
Want to use a simple and small embedded RTOS, nope

Why are they forcing my OS to be 64-bit. What if I don't want my OS to be 64-bit?
What if I want my OS to run on newer hardware but it's 32-bit only?

I can imagine Intel and motherboard manufacturers (a few years ago) saying the opposites (Why are people using old versions of Windows expecting us to play by their rules? Why can't people who like nostalgia use old hardware or virtual machines? Why are we letting legacy stuff prevent us from being more able to competing against ARM?).

The fact is, it's purely economics (money) - for Intel and motherboard manufacturers (and Microsoft), the cost of supporting older operating systems is higher than the profit gained.

Does it suck for us? Yes, for the short term it's going to be painful (but for the long term it's really not going to make a huge difference - you only need to adapt your code to handle 64-bit UEFI once).


Cheers,

Brendan

[zaval]

gosh, Octacone, instead of this butthurting, you could learn deeper on the subject and will know how it works in reality. What you wrote is a total bullshit.
Like this "brilliance".

Want to use a custom filesystem, nope, you have to use FAT 32 because M$ paid us

hell no, Octacone, you are freaking free to implement your own UEFI drvier with the pair of protocols implementing whatever File System you want! Or, implement it in your OS loader only without letting others to use it as would be the case in the former variant. Anyways, your custom filesystem needs to be implemented, do you realize this? And the specification doesn't forbid you of doing so. It's frigging "extensible" to fit every bullshit anyone wants to put there. why are you saying this silliness? They chose FAT because it's hell simple and fast and I'd say they would be stupid if they haven't chosen this FS for the system partition. "MS paid". LOL.

Want to use a good old BIOS, nope get used to UEFI

Wanna use BIOS, make your own system and use what you want. Computer vendors make their things for users and it's up to them what standards to stick up with, and you are free to buy this or not buy. Why should they satisfy whims of a bunch of perverts wanting BIOS/DOS or other fossilities? For what on the planet reason? Just think about it and give the answer to hundreds of vendors and millions of users looking at you.

Want to use a simple and small embedded RTOS, nope

*facepalm
UEFI specification in no way limits you what the OS to boot. Your OS just should be UEFI compatible. Have the OS loader, OS installer, install itself and create a boot option for itself. That's all. After taking control over, your OS could fly you away on Mars. Neither MS nor Intel did pay for preventing this.
If some crappy implementation doesn't allow you to do this - blame the vendor, call them and say how much you love them. Or buy a better weed.

[DiggDug]

Just to add further to this discussion...

Intel is only removing the 16/32 Bit booting and going straight to 64-Bit. That doesn't mean they are removing the 16/32 Bit registers. Only the mode of which the system boots up.

Also, with the UEFI, you have better access to hardware information all done for you while you are still in UEFI mode. So have your kernal ( or the UEFI file itself ) store this information, before you switch out of UEFI mode. ( That is If you plan on not using UEFI once the system is booted. )

you are freaking free to implement your own UEFI drvier with the pair of protocols implementing whatever File System you want! Or, implement it in your OS loader only without letting others to use it as would be the case in the former variant. Anyways, your custom filesystem needs to be implemented, do you realize this? And the specification doesn't forbid you of doing so. It's frigging "extensible" to fit every bullshit anyone wants to put there. why are you saying this silliness? They chose FAT because it's hell simple and fast and I'd say they would be stupid if they haven't chosen this FS for the system partition. "MS paid". LOL.

Just for clarification.
The UEFI FILE (I was implying the word FILE here, zaval couldn't understand the context of this thread, so now it's updated.) itself must be on the FAT12 ( Floppy ), FAT16 ( Floppy ), or FAT32 ( HD / USB / CD ) partition. But your OS can reside on another partition under your own file system.

In my tutorial, I shoved everything into one partition. But once you understand how it works, you will see that you can create two partitions. One with FAT32 on it ( hundred megs ) with the UEFI boot file on it, and the other partition set for your OS.

EDIT UPDATE : Also, I see some confusion in various forums about UEFI. The confusion is whether you are a hardware engineer or an operating system engineer. Because all of the info about UEFI seems to be lumped together.
GNU-EFI is geared for OS developers. TianoCore is geared for Hardware BIOS [Firmware] developers ( With OS development mixed in ). That is the simplest way for me to explain it. Although that may not be 100% accurate, that is how I have come to understand it at this time.
It was for this reason why I chose GNU-EFI in the tutorial. I am assuming this is also why OSDev.org website chose it as well for their tutorial. ( I could be wrong ?)

[zaval]

Just for clarification.
The UEFI itself must be on the FAT12 ( Floppy ), FAT16 ( Floppy ), or FAT32 ( HD / USB / CD ) partition. But your OS can reside on another partition under your own file system.

Well, no. UEFI by itself will reside in a so called Firmvare Volume on a NOR Flash device in most cases (on PCs, - for sure). There is such a specification, called PI, it covers this. Of course, implementers are free to choose their own format for the FW storage.
Firmware Volume is such a quite primitive format oriented on flash based devices, it has no directories, everything is straight thrown into a pile of files. But there is notion of "sections" inside a file. For example real PE executable files will be sections inside of those. But again, it's only how PI does it. And thus - Tianocore/EDK.
This excludes the earliest phase, the residing of which and the format depends on a CPU architecture. It's basically a blob hardcoded into a ROM. And the processor starts this code after Reset. I don't know how it's on x86, but on MIPS and ARM it's called ROM code, sometimes - mask ROM code, Boot ROM, BROM, it's a secretive thing CPU/SoC vendor produces. Then you (as a FW provider) include yourself in the chain of execution in a way this thing expects. And it starts FW. For example on MIPS Creator CI20 SBC, you just need to put a "magic" word (4 bytes) before your code and right after MBR on an SD card and ROM code happily load you in SRAM and will transfer control to you. Other vendors are more picky and not that talkative on what ROM code does expect. But it's hard to cover, because all they use uboot as a "FW" and they are required to open its code, so you easily can find out what their ROM code does expect. Anyway, the firmware by itself doesn't reside on a EFI System Partition (ESP).
This is ESP, where OS loaders and different vendor _bloat_ utilities reside, should be in one of the FAT formats. This is not considered UEFI by itself. And the fact of requiring the ESP FS format in no way limits an OS vendor to have their OS been put everywhere else. It's up to their OS loader to understand those rules and protocols.

In my tutorial, I shoved everything into one partition. But once you understand how it works, you will see that you can create two partitions. One with FAT32 on it ( hundred megs ) with the UEFI boot file on it, and the other partition set for your OS.

EDIT UPDATE : Also, I see some confusion in various forums about UEFI. The confusion is whether you are a hardware engineer or an operating system engineer. Because all of the info about UEFI seems to be lumped together.
GNU-EFI is geared for OS developers. TianoCore is geared for Hardware BIOS [Firmware] developers ( With OS development mixed in ). That is the simplest way for me to explain it. Although that may not be 100% accurate, that is how I have come to understand it at this time.
It was for this reason why I chose GNU-EFI in the tutorial. I am assuming this is also why OSDev.org website chose it as well for their tutorial. ( I could be wrong ?)

I am trying to create my own UEFI implementation for MIPS/ARM SBCs and I don't use either of those, - I have different needs. But I'd recommend using EDK - they do it better than GNU haha.

[BenLunt]

Just to say up front first, I love OSDev.org. Great work !!
I am writing this because I found some spotty info at best about how to get a UEFI USB Thumb Drive to boot on a UEFI Class 3 computer. So I wanted to share what I found out through a lot of trial and error.

Thank you DiggDug for this. I am in the process of updating my documentation on booting from UEFI and the information here should be helpful.

I have booted UEFI on 32-bit machines, but have yet to on a 64-bit machine. However, this is due to the fact that I haven't tried a 64-bit machine yet. Soon.

Thanks again,
Ben
- http://www.fysnet.net/osdesign_book_series.htm

[DiggDug]

Just for clarification.
The UEFI itself must be on the FAT12 ( Floppy ), FAT16 ( Floppy ), or FAT32 ( HD / USB / CD ) partition. But your OS can reside on another partition under your own file system.

Well, no. UEFI by itself will reside in a so called Firmvare Volume on a NOR Flash device in most cases (on PCs, - for sure). There is such a specification, called PI, it covers this. Of course, implementers are free to choose their own format for the FW storage.
Firmware Volume is such a quite primitive format oriented on flash based devices, it has no directories, everything is straight thrown into a pile of files. But there is notion of "sections" inside a file. For example real PE executable files will be sections inside of those. But again, it's only how PI does it. And thus - Tianocore/EDK.
This excludes the earliest phase, the residing of which and the format depends on a CPU architecture. It's basically a blob hardcoded into a ROM. And the processor starts this code after Reset. I don't know how it's on x86, but on MIPS and ARM it's called ROM code, sometimes - mask ROM code, Boot ROM, BROM, it's a secretive thing CPU/SoC vendor produces. Then you (as a FW provider) include yourself in the chain of execution in a way this thing expects. And it starts FW. For example on MIPS Creator CI20 SBC, you just need to put a "magic" word (4 bytes) before your code and right after MBR on an SD card and ROM code happily load you in SRAM and will transfer control to you. Other vendors are more picky and not that talkative on what ROM code does expect. But it's hard to cover, because all they use uboot as a "FW" and they are required to open its code, so you easily can find out what their ROM code does expect. Anyway, the firmware by itself doesn't reside on a EFI System Partition (ESP).
This is ESP, where OS loaders and different vendor _bloat_ utilities reside, should be in one of the FAT formats. This is not considered UEFI by itself. And the fact of requiring the ESP FS format in no way limits an OS vendor to have their OS been put everywhere else. It's up to their OS loader to understand those rules and protocols.

-SNIP-

I am trying to create my own UEFI implementation for MIPS/ARM SBCs and I don't use either of those, - I have different needs. But I'd recommend using EDK - they do it better than GNU haha.

Yea, I can't argue with anything your saying here, because as you said, you are using ARM and MIPS which has a totally different architecture compared to the x86_64. Everything I have spoken about is for the x86_64 only. In fact, this thread is only about the x86_64 UEFI Class 3. Not anything else. And the UEFI Class 3, requires a FAT system on the x86_64 for the bootable UEFI Class 3. Whether that is FAT12/FAT16 for Floppy and FAT32 for CD, USB or Hard Drive. I have yet to see any documentation saying otherwise. ( Although technically USB and CD could be considered removable Media, but as I have personally tested, neither FAT16 nor FAT12 would work for me on the USB Thumb Drive. FAT32 was the only one that worked for me. )

If you want proof of the FAT getting used for the UEFI partition formatting, read the UEFI Spec 2.7 PDF.
Source: http://www.uefi.org/specifications

Read section 13.3. It verifies this.

When you say "Well, no. UEFI by itself will reside in a so called Firmvare Volume on a NOR Flash device in most cases (on PCs, - for sure)." that is basically saying the same thing that this whole thread gives a tutorial on. How to setup a GPT Volume with an EFI Partition so that the EFI file, which is in the EFI/Boot/ folder, can be recognized by the Hardware. So not sure why you are disagreeing with me, even though you are saying the same thing I am, but in a different way.

Thank you DiggDug for this. I am in the process of updating my documentation on booting from UEFI and the information here should be helpful.

I have booted UEFI on 32-bit machines, but have yet to on a 64-bit machine. However, this is due to the fact that I haven't tried a 64-bit machine yet. Soon.

Thanks again,
Ben
- http://www.fysnet.net/osdesign_book_series.htm

You are welcome.

Keep in mind, this tutorial was only meant for the UEFI Class 3. None of this tutorial was meant for older UEFI versions. (Although you might could use it as a jumping point for older versions.) If you do follow this tutorial, you MUST have a Modern UEFI Class 3 based computer. The only reason I know this is because I also have a UEFI Class 2 ( CSM ) computer, and this tutorial failed to boot on it. But any of the Class 3 computers booted without issues. I have tested on 4 machines so far. 3 successful, 1 failed. The reason is because I do not have a 16-Bit boot loader as a fallback in this tutorial (or on my USB Thumb Drive for that matter). I am sure the kernel would boot if I had a 16-Bit boot loader, included with the UEFI setup, that would work on the older UEFI ( CSM ) computers.

Digg

[zaval]

When you say "Well, no. UEFI by itself will reside in a so called Firmvare Volume on a NOR Flash device in most cases (on PCs, - for sure)." that is basically saying the same thing that this whole thread gives a tutorial on. How to setup a GPT Volume with an EFI Partition so that the EFI file, which is in the EFI/Boot/ folder, can be recognized by the Hardware. So not sure why you are disagreeing with me, even though you are saying the same thing I am, but in a different way.

What you missed of what I said is what lays inside \efi\boot is not UEFI. BOOTX64 is an EFI application (OS loader, installer). I disagreed just because you said that UEFI needs to be on FAT. It doesn't. Firmware Volume is NOT EFI System Partition. It's another thing, out of the scope of the UEFI specification.

And the UEFI Class 3, requires a FAT system on the x86_64 for the bootable UEFI Class 3.

It's a requirement (not spefic to x64), but it still doesn't preclude support of other FSs. Read the section "Boot Mechanisms" (3.4, 3.4.1 for v2.4_ERRATA_B). Here is an excerpt:

The format of the file system specified is contained in Section 12.3. While the firmware must produce an EFI_SIMPLE_FILE_SYSTEM_PROTOCOL that understands the UEFI file system, any file system can be abstracted with the EFI_SIMPLE_FILE_SYSTEM_PROTOCOL interface.

FAT is only a default FS that is guaranteed to be supported by the FW. Still, anything else could be added.
See, the specification does only require that the implementation produced EFI_SIMPLE_FILE_SYSTEM_PROTOCOL for FAT FSs. But it doesn't say, no other FSs can be supported the very same way! Boot Manager looks for boot options and parsing Device Paths of those, uses supplied EFI_SIMPLE_FILE_SYSTEM_PROTOCOL instances "attached" to devices. It's up to the implementer what FS to support. *sigh

In general, if one wants to embed another FS in the FW, he/she supplies the UEFI driver that implements needed protocols and the FS in question. He/she installs it on a platform and voila - UEFI now can boot from that FS.
Also, all these bootx64.efi and company are just a fallback scenario (again, look at the "Boot Mechanisms" section). The OS loader could be anywhere you want, just create a proper boot option, containing right Device Path to your OS loader, and that's it. this is what UEFI booting is about. Yeah, unfortunately, the real implementations are way more whimsy.

[DiggDug]

When you say "Well, no. UEFI by itself will reside in a so called Firmvare Volume on a NOR Flash device in most cases (on PCs, - for sure)." that is basically saying the same thing that this whole thread gives a tutorial on. How to setup a GPT Volume with an EFI Partition so that the EFI file, which is in the EFI/Boot/ folder, can be recognized by the Hardware. So not sure why you are disagreeing with me, even though you are saying the same thing I am, but in a different way.

What you missed of what I said is what lays inside \efi\boot is not UEFI. BOOTX64 is an EFI application (OS loader, installer). I disagreed just because you said that UEFI needs to be on FAT. It doesn't. Firmware Volume is NOT EFI System Partition. It's another thing, out of the scope of the UEFI specification.

And the UEFI Class 3, requires a FAT system on the x86_64 for the bootable UEFI Class 3.

It's a requirement (not spefic to x64), but it still doesn't preclude support of other FSs. Read the section "Boot Mechanisms" (3.4, 3.4.1 for v2.4_ERRATA_B). Here is an excerpt:

The format of the file system specified is contained in Section 12.3. While the firmware must produce an EFI_SIMPLE_FILE_SYSTEM_PROTOCOL that understands the UEFI file system, any file system can be abstracted with the EFI_SIMPLE_FILE_SYSTEM_PROTOCOL interface.

FAT is only a default FS that is guaranteed to be supported by the FW. Still, anything else could be added.
See, the specification does only require that the implementation produced EFI_SIMPLE_FILE_SYSTEM_PROTOCOL for FAT FSs. But it doesn't say, no other FSs can be supported the very same way! Boot Manager looks for boot options and parsing Device Paths of those, uses supplied EFI_SIMPLE_FILE_SYSTEM_PROTOCOL instances "attached" to devices. It's up to the implementer what FS to support. *sigh

In general, if one wants to embed another FS in the FW, he/she supplies the UEFI driver that implements needed protocols and the FS in question. He/she installs it on a platform and voila - UEFI now can boot from that FS.
Also, all these bootx64.efi and company are just a fallback scenario (again, look at the "Boot Mechanisms" section). The OS loader could be anywhere you want, just create a proper boot option, containing right Device Path to your OS loader, and that's it. this is what UEFI booting is about. Yeah, unfortunately, the real implementations are way more whimsy.

I now see where the confusion is. First, You are refering to the older UEFI PDF version. To clarify, I have only been using UEFI 2.7 as my source, which has the latest changes and fixes. And Second, as I repeatedly said, I am ONLY talking about the UEFI Class 3 x86_64 hardware. And it's a different beast compared to the older UEFI versions. ( Older versions allow Legacy Booting, the Modern UEFI version does not. )

I see that you think I am talking about the Firmware, but I was not. I will quote a portion of what it says in the latest UEFI specs :

13.3 File System Format
The file system supported by the Extensible Firmware Interface is based on the FAT file
system. EFI defines a specific version of FAT that is explicitly documented and testable.
Conformance to the EFI specification and its associate reference documents is the only
definition of FAT that needs to be implemented to support EFI. To differentiate the EFI
file system from pure FAT, a new partition file system type has been defined.
EFI encompasses the use of FAT32 for a system partition, and FAT12 or FAT16 for
removable media.

Although I was not talking specifically about a LOAD FILE PROTOCAL or anything related to section 3 of the UEFI Specs that you thought I was talking about, but I was broadly hinting that the user who follows this tutorial on this thread, could take things a step further, and make a seperate partition and have the EFI file, that resides on the FAT32 partition, point to the other partition with their own File System on it that the OS Developer has designed. The tutorial on this thread has only been about what you need in order to boot on x86_64 hardware and where to place that file.

You seem to think I am lumping both the UEFI Firmware with the UEFI Boot File, which is not true. In this whole tutorial, I had been talking about the FILE itself that resides on the FAT32 partition. I was not talking about the UEFI Firmware.

I hope this clarifies it for you.

Digg

[zaval]

This gets hopeless... You seem unwilling to understand what I am writing. Apparently you don't understand parts about UEFI I am talking about. Reading "Boot mechanisms" would definitely help with this btw. Not because "i was thinking, you talk about it", - because it covers the subjects open in here. But well, forget it. Nice job anyway!
Just the last clarifaction on my side. The only reason I reacted was these words by you:

Just for clarification.
The UEFI itself must be on the FAT12 ( Floppy ), FAT16 ( Floppy ), or FAT32 ( HD / USB / CD ) partition. But your OS can reside on another partition under your own file system.

The bolded text is not true. It has nothing to do with the newer specification nor with x64. Actually there is nothing FS specific, specific for x64. In fact, the whole boot procedures must be the same across every supported platform.

And as I've said "UEFI itself" sits on NOR Flash chip without any file system on it. Well, there is FS, but it's not FAT. What it is, you can learn, if you want, from the PI specification, vol. 3, sections 2, 3. But again - it's out of the UEFI specification! And is not mandatory. It's just how EDK does it and how it is on so beloved by you Class 3 platforms.

Neither UEFI itself nor even an OS loader need to be on FAT of any kind. Even that "specific", yet another FAT, defined by UEFI for some incomprehensible reasons. What is really required is support of FAT by the FW implementation.

PS. I bet that Apple for example uses their own FS instead of FAT. I don't know actually, but wouldn't be surprised.

But I feel my clarifications go into nowhere. Don't perceive them as "disagreeing" or hinting that "you are not cool". I just am interested in this, work on it and made my humble rectification about your wrong statement. Ah and the ungrounded Octacone rant.

[DiggDug]

Just for clarification.
The UEFI itself must be on the FAT12 ( Floppy ), FAT16 ( Floppy ), or FAT32 ( HD / USB / CD ) partition. But your OS can reside on another partition under your own file system.

The bolded text is not true.

Of course that is not true if you are coming from a Firmware point of view. Your whole reply seems to suggest this. That line you quoted is obviously implying that I meant the file in that sentence. Who puts a Firmware on a partition ? Again, I was refering to the FILE. NOT the Firmware as I have said in my previous post. But I updated the posts for you since you can't seem to let it go.

As for the FAT system, you keep saying it is not required, and yet, the UEFI Specs PDF says it does, and then you come back and say the same thing the PDF said in the first place. I feel like we are going around in circles here. Since you are not grasping the context of this whole thread, you are combining info with this thread that has nothing to do with this thread at all. And that is where you seem to be confused.

The latest UEFI Speccs PDF says "EFI encompasses the use of FAT32 for a system partition, and FAT12 or FAT16 for removable media."

I do not understand how you can't seem to grasp that. So for your further reading, go here :
https://wiki.osdev.org/UEFI

I QUOTE
UEFI firmware expects UEFI applications to be stored in a FAT12, FAT16, or FAT32 file system on a GPT-partitioned disk.

Which totally agrees with what is said in the latest UEFI Specs PDF. And this whole thread has ONLY been about the GPT-Volume / EFI-Partitioned Disk, and that EFI file which is to be placed on that FAT32 EFI Partition, which is obvious since it's in the tutorial on how to set that up on the USB Thumb Drive in the first post of this thread. Even the title of this theads explains this is about the x86_64 Hardware. You claim you are making UEFI for MIPS and ARM, and using an older UEFI version, which is totally off topic of this thread. And thus, you are making this thread longer then it needs to be.

In the Forum Rules, read FORUM RULE 5.


Digg

[zaval]

Whatever, man.

As for the FAT system, you keep saying it is not required

This is the problem - you don't read or understand. I never said this. Oppositely - I said the specification does require its support. What I said is the OS loader file is not necessarily should be on FAT FS. Can you get it? that thing that you keep calling FILE or, sometimes, - UEFI and then blaming me I don't get the context. even this frigging FILE thing doesn't need to be on FAT. Per the spec, it's up to the implementer what FS support (in addition to FAT) they provide and thus what FS they choose for their volumes. Remember the Apple example.

Do I need to remind you that it wasn't me who came up and started to make wrong "clarifications"?

And if you ask me, then I tell you - I totally understand the context of this thread, - you managed to compile a "Hello world" efi app and run it. That's an incredible success. Congrats on this, wish you farther progress. I just have been reading this specification a little, for two years, to this date, and I saw wrong statements made here and reacted on them. I second to you again 1) there is nothing x64 specific about booting apart from the name for a file for the fallback scenario and removable media. 2) The specifications are almost bit by bit the same across the latest versions. At least regarding Boot. And if you don't believe me, then here is the excerpt from the latest version:

The format of the file system specified is contained in Section 13.3. While the firmware must produce an EFI_SIMPLE_FILE_SYSTEM_PROTOCOL that understands the UEFI file system, any file system can be abstracted with the EFI_SIMPLE_FILE_SYSTEM_PROTOCOL interface

So, who "can't grasp"? On UEFI, everything outside Boot and Runtime Services get support through protocols/interfaces. File systems as well. It's exactly this EFI_SIMPLE_FILE_SYSTEM_PROTOCOL that is in charge of this. Say you are a vendor, that produces computers and ship your FW with them. and installation media with your OS. You embed FancyFS support into your UEFI machine, - you make a UEFI driver for this and it, on intializing, attaches the instance of EFI_SIMPLE_FILE_SYSTEM_PROTOCOL to the FancyFS formatted volumes. For example those, found on removable devices, that were being inserted - the media with the OS mentioned above. let it be an SD card. On its insertion, your FW recognizes FancyFS volume and the driver attaches the FancyFS understanding EFI_SIMPLE_FILE_SYSTEM_PROTOCOL instance to this volume. Now, if the process gets to the point that the Boot Manager of the FW decides that this boot option is the only thing to do, or the user commands that through the "boot from a file" option, then the file found in the \efi\boot directory of the FancyFS formatted SD card, say bootaa64.efi, cause it's 64 bit arm, bitches, gets started. And there is no FAT at all! Because the FS is Fancy FS. UEFI deosn't dictate FS format for the media. It just requires FAT as the required provision that every FW implementation should supply. I depicted you a "removable" media boot scenario involving no FAT. It of course could be on an x64 machine too.

So the whole "confusion" is around you trying to make it look like you haven't said this:

Just for clarification.
The UEFI itself must be on the FAT12 ( Floppy ), FAT16 ( Floppy ), or FAT32 ( HD / USB / CD ) partition.

Or that it had some other sense than what it sounds like.
You made a wrong statement, I reacted on it. that's all. who needs to get over it?

And to be prefectly honest, this thread is not about "Boot USB Drive". There is just nothing to talk about with this respect. Booting from USB sticks was a "miracle" for BIOS systems. For UEFI, it's just an ordinary thing. It's a normal scenario for a proper UEFI implementation - support of removable storage devices. USB based too. They don't even need to be GPT partitioned. There could (and in fact, it's preferable, - should) be an option in the Boot Manager menu, called "Boot from a file", where user can direct FW to the specific file he/she wants to take control to. It just needs to be reachable in terms of UEFI. USB sticks are of such ilk. Even if they are just MBR formatted.
Your tutorial is basically a proof of concept that it's possible to compile the wiki "Hello World" app and run it from a USB stick using some UEFI capable computer.

[DiggDug]

I wish you the best in proper knowledge. Good day to you.

Digg