overdevelopment of virtual firmware [discussion]

[austanss]

I'd like to open up this thread to vent about my problems with virtual firmware, in particular OVMF for QEMU.

There's a few problems with them that can hinder the OS development experience.

1. Over-compatibility
2. Availability
3. Detachment

As for availability, the only problem I have in that regard is that OVMF is only distributed in RPM packages.

As for over-compatibillity...
The OVMF firmware comes with a CSM compatibility layer. Which is perfectly fine. It became a problem when I realized that OVMF actively allowed you to use text mode in UEFI applications when using the CSM module.

As for detachment...
It seems like it defeats the purpose of an emulator. When you are trying seriously to develop an OS, you want it to work on real hardware. So this means support for all new computers, as well as some slightly older computers (so you go as far back as 2007, after that it's a bit more effort for no more reward). QEMU and its related firmware (SeaBIOS/OVMF) give you too much compatibility, that it shocks you when it doesn't work on real hardware. Hell, it won't even work in fucking VirtualBox! This might anger some people who want to be serious about their OS, including me.

What are your thoughts?

[Yes, I'm talking to you, "Bing [Bot]".]

[nexos]

It should be in /usr/share/ovmf on Debian distros. If it is not, run

Code: Select all

sudo apt install ovmf 

On Debian distros

[austanss]

It should be in /usr/share/ovmf on Debian distros. If it is not, run

Code: Select all

sudo apt install ovmf 

On Debian distros

oh. well then i suppose i put in a lot of work for little in return. anyway, what are your thoughts on the other subjects? you have a relatively mature OS, you should have run into one of these problems.

[nexos]

For the detachment issue, I test in QEMU with OVMF. Then every so often, I test my kernel on my laptop. I haven't tested in Virtualbox. I need to do so. Also, my UEFI desktop from 10 years ago I don't think has GOP, that needs research. But I do agree that OVMF seems different then real hardware.

[austanss]

I guess this was an underwhelming discussion...

[nexos]

I guess SeaBIOS and OVMF are actually well designed and aren't insane like half of the BIOSes out there are .

[Octocontrabass]

It became a problem when I realized that OVMF actively allowed you to use text mode in UEFI applications when using the CSM module.

How exactly are you doing that?

Hell, it won't even work in fucking VirtualBox!

VirtualBox also uses OVMF...

I guess this was an underwhelming discussion...

After only two hours? I think you're giving up too soon.

OVMF [...] well designed

After running headfirst into several bugs surrounding the signedness of enums, I'm going to have to disagree with you there!

[zaval]

For the detachment issue, I test in QEMU with OVMF. Then every so often, I test my kernel on my laptop. I haven't tested in Virtualbox. I need to do so. Also, my UEFI desktop from 10 years ago I don't think has GOP, that needs research. But I do agree that OVMF seems different then real hardware.

if it's UEFI (more than EFI 1.2), there must be GOP, unless it's non-compliant.

[austanss]

Hell, it won't even work in fucking VirtualBox!

VirtualBox also uses OVMF...

OK, I did exaggerate there. The reason it worked in QEMU and not VirtualBox is because I had a bug that broke the bootloader if a serial port didn't exist. And when I gave VirtualBox a serial port, I have to admit it did work...

[zaval]

As for availability, the only problem I have in that regard is that OVMF is only distributed in RPM packages.

I am a Windows user, having no idea what that .rpm is inside. still, by uisng 7zip, it was super easy to extract the FW from that package.

As for the topic, I don't really get what is the problem? use them for exactly what you want and don't use for what you see is a "detachment" or "over-compatibility".

[nexos]

For the detachment issue, I test in QEMU with OVMF. Then every so often, I test my kernel on my laptop. I haven't tested in Virtualbox. I need to do so. Also, my UEFI desktop from 10 years ago I don't think has GOP, that needs research. But I do agree that OVMF seems different then real hardware.

if it's UEFI (more than EFI 1.2), there must be GOP, unless it's non-compliant.

Interesting, I'll have to check

After running headfirst into several bugs surrounding the signedness of enums, I'm going to have to disagree with you there!

Well, I guess not well designed, but better then a lot of firmware

[rdos]

My opinion is that an OS only running in an emulator is not a real OS.

I guess I'm biased though since I didn't have access to emulators and wrote one myself for really basic stuff.

Still, I always do tests on real hardware. If it doesn't work on real hardware then it's useless.

[bzt]

As for availability, the only problem I have in that regard is that OVMF is only distributed in RPM packages.

You can also compile it yourself. There are plenty of pre-compiled OVMF images on github (in .fd format), pick your choice.

The OVMF firmware comes with a CSM compatibility layer. Which is perfectly fine. It became a problem when I realized that OVMF actively allowed you to use text mode in UEFI applications when using the CSM module.

I don't understand your problem here. When you develop for UEFI, you shouldn't enable CSM in the first place.

It seems like it defeats the purpose of an emulator. When you are trying seriously to develop an OS, you want it to work on real hardware. So this means support for all new computers, as well as some slightly older computers

Virtual machines never meant to emulate actual hardware. They are just another (simulated) implementation of a hardware. Think about them as if you had more test beds (qemu with one config = one machine, qemu with another config = another test machine, VB = yet another machine to test your OS on etc.).

I regularly test my OS with qemu+bios, qemu+ovmf, bochs+bios, vb+bios, vb+ovmf. It must work on all of these VMs as well as on real machines. I wish I could test it on VMWare too, but I don't have that.

it shocks you when it doesn't work on real hardware.

It shouldn't shock you. A VM is just another machine. Just because your OS runs on one machine, it's not surprising that it won't run on another.

VirtualBox also uses OVMF...

Yes, a specially configured and compiled one. I've found many run-time differences between vanilla OVMF and the VB OVMF version. Which is good, because you can test two completely different machines with them

After running headfirst into several bugs surrounding the signedness of enums, I'm going to have to disagree with you there!

For different reasons, but I completely agree!

if it's UEFI (more than EFI 1.2), there must be GOP, unless it's non-compliant.

That's true! Hardware not supporting GOP should be considered an old legacy machine.

The reason it worked in QEMU and not VirtualBox is because I had a bug that broke the bootloader if a serial port didn't exist.

Chances are good you will run into the same problem on some of the real machines too! My advice is, make sure your OS runs on ALL virtual machines you have access to.

My opinion is that an OS only running in an emulator is not a real OS.

I've seen OSes specifically designed for VM only, using a minimal driver stack and tweaked specifically for a VM solution. But for a general purpose OS I agree.

Cheers,
bzt