EFI and the future of BIOS

[Combuster]

I was assuming that things like [eax*8+ebx+1234] wouldn't occur in real mode because it doesn't have segments capable of handling 32-bit addresses - it might well be possible to run the instruction, but the result would be an unusable address and therefore pointless, and that's why I don't expect to find that in video BIOS code.

If some bios developer had the idea of using GCC to generate tiny-model real mode code, then it will use 32-bit addressing exclusively. As long as the effective address does not overflow 16 bits, the instruction is valid, and works on real hardware in all circumstances. And it can save you some instructions or register pressure even if someone did decide to do the assembly manually. You just won't notice it happening if you use v8086 or someone else's emulator that wasn't designed with a built-in aneurysm.

Operand size prefix support are pretty much a requirement, otherwise you won't do 32-bit INs and OUTs

[DavidCooper]

Also, the one hour per instruction is a rediculous claim.. it's more like 5 minutes per instruction if you're framework isn't a complete pile. And depending on the instructions, some of them are much faster if you can copy/paste from a similar instruction. Yes, push took all of 20 seconds to write, pop took another 20. Call was a bit more complex as I had to call push, then update IP... ret was almost as hard calling pop, then updating ip. Add/sub/mul/div weren't really much harder.. the hardest part was making sure you had the correct src and dest (that encoding is a bit odd at first, but once you have it, it works for all instructions). Really none of the instructions are that difficult, especially if you write your framework in a somewhat organized manner.

I made a start on it last night, using mov eax,[ebx+4*eax] to convert the first instruction byte into an address to handle a specific instruction, and I then wrote enough code to handle about half the instruction set (the easy half where it only takes about ten instructions to emulate one instruction). It still looks as if a week should be enough to complete the job, but Brendan's right to point out the problems that may lie ahead, even if they only affect a small number of machines. What we need to be careful to avoid is running part of the code in a BIOS and then hitting a problem that results in a crash, so the emulator really needs to check the code the first time out to make sure it can handle it (and that means checking it a number of times with different inputs).

[Cognition]

We're trending towards native drivers for sure. Personally I think the best bet going forward to is to target GPUs that are integrated into CPUs and relatively well documented (Intel provides full documentation and AMD at least updates some register layout heads for XOrg currently). I don't think it's too unrealistic to expect new PCs going forward to have the integrated gpu doing most of the work for desktop and video acceleration, with a discrete card in there specifically for 3D work. All and all it should be a pretty positive change as it will once again be realistic to expect some level of hardware acceleration which sis accessible to the hobbyist OS developer. Yeah there will be a bit more ground work, but it shouldn't be anything insurmountable given what you'll need to have in place at that point.

Personally I plan on keeping VBE around as a fall back, but I'd also stress that the ability to switch graphics modes while the kernel is running probably isn't as crucial as a lot of people think it is.

[DavidCooper]

I was assuming that things like [eax*8+ebx+1234] wouldn't occur in real mode because it doesn't have segments capable of handling 32-bit addresses - it might well be possible to run the instruction, but the result would be an unusable address and therefore pointless, and that's why I don't expect to find that in video BIOS code.

If some bios developer had the idea of using GCC to generate tiny-model real mode code, then it will use 32-bit addressing exclusively. As long as the effective address does not overflow 16 bits, the instruction is valid, and works on real hardware in all circumstances. And it can save you some instructions or register pressure even if someone did decide to do the assembly manually. You just won't notice it happening if you use v8086 or someone else's emulator that wasn't designed with a built-in aneurysm.

Operand size prefix support are pretty much a requirement, otherwise you won't do 32-bit INs and OUTs

Thanks for the tip - I still suspect that they're so pushed for space that they'll be keen to avoid bloating their code with prefixes, but it should be easy enough to cover for the possibility. Rather than spending weeks or months covering all possibilities from the outset, I'm going to try to do it with the minimum amount of work and get the emulator to back out with a warning message (hidden from ordinary users - the OS will quietly switch to direct real mode for running BIOS code), and feedback from people testing it will indicate how many machines are causing the emulator problems and give specific information on which instructions are causing them. That will tell me whether it's worth going to the trouble of adding to the emulator to support those instructions - if many machines are affected, it clearly will be, but if it's only a few it might be a waste of time. Aiming for perfection invariably means that you never get anything finished, so you have to make sensible compromises.

[DavidCooper]

I'd also stress that the ability to switch graphics modes while the kernel is running probably isn't as crucial as a lot of people think it is.

On a new machine you're certainly right - now that I've tried it out properly I can see that even a cheap netbook can handle the screen memory so fast that there's absolutely no point in using anything less than an 8:8:8 colour mode.

[Owen]

EFI is becoming mainstream (slowly), and you'll be able to use GOP to set your mode, and nobody will have to worry about writing a specific driver for setting a screen mode, so in that case, either method is a waste of time.. unless of course you want it to work sometime soon on more than one chipset.

GOP only works at boot time

For supporting multiple GPUs: AMD have something called AtomBIOS, and nVIDIA have something similar. These are both bytecode architectures which the real mode BIOS and the operating system drivers invoke in order to perform certain card specific operations (therefore abstracting the driver from highly variable and rarely trodden portions of the chip)

Neither of these will get you acceleration, but they will get you mode setting (and more, e.g. RAM downclocking for power saving)

[Ready4Dis]

I apologize, I didn't realize it was a pre-boot environment protocol only. Well, I guess implenting the bytecode interpreter for AMD/Nvidia cards is a way to do it, however you'll still be stuck only supporting those cards that are supported (aka, not older cards). I will probably implement the AtomBIOS interpreter in my OS, but I'm still keeping my x86 emulator as a fallback so I can support many old and new cards. I will slowly add 'proper' support for specific cards, but until then, emulation takes care of the rest.

[~]

I fully expect that after everyone gets used to UEFI most of the old hardware (everything except CMOS/RTC?) will be removed from the chipset, and a PC BIOS layer won't be possible (without expensive full system emulation/virtualisation) because the hardware needed for compatibility doesn't exist anymore. I'd estimate that it won't be too long before we start seeing "legacy free UEFI" systems, starting from embedded systems and thin clients (maybe in as little as 2 years), and then moving up to mobile devices. After maybe 5 years "legacy free UEFI" might leap over to high-end servers, and eventually find it's way to normal desktop/server systems after that.

The tricky thing for OS developers is not just supporting UEFI, but also making sure their code doesn't rely on the existence of legacy hardware. For example, in the past (during boot, for starting other CPUs) I've assumed there's a PIT and assumed the PIT's IRQ is routed via. the PIC chips (until after the OS sets up IO APIC) - that's 2 assumptions that I wouldn't be comfortable with for UEFI for a start.

Aren't things becoming too dificult for a single person to achieve a practical development of a product?

If everything keeps being as standarized as regular PCs, it might not be too bad, but if PC hardware and specification become as closed and non-regular as in the case of mobile devices and their endless variations right from the base machine, then it would become a big problem for many people, maybe most of them. The ideal thing would be to standarize the whole PC instead of complicating it further. It's enough with all of the different hardware vendors and hundreds of different drivers.

In such a case, I think that completely emulating a classical PC would be very useful. First off, it could serve as the basis of a virtual PC platform, with absolutely all of its "hardware" normalized and publicly known, and could serve to simplify and turn any current machine, including tablets, into an easily programmable PC.

That can prove as useful and rewarding, as well as relevant, as creating yet another OS. On impossibility of creating hardware, let hardware vendors create it as they like, but attempt to make it look like a standarized machine (on top of another OS?) to get something actually done in real life and to mainly work with.

Of course that's just an option, but maybe it would be easier to allow a single person progress in one way instead of pretending to master so many emerging and highly complex new platform standards so easily every couple of years without even having mastered the previous one, and then suddenly having to potentially start over.

[Solar]

Single persons simply don't matter for the industry. There's Microsoft, there's Apple, Linux might cope or not as far as the hardware manufacturers are concerned, and they don't give a fiddly bit about the rest. Why should they?

[Cognition]

Honestly most legacy devices already have standardized replacements PIT -> HPET, PIC -> IOAPIC, PS2 KB/Mouse -> USB HID, etc. I wouldn't be terribly concerned unless Intel stopped publishing specs as they tend to more or less dictate what gets standardized and actually design a good amount of the chipsets that contain these devices.

[~]

Single persons simply don't matter for the industry. There's Microsoft, there's Apple, Linux might cope or not as far as the hardware manufacturers are concerned, and they don't give a fiddly bit about the rest. Why should they?

That's exactly the point. They should care if they expect to have any hope of making computing a little bit easier for developers in general. And of course they don't care. And so it looks like OS development is becoming less and less a practical achievable goal for individuals, generally speaking. It wasn't exactly so in the past. That wouldn't be a reason to desist, but it's a very important thing to keep in mind.

Honestly most legacy devices already have standardized replacements PIT -> HPET, PIC -> IOAPIC, PS2 KB/Mouse -> USB HID, etc. I wouldn't be terribly concerned unless Intel stopped publishing specs as they tend to more or less dictate what gets standardized and actually design a good amount of the chipsets that contain these devices.

Yes, as always it is good to know that the base PC machine is standard; but not beyond.

All modern existing hardware as a whole doesn't get near to the wider documentation as hardware in the past. That's the first point of difficulty. It could potentially turn a developer who wants to work in OS development into a "mediocre" or limited attempt of a driver developer. At least it seems to me that there are still too many different drivers to be done for video and acceleration, sound, network, etc., and of course new things like would be EFI/UEFI only machines; and all that work could only be done best if developers dedicated 100% of their time to OS and system-level development. Or their efforts maybe could be better aimed to bring practical benefits first to most users before trying yet another OS without thinking twice how much good it will be. For instance "migrating" more and more OS development techniques to user space for better software, and documentation about them, to make even regular users more familiar with that effort, among other things I might not even think about right now.

[Ready4Dis]

It's good and bad. Think about it this way: You are a new developer, and only want to support newer machines (UEFI) only. You decide on a 64-bit kernel... well, UEFI will load and run your kernel in 64-bit mode without jumping through hoops like you 'used' to have to do (like first booting into real mode, then jumping to pmode, finally jumping to long mode). So, it makes all the beginning parts easier, no need to write your own bootloader (unless you write it for UEFI), no jumping between modes, etc. Sure, you lose the ability to drop back to bios mode and read from a usb stick, but that is frowned upon anyways.

Why would companies reveal their documentation. What does that gain them financially? How much money have you brought in for dlink because of your OS? What about netgear? The answer is nothing, nada. So, what motivation do they have to release their resources too? None. Your OS is nothing to them, and probably will never be anything, how many OS's here have gained legitimate market share? Why in the world would they waste any effort on a lost cause? Yes, it'd be great if they did, and honestly wouldn't be hard for them, since they DO have documentation for m$ and others, but they really don't have a reason (business speaking).

The biggest reason right now, is due to linux, especially since they mostly support manufacturers who have open drivers, or open documentation. In this case, they are losing some money on potential customers, so do have a legitimate reason to want to release docs. Linux is pretty big at this point (not so much as a desktop OS, but), and they still have a hard time gaining support, you and me... we have about zero chance of talking a manufacturer into releasing us anything, and I don't see that changing anytime soon (at least not because of us).

[mutex]

Hi,

Not sure if that loading is exactly true. I think (maby mistaken) that there is no general loading capability in (U)EFI. You can load other EFI applications so i think you will end up writing an EFI application that will be your (first/second) stage loader, that then boots your kernel. This EFI application can of course use EFI services like disk, network and graphics support to do its work easier, but i think you will need to boot it yourself.

The UEFI bios will already be in 64 or 32 bit depending on EFI platform, but most work related to CPU/mode init (paging,segments,etc) will have to be done again anyway.

I have written some info on EFI under grub on the wiki. I been booting my kernel from EFI for some time now, and have also made some small EFI applications to test various stuff. See; http://wiki.osdev.org/GRUB#Grub_for_EFI

I like EFI, it reminds me of the old Sparc openboot loader from 10-15 years back Finally we can do cool stuff in "bios"...

best regards
Thomas