OSDev.org

Brendan wrote:Hi,

PearOs wrote:Out of curiosity would it be possible just to convert the Video Bios to 32bit assembly instructions and then boot that? Or is that entirely impossible? Thanks, Matt

It's theoretically possible to translate the code from 16-bit to 32-bit (or even from 16-bit 80x86 to ARM or something else). However, it'd be the most complex (and most error prone/buggy) method; and worse than all the other methods.

PearOs wrote:On the Interrupt Vector Table wiki, it says that in real mode the interrupt 0x10 offset 0x40 is x87 FPU Error. So how is Int 10 used in real mode if its a error? Or is this false info and Int 10, offset 0x40 is really the VBE Bios like people are saying?

Once upon a time the FPU was a separate chip (not built into the CPU at all), and FPU errors were routed through the PIC chips to ISA IRQ 0x13 (interrupt 0x75). This meant that you could get the interrupt long after the error occurred (which caused race conditions). It was also slow. Worse, when there's more than 1 FPU (e.g. modern multi-core systems) it doesn't work at all because there's no way to determine which FPU caused the error.

To fix all of this, for 80486 and later (where the FPU is built into the CPU) Intel introduced "native FPU exceptions", where FPU errors generate exception 0x10. This is enabled by setting a bit in CR0. All modern OSs use the "native FPU exceptions". Sadly (due to backward compatibility) the BIOS had to remain retarded, so the BIOS doesn't enable "native FPU exceptions" and uses the old and broken "FPU errors routed through IRQ13" mess.

PearOs wrote:I don't know what the UEFI boot services are.

Over the years the BIOS gained a lot of obsolete puss due to backward compatibility (A20 gate, FPU error handling, PIC chips, etc) and limitations that seemed OK at the time became very painful (640 KiB of RAM, 128 KiB of "device ROM" space, etc). To fix all of the problems with the BIOS, it's being replaced by something completely different called UEFI. UEFI provides "boot time services" (that an OS can only use during boot) and "run time services" (that an OS can use during boot or after boot). The idea is that when an OS has finished using UEFI's boot time services (e.g. the OS is ready to take control of the hardware itself) it calls an "exit boot services" function, which allows UEFI to free a lot of resources (RAM, etc) that it was using to provide the boot time services.

Of course for UEFI none of the obsolete puss needs to exist because it's all been replaced by something "better". Currently (because UEFI is still sort of new) most hardware still supports the obsolete puss (in case the hardware is used for BIOS); but it'd be a massive design flaw for any modern software to rely on the existence of any of the obsolete puss; including VBE (and the "legacy VGA" display memory window, and legacy VGA IO port routing, etc).


Cheers,

Brendan

Very very interesting Brendan, thank you for your reply. So on the subject of the 16bit VBE Code, why is it that someone couldn't just do a call in protected mode to where the VBE code is by looking it up in the IVT? Is there that big of a difference between real mode and protected mode? I mean there still are all the 16bit registers. What really causes the code to not work? Just curious. Thanks, Matt

PearOs wrote: Very very interesting Brendan, thank you for your reply. So on the subject of the 16bit VBE Code, why is it that someone couldn't just do a call in protected mode to where the VBE code is by looking it up in the IVT? Is there that big of a difference between real mode and protected mode? I mean there still are all the 16bit registers. What really causes the code to not work? Just curious. Thanks, Matt

VBE does have a protected mode interface, but it is broken in many BIOSes, so it cannot be relied on. This is probably due to Windows using the V86 interface.

Brendan wrote: Over the years the BIOS gained a lot of obsolete puss due to backward compatibility (A20 gate, FPU error handling, PIC chips, etc) and limitations that seemed OK at the time became very painful (640 KiB of RAM, 128 KiB of "device ROM" space, etc). To fix all of the problems with the BIOS, it's being replaced by something completely different called UEFI. UEFI provides "boot time services" (that an OS can only use during boot) and "run time services" (that an OS can use during boot or after boot). The idea is that when an OS has finished using UEFI's boot time services (e.g. the OS is ready to take control of the hardware itself) it calls an "exit boot services" function, which allows UEFI to free a lot of resources (RAM, etc) that it was using to provide the boot time services.

Of course for UEFI none of the obsolete puss needs to exist because it's all been replaced by something "better". Currently (because UEFI is still sort of new) most hardware still supports the obsolete puss (in case the hardware is used for BIOS); but it'd be a massive design flaw for any modern software to rely on the existence of any of the obsolete puss; including VBE (and the "legacy VGA" display memory window, and legacy VGA IO port routing, etc).

UEFI is written for OSes that have full native driver support, and is a load of crap for OSes that doesn't. This is especially the case for graphics where the BIOS offers run-time graphics services, while UEFI doesn't. That's why people need to hack it in order to use boot-services at run-time. This could be interesting for other devices than GOP as well.

rdos wrote:

Owen wrote: Until you realize that the GOP implementation relies upon the UEFI boot services, and that calling into those has a tendency to not work after ExitBootServices has reconfigured all the hardware to reflect the fact that its' handing off. For example, GOP mode setting might require a timeout, but of course you have taken control of the system timers...

Again, if the system is sane, GOP shouldn't rely on UEFI boot services that are hardware dependent.

Wut? GOP shouldn't rely on, oh, I don't know... PCI services?

rdos wrote: Also, as far as I understand the interface at this point, it is perfectly possible to replace UEFI services that deal with delays and timers with a native interface.

Good luck with that. How do you propose to find all the function calls that the GOP driver could be using? N.B. lots of them will be squirreled away inside function tables that the GOP driver has explicitly looked up.

rdos wrote:

Owen wrote: Or, boot services might somehow decide that a DHCP timer has expired, causing the network stack to try to send out a DHCP request. It might even succeed, configuring receive DMA and overwriting some portion of memory with random network packets.

It can't. The UEFI will not be linked to any IRQ, so it won't be notified about DHCP events.

GOP sets a timeout. While waiting for said timeout, the UEFI system discovers a DHCP timeout has expired.

The UEFI IP stack then trashes the configuration of your network card

rdos wrote:This is similar as in the BIOS case. You don't need the full BIOS running in order to support VESA BIOS video mode changes. It is typically enough to have a working int 8/1C timer interrupt.

The BIOS interface works because card vendors (obviously!) test it against Windows, and Windows needs VESA to work (The "starting windows" screen is displayed via VBE, plus don't forget the vesa.sys fallback driver)

rdos wrote:

Owen wrote: Hell, answer me this: how do you deal with the GOP code calling the UEFI memory allocator?

Change the UEFI memory allocator service (if needed).

How?

How do you propose to change the boot services table for the GOP driver, when you don't even know where it is?

What about when the GOP driver calls into the PCI driver?

Etc, etc...

What do you need to change the graphics mode for, anyway? Set the monitor's native resolution at boot, and to deal with crappy puke applications which need a fixed resolution and/or text mode, implement software scaling and text mode.

You can easily do a better job of scaling than most displays anyway...

Hi,

PearOs wrote:Very very interesting Brendan, thank you for your reply. So on the subject of the 16bit VBE Code, why is it that someone couldn't just do a call in protected mode to where the VBE code is by looking it up in the IVT? Is there that big of a difference between real mode and protected mode? I mean there still are all the 16bit registers. What really causes the code to not work? Just curious.

For real mode you're limited to 64 KiB segments, so code designed for real mode (including VBE and BIOS functions) has to change segment registers a lot. In real mode, when a segment register is loaded the CPU does "segment.base = value * 16". In protected mode, when a segment register is loaded the CPU does (GDT or LDT) table lookups instead, plus a lot of protection checks. The end result is that because segment register loads works differently (and because real mode code does a lot of segment register loads), you can't run real mode code in (16-bit) protected mode. For a simple example, if real mode code wants to access the BDA (BIOS Data Area) it will probably loads zero into a segment register, and in protected mode "zero" is always the NULL descriptor that can never be used.

Of course there are also other differences between protected mode and real mode (like how IDT/IVT lookups work).

For some of the problems it's possible to implement work-arounds. For a simple example, if real mode code has loaded zero into a segment register and then uses that segment register to access something, then you'll get a general protection fault, and your general protection fault handler can detect the problem and create/load a different descriptor with "base = 0" as a work-around. However, typically these work-arounds cause more problems (e.g. if the real mode code relies on the the value in the segment register and not just the segment's base).

In theory, it is possible to run some real mode code in 16-bit protected mode, if that real mode code doesn't do anything that causes problems that you've failed to work-around or can't work-around. With enough work implementing lots and lots of these work-arounds, you might be able to run 90% of the VBE code (in different video cards) using this method.

However, a good programmer knows that "works 90% of the time in theory" is unacceptable. Nobody is perfect and there will be bugs and things the programmer has overlooked (especially with something complex like a set of work-arounds for problems that are hard to foresee), so it might become "works 75% of the time in practice". Worse, the computers it does work on will be the computers that the programmer has been able to test/debug it on, leading to the programmer thinking their code works 95% of the time when it doesn't. Basically, a good programmer will only accept "works 100% of the time in theory" in the hope that one day their code will work 99% of the time in practice, and only bad programmers will do things like attempt to run real mode/VBE code in protected mode (unless it's a personal challenge and/or research rather than code that's actually intended to be used).

The same applies to attempting to rely on the existence of VBE on modern/future UEFI systems. A good programmer knows the risk of it not working is unacceptable, and only a bad programmer will accept the risk or ignore the risk. Because the service/s to setup a video mode in UEFI are "run time services", this means that for modern/future UEFI systems you have to setup a video mode during boot and you can't change video modes after boot (unless your OS has native video drivers); because if you attempt to do anything else the risk of it not working (now or in the future) is unacceptable (for a good programmer).

Now; if your OS is designed properly and therefore can't change video modes after boot on modern/future UEFI systems (unless your OS has native video drivers); then what about obsolete/current BIOS systems? In this case you can change video modes after boot if you're willing to implement enough emulation (e.g. using virtual 80x86 mode in protected mode or an emulator in long mode). The question is whether or not it's worth the hassle. To be perfectly honest, I don't think it is worth the hassle - it's easier/cleaner to setup a video mode during boot (e.g. when you're in real mode anyway) and forget about changing video modes after boot on obsolete/current BIOS systems; even though this can "work 100% of the time in theory".

Instead; I'd look into why you actually want to change video modes after boot to begin with. Typically there are only 2 reason. The first reason is that the user wants to reduce their resolution to get better frame rates out of a graphically intensive/3D game. In this case, without (very complex) native video drivers that support hardware acceleration the user won't be getting usable frame rates out of a graphically intensive/3D game regardless of what resolution they change the video mode to; and therefore you can ignore the first reason (as you need native video drivers for this anyway). The second reason is that the OS developer failed to provide resolution independence; and therefore applications need to know/care what the video mode is (and applications need to be able to set specific video modes or they don't work). For this case, fix your "no resolution independence" design flaw instead.

Basically, it's better to set a video mode during boot and forget about VBE and emulation (and UEFI) after boot; and just use a "previously setup framebuffer" after boot (with no way to change video modes after boot because you don't need to do that anyway).


Cheers,

Brendan

Also: You can reliably use VBE on UEFI systems, if the card supports it (probable for non-integrated solutions or systems with a compatibility support module) provided you use it the same way people use it on non-x86 systems: By creating a "Virtual Real-Mode Machine" with a sufficiently fleshed out BIOS implementation, then performing the standard option ROM initialization steps in it. If implemented right, the card will correctly hook the VBE interrupts, at which point you can use it by invoking VBE interrupts inside said emulator. You might want to steal the implementations of various BIOS functions from the Bochs BIOS as experimentally needed (e.g. quite clearly you need not support disk I/O).

The option ROM should successfully re-initialize the device into VGA compatibility text mode (after all, this is necessary to support warm boots), i.e., into a mode which the VESA BIOS can support.

You could probably do the same with an embedded UEFI GOP driver, and it would certainly be more reliable than attempting to do the same by copying the UEFI boot services state in memory.

Of course, you have to question the work necessary... its' probably on a similar order of magnitude as implementing an interpreter for AMD's ATOM bytecode, NVIDIA's equivalent, and a driver for the most recent Intel GPUs. All you're doing is mode switching, after all.

Brendan wrote: Now; if your OS is designed properly and therefore can't change video modes after boot on modern/future UEFI systems (unless your OS has native video drivers); then what about obsolete/current BIOS systems? In this case you can change video modes after boot if you're willing to implement enough emulation (e.g. using virtual 80x86 mode in protected mode or an emulator in long mode). The question is whether or not it's worth the hassle. To be perfectly honest, I don't think it is worth the hassle - it's easier/cleaner to setup a video mode during boot (e.g. when you're in real mode anyway) and forget about changing video modes after boot on obsolete/current BIOS systems; even though this can "work 100% of the time in theory".

Instead; I'd look into why you actually want to change video modes after boot to begin with. Typically there are only 2 reason. The first reason is that the user wants to reduce their resolution to get better frame rates out of a graphically intensive/3D game. In this case, without (very complex) native video drivers that support hardware acceleration the user won't be getting usable frame rates out of a graphically intensive/3D game regardless of what resolution they change the video mode to; and therefore you can ignore the first reason (as you need native video drivers for this anyway). The second reason is that the OS developer failed to provide resolution independence; and therefore applications need to know/care what the video mode is (and applications need to be able to set specific video modes or they don't work). For this case, fix your "no resolution independence" design flaw instead.

The major reason you want to be able to change video-mode is to support text mode. Emulating 80x25 text mode in 1920x1080 will be very slow, and if you cannot reduce the resolution to something more adequate, this emulation will not be useful. Of course, if you have a native video driver you could make it fast anyway, but that's not interesting for most hobby OSes that won't have one.

So, I think your argument is backwards. It is precisely because you don't have acceleration that you want to optimize video mode. With acceleration, there is usually no reason to reduce resolution.

As an example, I want to be able to show a 1920x1080 PNG image in full screen, I want my command interpreter to run in text mode, and I want a typical GUI application to run in 800x600 or similar. Without needing to reboot and select a new resolution.

Owen wrote: Wut? GOP shouldn't rely on, oh, I don't know... PCI services?

I don't change PCI configuration unless necessary (like when some address has not been allocated by BIOS, which is rate). Thus, GOP can use PCI as much as it wants.

Owen wrote: Good luck with that. How do you propose to find all the function calls that the GOP driver could be using? N.B. lots of them will be squirreled away inside function tables that the GOP driver has explicitly looked up.

Reference:

UEFI spec wrote:Pointers to the various global UEFI functions are found in the EFI_RUNTIME_SERVICES and EFI_BOOT_SERVICES tables that are located via the system table. Pointers to other functions defined in this specification are located dynamically through device handles.

IOW, the system table is global, and known to the OS boot-loader, and thus can be saved and manipulated.

Owen wrote: GOP sets a timeout. While waiting for said timeout, the UEFI system discovers a DHCP timeout has expired.

Not when the timer function is stubbed and replaced with a native version.

Owen wrote: How do you propose to change the boot services table for the GOP driver, when you don't even know where it is?

It's in the system table.

Hi,

rdos wrote:The major reason you want to be able to change video-mode is to support text mode.

Why on earth would anyone be stupid enough to bother with text mode? It's the most limited, worthless and obsolete video mode that's ever been possible. The only valid reason I can think of is nostalgia (e.g. attempting to look and act like a dumb terminal for the quickly approaching 50th anniversary of DEC's VT52).

Note: If you can't emulate 80*25 text mode quickly enough (even in 1920*1080), then blame your slow code.

rdos wrote:As an example, I want to be able to show a 1920x1080 PNG image in full screen, I want my command interpreter to run in text mode, and I want a typical GUI application to run in 800x600 or similar. Without needing to reboot and select a new resolution.

You want a pre-historic piece of crap?

I'd want all GUI applications to run in windows where the GUI itself uses whatever the monitor's native resolution is; I'd want to show the PNG image in full screen using whatever the monitor's native resolution is too. I'd also want the command interpreter to go back to 1960 where it belongs (or run in a window with nice anti-aliased scalable UTF-8 font if it absolutely must exist).


Cheers,

Brendan

Brendan wrote: Why on earth would anyone be stupid enough to bother with text mode? It's the most limited, worthless and obsolete video mode that's ever been possible. The only valid reason I can think of is nostalgia (e.g. attempting to look and act like a dumb terminal for the quickly approaching 50th anniversary of DEC's VT52).

Because some things are done much better on the command-line than in a GUI. Like compiling code, pinging an IP address, listing devices, starting applications.

Brendan wrote: Note: If you can't emulate 80*25 text mode quickly enough (even in 1920*1080), then blame your slow code.

There is no "quick enough" for text mode. At a bare minimum, you should be able to flip the entire screen in much less than a second, otherwise it's too slow.

Brendan wrote: I'd want all GUI applications to run in windows where the GUI itself uses whatever the monitor's native resolution is; I'd want to show the PNG image in full screen using whatever the monitor's native resolution is too. I'd also want the command interpreter to go back to 1960 where it belongs (or run in a window with nice anti-aliased scalable UTF-8 font if it absolutely must exist).

Window managers are retarded. The typical use of applicatiions is full-screen, not in some shitty window.

BTW, another reason I want to keep text-mode is that it is very handy when VBE or the video-card doesn't work. With text-mode, you can still boot your system. In addition to that, I certainly don't want the kernel debugger (or even crash manager) to run in a video mode requiring a functional video API with font rendering.

Hi,
(sorry about me being a bit lazy and not researching on this myself first, but before I spend hours just to figure out that it's not very practical:) while hacking on UEFI to use GOP at runtime has been discussed, is it possible to use GOP at boottime in a hobby OS without great hassle? I have never seen a hobby OS using (U)EFI yet.
(Furthermore, could one in theory abuse the UEFI network stack as an OS driver? )

rdos wrote:

Brendan wrote: Why on earth would anyone be stupid enough to bother with text mode? It's the most limited, worthless and obsolete video mode that's ever been possible. The only valid reason I can think of is nostalgia (e.g. attempting to look and act like a dumb terminal for the quickly approaching 50th anniversary of DEC's VT52).

Because some things are done much better on the command-line than in a GUI. Like compiling code, pinging an IP address, listing devices, starting applications.

And a graphical terminal emulator can do so much more (e.g. UTF-8)

rdos wrote:

Brendan wrote: Note: If you can't emulate 80*25 text mode quickly enough (even in 1920*1080), then blame your slow code.

There is no "quick enough" for text mode. At a bare minimum, you should be able to flip the entire screen in much less than a second, otherwise it's too slow.

If you can't redraw a 1080p image at 60Hz into a linear framebuffer, something is very wrong with your code

rdos wrote:

Brendan wrote: I'd want all GUI applications to run in windows where the GUI itself uses whatever the monitor's native resolution is; I'd want to show the PNG image in full screen using whatever the monitor's native resolution is too. I'd also want the command interpreter to go back to 1960 where it belongs (or run in a window with nice anti-aliased scalable UTF-8 font if it absolutely must exist).

Window managers are retarded. The typical use of applicatiions is full-screen, not in some shitty window.

BTW, another reason I want to keep text-mode is that it is very handy when VBE or the video-card doesn't work. With text-mode, you can still boot your system. In addition to that, I certainly don't want the kernel debugger (or even crash manager) to run in a video mode requiring a functional video API with font rendering.

UEFI GOP does not support a "hardware text mode"

Every UEFI implementation I've seen actually implements the text output protocol by implementing text mode in software.

They have absolutely no issues with being "fast enough" for near all intents and purposes.

Hi,

rdos wrote:

Brendan wrote:Why on earth would anyone be stupid enough to bother with text mode? It's the most limited, worthless and obsolete video mode that's ever been possible. The only valid reason I can think of is nostalgia (e.g. attempting to look and act like a dumb terminal for the quickly approaching 50th anniversary of DEC's VT52).

Because some things are done much better on the command-line than in a GUI. Like compiling code, pinging an IP address, listing devices, starting applications.

No, you're confused. Applications for compiling code, pinging an IP address, listening to devices and starting applications are better/easier for users to use if they're designed for a GUI (or at least used via. a GUI - e.g. compiler used from an IDE). The problem is that programmers are lazy and support for GUI is more work; but this has nothing to do with what is/isn't better for the only people that matter (the users).

The other issue is that scripting languages for GUIs didn't exist 50 years ago, and some people (far too many actually) are too stupid to figure out how to do scripting for GUIs.

rdos wrote:

Brendan wrote:Note: If you can't emulate 80*25 text mode quickly enough (even in 1920*1080), then blame your slow code.

There is no "quick enough" for text mode. At a bare minimum, you should be able to flip the entire screen in much less than a second, otherwise it's too slow.

You can't flip the screen in text mode because text mode is too limited - the closest you can do is flip the rows of characters. Despite this; if your so incompetent that you can't write code to flip rows of characters (or flip the screen) that takes less than an entire second then you should stop pretending to be a programmer.

rdos wrote:

Brendan wrote:I'd want all GUI applications to run in windows where the GUI itself uses whatever the monitor's native resolution is; I'd want to show the PNG image in full screen using whatever the monitor's native resolution is too. I'd also want the command interpreter to go back to 1960 where it belongs (or run in a window with nice anti-aliased scalable UTF-8 font if it absolutely must exist).

Window managers are retarded. The typical use of applicatiions is full-screen, not in some shitty window.

No; the typical use of applications is whatever the user feels like (full screen, in a window, tiled, spread across multiple monitors, minimised, etc). Your code is designed by a fool, so you're attempting to pretend that users want code designed by a fool (despite a huge amount of evidence to the contrary).

rdos wrote:BTW, another reason I want to keep text-mode is that it is very handy when VBE or the video-card doesn't work. With text-mode, you can still boot your system. In addition to that, I certainly don't want the kernel debugger (or even crash manager) to run in a video mode requiring a functional video API with font rendering.

If VBE doesn't work you can still fall back to 640*480 with 16 colours. If the video card doesn't work then neither will text mode. In either case a sane OS would support "headless" systems (e.g. using network and/or serial) so no video at all isn't necessarily a problem. Of course if something is going to go wrong it's better to find out during boot, rather than when the OS is running and user's data may be lost/corrupted because of an obscure bug in your real mode emulation that's needed for the "VBE mode switching" that you failed to have the intelligence to avoid.

rdos wrote:In addition to that, I certainly don't want the kernel debugger (or even crash manager) to run in a video mode requiring a functional video API with font rendering.

I agree with you - if I had to put up with RDOS's "hideous hack-fest" kernel I wouldn't let anything except the kernel debugger and crash manager use video.


Cheers,

Brendan

Owen wrote: UEFI GOP does not support a "hardware text mode"

Every UEFI implementation I've seen actually implements the text output protocol by implementing text mode in software.

They have absolutely no issues with being "fast enough" for near all intents and purposes.

It seems like you are right about the text output protocol. It seems to be a software construction where it is possible to emulate the whole thing. There is not even an specification of the buffer address or pixel organisation,

But that doesn't invalidate my previous claims. The best video-mode to emulate text mode with is 640x350, which gives 8x14 cells. You can also use 640x480 with 8x19 cells. These resolutions have 0.3 megapixels. 1920x1080 is 2 megapixels. That means running in the lower resolution is about 7 times faster. Running in text mode is about 8x14 (112) times faster.

Brendan wrote: No; the typical use of applications is whatever the user feels like (full screen, in a window, tiled, spread across multiple monitors, minimised, etc). Your code is designed by a fool, so you're attempting to pretend that users want code designed by a fool (despite a huge amount of evidence to the contrary).

You forgot one. On multiple virtual full screens with fast switching. That means you don't need to move or resize windows, and also don't need the window frame which is ugly and uses screen space.