Changing to VESA in protected mode

[Teln0]

I've searched on the wiki and the only thing that I found is how to do this in real mode. They said "it's more complicated on protected mode, and that I have to do my own drivers". If somebody can give my an advice or link me to some information it would be much appreciated.

[songziming]

I'm not very familiar with VESA. But if your only purpose is to draw pixels, then you can do that without calling VESA functions.

What VESA does is mode-setting, i.e. changing width/height/depth. One the mode-setting is done, you can draw pixels by writing to framebuffer memory.

My OS runs under 64-bit and use GRUB to boot. I can tell grub to set desired width/height/depth, and grub will pass the video mode info (like buffer address/bpp/pitch/etc) to my kernel.

If your OS doesn't use grub, then you must have a custom bootloader which runs under real mode. You can switch mode while under real mode, and store mode info returned by VESA in a global data structure. After switched into protected mode, your C code can retrieve those info and print images on the screen.

[Teln0]

If I understand it right, the graphics mode can be chosen in real mode, and it can't be chosen in protected mode anymore ? Isn't there a way to do this in protected mode ?

[Octocontrabass]

How often do you need to change the graphics mode? This is a lot of work for something that you probably won't use very often.

What kind of hardware will you be using? If it's recent, it's using UEFI and probably doesn't support VBE at all.

[MollenOS]

The problem is, if you want to use VESA to do mode switching in protected mode you have two options;

1) Implement V8086 mode, where you temporarily drop back to 16 bit mode to invoke BIOS services that can perform the switch for you, this can only be done if your OS runs in 32 bit mode, as V8086 mode is not available in 64 bit mode.
2) Use VESA Protected Mode interface, which I have no idea where is documented, or whether or not it's supported at all on any graphic cards these days. So I can't really recommend this method, as it probably not a feasable solution.

Whats left is to do the switch while you are in 16 bit mode before going to 32/64 bit mode.

[Teln0]

Alright, I'll do the switch in real mode.

What kind of hardware will you be using? If it's recent, it's using UEFI and probably doesn't support VBE at all.

I'm using qemu-i386 to test my OS

[mallard]

To use "better than VGA" modes in protected mode your options are pretty much as follows:

1. Set the mode using VBE before switching to pmode. Probably the easiest option, but not very flexible.

2. Temporarily switch back to realmode to set the mode via VBE. Fairly easy, but risks losing interrupts which other device drivers may be relying on. Probably OK during the early stages of boot.

3. Use V86 mode to set the mode via VBE. Starting to get a bit complicated. Not possible in long mode.

4. Use an x86 emulator to set the mode via VBE. A fairly easy to use emulator designed specifically for this purpose exists (libx86emu) and works well, but if you want to build your own it'll be a lot of work. Fully compatible with all CPU modes and can even be made to work on non-x86 hardware.

5. Use native drivers. Requires a fair amount of work per graphics card (family), but will allow you to use features not available via VBE (extra resolutions, timings, etc.).

6. Use GOP. Only available to the bootloader, very inflexible, but possibly your only option (except for native drivers) on the most modern hardware.

7. Use a multiboot-compliant bootloader (e.g. GRUB) and request a mode via the graphics fields in the multiboot header. The bootloader may use VBE, GOP or some other method, but that doesn't matter to your code. Very inflexible since the mode is "hard-coded" into the kernel image and there's no easy way to recover if the bootloader can't/doesn't set a mode you're compatible with.

If anyone knows any other methods, feel free to expand...

[max]

From my experience I would recommend as an easy approach to get you started to set the video mode by dropping back to real mode, setting the mode you want, before you switch to protected mode (and before you do any tasking stuff etc). Then provide information about the video buffer to your OS. This would also be kind of compatible if you decide to switch to UEFI, then you could instead just pass that frame buffer.

I've implemented VM8086 - it's kind of a pain and probably as MollenOS said not worth the work just so you can set the video mode one time.

[Octacone]

I would advise against switching back to real mode. You loose interrupts and paging, if you have a 32 bit PAE higher half kernel like I do, switching back to real mode from that state would be a huge pain.
My personal choice is a virtual 8086 monitor, I think it is well worth the "hassle". It allows you to run 16 bit programs in protected mode with paging support. You can use it for other things too, not just the mode switching.
If you think you don't have enough skill for such a task, you can always switch it right on, while still in real mode before switching to protected mode.

[SpyderTL]

If you are using GRUB to load your OS, then let GRUB set the video mode for you.

If you are writing your own boot loader, then call the BIOS INT 10h methods to change the video mode BEFORE switching to 32-bit mode.

Those two options are very easy to implement. The other options are much more difficult, and probably best left until you have a stable OS.

But, of course, the choice is yours.

[bellezzasolo]

I would advise against switching back to real mode. You loose interrupts and paging, if you have a 32 bit PAE higher half kernel like I do, switching back to real mode from that state would be a huge pain.
My personal choice is a virtual 8086 monitor, I think it is well worth the "hassle". It allows you to run 16 bit programs in protected mode with paging support. You can use it for other things too, not just the mode switching.
If you think you don't have enough skill for such a task, you can always switch it right on, while still in real mode before switching to protected mode.

Not really. I've dropped from a higher half 64 bit kernel. That's a bit more involved, as you need to drop through compatibility mode, so that you can turn off paging. You need to identity map the low MB, but only while you're using this (so early in boot). If your kernel is 32 bit, you just jump into low memory (where your real mode code is), turn off paging, PMode, and voila! When you turn paging back on, CR3 is unchanged. If you're dropping out of 64 bit, go through compatibility mode, and you eventually reach real mode. It's not a huge pain, the code is just a bit long.

Code: Select all

typedef struct _CHAIOS_REALMODE_MODULE {
	uint32_t jmp;		//16 bit jump, if standalone.
	char sig[8];		//Signature. Must be "CHAIOS16"
	char modName[8];	//name of the module
	uint32_t base;
	uint32_t entry32;	//32 bit entry point address
	uint64_t entry64;	//64 bit entry point address
}CHAIOS_REALMODE_MODULE, *PCHAIOS_REALMODE_MODULE;

Code: Select all

;/********************************************************** 
;ChaiOS 0.05
;CC0 license
;
;Project: ChaiOS
;File: chaimodule16.asm
;
;Description: ChaiOS 16 Bit Module - Common Macros. Include at top of modules (refer to documentation)
;**********************************************************/

;CHAIOS16 header - arg1 - base address, arg2 - Module name ("......", max 7 chars)
;Use this at the file beginning - then entry point is modentry
%macro header 2
BITS 16
section .text

%strlen modnamelen %2

%if modnamelen > 7
%error Module Name %2 is too long (%[modnamelen] chars, the max is 7)
%endif

org %1
start:
jmp modentry
align 4
db "CHAIOS16"
db %2
TIMES 8-modnamelen db 0
dd %1
dd ChaiOSentry32
dq ChaiOSentry64

%endmacro

;****************************************************************************
;Put this at the end of the file. This handles 32/64 bit stuff. No parameters
;If necessary, define prologue and epilogue functions - func_prologue_32, func_prologue_64,
;func_epilogue_32 and func_epilogue_64 - as macros

%macro footer 0

;Default macros, if macros not defined
%ifnmacro func_prologue_32
%macro func_prologue_32 0
nop
%endmacro
%endif

%ifnmacro func_prologue_64
%macro func_prologue_64 0
nop
%endmacro
%endif

%ifnmacro func_epilogue_32
%macro func_epilogue_32 0
nop
%endmacro
%endif

%ifnmacro func_epilogue_64
%macro func_epilogue_64 0
nop
%endmacro
%endif

BITS 32
ChaiOSentry32:
pushad
;Prologue function
func_prologue_32
cli
sgdt [gdtold]
sgdt [gdtnew]
sidt [idtold]
mov DWORD[os_bits], 32
;Now use existing GDT, but load it for non-paging
mov esi, [gdtold.base]
xor ecx, ecx
mov cx, [gdtold.limit]
add cx, 1
lea edi, [GDT]
rep movsb
lea eax, [GDT]
mov [gdtnew.base], eax
lgdt [gdtnew]
mov ax, 0x30	;Data 16 segment
mov WORD[savess], 0x10
call 0x28:Entry16
lgdt [gdtold]
lidt [idtold]
;Epilogue function
func_epilogue_32
popad
ret

BITS 64
bits32addr:
dq ChaiOSentry64.bits32
dw 0x28
ChaiOSentry64:
push rbx
push rsi
push rcx
push rdi
push rax
;Prologue function
func_prologue_64
cli
sgdt [gdtold]
sgdt [gdtnew]
sidt [idtold]
;Now use existing GDT, but load it for non-paging
mov rsi, [gdtold.base]
xor rcx, rcx
mov cx, [gdtold.limit]
add cx, 1
lea rdi, [GDT]
rep movsb
lea rax, [GDT]
mov [gdtnew.base], rax
mov [saveesp], rsp
lgdt [gdtnew]
jmp far [bits32addr]
BITS 32
.bits32:
mov ax, 0x30
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax

mov DWORD[os_bits], 64
;We are in compatibliity mode. Now we need to move to legacy mode
mov eax, cr0
and eax, 0x7FFFFFFF
mov cr0, eax
jmp 0x28:.pmode
.pmode:
mov ax, 0x50	;Data 16 segment
mov WORD[savess], 0x30
call 0x48:Entry16_64
mov eax, cr0
or eax, 0x80000000
mov cr0, eax
jmp 0x8:.end
BITS 64
.end:
mov rsp, [saveesp]
mov ax, 0x10
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
lgdt [gdtold]
lidt [idtold]
;Epilogue function
func_epilogue_64
pop rbx

pop rax
pop rdi
pop rcx
pop rsi
pop rbx
ret

BITS 16 
idt_real:
	dw 0x3ff		; 256 entries, 4b each = 1K
	dq 0			; Real Mode IVT @ 0x0000
 
savcr0:
	dq 0			; Storage location for pmode CR0.
saveesp:
	dq 0
savess:
	dw 0
savecs:
	dw 0
 
Entry16:
        ; We are already in 16-bit mode here!
 
	cli			; Disable interrupts.
	mov [saveesp], esp
	; Need 16-bit Protected Mode GDT entries!
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	mov ss, ax
 
	; Disable paging (we need everything to be 1:1 mapped).
	mov eax, cr0
	mov [savcr0], eax	; save pmode CR0
	and eax, 0x7FFFFFFe	; Disable paging bit & enable 16-bit pmode.
	mov cr0, eax
 
	jmp 0:GoRMode		; Perform Far jump to set CS.
 
GoRMode:
	mov sp, 0x9000		; pick a stack pointer.
	mov ax, 0		; Reset segment registers to 0.
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	mov ss, ax
	lidt [idt_real]
	sti			; Restore interrupts -- be careful, unhandled int's will kill it.
	call modentry
	cli
	lidt [idtold]
	mov bx, [savecs]
	mov eax, [savcr0]
	mov cr0, eax
	jmp 0x8:.restored
	BITS 32
	.restored:
	mov ax, [savess]
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	mov ss, ax
	mov esp, [saveesp]
	retf

	BITS 16
Entry16_64:
        ; We are already in 16-bit mode here!
 
	cli			; Disable interrupts.
 
	; Need 16-bit Protected Mode GDT entries!
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	mov ss, ax
	mov [saveesp], esp
 
	; Disable paging (we need everything to be 1:1 mapped).
	mov eax, cr0
	mov [savcr0], eax	; save pmode CR0
	and eax, 0x7FFFFFFe	; Disable paging bit & enable 16-bit pmode.
	mov cr0, eax
 
	jmp 0:GoRMode_64		; Perform Far jump to set CS.
 
GoRMode_64:
	mov sp, 0x9000		; pick a stack pointer.
	mov ax, 0		; Reset segment registers to 0.
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	mov ss, ax
	lidt [idt_real]
	sti			; Restore interrupts -- be careful, unhandled int's will kill it.
	call modentry
	cli
	lidt [idtold]
	mov bx, [savecs]
	mov esp, [saveesp]
	mov eax, [savcr0]
	mov cr0, eax
	jmp 0x28:.restored
	BITS 32
	.restored:
	mov ax, [savess]
	mov ds, ax
	mov es, ax
	mov fs, ax
	mov gs, ax
	mov ss, ax
	retf
	

section .data

gdtold:
.limit: dw 0
.base: dq 0
idtold: dq 0,0

gdtnew:
.limit: dw 0
.base: dq 0

GDT:
TIMES 16 dq 0

os_bits dd 0

%endmacro

;/******************************************
;Runtime functions used for OS interface.
;Please use these functions sparsely,
;since the overhead of mode changing is huge.
;******************************************/

;callsystem: performs a system call.
;Arg1: intN. Arg2: call number. 
%macro callsystem 2-5 0

;Call the pmode OS.
mov eax, [os_bits]
cmp eax, 64
jg %%unsupported
je %%bits64
cmp eax, 32
je %%bits32
;Something's gone wrong
jmp %%unsupported

%%bits32:
cli
mov BYTE[interrupt+1], %0
lidt [idtold]
mov bx, [savecs]
mov eax, [savcr0]
mov cr0, eax
jmp 0x8:.restored
BITS 32
.restored:
mov ax, [savess]
mov ds, ax
mov es, ax
mov fs, ax
mov gs, ax
mov ss, ax
mov esp, [saveesp]
lgdt [gdtold]
mov eax, %1
mov rbx, %2
mov rcx, %3
mov rdx, %4
.interrupt:		;Calls the OS
int 0

;Now return back to 16 bit land

;/*********************
;These were still under development
;*********************/

cli
hlt

jmp %%end

%%bits64:


%%unsupported:
;An unsupported (>64 bit or invalid) OS. Don't know how we got here.
cli
hlt


%%end

%endmacro

Code: Select all

%include "chaimodule16.asm"

header 0x1000, "VESAVBE"
modentry:
...

footer

[Schol-R-LEA]

Alright, I'll do the switch in real mode.

What kind of hardware will you be using? If it's recent, it's using UEFI and probably doesn't support VBE at all.

I'm using qemu-i386 to test my OS

Uhm, that's a separate issue, unless you happen to be using an emulated BIOS - if it is virtualizing based on the host's BIOS (whether Legacy or UEFI), it will have the same support as the underlying hardware. I am not really certain how the BIOS selection works in QEMU - can anyone give more details?

Now, I should mention that both virtualization or emulation of the video hardware come with their own issues. While QEMU does have basic support for emulating a generic VESA video card, as well as some more specific ones such as the Cirrus cards (circa 1998, I think - before GPUs were common, certainly), I don't know if any of those emulation modes supports VESA Protected Mode Interface. Even if one does, it is almost certainly emulating a card from at least a decade and a half ago - it would be of little use in targeting modern hardware except as a baseline.

Virtualizing the live hardware is even more problematic, as the necessary hardware support for virtualizing the video hardware is relatively new - if your hardware is more than five years old, it may not support it. Worse, you really can't share that virtualization with the host OS - you would need a second video card dedicated for use by the virtual system.

Again, I am a bit out of my depth on this topic, so if anyone can correct my errors or elaborate on the details, I would be grateful.

[Octocontrabass]

Uhm, that's a separate issue, unless you happen to be using an emulated BIOS - if it is virtualizing based on the host's BIOS (whether Legacy or UEFI), it will have the same support as the underlying hardware. I am not really certain how the BIOS selection works in QEMU - can anyone give more details?

You can't virtualize the host's firmware. QEMU uses a legacy BIOS by default, but it also supports UEFI.

I don't know if any of those emulation modes supports VESA Protected Mode Interface.

It seems to support only the functions for changing the display start address and the window locations. The complete protected mode interface described in VBE 3.0 is not supported.

[Schol-R-LEA]

Uhm, that's a separate issue, unless you happen to be using an emulated BIOS - if it is virtualizing based on the host's BIOS (whether Legacy or UEFI), it will have the same support as the underlying hardware. I am not really certain how the BIOS selection works in QEMU - can anyone give more details?

You can't virtualize the host's firmware. QEMU uses a legacy BIOS by default, but it also supports UEFI.

Odd... I don't know where I got the impression is could, then. Thank you for the correction.