in-kernel vm86 mode

[xvedejas]

Vm86 mode is generally used for running tasks separate from the kernel, but what if I want to run some of my kernel's code in vm86 mode? What I need is for my kernel to send the BIOS an interrept (for VESA) then immediately return to protected mode. All the guides I have found for vm86 mode talk about ring 3 and setting up a TSS, I'm wondering if I can accomplish the same thing without this if I plan on doing it all in ring 0 and within my kernel.

The simplest answer is the best one. If you can point me to docs describing how to do what I ask, that works too, though I haven't found any yet

Thanks

[Martijn]

All the guides I have found for vm86 mode talk about ring 3 and setting up a TSS, I'm wondering if I can accomplish the same thing without this if I plan on doing it all in ring 0 and within my kernel.

How about reading the Intel manuals.

Intel 3A: 15.2.1 Enabling Virtual-8086 Mode
...when checking privileged instructions, on page accesses, or when performing other permission checks. (Virtual-8086 mode always executes at CPL 3.)

[Cognition]

It's feasible to preform the task you're talking about in vm86 mode at CPL3, simply create a virtual monitor and set the TSS to operate at the proper address. Article wise it's almost completely covered in the Virtual Monitor page in the wiki here. The links at the bottom should provide you with code for some working examples, other then that the TSS is covered in depth in the Intel SDM.