The future method of doing VM86?

[SANiK]

Thanks to:

• Candy for giving me the notes (And for reading those x86 docs for me ;P)
• froggey for the inline asm basics
• geist for the patience (Went insanse 2 days into the project...) =P

Introduction
It all started with a hex editor, a list of assembly instructions, and NASM. I began to question how different was 16-bit code to 32-bit code. After doing some examples, I started to notice patterns, and one of them is the usage of the 0x66 prefix. I was hesitant at first to actually write a 16-to-32 bit converter. What got me started were rumors of 64-bit CPUs having no or a buggy VM86 mode. Then, I had a talk with Candy on IRC and thanks to his encouragements, I finally began writing the converter.

The theory
The theory is that Intel kept the opcodes the same between 16 and 32 bit modes. The only difference is, if an opcode has a 0x66 before it, IN 16 BIT MODE it means that the registers for the current opcodes are 32 bits. If an opcode IN 32 BIT MODE has a 0x66 in front of it, this means to use 16 bit register sizes.
And the whole idea is to keep doing this based on the instruction types. Some instructions will need to be emulated (like ones that use the segment registers) or deal with memory offsets (memory offsets rely on the DS segment register). And one common argument is that this is slow. It's not slow because 16 bit programs were made to be slow (As Candy said). Another argument is that this is nuts due to all the X86 opcodes. In reality, the major portion of X86 opcodes are the MMX and SSE add-on opcodes which the BIOS interrupts don't use.
Also, one does not completely emulate each instruction. Out of all the 256 opcodes, I'll probably end up emulating only 15 of them. All I do is apply certain prefixes to the opcode and store it in a buffer, then on every loop cycle I execute the buffer.

And for those that haven't noticed, VMWare uses translating instead of emulating. This is why it's so fast compared to Boch's.

The actual translator
So far, I'm able to run a Hello World example written in 16 bit.
I've been working on adding more instructions to be hopefully execute an interrupt. (So far so good).

Here is a picture of it executing an interrupt:


Here is a picture of how SIMPLE the source is: (Notice that all that's needed to be done for most of the opcodes is for them to be "reformatted" which takes a nanosecond)


Purpose
Well, the whole basic purpose of this is to get this thing to work.
But I would also like to hear opinions or comments on how this may have changed your opinions on things.

Thank you and you have been a great audience.
Best of luck to yah geist!

[B.E]

the way you test each byte is not the quickest way to do it.

You could use an array of function pointers and asign each byte code (array element) to a function and then call that array elemant with the byte it working at.

ps. please tell me if you don't understand and I will give an example

[SANiK]

I understand what you mean, but I'll have to use ASM to implement that. If I do this in C, the push/pop inlines will get screwed up.
Everytime you call a function, the return address of the function is pushed to the stack. If all that the function does is push AL onto the stack, then once the function attempts to return, it will jump back to whatever AL was.

Anyway, don't fear. I'll have to port it to ASM sooner or later.

[B.E]

You can do it in C. Use the following varable prototype

Code: Select all

int (* var[256])(/*any Parameters */)

var - name of a varable
256 - size of the array

In the initization of the program just assign the function to the var[element]

for example

Code: Select all

var[0] = function;

so to call the function all you would would have to do this

Code: Select all

var[0]();

and that is it. no case statment or if statments

if you are unclear of what i talking about maybe you could give me the source to the program.

[SANiK]

Umm, that'll screw up the stack and I said that I understood what you ment.

[aladdin]

And for those that haven't noticed, VMWare uses translating instead of emulating. This is why it's so fast compared to Boch's.

and what is the difference between translating and emulationg :-\ ??
I thought emulators just translate opcodes from emulated platform to host platform ::) ?!

[Pype.Clicker]

and what is the difference between translating and emulationg :-\ ??
I thought emulators just translate opcodes from emulated platform to host platform ::) ?!

when doing emulation, you have a loop that picks an opcode, execute instruction to modify the state of the virtual machine, then picks another opcode, etc.

a code translator, comparatively, will generate native code for a block of "initial code" so that one can run it as a block (thus improving performances in case of "hot spots" in the code). Even more advanced "emulators" such as Nintendo64 emulators identify *patterns* within initial code and replace them with high-level function calls (like you could replace a call to INT 0x13 with a call block_device_emulation).

[SANiK]

I am getting closer and closer to actually entering VESA mode from PMODE. In VMWare, it's beginning to send port messages to the video card. I just need to fix up the stak management to prevent it from clashing with the translator itself. Maybe I am going to use a virtual stack?

[Pype.Clicker]

Maybe I am going to use a virtual stack

That'd make sense ... even if you're writing translations, i guess it'll be easier to keep "call" and "returns" between translated chunks on an array than on the real stack and have code that do
- check if the called chunk has already been translated
- jump to the translated address.

yet, once you "call" the next translated chunk, it's simply some IA-32 code calling some more IA-32 code ... i fail to see how it could mess with the "real" stack.

[SANiK]

I want to release the source in hope that it will get done quicker.
How many would be interested? ... But first I have to clean up my junk =P (You guys would kill me if you saw that I used a pointer to attach inside the while loop and then write using it)

[B.E]

I would be interested in the release of the source code

[SANiK]

(I took the source down since it's COMPLETELY wrong)

Currently I execute one opcode per iteration. (This is for testing reasons). Later on, I want to keep converting opcodes till I reach an emulated opcode, then execute the whole thing.

ONE problem exists (which I had previously ignored). The r/m byte allows memory locations to be used based on the DS register. Thing is, the code uses an emulated DS register and not the real DS register. So, this means that ALL the r/m opcodes need to be rewritten. Since I pass them by just adding the 0x66 prefix, they still use the DS register.

The only solution would be (other than emulating every opcode) if it is possible to enable read and writes to the DS register in PMODE.

[Pype.Clicker]

see http://www.mega-tokyo.com/forum/index.p ... 7#msg62747
for latest results ...

[proxy]

any chance that the souce for this translation engine will be open sourced?

I like the idea a lot and I suppose I could do it myself (I have a lot of experience writting emulators and such), but if the work is already done...why reinvent the wheel

proxy

[SANiK]

Sorry for being away or such a long time.

The emu (the last time I checked) enters VESA correctly.
Then I tried to execute the interrupt to get back OUT of VESA into regular text mode. Well, it seems the emulated stack got corrupted.

Later I found this opcode to blame:
/*
vm_sreg[] = array of segment registers
vm_override() = function which allows one to set the default segment in case there is NOT a segment prefix
vm_iw() = reads the next word
vm_ib() = reads the next byte
vm_idw() = reads the next dword
*/

case 0xA2: //Opcode A2
print("MOV moffs8, AL\n");
*((byte*)((vm_sreg[vm_override(DS)] << 4) + vm_ib()))=*vm_reg8[AL];
break;

Look what the reference that I use tells me:


It says "A2 ib" meaning, that the OFFSET is encoded in a byte.
Well, that's wrong. After 2 weeks, Therx showed me the Intel manuals and they say the offset is encoded in a word (16 bit mode) and a dword (32 bit mode).

So now, I have to go back and recheck everything due to an idiot who doesn't know how to correctly make a look up chart.
I have found so many little mistakes due to his incorrect information, and there must be more cause it still doesn't switch to text mode correctly.

Does anyone here wish to help me fix this beast?
It's an interpretter now, and not much of a translator, but I will eventually make a translator after I can easily make a Look Up Table for the Mod-r byte.

Here's the BUGGY code:
http://jamesseph.phpwebhosting.com/users/sanik/vm.c

If you are willing and able, then PM here so I can give you the 3 extra files needed to make this work.