OSDev.org

For some reason, when my bootloader code switches to protected mode, VMWare crashes.  I have disabled all interrupts including NMI, and I've checked and rechecked the code, but it keeps crashing VMWare (2.0.4 under Windows 2000 - Gives memory access violation errors then dumps core).

When I boot the bootcode under a real machine it works fine, and I have a loop that continuously writes characters to the screen so if I got a triple-fault, etc.. and some unknown interrupt code was being executed I would know because it would crash the system.  On VMWare, it doesn't even print the first character, not even the debugging characters I insert into the frame buffer before I do the pmode switch.  Does anyone have a clue?  Has anyone seen this before?

Thanks all,
Breckin

do you have your es/fs/gs/ss etc set correctly? bugs like this can be fun to track down, especially when they work on a real machine and fail on vmware, etc. it could be your stack or something. Quite often it might point to something fundamentally wrong, that just happens to work on a real machine, but a more stricter emulator will catch.

Thanks!  I found the problem, it appears I Was looking at an incorrect version of the 80386 Programmers Reference Manual.  The type entries in the GDT as listed under that model only differed by code/data and system, code and data segments had the same type bits (the higher order bit in the type field was listed as the same).  So I took a look at the linux source code and I noticed that the bootloader used a different type field from mine.  Similar, but different.  Hmmm....  Anyway, I ended up downloading the new IA-32 System Programmers Manual and lo and behold it had the right stuff in it.  Worked like a charm.

As for why that would crash VMWare... It seems that VMWare can't handle OSes that do not handle Exceptions.  Especially double faults and things like that.  An unhandled exception would normally reset the processor, and I think that's what VMWare tries to do, but it fails miserably at it.  One tip though, under settings (or wherever) turn on verbose debugging information if you're doing your development under VMWare.  Then, when you tank the program, look at the VMWare log file in your VMWare directory, it has a ring buffer  and if you scroll to the bottom you'll find the segment:offset of the code line and the exception that was generated unlong with the error code (if any).  A little cryptic at first, but if you're wriiting an operating system you should probably have a pretty good idea of what it all means.,  

Anyway, just a heads up for anyone who might be experiencing these problems...

... Now if I could only figure out why I can't write data to my dataseg (no GP#, just can't read anything back that I put in)  :-/

Well I fixed that problem too, didn't take me too long to track it down. ?I left out the last byte if the descriptor for the dataseg, so it was using a junk value for bits 24-31 of the base. ?I might have EVENTUALLY found some valid memory somewhere in the 1-3GB range (if I had that much memory, that is). ?

So now I'm in protected mode, writing to memory, not getting protection faults ?My next step is to test the A20 code I used (used PS/2 compatible code, should work on most new systems right? ?I hope so, because it's four lines of code instead of twenty-four), and then import some old code I wrote a few weeks ago into the bootloader to read the next sector off the disk then jump to it. ?Then we'll be out of the bootsector code finally (that is until I have to modify it to work with anything other than 1.44 MB floppies )

I'm having problems with vmware too. Just after I JuMP to set cs:ip after switching to pmode.

Did you need to set up and IDT to make it work?

you dont need an IDT to switch to pmode, you do need a valid GDT tho.

my gdt seems fine.. but the jump to $0x8:offset is what causing a gpf...

then either 'offset' is out of the boundaries, or 0x08 isn't a valid code segment.

yeah i know, either one of the two

anyway, 8 is a valid descriptor in my gdt, i'm 99% sure about it

i prefixed the jump with a 0x66 byte and i'm using "biew" to dissasemble it and it appears kinda broken, but vmware debug reports well the seg:off

nothing wrong with the instruction being located at 0x8:offset ? did you disable interrupts (hum, i guess you did ?
maybe try it with a leading nop at offset ...

i'm basically doing this:

.bits16
.... code...
.code32
   D32 ljmp $0x08, $start32

start32:
movl $0x10, %eax

i start this boot stage at 07c0:0 so my offset is 0, $0x10 is a valid data segment descriptor...

where you say i put this nop? right before the jmp or the mov?

.bits 16 should be .code16

D32 is a macro for 0x66 and yes, i have interrupts disabled, and irq's 0-0xF to int's 0x20-0x2F

i load gdt like this:

A32 lgdt gdtptr

where A32 equals 0x67