Question about which tools to use, bugs, the best way to implement a function, etc should go here. Don't forget to see if your question is answered in the wiki first! When in doubt post here.
We can't just guess what your problem is if you give so little detail. However, in my personal experience, VMware has no issues with VESA or the BIOS in general.
What does your 512 byte boot sector do? How are you printing the text after the boot sector? Are you using the BIOS for that, too? If not, my first guess would be you don't correctly set the attributes, and you're just lucky that VirtualBox does it for you.
As for crashing when trying to set a VESA mode, nobody can guess what your problem is without more detail.
You know your OS is advanced when you stop using the Intel programming guide as a reference.
so i looked into my code and where i put my page table into memory and i found out that it is in the bios area and writing there is bad but virtualbox doesnt seem to care.