OSDev.org

Hey all just want to get your opinions on using virtual machines for testing (VMware and Virtual Box). I always test out my OS on real hardware however it takes up atleast 2 minutes for every test run (which really adds up). Do any of you use virtual machines to test out your operating system? If so which virtual machine software and how exactly do you go about configuring it. Any resources on how to configure virtual machines to run raw binaries would be awesome.

Thanks,
bigboyav

I would say that the majority of people here use some kind of emulator for the bulk of their development work, and only do testing on live hardware once the VM version works; there are some caveats about doing this, but it makes the turn-around time much faster.

The wiki has pages about Emulators in general, as well as about specific emulators/virtualizers such as Bochs, VirtualBox, and QEMU. While Bochs was the most widely used for a long time, being the easiest to configure and use, there seems to be a trend towards using QEMU lately - since Bochs is a pure emulator, rather than a virtualizer, it tends to run very slowly, and have a lot of places where it doesn't quite match the real hardware.

QEMU is more complex to configure than either Bochs or VirtualBox, but more flexible in both what hardware it can virtualize, and what types of processors or configurations it can emulate. If you are looking to use a platform other than the PC, such as the common ARM-based SBCs (Raspberry Pi, FriendlyElec NanoPC, Libre Renegade, Asus Tinkerboard), or less widely-used CPUs such as the MIPS or RISC-V, then QEMU is usually the best bet. It is also good for testing multiple different PC load-outs in a quick and painless fashion.

The biggest headache of using an emulator or virtualizer is when testing drivers for real video hardware; while newer Intel and AMD processors can virtualize video cards, they can't do it seamlessly and the configuration is a nightmare. It generally will require you to have two separate video cards, one for the host system and another for the virtual one, and either separate monitors or a KVM switch for the display. While emulated video card configurations are available, they are generally for much older hardware (with the default on QEMU being the Cirrus graphics card circa 2000) and often don't quite match the real hardware anyway.

This is less of a problem with cross-emulation, as you aren't virtualizing real hardware at all, but again, the emulation is rarely perfect and you may find that your driver for, say, Mali GPUs doesn't work correctly on the actual SBCs.

Schol-R-LEA wrote:While Bochs was the most widely used for a long time, being the easiest to configure and use, there seems to be a trend towards using QEMU lately [...]
QEMU is more complex to configure than either Bochs or VirtualBox

Do you really think so? I gave up on bochs relatively early because I hated having to write config files that included ridiculous mandatory stuff like disk geometry. Configuring QEMU, on the other hand was (and still is) as simple as calling "qemu -hda disk.img" if you just want to get that disk image booted. I'll admit that meanwhile QEMU has gained a massive amount of functionality like additional hardware emulation that can be configured in detail; and if you want to control everything, the QEMU command line gets just as complex, if not more, as bochs config files. But you can still keep it very simple if all you need is just some standard PC.

The advantage that bochs still had back then was that it provided more useful debug messages for the typical errors that beginners tend to make. It's quite possible that it's still better at this, though it doesn't really matter for me any more.

For me, VirtualBox has been the most annoying of all because it doesn't directly boot raw images, and VDI images have an UUID that makes it impossible to just overwrite your disk image with a newly generated image for the next build of your OS. It will complain about the mismatch and you'll have to remove and re-add the image. (Overwriting with "qemu-img convert -n" does the trick, but...)

For bochs, you don't have to write config files. You can use it's interactive menu system. For a config example, just google bochsrc, but config commands are usually easy to remember and straightforward. A big plus that you can provide a memory map for it's built-in debugger, which is superiror to gdb when it comes to debugging low-level software (things like "page", "info gdt", "info idt" etc.). Out of the three, this is the best for debugging, no doubt.

About qemu, I agree, the defaults are sane, and with the machine profile support (the -M switch) it's not that hard to configure as it used to be. You still can specify every aspect of your emulated hardware in the command line, which sometimes can be confusing, contradictional, non-intuitive, and turns out to be undocumented. Unlike the other two, it lacks a proper debugger, but who cares when you can activate a gdb-server with "-S -s"? Few examples from my makefile which can be useful: For VirtualBox, I agree that the lack of support for raw disk images is a big disadvantage, and that's not the only one. I use "VBoxManage convertfromraw" to create VDI, and I have also found a solution to the more-than-f*ucking-annoying-have-you-developers-ever-tried-this-sh*t UUID problem: "VBoxManage internalcommands sethduuid". It's built-in debugger is just a bit non-trivial, most commands work as expected, but minimal. Provided you could figure out how to summon that "Debug" menu. I have found lots of bugs in emulated VB (specially under mac). I have a feeling they don't want to truly emulate a hardware, but something which is good enough for running the mainstream OSes in a vm.

On maintenance, bochs is a fully software emulator (slow, but always works), while qemu and VB use a kernel module (with hardware acceleartion you got better performance, some module dependency hell and maybe security backdoors too). Usually kernel module installation works just out-of-the-box, but if VB's kdms fails you, then God help you.

Out of the the three, IMHO qemu is the winner because:
- it is supported on most platforms (win, linux, macosx) No need for special modules under linux (KVM is usually already included in every distro)
- it's SDL backend is simple, fast, reliable and available on all platforms (but lacks menus, which I don't care btw)
- it is the fastest by far (bootup time and overall emulation)
- it supports the widest hardware+firmware spectrum (not just many PC devices, but also other architectures)
- if you don't like command line, you have lots of GUI lunchers to choose from. You can also write one in python in literally 10 minutes.

Regardless, I test my OS on real hardware only when it works without problems on all of them.

Cheers,
bzt