OSDev.org

For a couple weeks now, I have been stuck trying to implement mouse support in my OS. I run my OS both through vmware on my machine (with a usb mouse) and on a separate, older, system with both usb and PS/2 mice, but haven't been successful on either system. Because I have 2 types of mice, I have been trying to implement both a basic PS/2 driver through IRQ 12 and to implement enough of USB to get the mouse working, and I have a few questions about both methods:

1. In general, how will a usb mouse show up in vmware? (usb mode, legacy mode, etc) what IRQ will it use given that it is a USB mouse?
2. In implementing USB, I have located 2 PCI controllers, the first controlling bus 0 and the second controlling buses 1-73, I'm not sure if there are more after that. I've been using the Enhanced Configuration Mechanism defined in the wiki article about PCI Express to scan through buses and devices looking at the class codes, but every one seems to be 0xFF. Does this indicate that no devices are attached?
3. I have tried setting the 'Compaq status' byte as indicated in the Mouse Input wiki article by reading the status byte, running: and then setting the new byte. I've confirmed that this new byte is accepted, but afterward I no longer receive IRQ's from the keyboard. Am I somehow effecting the keyboard given that I am communicating with ports 0x64 and 0x60 to send and receive the status byte?

I have never tested USB mice, or vmware, so I can't say anything abou #1 or #2, but it seems pretty clear that you should get the pure PS2 mouse working first and worry about the others afterward. You can put in a printf to find out what IRQ you get after you turn off USB->PS2 compatibility mode. Then please tell us all the answer. In general, PCI devices (including USB) get assigned a shared IRQ (A through D) by the BIOS at boot time.

As far as #3 goes, BTR and BTS are not the best opcodes to use to set/clear those two bits -- AND and OR are better. But if those are the only two instructions you use on the compaq status byte, then they should give you the right answer.

If you accidentally changed any other bits in the compaq status byte, or if you received a funny byte (specifically bit 0, or bit 4 -- you should examine the byte and see what those bits are set to!) those bits can kill your keyboard. The other things that can stop you from receiving IRQ1:
if your IRQ1 handler failed to send an EOI to the PIC
if you still have unread bytes on port 0x60 -- you only get an IRQ1 for the first one, unless you do a trick.