Bochs' USB emulation

[BenLunt]

Hi guys,

If you have been following along with the few USB related threads lately, you might know that I have been updating the Bochs USB emulation to include a more accurate emulation as well as showing many more error checks.

My new code now checks for:
1) Speed indicator in the Transfer Descriptor matching the device speed.
2) Max packet size check, checking that the amount requested/sent is within limits.
3) Checking that the first packet sent is the GetDescriptor Request with a length equal to or less than the max packet size.
4) Checking the command length in the USB SCSI emulation.
5) Adding function to the USB SCSI emulation.
6) Checking command parameters within various emulations (SCSI (BBB), etc.)
7) Implemented "Boot Protocol" for the (HID) Mouse emulation.

As well as other checks and additions.

However, before I push it to the Bochs website as a pull-request, I need to test it further. If anyone here has a bootable image file of their OS with USB supported, I am requesting a copy of this image to further test my additions.

If you would like, please post a link here, or PM me for instructions on sending one to me via email. Please include a few details of which controller(s) and device(s) you support. A bochsrc.txt file would be best.

Thank you,
Ben

[BenLunt]

Thank you to those who sent me (a link to) their image files.

I have added quite a bit of function to the USB emulation as well as fixed some problems. I have tested with various versions of Windows from quite old to fairly new.

However, I would like to test with a Linux variation as well. I have and older version of Ubuntu (v12.xx), so I also grabbed a new version (v22.10). However, it is going on 20 minutes and still hasn't booted to any place I can see what is happening.

I am wondering if any of you have booted a (recent) *nix variation in Bochs that boots fairly quickly. If so, would you please point me to that resource?

Thank you,
Ben

[sounds]

I just tried a Gentoo minimal installation CD, maybe it's fast enough?

It takes 30sec just to decompress the kernel, reaches init by 90sec and about 25 minutes after that to decompress the files on its squashfs.

Here's the bochsrc I used:

Code: Select all

display_library: sdl2
plugin_ctrl: unmapped=1, biosdev=1, speaker=1, extfpuirq=1, parallel=1, serial=1

romimage: file=$BXSHARE/BIOS-bochs-latest

cpu: count=1, ips=60000000, model=bx_generic, reset_on_triple_fault=1, cpuid_limit_winnt=0, ignore_bad_msrs=1, mwait_is_nop=0

cpuid: level=6, stepping=3, model=3, family=6, vendor_string="GenuineIntel", brand_string="              Intel(R) Pentium(R) 4 CPU        "
cpuid: mmx=1, apic=xapic, simd=sse2, sse4a=0, misaligned_sse=0, sep=1, movbe=0, adx=0
cpuid: aes=0, sha=0, xsave=1, xsaveopt=1, x86_64=1, 1g_pages=1, pcid=1, fsgsbase=1
cpuid: smep=1, smap=1, mwait=1

memory: host=256, guest=256

vgaromimage: file=$BXSHARE/VGABIOS-lgpl-latest-cirrus

boot: cdrom

ata0: enabled=1, ioaddr1=0x1f0, ioaddr2=0x3f0, irq=14
ata0-master: type=none
ata0-slave: type=cdrom, path="install-amd64-minimal-20230129T164658Z.iso", status=inserted
ata1: enabled=1, ioaddr1=0x170, ioaddr2=0x370, irq=15
ata1-master: type=none
ata1-slave: type=none
ata2: enabled=0
ata3: enabled=0

print_timestamps: enabled=0
debugger_log: -

pci: enabled=1, chipset=i440fx, slot1=pcivga

clock: sync=none, time0=local

floppy_bootsig_check: disabled=0

panic: action=fatal
error: action=report
info: action=report
debug: action=ignore

debugger_log: -

com1: enabled=1, mode=file, dev=/dev/stderr

vga: update_freq=20

keyboard: type=mf, serial_delay=250, paste_delay=100000, user_shortcut=none
mouse: enabled=0

private_colormap: enabled=0
usb_uhci: enabled=1
usb_ohci: enabled=1, port1=keypad

[BenLunt]

Just a note to say that the first set of updates to the Bochs USB emulation has been merged.

This update was targeted toward those who are starting their USB programming.

For example, Bochs will now monitor the toggle bit in the Control and Bulk pipes, letting you know if you do not have the toggle bit correct. Other checks and balances are done to make sure you are sending the correct values in your packets, such as a valid Max Packet Size, etc.

My next update (no ETA) will have many more checks as well as a much enhanced xHCI emulation.

If your USB code use to work and now with this new emulation, it does not, if you cannot find the reason, please let me know.
(Please note that you may need to turn DEBUGing on to see some of the checks and balances performed)

Thank you,
Ben
- https://www.fysnet.net/the_universal_serial_bus.htm

[sounds]

Will try it out and post if I have any questions. Thanks, Ben.

[Gigasoft]

Did you fix the error where the UHCI controller continues down a queue even when a TD fails? I have also found another error, where the XHCI controller does not post a port status change event when a port has finished resetting.

[BenLunt]

Did you fix the error where the UHCI controller continues down a queue even when a TD fails?

Yes, I believe so. The UHCI had a considerable re-write. If you find that it does not, please let me know.

I have also found another error, where the XHCI controller does not post a port status change event when a port has finished resetting.

I thought I had this one already done, though it appears that I only did at a Power change. I will look in to this again. Thanks for the reminder.

Ben

[BenLunt]

I have also found another error, where the XHCI controller does not post a port status change event when a port has finished resetting.

I thought I had this one already done, though it appears that I only did at a Power change. I will look in to this again. Thanks for the reminder.

I believe I have fixed this issue as well. It will be pushed to the Bochs source tree at a later date. Thanks again for the reminder.

Please remember that for this event to be posted to the event ring:
1) the Halted bit must be zero
2) any of the following change bits transition from 0 to 1: CSC, PEC, WRC, OCC, PRC, PLC, CEC
3) all of the change bits have been reset to zero before another event will be posted.

i.e.: A change event will only be posted to a running schedule once when any of the seven listed bits transition from 0 to 1. If any of the bits transition from a 0 to 1 and any of the other bits are still set, the HC will not post a change event. All change bits must be set back to zero before a new change event will take place. (xHCI 1.0, Section 4.19.2)

(Note: Any of the bits could have been set *before* the Halted bit became zero, though the Event won't be posted until the halted bit becomes zero, as long as you don't clear all of the bits before then.)

Thanks,
Ben
- https://www.fysnet.net/the_universal_serial_bus.htm

[BenLunt]

Just a quick note to say that I have updated the Bochs USB emulation with a few enhancements and fixes. One of the enhancements now emulates UASP for disk drives and CD-ROMs, both high- and super-speed devices. Please note that it is experimental, though I have tested it on various modern OSes and have found no errors.

If you test your code and find errors or irregularities, please let me know.

The documentation at the sourceforge.io site hasn't caught up to the new documentation yet, but you can view the latest DocBook for the new items. The USB docs start at Line 4937. See line 5022 and 5090 for more on the UASP addition.

As always, thank you to everyone here in this forum, those who have sent PMs, and those who have directly emailed me with comments and corrections,
Ben
- https://www.fysnet.net/the_universal_serial_bus.htm

[BenLunt]

Hi,

Just a small update to announce new updated HID support in Bochs. You can now specify a model for the mouse.

Code: Select all

usb_uhci: enabled=1, port1=mouse, options1="speed:full, model:m228" # 3-byte report packet, 2 button, 8-bits

The model can be one of the following: 'm228', 'm338', 'm3312', 'm3316', 'm338phy', or 'm33xDebug'.

The number is decoded as, the m3312 for example, as 3 button, 3 coords, 12-bits. This means that there are 3 buttons, with three 12-bit coordinates; X, Y, and Wheel.

The m338phy includes a Physical Descriptor, a descriptor that defines what physical body part is used for each button, etc. This feature is rarely used, in fact, I have never seen a (normal) mouse have a physical descriptor. However, I thought it would be nice to test your parser.

Speaking of testing your parser, the m33xDebug model has a deliberately irregular Report Descriptor.

Code: Select all

byte 0:  YYYYYYY0  10-bit Y displacement (-512 -> 511)
byte 1:  WWWW0YYY  8-bit Wheel displacement (-128 -> 127)
byte 2:  0B00WWWW  bit 6 is Button #2 (right button)
byte 3:  XXXXX0B0  9-bit X displacement, bit 1 is Button #1 (left button)
byte 4:  0B00XXXX  bit 6 is Button #3 (middle button)

It is not anywhere near normal, the buttons aren't together, nor are they consecutive, as well as the coords are not byte aligned, and are each of different sizes. This is to help you test your HID Report Descriptor parsing code. Note, that it even uses the rarely used PUSH/POP 'instructions' in the descriptor. As Yoda might say, A robust test it is.

The keyboard emulation has been updated a bit too. It now handles more than one simultaneous keypress/release and will generate a "phantom packet" if too many simultaneous keys are pressed (more than 6).

Anyway, the updated and current documentation is still in docbook format (user.dbk), so it is a little difficult to read, but I am hoping that the maintainers will update the HTML files soon. Also, unless you can build from the source code, the executable is not yet available. If you are running a Windows Host, I can send you a 32- or 64-bit executable, upon request. I use Win10 as a Host.

If you find any problems or irregularities (besides the deliberate one), please let me know. (There is also a small fix pertaining to zero-byte short-packet detection.)

Thanks,
Ben
- https://www.fysnet.net/the_universal_serial_bus.htm

[MollenOS]

These updates are amazing Ben! They have uncovered several issues with my USB stack, and I super appreciate this! Even though getting proper toggle management is a bit of a pain.

Especially the HID improvements!

[BenLunt]

Thank you for your kind words.

[stlw]

Ben I think you have noticed Volker's comment:

"Please revert speed handling. Setting the speed for a device should be optional again. An existing config using the disk device now panics, here since the speed option is not set. The device defaults to full speed if not set, so no panic should happen."

I suggest to fix it because reverting looks not trivial now ...

[BenLunt]

Ben I think you have noticed Volker's comment:

"Please revert speed handling. Setting the speed for a device should be optional again. An existing config using the disk device now panics, here since the speed option is not set. The device defaults to full speed if not set, so no panic should happen."

I suggest to fix it because reverting looks not trivial now ...

Hi,

I did not see his comment. I think I currently have it default to low-speed, but I will have a look and see.
...
Yes, I have it default to Low-speed. This is a simple fix.

Please note that it might throw a PANIC on the xhci if the wrong port is given. For example, the following will throw a PANIC.

#assume a four port hub, with first two as USB3 and the last two as USB2
usb_xhci: port1=disk, options1="path:hdd.img, proto:bbb"

On the xhci, the first two ports must not use a speed less than SUPER. Therefore, the new code checks for this and a proper declaration should be:

usb_xhci: port3=disk, options3="path:hdd.img, proto:bbb"

The first two ports must be SUPER speed while the second two ports must be LOW-, FULL-, or HIGH-speed. If a speed is not given, it will default to a speed less than SUPER, which requires the declaration to use a port in the second half of the port set. My new code checks for this.

Please ask Volker if this is the case.

Thanks,
Ben

[BenLunt]

Hi everyone,

It has been a little while. Now that the snow has stopped falling, the ground is dry, and I don't have to worry about keeping the wood stove fed, I have been working on other projects out in the warm sun. :-) Anyway, I thought I would post an update to my USB work.

I have decided to Fork Bochs to include my USB emulation additions. With discussion and deep thought, it seems to be a better idea than to pepper the original source with my additions. Please understand though, that I am not trying to replace Bochs, I am simply creating a Fork to house my additions. At a later time, I hope the original Bochs team may wish to add my additions.

My source (fork) and documentation can be found at https://www.fysnet.net/bochs/index.php (https://github.com/fysnet/Bochs-USB-Edition).

The plan is to display all ports and relevant information at given times, for example when the start of frame happens.

Currently, only the XHCI has been (somewhat) implemented, the UHCI has been started, and the OHCI and possibly EHCI are on future plans.

Place the following line in your bochsrc.txt file:

Code: Select all

usb_debug: type=xhci, reset, enable, start_frame, doorbell, event, non_exist

Only one type can be used at a time/per emulated session (currently xhci only).
The 'reset' tag triggers the debugger when a port resets. (currently not implemented)
The 'enable' tag triggers the debugger when a port is enabled. (currently not implemented)
The 'start_frame' tag triggers the debugger at start of frame. (currently not fully implemented)
The 'doorbell' tag triggers the debugger when a doorbell is run.
The 'event' tag triggers the debugger when an event TRB is inserted in an event ring.
The 'non_exist' tag triggers the debugger when the guest writes to the first byte of the first non-existant port register set. (for debugging)

Currently, only the following should be used:

Code: Select all

usb_debug: type=xhci, doorbell, event

With the 'doorbell' tag, the debugger will trigger when the command doorbell is rung, displaying the command ring.
With the 'event' tag, the debugger will trigger when an event TRB is inserted in an interrupters event ring.

You can then view any TRB in the said ring, actually changing items within a TRB that hasn't been executed yet, or at least viewing the TRB.

It is highly experimental and needs a lot of work, but since I won't be putting much into it until late fall/early winter when the temperatures drastically fall again, I thought someone here might benefit from it. Also, if someone wants to add function and/or continuing my work, please do so by submitting pull requests.

Since I target Windows, all additions are only for Windows platforms. My Fork includes Windows executables for those who don't want to build the source themselves.

Again, please don't think I am taking over the Bochs project...this is far from. I am simply adding a USB debugger to the project (via my fork) and hopefully later down the road when it is tested and functions as expected, it can be merged into the original project.

Thanks again for all of you that have helped in one way or another. Of all my hobbies, this is the most enjoyable, and this forum and its contributors have made it happen.

Thanks so much,
Ben
- https://www.fysnet.net/bochs/index.php

P.S. There is a USB button now in the tool bar. It is used to stop the emulation and trigger the USB debugger. It is not fully implemented yet and probably should not be used, though it won't hurt to do so. If after clicking on the button, if the emulation isn't ready to display information (the controller isn't in the run state yet), squiggly lines will show on the button indicating that it is waiting for the controller to become available. Again, lots of work needs to be done. Please be patient.