OSDev.org

Hey all, I've been working on adding a PCI + virtio layer to xv6 (a teaching OS). I've been using the osdev pages on PCI and virtio + the virtio 1.0 specification to build this. I've also been looking at existing implementations in minix, linux and a hobby operating system linked to in the osdev virtio page.

I've been having issues with configuring the virtio device itself. The specification talks about finding the VIRTIO_PCI_CAP_COMMON_CFG in the PCI capabilities list, figure out which BAR it is mapped to, and that's where you'll find the configuration structure. When I read from the address obtained from the BAR (BAR4), I don't get any valid values. I am running the OS in a debug build of qemu, and setting up breakpoints in the appropriate memory region handlers(virtio_pci_common_read/write), I don't see any of them being hit.

I tried the legacy interface of using the io space address in BAR0 (always the address 0xC000), but using the in/out instructions and reading/writing from the registers specified in the specification (and minix implementations), I do not get any valid values (it's all 1s). Setting up breakpoints in the functions handling the reads and writes to this region also was a dead end.

The only way I am able to configure the device is by using the VIRTIO_PCI_CAP_PCI_CFG and setting up the window to read and write values. The specification mentions that this is not the preferred way to configure the device. This also does not let me read device specific configuration for the network device which contains the mac address.

I apologize if I missed anything obvious. I've been banging my head against this problem for a while now and I would appreciate some help pointing me in the right direction .

My VirtIO network code works pretty well in VirtualBox, so here are the steps I'm currently using:

• Find a PCI device with Vendor/Device 1af4/1000.
• Get BAR0 for that device.
• Write 0x01 to BAR0+0x12 to Acknowledge that you've discovered the device.
• Write 0x03 to BAR0+0x12 to Notify the host that the driver is loaded.
• Write 0x00000000 to BAR0+0x0E to select Queue 0.
• Write 0x00000100 to BAR0+0x08 to set the Queue Address to 0x00100000.
• Write 0x00000001 to BAR0+0x0E to select Queue 1.
• Write 0x00000120 to BAR0+0x08 to set the Queue Address to 0x00120000.
• Set up buffer chains in memory for both Queues.
• Write 0x00010020 to BAR0+0x04 to set the Guest Feature flags.
• Write 0x07 to BAR0+0x12 to Notify the host that the driver is ready.
• Write 0x00 to BAR0+0x10 to Notify the host that the Queue 0 has been modified by the driver.

Hopefully this helps track down your issue. If not, let us know where you start having problems.

bastion wrote:I've been having issues with configuring the virtio device itself. The specification talks about finding the VIRTIO_PCI_CAP_COMMON_CFG in the PCI capabilities list, figure out which BAR it is mapped to, and that's where you'll find the configuration structure. When I read from the address obtained from the BAR (BAR4), I don't get any valid values.

That seems very strange. You should at least be able to read some static information such as the MAC without any further interaction with the device. Maybe you're just reading from a wrong address? Reading the common capability first and using the BAR indices and offsets from that is certainly the correct behavior.

FWIW here is my virtio implementation (standard transport in StandardPciTransport, legacy in LegacyPciTransport). The interesting method is finalizeFeatures().

Thank you for the replies. Turns out, there is something more to the issue than just the PCI config space.

I'm working on a 64 bit port of a 32 bit OS. And I tired to run my code in the 32 bit version and got both the IO space virito configuration and memory mapped configuration working. But the same routines in the 64 bit version aren't. It could be something to do with the way the kernel's page table is set up or how the IOAPIC is set up, but I don't think it's related to the PCI config space or the virtio implementation.

I'd been working on this for a long time now, and seeing that the device reported queue size of 256 for the rx and tx queues was one of the happiest in the last month

Now I've got to figure out why it isn't working in the 64 bit version.