MMIO not being set on qemu with xHCI

[slammar]

I have made a proof of concept bootloader that writes a value to somewhere in memory, and then sets the base address register of the xhci device to point to that location so that the value previously written should be overwritten by the xhci when read through MMIO. However, it only works on VirtualBox and not in qemu, and I don't have any clue why.

This is the code (I made it as simple as possible, but still feels somehow large):

Code: Select all

PCI_CONFIG_ADDRESS equ 0xCF8
PCI_CONFIG_DATA equ 0xCFC

PCI_BAR_0_OFFSET equ 0x10
PCI_BAR_1_OFFSET equ 0x14

%macro write_pci_register 0
    mov dx, PCI_CONFIG_ADDRESS
    out dx, eax
    mov eax, ebx
    mov dx, PCI_CONFIG_DATA
    out dx, eax
%endmacro

; PCI_ADDRESS:
; ; Enable Bit    Reserved    Bus Number  Device Number   Function Number Register Offset
; ; 31            30 - 24     23 - 16     15 - 11         10 - 8          7 - 0
; QEMU: 1_0000000_00000000_00010_000_00000000b --> 0x8000_1000
; VBOX: 1_0000000_00000000_00110_000_00000000b --> 0x8000_3000

PCI_DEV_ADDRESS equ 0x8000_1000

; BAR_0:
; QEMU: 1111111111111111_1100000000000100
; VBOX: 1111111111111111_0000000000000000

org 0x7C00
bits 16

mov ax, 0x0500 >> 4
mov ss, ax
mov esp, 0x7C00 - 0x0500
mov ax, 0
mov ds, ax

; Disable cache
mov eax, cr0
and eax, 1011_1111_1111_1111__1111_1111_1111_1111b
mov cr0, eax
invd

; Write anything to the MMIO address space
mov ax, 0x1000
mov es, ax
mov byte [es:0x00], 0x7D

; Set MMIO
mov eax, PCI_DEV_ADDRESS | PCI_BAR_0_OFFSET
mov ebx, 0x10000
write_pci_register
mov eax, PCI_DEV_ADDRESS | PCI_BAR_1_OFFSET
mov ebx, 0x00
write_pci_register

; Read the value back
mov ax, 0x1000
mov es, ax
mov byte al, [es:0x00]

; Print anything to screen if the value changed
cmp al, 0x7D
je end
mov ax, 0xB800
mov es, ax
mov word [es:0x00], 0x3300 | 'X'

end:
cli
hlt
jmp end

times 510-($-$$) db 0
dw 0xAA55

This code is supposed to print something in screen if the value effectively changed, which only happens with virtualbox.

These are the commands I use to test with qemu:

Code: Select all

nasm -l kernel.lst kernel.asm ; \
qemu-system-x86_64 \
    -device qemu-xhci \
    -drive format=raw,file=kernel

These are the commands I use to test with VBox:

Code: Select all

VBoxManage createvm \
    --name "X" \
    --ostype "Other" \
    --register \
    --basefolder "$(pwd)/vbox" \
    --uuid "e0b08add-d834-4af5-89e8-05abec11aa78" \
    --default ; \
VBoxManage modifyvm "e0b08add-d834-4af5-89e8-05abec11aa78" --usbxhci on ; \
nasm -l kernel.lst kernel.asm ; \
dd if=/dev/zero of=kernel.raw bs=1024 count=2048 ; \
dd if=kernel of=kernel.raw conv=notrunc ; \
VBoxManage convertfromraw kernel.raw kernel.vdi --format VDI ; \
VBoxManage storageattach "X" \
    --storagectl "IDE" \
    --port 0 \
    --device 0 \
    --type hdd \
    --medium  "$(pwd)"/kernel.vdi ; \
VBoxManage startvm "X"

The PCI device address is hardcoded to make the code simpler.

VBox version: 6.1.0 r135406
Qemu version: 4.1.0

[Octocontrabass]

Code: Select all

; Disable cache
mov eax, cr0
and eax, 1011_1111_1111_1111__1111_1111_1111_1111b
mov cr0, eax
invd

Clearing CR0.CD (bit 30) enables the cache. I'm not sure how much of an effect this has under emulators, since they have no reason to emulate the cache.

The INVD instruction violates cache coherency. If you want to flush the cache, use an instruction that flushes the cache without violating cache coherency, such as WBINVD.

Code: Select all

; Set MMIO
mov eax, PCI_DEV_ADDRESS | PCI_BAR_0_OFFSET
mov ebx, 0x10000
write_pci_register
mov eax, PCI_DEV_ADDRESS | PCI_BAR_1_OFFSET
mov ebx, 0x00
write_pci_register

I'm no expert on PCI, but I don't think you're supposed to map a device's MMIO to an address that's already mapped to something else (RAM).

[slammar]

Clearing CR0.CD (bit 30) enables the cache. I'm not sure how much of an effect this has under emulators, since they have no reason to emulate the cache.

I changed it to

Code: Select all

or eax, 0100_0000_0000_0000__0000_0000_0000_0000b

but the result remains the same.

The INVD instruction violates cache coherency. If you want to flush the cache, use an instruction that flushes the cache without violating cache coherency, such as WBINVD.

I invalidated the cache before performing any memory operation, so it shouldn't matter. Anyways, I tried using WBINVD instead and nothing happened.

I'm no expert on PCI, but I don't think you're supposed to map a device's MMIO to an address that's already mapped to something else (RAM).

Is not mapped to anything. This article from Osdev wiki states that memory from 0x00007E00 to 0x0007FFFF is guaranteed free for use. 0x10000 is in that range, so there shouldn't be any problems.

[Klakap]

Why you remapping MMIO? Are you want write xHci driver in real mode? It isn't very good idea. From my known you cant remap MMIO to anything. Anyway, using low memory is dangerous thing because you can easy cross BIOS things.

If you want to read your OS from USB, better idea is use BIOS interrupt.

[Octocontrabass]

I invalidated the cache before performing any memory operation, so it shouldn't matter.

The BIOS interrupt handlers will perform memory operations too.

Is not mapped to anything. This article from Osdev wiki states that memory from 0x00007E00 to 0x0007FFFF is guaranteed free for use. 0x10000 is in that range, so there shouldn't be any problems.

According to that article, it's mapped to RAM. You need an address that's mapped to nothing at all. You can't map two things (RAM and PCI MMIO) to the same address.

[slammar]

According to that article, it's mapped to RAM. You need an address that's mapped to nothing at all. You can't map two things (RAM and PCI MMIO) to the same address.

This might seem like a dumb question but How do I know where can I map PCI MMIO?

Thanks in advance.

[nullplan]

According to that article, it's mapped to RAM. You need an address that's mapped to nothing at all. You can't map two things (RAM and PCI MMIO) to the same address.

This might seem like a dumb question but How do I know where can I map PCI MMIO?

Thanks in advance.

You ask BIOS for a memory map. It will tell you about all memory available, and wherever there is nothing, you can map PCI MMIO. Note that you shouldn't have to do this, since the BIOS will initialize the PCI bus before booting your code, so the MBARs are likely already set up correctly. You might not be able to use these MBARs from real mode, but then, you might not be able to reach the unused memory areas, either. Real mode is only able to address 20 bits (and a half) of address space, and the times when 1MB was a lot of memory are long gone. Besides, switching to 32-bit protected mode takes about five instructions, so Just Do It!