XHCI working on emulator but not on real hardware

[SanderR]

Hello everyone,

After allmost a year I finally managed to make me XHCI driver work on the BOCHS and QEMU emulator.
Now I tried to make it work on real hardware.
The first problem which I found was that the ENABLE_SLOT command gave a 1 as completioncode but a 0 as slotnumber (I assumed it would start at 1).
After fixing this, the code runs smooth untill line 1126 ( GET_DESCRIPTOR on EndPoint0 ring) where it gives completioncode 0.
Could someone please tell me what I am doing wrong?

Link to sourcecode:
https://github.com/AdeRegt/SanderOSUSB/ ... dev/xhci.c

[BenLunt]

Without looking at your code--sorry I don't have the time at the moment--here are a few ideas:

1) Timing is always an issue. The emulators don't care the timing between a command and an event, while real hardware does.
2) Is your real hardware 64-bit and if so, have you initialized the correct high-order registers as well as checking if the controller supports 64-bits.
3) When writing the xHCI_INTERRUPTER_ADDRESS register, it must be the last of the initialization phase. i.e.: you must initialize all other parts of the interrupters before you write this register.
4) in the IRQ, you must acknowledge the interrupt (xHC_OPS_USBStatus) *before* you write to the xHC_INTERRUPTER_IMAN:IP bit.

All of these items are not really managed within an emulator, but are very much so on real hardware.

Hope this helps,
Ben

[SanderR]

Thank you for your answer.

1) Is this before or after the run bit is set?
2) Yes, my real hardware is 64bit. the CPU is in 32bit state. All 64bit side of the registers are 0.
3) This is the last thing I do before I set the RUN-bit on 1.

Code: Select all

	// DCBAAP
	printf("[XHCI] Setting up DCBAAP\n");
	unsigned long btc[20] __attribute__ ((aligned (0x100)));
	((unsigned long*)bcbaap)[0] |= (unsigned long)&btc;
	((unsigned long*)bcbaap)[1] = 0;
	
	// setting first interrupt enabled.
	if(1){
		printf("[XHCI] Setting up First Interrupter\n");
		unsigned long iman_addr = rtsoff + 0x020;
		((unsigned long*)iman_addr)[0] |= 0b10; // Interrupt Enable (IE) – RW
		sleep(50);
	}
	printf("[XHCI] Use interrupts\n");
	((unsigned long*)usbcmd)[0] |= 4;
	sleep(50);

I created the interrupt code because I have not made this before your post.
4) Yes, I am doing this:

Code: Select all

void irq_xhci(){
	unsigned long xhci_usbsts = ((unsigned long*)usbsts)[0];
	if(xhci_usbsts&4){
		printf("[XHCI] Host system error interrupt\n");
	}
	if(xhci_usbsts&8){
		printf("[XHCI] Event interrupt\n");
	}
	if(xhci_usbsts&0x10){
		printf("[XHCI] Port interrupt\n");
	}
	((volatile unsigned long*)&interrupter_1)[0] = 0xCD;
	
	((unsigned long*)usbsts)[0] |= 0x8;
	((unsigned long*)usbsts)[0] |= 0b10000;
	unsigned long iman_addr = rtsoff + 0x020;
	((unsigned long*)iman_addr)[0] |= 1;
	outportb(0xA0,0x20);
	outportb(0x20,0x20);
}

I found out that what I wrote in my first post was not entirely right.
Because when I was checking where the event_queue I found out that on real hardware they add an event object when a port is attached.
I wrote a small code to find an empty entry and wait until it is filled. this works on the emulators and on real hardware.
The real hardware is now returning a 0 as a return code.
The real hardware is not printing "event interrupt" while it does when a port connects the event rises.
With this information, is it correct to assume that there is something wrong with the command queue?

[iloveosdev]

-

[SanderR]

Hello!

Well, welcome

Thank you for your reply.
I have no scratchpad entries. I just added them but this did not help.

https://github.com/AdeRegt/SanderOSUSB/ ... dev/xhci.c
is my sourcecode

[iloveosdev]

-

[SanderR]

Hello,

Thank you for your answer.
I did set the CONFIG to 10

The port enumeration is not a problem.
The code detects an attached device.
The problem is at line 1016. We ask for a slotid but the system does not issue an interrupt on real hardware (while it does on emulators) and keeps hanging in the loop defined at line 619 waiting for the command to complete.

[BenLunt]

What is the IRQ number?

Does your real hardware have an APIC, ACPI, and AML?

You may have to run through the ACPI asking it for the correct IRQ number? Later machines actually use numbers in the 21->23 range.

If you don't have a working ACPI parser and/or AML processor, you can "cheat" and place a DEBUG line in most of your ISRs (Interrupt Service Routines), then see which ISR prints the debug when it fires. Then, hard code this IRQ. It isn't something that you want to leave in, or rely upon, but at least it will get you up and going.

Ben

[SanderR]

Thank you for your answer.

After turning the "RUN" bit on, there is an interrupt triggered and on USBSTATUS register the "PORT CHANGE" bit is set on (system prints

[XHCI] Port interrupt

and

[XHCI] Event interrupt

see at https://github.com/AdeRegt/SanderOSUSB/ ... hci.c#L643 )
So I think the interrupt is caught. but after this I see no event.

[SanderR]

I have some new things I have discovered which I wanted to share with you.
Maybe it gives you an idea of what I am doing wrong.

So I did add the scratchpad buffers but I also decided to test the commandring with a NOOP command.
This command was working on real hardware with interrupts off. this means the command ring is working, but somehow does not accept me enable_port command.

[BenLunt]

Sorry, I have been away for a little while. May I ask the status of your project? Have you been able to find out what is not working?

Ben

[SanderR]

Hello Ben,

Welcome back!
My status is:
I set up a scratchpad buffer. On real hardware 1, it does accept the scratchpad buffer. on real hardware 2 is turn the screen black. (probably used too large buffer).
On real hardware1 this does not result in responding to the ENABLE_SLOT command.

I checked if the command-ring was functioning as it should be. And it did: I sent a NOOP command and XHCI responded on real hardware 1 and 2 and on bochs (qemu did not have this feature implemented).

I checked if the device type was maybe invalid. but the device type-id inside the extended capabilities are the same (type 0).

So the problem is: The system does not execute the ENABLE_SLOT command offered in the commandring.
The code is: https://github.com/AdeRegt/SanderOSUSB/ ... dev/xhci.c

I hope you have some ideas because currently im not having any idea what it could be.

[BenLunt]

Throughout the code you use things like:

Code: Select all

1104      unsigned char offsetA = deviceid!=XHCI_DEVICE_BOCHS?32:64;
1105      unsigned char offsetB = deviceid!=XHCI_DEVICE_BOCHS?64:128;

This means you are hard coding the the slot size depending on if you are using QEMU or BOCHS.

There are two things wrong with this.
- The controller will tell you what size the slot is. You need to simply calculate this size in your initialization routine and then use that saved value everywhere else. There is no need to hard code it like you have here.
- Second, on real hardware (which will have a completely different deviceid value), does it do what QEMU does or what BOCHS does?
Get the size of the context/slots/etc. from the Configuration registers, bit 2 of HCCPARAMS1.

Starting at line 1038, you "scroll" through the register set checking for connections. You shouldn't do it this way.
Each register set may have a paired "other-speed" register set. The xHCI has a mechanism to tell you which set is paired to which set.

For example, let's say that port 0 register set is a USB3 set, and port 3 register set is a USB2 set, paired to port 0.
The way your code currently handles the register sets, if a device is attached to port 0, you enumerate the device. Then you scroll through to the next set, and so on.

Use HCCPARAMS1 to get the context size. Then you don't have to check to see what hardware you are running on.
Use the xHCI mechanism to get the Port Routing and Control specification. This will tell you which register set belongs to which physical socket. One of the HCCPARAMS registers will tell you how many register sets are present, not how many sockets are present. Don't assume that every USB3 set will have a paired USB2 set, and visa-versa. It is perfectly legal to have a USB2 only socket. Also, don't assume that all the USB3 sets are consecutive, before or after the USB2 sets. The Port Routing and Control sequence can have any sequence of sets, consecutive or not.

Also, I noticed at line 414, you get the cycle bit, but return a different value depending on the deviceid you are using. Your code initializes the cycle bit for all rings. Therefore, your code should known what the initial cycle bit will be. There should be no need for that tidbit of code.

Work with these items a bit and see what you come up with.

Hope that helps,
Ben

[SanderR]

Hello Ben!

Thank you for your reply.
In the meantime, I was doing some research on my scratchpad buffer and I managed to proceed until I need to communicate with the local ring. (GET_DESCRIPTOR) which does not accept a NOOP, this is new for me. still, the code works on emulators but not on real hardware.

From this point, things are not working anymore on real hardware: https://github.com/AdeRegt/SanderOSUSB/ ... ci.c#L1230

[BenLunt]

I was doing some research on my scratchpad buffer...

At this point, you can totally forget about the scratchpad buffer. The only thing you have to remember is the 1st pointer in the array of pointers is the scratchpad pointer and must point to a specified size of free (physical) memory. Other than that, completely forget about the scratchpad. The scratchpad space is used by the hardware for save and restore as well as other things it might wish to use it for. Your code does not/should not rely upon any fact about the scratchpad buffer.

If I get a chance, I will look through your code again.

Ben