Problem receiving IRQ in driver

[onlyonemac]

Hi,

I've got, as part of a driver, the following code:

Code: Select all

static void fdc_wait_for_irq()
{
	while (instance->irq == FALSE)
	{
		// wait
	}
}

void irq6_handler(uint8_t interrupt, uint32_t data)
{
	if (interrupt != 0x26)
	{
		return;
	}

	((instance_data*) data)->irq = TRUE;
}

I've confirmed that data in irq6_handler is equal to (uint32_t) instance. irq6_handler is being called from an interrupt dispatcher in my kernel, and is known to work correctly. Here also is a disassembly of both functions:

Code: Select all

00000513 <fdc_wait_for_irq>:
 513:    55                       push   %ebp
 514:    89 e5                    mov    %esp,%ebp
 516:    e8 fc ff ff ff           call   517 <fdc_wait_for_irq+0x4>
 51b:    81 c2 02 00 00 00        add    $0x2,%edx
 521:    90                       nop
 522:    8b 82 00 00 00 00        mov    0x0(%edx),%eax
 528:    8b 40 10                 mov    0x10(%eax),%eax
 52b:    85 c0                    test   %eax,%eax
 52d:    74 f3                    je     522 <fdc_wait_for_irq+0xf>
 52f:    90                       nop
 530:    5d                       pop    %ebp
 531:    c3                       ret

Code: Select all

00000582 <irq6_handler>:
 582:    55                       push   %ebp
 583:    89 e5                    mov    %esp,%ebp
 585:    83 ec 04                 sub    $0x4,%esp
 588:    e8 fc ff ff ff           call   589 <irq6_handler+0x7>
 58d:    05 01 00 00 00           add    $0x1,%eax
 592:    8b 45 08                 mov    0x8(%ebp),%eax
 595:    88 45 fc                 mov    %al,-0x4(%ebp)
 598:    80 7d fc 26              cmpb   $0x26,-0x4(%ebp)
 59c:    75 0c                    jne    5aa <irq6_handler+0x28>
 59e:    8b 45 0c                 mov    0xc(%ebp),%eax
 5a1:    c7 40 10 01 00 00 00     movl   $0x1,0x10(%eax)
 5a8:    eb 01                    jmp    5ab <irq6_handler+0x29>
 5aa:    90                       nop
 5ab:    c9                       leave  
 5ac:    c3                       ret

The problem is that fdc_wait_for_irq never returns. I thought it was possible that either the handler or the loop was being optimised away, but the disassembly shows otherwise. The loop is being returned to correctly after the interrupt. Reading back the value from ((instance_data*) data)->irq inside the interrupt handler indicates that it is being set to TRUE correctly. Why might the loop not be exiting?

Thanks,

onlyonemac

[linuxyne]

The irq field is at offset 0x10 within the structure.
Assuming that 'instance' is stored at a linear address 0xa7000, you seem to be confirming (with evidence) that, after irq() wrote 1 to 0xa7010, wait() continued to read 0 from 0xa7010.

Other conditions:
The memory accesses are not optimised out. wait() and irq() both ran on the same cpu.

* Are there more than one cpus?
* Do wait() and irq() operate within the same virtual address space?
* Any unexpected memory overwrites?

[onlyonemac]

Assuming that 'instance' is stored at a linear address 0xa7000, you seem to be confirming (with evidence) that, after irq() wrote 1 to 0xa7010, wait() continued to read 0 from 0xa7010.

Yes. Also, irq6_handler() read 1 from the same address that fdc_wait_for_irq() was continuing to read 0 from. (Except I believe that "instance" is not stored at linear address 0x000A7000. The actual structure is at address 0x03002190 if I remember correctly (it's dynamically allocated by the kernel) and the global pointer "instance" is stored somewhere around 0x00020000 (exact location depends on the size of the kernel and the size of all the drivers, but it's stored somewhere in the driver's binary). Care to explain how you conclude that "instance" is at address 0x000A7000?) Edit: never mind I see you're just using that as an example.

* Are there more than one cpus?
* Do wait() and irq() operate within the same virtual address space?

There is one CPU and both functions have the same address space, yes.

[onlyonemac]

* Any unexpected memory overwrites?

No (although I may have missed something). I'm pretty sure that everything I think is allocated somewhere really is allocated, and my stack is not overflowing. I know as a fact that the structure to which "instance" points is allocated, and that nothing else is allocated there.

[linuxyne]

There is one CPU and both functions have the same address space, yes.

* Any unexpected memory overwrites?

No (although I may have missed something). I'm pretty sure that everything I think is allocated somewhere really is allocated, and my stack is not overflowing. I know as a fact that the structure to which "instance" points is allocated, and that nothing else is allocated there.

Understood.

After setting the field to 1, irq() can setup a debug register to force the cpu to 'break' on write on that field, in case we want to capture any unexpected overwrites.

Added quotes.

* If this is an emulator, examining the location through its debugger should shows us the actual contents of the memory from outside of the OS.

* Dumping the entire 'instance', inside both irq() and wait(), should show if the discrepancy between their views of it is wider than just a single field.

[onlyonemac]

* Dumping the entire 'instance', inside both irq() and wait(), should show if the discrepancy between their views of it is wider than just a single field.

OK so I tried dumping instance->path (and "((instance_data*) data)->path" inside irq6_handler()), which is a pointer to a string and the field directly before instance->irq, and it had the same correct non-zero value in both places. I should try adding a field after instance->irq (currently irq is the last field in struct instance_data) and see what happens with its value... EDIT: Done and it works correctly.

[linuxyne]

* Dumping the entire 'instance', inside both irq() and wait(), should show if the discrepancy between their views of it is wider than just a single field.

OK so I tried dumping instance->path (and "((instance_data*) data)->path" inside irq6_handler()), which is a pointer to a string and the field directly before instance->irq, and it had the same correct non-zero value in both places. I should try adding a field after instance->irq (currently irq is the last field in struct instance_data) and see what happens with its value... EDIT: Done and it works correctly.

Did you mean to say that the fields surrounding irq are not affected (but irq is, as we already know)?
Or, that after adding an extra field after irq, the rw to the irq field started working as expected?

[onlyonemac]

Did you mean to say that the fields surrounding irq are not affected (but irq is, as we already know)?
Or, that after adding an extra field after irq, the rw to the irq field started working as expected?

The fields around irq are not affected.

Get this: I added a new field, "test", at the end of the structure, and tried writing to it from the irq handler and it worked. In other words:

Code: Select all

<somewhere in initialisation routine>

instance->test = 0x01234567;

Code: Select all

static void fdc_wait_for_irq()
{
	while (instance->irq == FALSE)
	{
		if (instance->test == 0x89ABCDEF)
		{
			sys_log("FDC: true");
		}

		// wait
	}
}

Code: Select all

void irq6_handler(uint8_t interrupt, uint32_t data)
{
	// don't let anyone else attach us as their interrupt handler
	if (interrupt != 0x26)
	{
		return;
	}

	((instance_data*) data)->irq = TRUE;
	((instance_data*) data)->test = 0x89ABCDEF;
}

The system log gets flooded with "FDC: true". In other words, the value of instance->test has clearly changed but not instance->irq.

[linuxyne]

Get this: I added a new field, "test", at the end of the structure, and tried writing to it from the irq handler and it worked.

Code: Select all

...

The system log gets flooded with "FDC: true". In other words, the value of instance->test has clearly changed but not instance->irq.

That, is quite interesting. At this stage, I would step through bochsdbg, placing pb breakpoints. And carry out a code review.

Edit0: Are we sure that the wait() did not actually return (where irq got set to FALSE) and get invoked again (with irq set to FALSE), very quickly?
I think not, but the question arose since the activities of wait() and its caller are not visible here.

[onlyonemac]

Edit0: Are we sure that the wait() did not actually return (where irq got set to 0 again) and get invoked again, very quickly?
I think not, but the question arose since the activities of wait() and its caller are not visible here.

Hmm... I just checked and it actually did return and get called again. It seems to be hanging on the next interrupt though. According to a debug log in my kernel's interrupt dispatcher, the second interrupt never arrives. I had thus assumed that the first interrupt wait was never returning, but it actually is and it seems that the issue is that the second interrupt is not arriving. Whether that's because I'm expecting the FDC to send an interrupt at the wrong time or I haven't acknowledged the previous interrupt correctly I don't know.

[onlyonemac]

Here's my interrupt dispatcher. It gets called from an assembly wrapper for each interrupt, where the assembly wrapper is the routine pointed to by the IDT.

Code: Select all

void irq_handler(uint32_t irq)
{
	if (interrupt_handler_table[irq] != NULL)
	{
		interrupt_handler_table[irq](irq, interrupt_handler_data_table[irq]);
	}

	if (irq >= 0x20 && irq < 0x30)
	{
		if (irq >= 0x28)
		{
			hardware_outb(0x00A0, 0x20);
		}

		hardware_outb(0x0020, 0x20);
	}
}

interrupt_handler_table and interrupt_handler_data_table store the function pointer and custom data parameter, respectively, for whatever code wants to respond to the interrupt (in this case, the aforementioned irq6_handler() function). irq is passed from the assembly wrapper and is known to be correct.

Does this look OK?

[linuxyne]

. . . the issue is that the second interrupt is not arriving. Whether that's because I'm expecting the FDC to send an interrupt at the wrong time or I haven't acknowledged the previous interrupt correctly I don't know.

* It may also be a synchronization problem.
* Are interrupts from other devices, such as timer and keyboard, being serviced properly?
* Is wait() being called inside a loop which processes IO? If so, what is the request, for which the driver configured the controller before calling into wait()?

Here's my interrupt dispatcher. It gets called from an assembly wrapper for each interrupt, where the assembly wrapper is the routine pointed to by the IDT.

Code: Select all

void irq_handler(uint32_t irq)
{
	if (interrupt_handler_table[irq] != NULL)
	{
		interrupt_handler_table[irq](irq, interrupt_handler_data_table[irq]);
	}

	if (irq >= 0x20 && irq < 0x30)
	{
		if (irq >= 0x28)
		{
			hardware_outb(0x00A0, 0x20);
		}

		hardware_outb(0x0020, 0x20);
	}
}

interrupt_handler_table and interrupt_handler_data_table store the function pointer and custom data parameter, respectively, for whatever code wants to respond to the interrupt (in this case, the aforementioned irq6_handler() function). irq is passed from the assembly wrapper and is known to be correct.

Does this look OK?

That does look okay. The prevention or channelling of nested interrupts might be in the asm wrapper, I think.

[SpyderTL]

Where is the code that sets "irq" back to FALSE?

The FDC wiki article actually points out that an interrupt could happen before that code runs, which would mean that you lose the second interrupt.

You have to set it to FALSE before sending any commands to the controller, handle the interrupt, read any data in the FDC data buffer, then set the irq flag to FALSE again, then return from your interrupt, so that another interrupt can't occur before you have set the flag to FALSE.

I don't use interrupts very often, but when I do, I use a counter instead of a flag, and I store the value before sending any commands, and I wait for the counter to change, and then I increment my stored value, so that two or more back-to-back interrupts will be handled the correct number of times.

This may solve your problem, as well.

[onlyonemac]

* It may also be a synchronization problem.

Definitely not that, I specially made sure to avoid it. I set irq to FALSE before I send the command to the FDC.

* Are interrupts from other devices, such as timer and keyboard, being serviced properly?

Currently I'm not receiving any other interrupts.

* Is wait() being called inside a loop which processes IO? If so, what is the request, for which the driver configured the controller before calling into wait()?

It's the recalibrate command that's hanging at the moment; if someone could suggest another command that I can try that would be great.

The prevention or channelling of nested interrupts might be in the asm wrapper, I think.

Currently I'm not doing anything to avoid receiving nested interrupts. I can receive a second interrupt while the first one is still being processed. Should I try to avoid this (e.g. cli on entry to the asm wrapper and sti before iret)?

[SpyderTL]

Currently I'm not doing anything to avoid receiving nested interrupts. I can receive a second interrupt while the first one is still being processed. Should I try to avoid this (e.g. cli on entry to the asm wrapper and sti before iret)?

Either handle your interrupt entirely inside your interrupt handler (which prevents nested interrupts), or make your interrupt flag a counter, and separately keep track of which interrupt you are currently handling.

Specifically, for your code, you would have to set the IRQ flag back to FALSE inside your interrupt handler, before your IRET statement.