Page 1 of 2
Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 9:16 am
by onlyonemac
Hi,
I've got, as part of a driver, the following code:
Code: Select all
static void fdc_wait_for_irq()
{
while (instance->irq == FALSE)
{
// wait
}
}
void irq6_handler(uint8_t interrupt, uint32_t data)
{
if (interrupt != 0x26)
{
return;
}
((instance_data*) data)->irq = TRUE;
}
I've confirmed that data in irq6_handler is equal to (uint32_t) instance. irq6_handler is being called from an interrupt dispatcher in my kernel, and is known to work correctly. Here also is a disassembly of both functions:
Code: Select all
00000513 <fdc_wait_for_irq>:
513: 55 push %ebp
514: 89 e5 mov %esp,%ebp
516: e8 fc ff ff ff call 517 <fdc_wait_for_irq+0x4>
51b: 81 c2 02 00 00 00 add $0x2,%edx
521: 90 nop
522: 8b 82 00 00 00 00 mov 0x0(%edx),%eax
528: 8b 40 10 mov 0x10(%eax),%eax
52b: 85 c0 test %eax,%eax
52d: 74 f3 je 522 <fdc_wait_for_irq+0xf>
52f: 90 nop
530: 5d pop %ebp
531: c3 ret
Code: Select all
00000582 <irq6_handler>:
582: 55 push %ebp
583: 89 e5 mov %esp,%ebp
585: 83 ec 04 sub $0x4,%esp
588: e8 fc ff ff ff call 589 <irq6_handler+0x7>
58d: 05 01 00 00 00 add $0x1,%eax
592: 8b 45 08 mov 0x8(%ebp),%eax
595: 88 45 fc mov %al,-0x4(%ebp)
598: 80 7d fc 26 cmpb $0x26,-0x4(%ebp)
59c: 75 0c jne 5aa <irq6_handler+0x28>
59e: 8b 45 0c mov 0xc(%ebp),%eax
5a1: c7 40 10 01 00 00 00 movl $0x1,0x10(%eax)
5a8: eb 01 jmp 5ab <irq6_handler+0x29>
5aa: 90 nop
5ab: c9 leave
5ac: c3 ret
The problem is that fdc_wait_for_irq never returns. I thought it was possible that either the handler or the loop was being optimised away, but the disassembly shows otherwise. The loop is being returned to correctly after the interrupt. Reading back the value from ((instance_data*) data)->irq inside the interrupt handler indicates that it is being set to TRUE correctly. Why might the loop not be exiting?
Thanks,
onlyonemac
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 10:22 am
by linuxyne
The irq field is at offset 0x10 within the structure.
Assuming that 'instance' is stored at a linear address 0xa7000, you seem to be confirming (with evidence) that, after irq() wrote 1 to 0xa7010, wait() continued to read 0 from 0xa7010.
Other conditions:
The memory accesses are not optimised out. wait() and irq() both ran on the same cpu.
* Are there more than one cpus?
* Do wait() and irq() operate within the same virtual address space?
* Any unexpected memory overwrites?
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 10:53 am
by onlyonemac
linuxyne wrote:Assuming that 'instance' is stored at a linear address 0xa7000, you seem to be confirming (with evidence) that, after irq() wrote 1 to 0xa7010, wait() continued to read 0 from 0xa7010.
Yes. Also, irq6_handler() read 1 from the same address that fdc_wait_for_irq() was continuing to read 0 from.
(Except I believe that "instance" is not stored at linear address 0x000A7000. The actual structure is at address 0x03002190 if I remember correctly (it's dynamically allocated by the kernel) and the global pointer "instance" is stored somewhere around 0x00020000 (exact location depends on the size of the kernel and the size of all the drivers, but it's stored somewhere in the driver's binary). Care to explain how you conclude that "instance" is at address 0x000A7000?) Edit: never mind I see you're just using that as an example.
linuxyne wrote:* Are there more than one cpus?
* Do wait() and irq() operate within the same virtual address space?
There is one CPU and both functions have the same address space, yes.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 10:56 am
by onlyonemac
linuxyne wrote:* Any unexpected memory overwrites?
No (although I may have missed something). I'm pretty sure that everything I think is allocated somewhere really is allocated, and my stack is not overflowing. I know as a fact that the structure to which "instance" points is allocated, and that nothing else is allocated there.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 11:02 am
by linuxyne
onlyonemac wrote:
There is one CPU and both functions have the same address space, yes.
onlyonemac wrote:
linuxyne wrote:
* Any unexpected memory overwrites?
No (although I may have missed something). I'm pretty sure that everything I think is allocated somewhere really is allocated, and my stack is not overflowing. I know as a fact that the structure to which "instance" points is allocated, and that nothing else is allocated there.
Understood.
After setting the field to 1, irq() can setup a debug register to force the cpu to 'break' on write on that field, in case we want to capture any unexpected overwrites.
Added quotes.
* If this is an emulator, examining the location through its debugger should shows us the actual contents of the memory from outside of the OS.
* Dumping the entire 'instance', inside both irq() and wait(), should show if the discrepancy between their views of it is wider than just a single field.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 12:18 pm
by onlyonemac
linuxyne wrote:* Dumping the entire 'instance', inside both irq() and wait(), should show if the discrepancy between their views of it is wider than just a single field.
OK so I tried dumping instance->path (and "((instance_data*) data)->path" inside irq6_handler()), which is a pointer to a string and the field directly before instance->irq, and it had the same correct non-zero value in both places. I should try adding a field
after instance->irq (currently irq is the last field in struct instance_data) and see what happens with its value... EDIT: Done and it works correctly.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 12:44 pm
by linuxyne
onlyonemac wrote:linuxyne wrote:* Dumping the entire 'instance', inside both irq() and wait(), should show if the discrepancy between their views of it is wider than just a single field.
OK so I tried dumping instance->path (and "((instance_data*) data)->path" inside irq6_handler()), which is a pointer to a string and the field directly before instance->irq, and it had the same correct non-zero value in both places. I should try adding a field
after instance->irq (currently irq is the last field in struct instance_data) and see what happens with its value... EDIT: Done and it works correctly.
Did you mean to say that the fields surrounding irq are not affected (but irq is, as we already know)?
Or, that after adding an extra field after irq, the rw to the irq field started working as expected?
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 12:53 pm
by onlyonemac
linuxyne wrote:Did you mean to say that the fields surrounding irq are not affected (but irq is, as we already know)?
Or, that after adding an extra field after irq, the rw to the irq field started working as expected?
The fields around irq are not affected.
Get this: I added a new field, "test", at the end of the structure, and tried writing to it from the irq handler and it worked. In other words:
Code: Select all
<somewhere in initialisation routine>
instance->test = 0x01234567;
Code: Select all
static void fdc_wait_for_irq()
{
while (instance->irq == FALSE)
{
if (instance->test == 0x89ABCDEF)
{
sys_log("FDC: true");
}
// wait
}
}
Code: Select all
void irq6_handler(uint8_t interrupt, uint32_t data)
{
// don't let anyone else attach us as their interrupt handler
if (interrupt != 0x26)
{
return;
}
((instance_data*) data)->irq = TRUE;
((instance_data*) data)->test = 0x89ABCDEF;
}
The system log gets flooded with "FDC: true". In other words, the value of instance->test has clearly changed but not instance->irq.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 1:01 pm
by linuxyne
onlyonemac wrote:
Get this: I added a new field, "test", at the end of the structure, and tried writing to it from the irq handler and it worked.
The system log gets flooded with "FDC: true". In other words, the value of instance->test has clearly changed but not instance->irq.
That, is quite interesting. At this stage, I would step through bochsdbg, placing pb breakpoints. And carry out a code review.
Edit0: Are we sure that the wait() did not actually return (where irq got set to FALSE) and get invoked again (with irq set to FALSE), very quickly?
I think not, but the question arose since the activities of wait() and its caller are not visible here.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 1:16 pm
by onlyonemac
linuxyne wrote:Edit0: Are we sure that the wait() did not actually return (where irq got set to 0 again) and get invoked again, very quickly?
I think not, but the question arose since the activities of wait() and its caller are not visible here.
Hmm... I just checked and it actually did return and get called again. It seems to be hanging on the next interrupt though. According to a debug log in my kernel's interrupt dispatcher, the second interrupt never arrives. I had thus assumed that the first interrupt wait was never returning, but it actually is and it seems that the issue is that the second interrupt is not arriving. Whether that's because I'm expecting the FDC to send an interrupt at the wrong time or I haven't acknowledged the previous interrupt correctly I don't know.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 1:19 pm
by onlyonemac
Here's my interrupt dispatcher. It gets called from an assembly wrapper for each interrupt, where the assembly wrapper is the routine pointed to by the IDT.
Code: Select all
void irq_handler(uint32_t irq)
{
if (interrupt_handler_table[irq] != NULL)
{
interrupt_handler_table[irq](irq, interrupt_handler_data_table[irq]);
}
if (irq >= 0x20 && irq < 0x30)
{
if (irq >= 0x28)
{
hardware_outb(0x00A0, 0x20);
}
hardware_outb(0x0020, 0x20);
}
}
interrupt_handler_table and interrupt_handler_data_table store the function pointer and custom data parameter, respectively, for whatever code wants to respond to the interrupt (in this case, the aforementioned irq6_handler() function). irq is passed from the assembly wrapper and is known to be correct.
Does this look OK?
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 3:57 pm
by linuxyne
onlyonemac wrote:
. . . the issue is that the second interrupt is not arriving. Whether that's because I'm expecting the FDC to send an interrupt at the wrong time or I haven't acknowledged the previous interrupt correctly I don't know.
* It may also be
a synchronization problem.
* Are interrupts from other devices, such as timer and keyboard, being serviced properly?
* Is wait() being called inside a loop which processes IO? If so, what is the request, for which the driver configured the controller before calling into wait()?
onlyonemac wrote:Here's my interrupt dispatcher. It gets called from an assembly wrapper for each interrupt, where the assembly wrapper is the routine pointed to by the IDT.
Code: Select all
void irq_handler(uint32_t irq)
{
if (interrupt_handler_table[irq] != NULL)
{
interrupt_handler_table[irq](irq, interrupt_handler_data_table[irq]);
}
if (irq >= 0x20 && irq < 0x30)
{
if (irq >= 0x28)
{
hardware_outb(0x00A0, 0x20);
}
hardware_outb(0x0020, 0x20);
}
}
interrupt_handler_table and interrupt_handler_data_table store the function pointer and custom data parameter, respectively, for whatever code wants to respond to the interrupt (in this case, the aforementioned irq6_handler() function). irq is passed from the assembly wrapper and is known to be correct.
Does this look OK?
That does look okay. The prevention or channelling of nested interrupts might be in the asm wrapper, I think.
Re: Problem receiving IRQ in driver
Posted: Fri Aug 05, 2016 7:25 pm
by SpyderTL
Where is the code that sets "irq" back to FALSE?
The FDC wiki article actually points out that an interrupt could happen before that code runs, which would mean that you lose the second interrupt.
You have to set it to FALSE before sending any commands to the controller, handle the interrupt, read any data in the FDC data buffer, then set the irq flag to FALSE again, then return from your interrupt, so that another interrupt can't occur before you have set the flag to FALSE.
I don't use interrupts very often, but when I do, I use a counter instead of a flag, and I store the value before sending any commands, and I wait for the counter to change, and then I increment my stored value, so that two or more back-to-back interrupts will be handled the correct number of times.
This may solve your problem, as well.
Re: Problem receiving IRQ in driver
Posted: Sat Aug 06, 2016 12:53 am
by onlyonemac
Definitely not that, I specially made sure to avoid it. I set irq to FALSE before I send the command to the FDC.
linuxyne wrote:* Are interrupts from other devices, such as timer and keyboard, being serviced properly?
Currently I'm not receiving any other interrupts.
linuxyne wrote:* Is wait() being called inside a loop which processes IO? If so, what is the request, for which the driver configured the controller before calling into wait()?
It's the recalibrate command that's hanging at the moment; if someone could suggest another command that I can try that would be great.
linuxyne wrote:The prevention or channelling of nested interrupts might be in the asm wrapper, I think.
Currently I'm not doing anything to avoid receiving nested interrupts. I can receive a second interrupt while the first one is still being processed. Should I try to avoid this (e.g. cli on entry to the asm wrapper and sti before iret)?
Re: Problem receiving IRQ in driver
Posted: Sat Aug 06, 2016 1:59 am
by SpyderTL
Currently I'm not doing anything to avoid receiving nested interrupts. I can receive a second interrupt while the first one is still being processed. Should I try to avoid this (e.g. cli on entry to the asm wrapper and sti before iret)?
Either handle your interrupt entirely inside your interrupt handler (which prevents nested interrupts), or make your interrupt flag a counter, and separately keep track of which interrupt you are currently handling.
Specifically, for your code, you would have to set the IRQ flag back to FALSE inside your interrupt handler, before your IRET statement.