OSDev.org

Posted: **Sun Jun 27, 2021 3:15 pm**

I'm trying to debug a triple fault in my virtual memory manager. However, for some odd reason qemu is flooding my interrupt log with lines similar to this, always being vector 68:

303: v=68 e=0000 i=0 cpl=0 IP=0038:000000007f11250c pc=000000007f11250c SP=0030:000000007fea35a0 env->regs[R_EAX]=0000000000000000
RAX=0000000000000000 RBX=0000000000008024 RCX=0000000000008024 RDX=0000000000000004
RSI=0000000000000004 RDI=000000007fea3708 RBP=000000007fea3708 RSP=000000007fea35a0
R8 =0000000000000000 R9 =000000007fea365f R10=000000007fec03d7 R11=0000000000000010
R12=0000000000000000 R13=000000007f1172aa R14=0000000000000001 R15=0000000000000004
RIP=000000007f11250c RFL=00000202 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
CS =0038 0000000000000000 ffffffff 00af9a00 DPL=0 CS64 [-R-]
SS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
DS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
FS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
GS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT= 000000007f9de000 00000047
IDT= 000000007f18d018 00000fff
CR0=80010033 CR2=0000000000000000 CR3=000000007fc01000 CR4=00000668
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000000 CCD=000000007fea3580 CCO=EFLAGS
EFER=0000000000000d00

This was from my most recent run of qemu. When I use -no-reboot, qemu haults the VCPU and then never actually shuts down the system; it enters this endless interrupt cycle. On one run this interrupt fired over 80000 times before I terminated it. I'm pretty sure that if I left it going long enough it would fill my disk with this alone. Its generated files that are over 200 MB and this is getting ridiculous.
I'm pretty sure vector 68 isn't bound to anything. Or, at least, nothing that should be triggering like this. Out of the 6,387 line file qemu generated this run, the *only* two interrupts that differed were these two:

check_exception old: 0xffffffff new 0xe
220: v=0e e=0002 i=0 cpl=0 IP=0008:0000000000253c97 pc=0000000000253c97 SP=0010:0000007fffffbf40 CR2=0000007fffffbf48
RAX=0000008000002038 RBX=000000007fea3828 RCX=0000000000360f10 RDX=0000000000000000
RSI=000000800000e0c8 RDI=0000008000002038 RBP=000000007fb7e014 RSP=0000007fffffbf40
R8 =0000000000000004 R9 =000000000000000a R10=000000800000e120 R11=0000000000000059
R12=000000014000f070 R13=000000007fea3758 R14=000000007fea3490 R15=000000007fea3238
RIP=0000000000253c97 RFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
CS =0008 0000000000000000 ffffffff 00af9b00 DPL=0 CS64 [-RA]
SS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
DS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
FS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
GS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT= 000000000007b000 00000017
IDT= 000000007f18d018 00000fff
CR0=80010033 CR2=0000007fffffbf48 CR3=0000000000002000 CR4=00000668
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000006028 CCD=0000007fffffbf40 CCO=SUBQ
EFER=0000000000000d00
check_exception old: 0xe new 0xe
221: v=08 e=0000 i=0 cpl=0 IP=0008:0000000000253c97 pc=0000000000253c97 SP=0010:0000007fffffbf40 env->regs[R_EAX]=0000008000002038
RAX=0000008000002038 RBX=000000007fea3828 RCX=0000000000360f10 RDX=0000000000000000
RSI=000000800000e0c8 RDI=0000008000002038 RBP=000000007fb7e014 RSP=0000007fffffbf40
R8 =0000000000000004 R9 =000000000000000a R10=000000800000e120 R11=0000000000000059
R12=000000014000f070 R13=000000007fea3758 R14=000000007fea3490 R15=000000007fea3238
RIP=0000000000253c97 RFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
CS =0008 0000000000000000 ffffffff 00af9b00 DPL=0 CS64 [-RA]
SS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
DS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
FS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
GS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT= 000000000007b000 00000017
IDT= 000000007f18d018 00000fff
CR0=80010033 CR2=000000007f18d0f8 CR3=0000000000002000 CR4=00000668
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000006028 CCD=0000007fffffbf40 CCO=SUBQ
EFER=0000000000000d00
check_exception old: 0x8 new 0xe

I dug this out of the file using ripgrep with the regexp "v=[012345789]". I don't find this a satisfactory solution though. Does anyone know why Qemu is behaving this way? I'm using QEMU emulator version 6.0.0.
My second question regards debugging. Is there any way in GDB I can generate a backtrace right before an exception occurs, but not when it does? The reason I ask is because once CR2 changes -- or when an exception in general occurs -- my backtrace becomes corrupted and I can't follow the trace back to find out where I need to go. I'm additionally suffering this weird GDB bug where after a short period of using the "step" command GDB starts behaving like step with "next". This is extremely frustrating because when this happens I get pulled through the muck of the Rust libcore and liballoc libraries, which is a bunch of debugging noise I don't need. I just want to look at my code and the dependencies it has; I don't need or care about libcore or liballoc since the likelihood of either of those libraries generating any kind of CPU exception is pretty tiny given that they're used by every rust project in existence. Have any of you suffered this problem and, if so, how did you resolve it?

Posted: **Sun Jun 27, 2021 6:49 pm**

Ethin wrote:I'm pretty sure vector 68 isn't bound to anything.

OVMF uses it for the timer IRQ, but I can't explain why OVMF would be running after you've told QEMU to halt on a triple fault.

Ethin wrote:Out of the 6,387 line file qemu generated this run, the *only* two interrupts that differed were these two:

The first is a page fault because your stack isn't present in the page tables. The second is a double fault because OVMF's IDT isn't present in the page tables.

Posted: **Sun Jun 27, 2021 9:52 pm**

I don't get why OVMF would be running then. Hell, I don't get why I'd need its IDT. I've got my own.

I'll try it with a release build of OVMF -- I had a debug one to (hopefully) make debugging my kernel simpler.

Posted: **Sun Jun 27, 2021 10:54 pm**

Ethin wrote:Hell, I don't get why I'd need its IDT. I've got my own.

Do you? The IDTR in the logs you've posted looks an awful lot like it's pointing to the OVMF IDT instead of yours.

Posted: **Mon Jun 28, 2021 12:25 am**

Octocontrabass wrote:
Ethin wrote:Hell, I don't get why I'd need its IDT. I've got my own.
Do you? The IDTR in the logs you've posted looks an awful lot like it's pointing to the OVMF IDT instead of yours.

Yes. I think so. I load my gDT and IDT before my VMM is initialized. I've tried moving it up quite far in the initialization chain. And it still triple faults (I haven't yet allocated a kernel heap). The registers look like this (from the qemu monitor):

RAX=0000008000001fd8 RBX=000000007ff156b8 RCX=0000000000359f08 RDX=0000000000000000
RSI=000000800000e068 RDI=0000008000001fd8 RBP=000000007fb7e014 RSP=0000007fffffbee0
R8 =0000000000000004 R9 =000000000000000a R10=000000800000e0c0 R11=0000000000000059
R12=000000014000f070 R13=000000007ff155e8 R14=000000007ff15320 R15=000000007ff150c8
RIP=000000000024f817 RFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 ffffffff 00cf9300
CS =0008 0000000000000000 ffffffff 00af9b00
SS =0010 0000000000000000 ffffffff 00cf9300
DS =0010 0000000000000000 ffffffff 00cf9300
FS =0030 0000000000000000 ffffffff 00cf9300
GS =0030 0000000000000000 ffffffff 00cf9300
LDT=0000 0000000000000000 0000ffff 00008200
TR =0010 00000000002fe8b8 00000067 00008900
GDT= 00000000002fe930 0000001f
IDT= 00000000002fe9c0 00000fff
CR0=80010032 CR2=000000000000bede CR3=0000000000002000 CR4=00000668
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d00
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000 XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000 XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000 XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000 XMM07=00000000000000000000000000000000
XMM08=00000000000000000000000000000000 XMM09=00000000000000000000000000000000
XMM10=00000000000000000000000000000000 XMM11=00000000000000000000000000000000
XMM12=00000000000000000000000000000000 XMM13=00000000000000000000000000000000
XMM14=00000000000000000000000000000000 XMM15=00000000000000000000000000000000

My kernel shows:

[INFO] [kernel] Loading descriptor tables and enabling interrupts
[INFO] [libk::gdt] Loading GDT
[DEBUG] [libk::gdt] Loading GDT at addr 0x2fe930: GlobalDescriptorTable { table: [0, 49428545226735615, 150838860841063, 0, 0, 0, 0, 0], next_free: 4 }
[INFO] [libk::gdt] Setting CS
[DEBUG] [libk::gdt] CS at addr 0x2fe978: SegmentSelector { index: 1, rpl: Ring0 }
[INFO] [libk::gdt] Loading TSS
[DEBUG] [libk::gdt] TSS at addr 0x2fe8b0, TSS selector at 0x2fe97a: TSS = Lazy { cell: Once { data: TaskStateSegment { reserved_1: 0, privilege_stack_table: [VirtAddr(0x0), VirtAddr(0x0), VirtAddr(0x0)], reserved_2: 0, interrupt_stack_table: [VirtAddr(0x301ea0), VirtAddr(0x311ea0), VirtAddr(0x359ea0), VirtAddr(0x0), VirtAddr(0x0), VirtAddr(0x0), VirtAddr(0x0)], reserved_3: 0, reserved_4: 0, iomap_base: 0 }}, init: ".." }, TSS selector = SegmentSelector { index: 2, rpl: Ring0 }
[DEBUG] [libk::gdt] TSS at addr 0x2fe8b0 with TSS selecter at addr 0x2fe97a loaded
[DEBUG] [libk::gdt] Changed TR; old: 0, RPL_0 | TI_GDT, new: 10, 0x10
[INFO] [libk::interrupts] Loading IDT
[DEBUG] [libk::interrupts] IDT loaded. IDT at 0x2fe9c0: [snipped - IDT is huge]
[DEBUG] [libk::interrupts] Changed IDT: old: 7F274018 with limit FFF, new: 2FE9C0 with limit FFF

I don't use an STI/CLI instruction -- I'd assume that interrupts are (already) enabled... Does UEFI disable those after exiting boot services?

Posted: **Mon Jun 28, 2021 9:13 am**

Ethin wrote:I don't use an STI/CLI instruction -- I'd assume that interrupts are (already) enabled... Does UEFI disable those after exiting boot services?

The UEFI spec isn't clear on whether interrupts will be enabled after exiting boot services. If they are enabled, wouldn't you want to disable them while you're setting up the GDT and IDT and interrupt controllers?

Posted: **Mon Jun 28, 2021 11:28 am**

Yes, but its good to have them if I forget to map something. E.g.: I need to map the APIC memory area to use it because it isn't by default in my pagetables.

Posted: **Mon Jun 28, 2021 11:31 am**

Ethin wrote:Yes, but its good to have them if I forget to map something. E.g.: I need to map the APIC memory area to use it because it isn't by default in my pagetables.

What do you mean by that? Exceptions are still able to occur if EFLAGS.IF is 0.

Posted: **Mon Jun 28, 2021 2:31 pm**

nexos wrote:
Ethin wrote:Yes, but its good to have them if I forget to map something. E.g.: I need to map the APIC memory area to use it because it isn't by default in my pagetables.
What do you mean by that? Exceptions are still able to occur if EFLAGS.IF is 0.

Oh, I forgot about that. Whoops?

OSDev.org

Qemu: interrupt vector 68 flood? And a question on GDB

Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB

Re: Qemu: interrupt vector 68 flood? And a question on GDB