Could you check if my interrupt wrapper makes sense?

[Biogenique]

Hello,

I'm starting to work on interrupts for a 64bits OS written in Nim and I'm unsure if my interrupt caller is correct or not, as I get weird values from the interrupt stack frame.
Right now I have an interrupt table setup and the interrupt gets called successfully, although I get infinite interrupts I'm not too worried about that.

What I'd like to know, before going further, is if my interrupt calling logic makes sense because I'm bad at assembly.

The full code is here:
https://pastebin.com/ie4NxYiZ

The part I'm unsure about is the logic of my assembly:
```
push %rax
push %rcx
push %rdx
push %r8
push %r9
push %r10
push %r11

enter $16, $0
mov %rsp, %rax
add $80, %rax
push [%rax]
add $32, %rax
push %rax
call errorHandler
leave

pop %r11
pop %r10
pop %r9
pop %r8
pop %rdx
pop %rcx
pop %rax

iretq
```

The logic is the following:
I push all the volatile registers so they don't get lost if the callee uses them.

With enter, I assign 16 bytes to the stack frame of the callee, because I intend to pass it a pointer to the start of the interrupt information, and an error code (uint64).
Then I get the address that's 80 bytes above RSP. Because I subtracted 16 bytes to RSP with enter, and 7*8 bytes by pushing the registers, and the error code should be 8 bytes above that. So 16+56+8=80
I push the value at that address so it becomes the second argument passed to my caller.
Then I go up further 32 bytes, because the interrupt information is 40 bytes total in amd64.
And I push that address as a pointer to the interrupt information struct.

Then I call my error handler, leave it, pop all the volatile registers and return from my interrupt.


Does that make sense?

[Octocontrabass]

I push all the volatile registers so they don't get lost if the callee uses them.

Assuming you're using the System V psABI, you missed RSI and RDI. Also, you typically want to push all the GPRs so your interrupt handler has access to the complete state of the interrupted program.

With enter, I assign 16 bytes to the stack frame of the callee, because I intend to pass it a pointer to the start of the interrupt information, and an error code (uint64).

You shouldn't use ENTER unless you're absolutely desperate for every last byte. It's much slower than the equivalent PUSH/MOV/SUB instructions. Stack frames are optional anyway.

Then I get the address that's 80 bytes above RSP. Because I subtracted 16 bytes to RSP with enter, and 7*8 bytes by pushing the registers, and the error code should be 8 bytes above that. So 16+56+8=80
I push the value at that address so it becomes the second argument passed to my caller.
Then I go up further 32 bytes, because the interrupt information is 40 bytes total in amd64.
And I push that address as a pointer to the interrupt information struct.

Function parameters are passed in registers, not on the stack. You don't need to pass the error code separately, it should already be part of your interrupt information struct. For interrupts that don't include an error code, you can push a dummy error code to keep the stack layout consistent.

Then I call my error handler,

The direction flag must be clear before calling the function. I didn't check whether your stack is correctly aligned according to the ABI.

pop all the volatile registers and return from my interrupt.

You need to remove the error code from the stack before you return.

Does that make sense?

You're missing stubs to push the interrupt vector on the stack so your interrupt information struct can include it. You don't need separate wrappers for interrupts with and without error codes if you use those stubs to fix the difference.

[nullplan]

Assuming you're using the System V psABI, you missed RSI and RDI. Also, you typically want to push all the GPRs so your interrupt handler has access to the complete state of the interrupted program.

Seconded. If even mister efficiency himself, Linus Torvalds, is pushing all registers, then doing that is probably the right thing to do.

There is also the matter of the stack alignment. Some time ago, another user here showed clang badly miscompile a kernel with -O0 to misalign the stack pointer. Basically, this means you cannot know the stack pointer alignment on entry, and must verify it yourself. I would write something along these lines:

Code: Select all

pushq %r15
pushq %r14
[...]
pushq %rax
movq %rsp, %rsi
andq $-16, %rsp
cmpq %rsp, %rsi
jnz 1f
movq %rsp, %rdi
movl $21*8, %ecx
rep movsb # I am not keen on finding out if overlapping movsq works
1:

After that, the stack is definitely aligned. You now only need to set RDI to RSP and call the error handler function. The length given to ECX is enough to cover all 15 saved GPRs (RSP is not saved there, right?) plus error code, plus the five words for the interrupt stack frame.

The direction flag must be clear before calling the function.

Oh yeah, that too.

[Octocontrabass]

Basically, this means you cannot know the stack pointer alignment on entry, and must verify it yourself.

In protected mode, yes, but this is long mode. In long mode, the CPU aligns the stack pointer before it pushes the values for the interrupt stack frame.

I would write something along these lines:

I wouldn't. Modern x86 CPUs have caches that should absorb most of the misalignment penalties, and REP MOVSB will be extremely slow with the source and destination so close together.

Assuming protected mode instead of long mode, I would tell the compiler to relax its alignment requirements, then write something along these lines:

Code: Select all

pushal
movl %esp, %ebp
andl $~0x3, %esp
pushl %ebp
call interrupt_handler_function_name
movl %ebp, %esp
popal

For GCC, that would mean packing the struct and compiling with "-mpreferred-stack-boundary=2". Similar code will work for stricter stack alignment, but the stack must be aligned after the parameters are pushed, so you'll need to do more than just changing the "andl $~0x3, %esp" line. (Actually, I'd use "-mregparm=3" and pass parameters in registers...)

I am not keen on finding out if overlapping movsq works

It does. It might even be faster than REP MOVSB.

[nullplan]

In protected mode, yes, but this is long mode. In long mode, the CPU aligns the stack pointer before it pushes the values for the interrupt stack frame.

Huh. I never knew that. I just looked it up and you're right, RSP is aligned to a 16-byte boundary prior to the first push. So as long as the number of words pushed is even, the stack is correctly aligned for C. There are five words in the interrupt frame, one more for the error code, and then fifteen registers to push. So push one more word, e.g. the vector number and it'll all work out.

I wouldn't. Modern x86 CPUs have caches that should absorb most of the misalignment penalties, and REP MOVSB will be extremely slow with the source and destination so close together.

ABI being what it is, however, I know the stack must be 16-bytes aligned before the call instruction.

That 16-byte alignment before the call also attends to 32-bit implementations, and has for decades now. Yeah, it is likely nothing bad will happen but undefined behavior is undefined.

However, I do not accept the "extreme" slowness of REP MOVSB as an argument. Because it is a 168 byte move with predictable addresses. I'd wager that there are very few situations in which the overhead of this instruction in this place would ever matter, and none that pertain to hobby operating systems.

[Octocontrabass]

ABI being what it is, however, I know the stack must be 16-bytes aligned before the call instruction.

Which is why I still align the stack before the function call but leave the data already on the stack where it is.

I'd wager that there are very few situations in which the overhead of this instruction in this place would ever matter, and none that pertain to hobby operating systems.

That's fair.

One other thing I hadn't considered is that, in 32-bit mode, you have to put the data back where you found it before the IRET instruction.

[nullplan]

One other thing I hadn't considered is that, in 32-bit mode, you have to put the data back where you found it before the IRET instruction.

Ooh, right, because ESP is only pushed if CPL changes. I suppose in this case your idea is probably roughly the right thing to do.

[iProgramInCpp]

Stack frames are optional anyway.

Sure, as long as you never want to get a stack trace from your OS, ever.

While you can get away without creating a stack frame in the interrupt dispatcher, you may want to anyway, to get a proper trace. If you don't, the instruction where the exception was triggered will be skipped when navigating the stack trace.

[Octocontrabass]

Sure, as long as you never want to get a stack trace from your OS, ever.

You can still get a stack trace with help from the ".eh_frame" section. It's not as easy as using a frame pointer, but OS development isn't always about doing things the easy way...

While you can get away without creating a stack frame in the interrupt dispatcher, you may want to anyway, to get a proper trace. If you don't, the instruction where the exception was triggered will be skipped when navigating the stack trace.

You don't need a stack frame for that either. Or at least, you don't need a frame pointer stored in RBP - your exception handler is going to have a hard time handling the exception with no way to access to the stack frame.

[iProgramInCpp]

You don't need a stack frame for that either.

I forget that you don't strictly need to see what's above the actual exception interrupt. If you do, I guess you gotta simulate that other method of adding a stack frame.

[iProgramInCpp]

OS development isn't always about doing things the easy way...

Sure, but if you intentionally complicate yourself pointlessly, you are going to have a worse time for no reason.