[SOLVED] How to handle C++ function in IRQ handler

[wichtounet]

Hi,

My kernel is made of a micro kernel in assembly that boots up to the long mode, installs GDT, IDT, IRQs and then calls the C++ kernel. Both the assembly and C++ parts are running in Ring 0, there is no user code and I'm running in single processor mode.

In my kernel I'm using several IRQs (keyboard, PIT, IDE controllers). What I've done is that I have one assembly routine for each possible IRQs and then an array of handlers that are addresses of functions or 0 if the C++ kernel did not request something at this point.

Here the macro used to generate the various IRQ routines:

Code: Select all

%macro CREATE_IRQ 1
_irq%1:
    ; Disable interruptions to avoid being interrupted
    cli

    mov rax, [irq_handlers + 8 *%1]

    ; If there are no handler, just send EOI
    test rax, rax
    je .eoi

    ; Call the handler
    call rax

    .eoi:

    mov rax, %1 ; IRQ number
    cmp rax, 8
    jl .master

    ; If IRQ 8 -> 15, send EOI to PIC2
    mov al, 0x20
    out 0xA0, al

    .master:

    ; Send EOI to PIC1
    mov al, 0x20
    out 0x20, al

    iretq
%endmacro

With that, I had some issues with registers being trashed by this routine and this was causing me page faults.

So I decided, to save rax after cli and restore it before iretq and save registers (rbx,rcx,rdx,rsi,rdi,r8,r9) before the call and restore them after the call. However, when I do that, the iretq instruction throws a General Protection Fault.

If I save less registers, some things are improved (no page faults), but it seems that my ATA interrupt is never called as I'm waiting forever for an ATA read.

So, my question is, what is a safe way to call a C++ function from an interrupt handler ? Or is there something specific that I'm doing wrong here ?

Thank you

[jnc100]

You need to save all registers that are changed in the handler at the start of the interrupt handler and restore them at the end (prior to iret) in the reverse order. In particular, here you are changing rax, and so have to preserve it. If you are using the AMD64 ABI (which I am assuming you are) then you should also assume that your C++ code will trash rax, rcx, rdx, rdi, rsi, r8-11 (and possibly rsp - although this isn't specifically mentioned in the ABI docs, and only a really broken C++ implementation would change it, but there are no guarantees) and therefore save these. Finally, you need to be sure that you are using a cross compiler (I see from your repository Makefile that you just call 'g++') and that you disable the red zone when compiling in 64-bit mode as this is notorious for breaking the stack with interrupt handlers.

Regards,
John.

[wichtounet]

Hi John,

You need to save all registers that are changed in the handler at the start of the interrupt handler and restore them at the end (prior to iret) in the reverse order. In particular, here you are changing rax, and so have to preserve it.

Ok, that makes sense.

If you are using the AMD64 ABI (which I am assuming you are) then you should also assume that your C++ code will trash rax, rcx, rdx, rdi, rsi, r8-11 (and possibly rsp - although this isn't specifically mentioned in the ABI docs, and only a really broken C++ implementation would change it, but there are no guarantees) and therefore save these.

I will read the ABI in more details to find out what registers I have to save. I took the list from the internet but apparently, I had to wrong ABI.

Finally, you need to be sure that you are using a cross compiler (I see from your repository Makefile that you just call 'g++')

Ok, I will try with a cross-compiler. I thought that just using some parameters would make it, but you're right, I should use a cross compiler.

and that you disable the red zone when compiling in 64-bit mode as this is notorious for breaking the stack with interrupt handlers.

Gosh, I never heard of this before. I will give it a look.

Thanks a lot for all that. I will try all of this and I'll come back later. I've also seen that there are other problems in the way some interrupts are handled, so that will solve other problems too.

[jnc100]

Also, I forgot to mention that you should probably also disable mmx, sse etc when compiling kernel code - see http://wiki.osdev.org/Creating_a_64-bit ... #Compiling and http://wiki.osdev.org/Bare_Bones#Writin ... in_C.2B.2B and try amalgamating the various compiler options.

Regards,
John.

[wichtounet]

Ok, so now I've built a cross compiler and my compile command looks like that:

x86_64-elf-g++ -masm=intel -Iinclude/ -O1 -std=c++11 -nostdlib -fno-exceptions -fno-rtti -ffreestanding -lgcc -mno-red-zone -mno-mmx -mno-sse -mno-sse2 -mno-sse3 -mno-3dnow -c src/shell.cpp -o shell.o

Is that better ?

And I have one more question, is it really necessary to use --without-headers when building the cross compiler ? Does that mean that I cannot use a single thing of the C++ STL ? (I'm not complaining, just want to be sure).

Thank you

P.S. I haven't had the time to do any tests since there are other problems that have to be corrected in my code as well.

[sortie]

The --without-headers headers option is deprecated, but really means --without-sysroot. [Edit: This appears to be wrong, please consult the documentation rather than me.] Normally you provide --with-sysroot to your cross-compiler so it knows where to find headers, libraries, etc. However, as you are making an entirely new operating system, you do not have a user-space yet and don't have anything to put in the system root. As you progress and begin developing a user-space, you should invest time in creating an OS-Specific Toolchain, which means you modify binutils and gcc and teach them what exactly your OS is. At the same time, you introduce a system root in which your store your libc if you have one, port one, or write on yourself.

However, you cannot use the C++ standard library (libstdc++) out of the box as it depends on the libc (you do you not have yet). I don't know about integrating it into kernels, though, you'll have to ask somebody else about that. The good news is that you only need libsupc++ in your kernel to enable a bunch of C++ runtime features (like exceptions and other things). I never quite looked into this, yet, but you'll probably need to build it like you do with libgcc during the cross-compiler build.

I see you are passing -lgcc when compiling individual .o files. This doesn't make sense as the -l options are only used at link time, so you should remove them from any -c command line. There is value in simplifying your compile commands.

[wichtounet]

The --without-headers headers option is deprecated, but really means --without-sysroot. Normally you provide --with-sysroot to your cross-compiler so it knows where to find headers, libraries, etc. However, as you are making an entirely new operating system, you do not have a user-space yet and don't have anything to put in the system root. As you progress and begin developing a user-space, you should invest time in creating an OS-Specific Toolchain, which means you modify binutils and gcc and teach them what exactly your OS is. At the same time, you introduce a system root in which your store your libc if you have one, port one, or write on yourself.

However, you cannot use the C++ standard library (libstdc++) out of the box as it depends on the libc (you do you not have yet). I don't know about integrating it into kernels, though, you'll have to ask somebody else about that. The good news is that you only need libsupc++ in your kernel to enable a bunch of C++ runtime features (like exceptions and other things). I never quite looked into this, yet, but you'll probably need to build it like you do with libgcc during the cross-compiler build.

Thank you for this very clear explanation

I will try to add these features in the future. As an avid user of C++, I was probably too much enthusiastic about using all of its features, especially the STL.

I see you are passing -lgcc when compiling individual .o files. This doesn't make sense as the -l options are only used at link time, so you should remove them from any -c command line. There is value in simplifying your compile commands.

You're absolutely right, that is a stupid mistake of me

[sortie]

I am looking a bit into libsupc++, btw. I noticed there was a wiki article and I don't trust its accuracy as it appears to be severely outdated, so I'll verify its information and possibly delete stuff from it or merge it into the GCC Cross-Compiler tutorial. I appear I was wrong, --without-sysroot doesn't seem to work, but --without-headers do. I need to look more closely into this.

[sortie]

After examining libsupc++ and attempting to build a freestanding libsupc++ for i586-elf using gcc 4.8.2, I must unfortunately announce that I have failed. It would appear that i586-elf is indeed too freestanding for libsupc++, or rather the libstdc++-v3 build system. The configure script for libstdc++-v3 scans through headers such as <stdio.h> and searches for dlopen (and other things) in various system libraries, however, the configure script correctly recognizes it should not do such tests and simply decides to error out upon encountering this paradox. I was unable to work-around these issues, indeed the approach suggested by the older libstdc++ article appears to have bitrotted away. The --disable-hosted-libstdcxx configure option that was supposed to save the day only appears to work on actually hosted targets. I successfully used this --disable-hosted-libstdcxx option when targeting my native x86_64-linux-gnu platform and it actually only installed a libsupc++.a into the compiler prefix and only a small subset of freestanding C++ headers - so there's hope. After examining the libsupc++.a for x86_64-linux-gnu using nm(1), I saw a few references to libc (strcmp, malloc, free), a bunch of references to libgcc and a few problematic pthread_ calls. As a work-around you can probably use a i386 Linux libsupc++.a library in your kernel, but the pthread symbol references suggest you might be pulling in some bad Linux stuff.

Anyways, my recommendation is that you don't use libsupc++ and the provided C++ headers in a kernel built with i586-elf. You can probably get it working if you add a working libc into your system root, and you use an OS-Specific Toolchain that has a libstdc++-v3 port, and if you link your libc (a special kernel version) into your kernel. The real solution here is to get the GCC developers (or better yet - do it yourself) to fix --disable-hosted-libstdcxx not working properly on *-elf platforms. I might just look into this if I get some time, or perhaps just send a bug report to the GCC team.

[wichtounet]

I've been able to compile my kernel using the cross compiler and after having saved/restored the correct registers from the ABI (according to AMD ABI 0.99.6, there is no need to save rsp and rbp), it works really well

Thank a lot for you help guys.

Anyways, my recommendation is that you don't use libsupc++ and the provided C++ headers in a kernel built with i586-elf. You can probably get it working if you add a working libc into your system root, and you use an OS-Specific Toolchain that has a libstdc++-v3 port, and if you link your libc (a special kernel version) into your kernel. The real solution here is to get the GCC developers (or better yet - do it yourself) to fix --disable-hosted-libstdcxx not working properly on *-elf platforms. I might just look into this if I get some time, or perhaps just send a bug report to the GCC team.

I will follow your recommendation