fxsave/fxrstor causes a reboot

[kemosparc]

Hi,

I am trying to enable sse so I can use the xmm registers in fast memory copy.

I have a problem that whenever I call fxsave on qemu/kvm the machine reboots.

I tried it on bochs and it generates a lot of errors with the same message

Code: Select all

08153109086e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109145e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109204e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109263e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109322e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109381e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109440e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109499e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109558e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109617e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109676e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109735e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109794e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109853e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109912e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153109971e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153110030e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153110089e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
08153110148e[CPU0  ] write_linear_xmmword_aligned(): #GP misaligned access
(0).[8153110175] [0x00000001c3c3] 0008:000000000001c3c3 (unk. ctxt): call rax                  ; ffd0
08153110175e[CPU0  ] exception(): 3rd (14) exception with no resolution, shutdown status is 00h, resetting
Next at t=8153110176
(0) [0x0000fffffff0] f000:fff0 (unk. ctxt): jmpf 0xf000:e05b          ; ea5be000f0

I checked the address using objdump and it is the fxsave.

I have initialized sse following OSdev wiki as follows:

Code: Select all

mov eax, cr0
and ax, 0xFFFB	
or ax, 0x2
mov cr0, eax
mov eax, cr4
or ax, 3 << 9
mov cr4, eax

I created a 512 bytes aligned buffer as my fxregion and I have printed its address to find it 0x2f68000, so it is aligned.

I call the following inside my interrupt handler:

Code: Select all

extern "C" void idt_handler(InterruptContext * p_interruptContext)
{
    asm volatile(" fxsave %0; "::"m"((uint64_t)fxsave_region));
    .
    .
    .
    .
    .
    .
    .
    .
    .
    asm volatile(" fxrstor %0; "::"m"((uint64_t)fxsave_region));
}

Actually, what I want to do at the end is to save the xmm[0-7] registers across interrupts to be able to use them in different processes.
If any body can point out what might be the problem, I highly appreciate it.

Thanks.
Karim.

[Octocontrabass]

I have a problem that whenever I call fxsave on qemu/kvm the machine reboots.

Your log shows a page fault that turns into a triple fault, so it's safe to say that's why it reboots.

I checked the address using objdump and it is the fxsave.

I created a 512 bytes aligned buffer as my fxregion and I have printed its address to find it 0x2f68000, so it is aligned.

Now set a breakpoint on that fxsave and make sure it's actually trying to use your buffer.

Code: Select all

asm volatile(" fxsave %0; "::"m"((uint64_t)fxsave_region));

I'm not an expert in GCC's inline assembly, but this looks wrong to me. You should compare with a reliable source (e.g. Linux) and try to figure out why it's different from yours.

[gerryg400]

Code: Select all

asm volatile(" fxsave %0; "::"m"((uint64_t)fxsave_region));

This may or may not be correct depending on what fxsave_region actually is. You may need to indirect off fxsave_region with * fxsave_region like this.

Code: Select all

asm volatile(" fxsave %0; "::"m"((uint64_t)(*fxsave_region)));

As I understand it the "m" takes the address of its operand. So it the operand is already a pointer then usually you need a *.

[hgoel]

What are the odds! I just encountered this exact same issue. It's a simple fix really. You want the instruction to use the address stored at that location, so instead you make the inline assembly:

Code: Select all

__asm__ volatile("fxsaveq (%0)" :: "r"(target) : );

(remove the 'q' if not targeting 64-bit)

[Boris]

Hi,
fxsave/store needs a 16 byte aligned address.
Most of spacial instructions needs 16 bytes alignment, I'd suggest you to be sure your kmalloc yields 16 bytes aligned addresses.

Secondly, fxsave/story is a heavy instruction. I don't use them in my interrupts handlers . I do use them in tasks switches.

Tip: if you need to align your stack, you can do
and 0xFFFFFFF0 , esp

[kemosparc]

Hi all,

Thanks a lot for all your replies.

The memory region I am using is 16-bit aligned, as I said earlier I print it and it has the least right significant digits as zeros. e.g. "0xXXXXX000".


I tried the following as gerryg400 recommended

Code: Select all

asm volatile(" fxsave %0; "::"m"((uint64_t)(*fxsave_region)));

but it gave the following compilation error:

Code: Select all

sr.cpp:25:66: error: memory input 0 is not directly addressable
     asm volatile(" fxsave %0; "::"m"((uint64_t)(*fxsave_region)));    

The only one that works is the one provided by hgoel0974.

Code: Select all

__asm__ volatile("fxsaveq (%0)" :: "r"(target) : );

So I want to verify with you that what I am doing is correct.

I am in Long mode, and I allocate a 512 4k-aligned uint8_t * region for every task I have, and then in side the interrupt handler upon switching between tasks I do the fxsave and fxstor to preserve the xmm registers between task. so my code looks like that now after adopting hgoel0974 recommendation:

Code: Select all

extern "C" void idt_handler(InterruptContext * p_interruptContext)
{
    // Save current task sse
    uint64_t fxsave_region =  (uint64_t) kernel.getTaskManager->getCurrentTask()->getfxregion();
    asm volatile("fxsaveq (%0)" :: "r"(fxsave_region) : );                
    asm volatile("finit;");
    .
    .
    .
    .
    .
    .
    // Restore sseafter switching to the new task.
    fxsave_region =  (uint64_t) kernel.getTaskManager->getCurrentTask()->getfxregion();
    asm volatile("fxrstorq (%0)" :: "r"(fxsave_region) : );                

hgoel0974, can you please confirm that fxsaveq/fxrstorq achieves my target; I googled fxsaveq (with q at the end) and I could not find any documentation. In fact few examples I could find on fxsave/fxstor.

Another question that I have is that is using fxsave/fxrstor instruction equivalent to pushing xmm on the stack within my interrupt handler? using something like the following macros:

Code: Select all

%macro push_xmm 1
    sub rsp, 16
    movdqu [rsp], xmm%1
%endmacro
%macro pop_xmm 1
    movdqu  xmm%1, [rsp]
    add     rsp, 16
%endmacro

At the end of the day my target is to use the xmm% registers to build a faster version for memcpy. I tried that in userland on Linux and it is much faster than the standard memcpy provided by linux. Am I doing the right thing ??

Thanks a lot for your reply.
Karim.

[iansjack]

Have you examined the generated code? Have you single-stepped through it? One of these should make it obvious what the problem is.

[Octocontrabass]

You should at least try to tell GCC when your inline assembly modifies one of its operands.

Code: Select all

asm volatile("fxsaveq %0\n\t" : "=m" (*fxsave_region) );

For reasons that aren't entirely clear to me, you need to use a struct for your fxsave_region variable if you're using Intel syntax. I'm not sure how this affects GCC's optimizer. (You may want to use a struct anyway, so you can modify the saved FPU state from within your interrupt handler.)

Code: Select all

typedef struct fpustate_t
{
    uint8_t data[512];
}fpustate_t;

fpustate_t fxsave_region = ...

I googled fxsaveq (with q at the end) and I could not find any documentation.

Try fxsave64, it's another name for the same instruction.

[Boris]

Hi,
i use
asm volatile("fxrstor %0" ::"m"(mFxSave));
Where mFxsave is :
char mFxSave[512];

I have no problems whatsoever.
Be sure to zero that field when you first create it .
Because fxsave is not zeroing reserved fields, which will make you having trouble with fxrstor

[sebihepp]

I am using this in my code:

Code: Select all

asm volatile ("fxsave (%%eax)" : : "a" (pFPUState) : );

And to load:

Code: Select all

asm volatile ("fxrstor (%%eax)" : : "a" (pFPUState) : );

But these instructions only work if SSE is available and enabled. Did you initialize the FPU and set the CR0 and CR4 registers?

Edit:
pFPUState is a pointer to a 512(SSE)+36(FPU) Bytes memory location.