General Protection Fault when Setting WC via MTRR

[Absobel]

I've been stuck on this problem for more than a week. I read the wiki page on MTRR and the relevant AMD64 manual section (volume 2, 7.7), i searched on almost every random forum/reddit post and it still doesn't work. I'm trying to make my framebuffer WC to make graphics faster, but I keep getting a General Protection Fault. Here is my code (it's in rust but I don't think there is anything too complicated to understand?) :

Code: Select all

fn free_mtrr_pair() -> Option<(usize, usize)> {
    let mttr_pair_reg_nb = mtrr_pair_reg_nb();
    let mut mttr_pair_reg = None;
    for i in 0..mttr_pair_reg_nb {
        let mask_reg = 0x201 + i * 2;
        let valid_bit = unsafe { readmsr(mask_reg, 11..=11) };
        if valid_bit == 0 {
            mttr_pair_reg = Some((mask_reg - 1, mask_reg));
            break;
        }
    }
    mttr_pair_reg
}

pub fn set_mtrr_wc(addr: usize, size: usize) -> Result<(), MsrError> {
    if !has_msr_support() {
        return Err(MsrError::NoMsrSupport);
    }
    if !has_wc_type_support() {
        return Err(MsrError::NoWCTypeSupport);
    }

    enable_mtrr(); // It's already enabled by default but well

    // Use the MTTR pair to set the WC memory type to the given address range
    let (base_reg, mask_reg) = free_mtrr_pair().ok_or(MsrError::NoFreeMtrPair)?;
    // size must be aligned to a boundary of a power of two and not be bigger than 52 bits
    let mask = !(size.next_power_of_two() - 1) & ((1 << 52) - 1);

    x86_64::instructions::interrupts::without_interrupts(move || unsafe {
        writemsr(base_reg, 12..=51, addr >> 12).unwrap(); // Set the base address
        writemsr(base_reg, 0..=7, WC_MEMORY_TYPE).unwrap(); // Set the memory type (WC_MEMORY_TYPE = 1)
        writemsr(mask_reg, 12..=51, mask >> 12).unwrap(); // Set the mask                   // <-- THIS IS THE LINE CAUSING THE GPF
        writemsr(mask_reg, 11..=11, 1).unwrap(); // Set the valid bit
    });

    Ok(())
}

In theory my writemsr and readmsr functions work correctly, I tested them. Maybe somehow I missed something but I don't think so?
I checked all the values and the mask and everything so many times, I calculated the values I should have in the register by hand and compared them with what I'm writing in the registers and everything seems fine. I really can't find why it doesn't work
(I swear if it's something dumb I'll remove my hippocampus with a table spoon)

Random informations that might or might not be relevant :
- I boot on UEFI using grub (the BootServices already are exited by the time my code is ran)
- I just have basic temporary paging yet, just enough huge pages to reach the framebuffer address given to me
- Here is my GitHub repo

[Octocontrabass]

Code: Select all

        writemsr(mask_reg, 12..=51, mask >> 12).unwrap(); // Set the mask                   // <-- THIS IS THE LINE CAUSING THE GPF

When a processor implements less than the architecturally-defined address size of 52 bits, the unused bits are reserved, and setting reserved bits in any MSR causes a general protection fault.

[Absobel]

Used :

Code: Select all

cat /proc/cpuinfo | grep "address"  

And got :

Code: Select all

address sizes	: 43 bits physical, 48 bits virtual

Changed it and now it works.

I feel kinda dumb but not too dumb either so it's fine. And I found a lot of bugs by searching to solve this one so yeah, worth it I guess

Now my framebuffer is quicker, instead of having one line every two seconds, I have two new lines every second. There must be some other problem elsewhere but well

[Octocontrabass]

There must be some other problem elsewhere but well

You're not reading the framebuffer, are you? That's going to be extremely slow no matter how you do it.

How close together (spatially and temporally) are your writes? The CPU has a limited number of write-combining buffers, and anything that spreads out your writes can cause the buffers to be emptied before you've really filled them.

Which CPU instructions are you using to write the framebuffer? Some instructions that are fast with WB memory are not so fast with WC memory.

What kind of display adapter are you using? Some of them are known to be pretty slow when using the MMIO framebuffer set up by the firmware.

[Absobel]

Ah, I begin to see the problem. I am indeed both reading from the framebuffer and doing read write quickly in time instead of doing everything once, this is what double buffering is for I suppose ?

I don't actually know what instructions my rust code uses but I'm essentially just interpreting the whole framebuffer as a big slice and reading/writing from/into it.

I'm using GOP provided by UEFI

[Octocontrabass]

Ah, I begin to see the problem. I am indeed both reading from the framebuffer and doing read write quickly in time instead of doing everything once, this is what double buffering is for I suppose ?

Yep, you need to double buffer to avoid expensive framebuffer reads.

I don't actually know what instructions my rust code uses but I'm essentially just interpreting the whole framebuffer as a big slice and reading/writing from/into it.

Replace the framebuffer reads with backbuffer reads, add backbuffer writes to the framebuffer writes, and you're good to go. It might not give you the best performance, but it's the simplest way to avoid copying the entire backbuffer when the changes are small, and it should still be pretty fast in most cases.

I'm using GOP provided by UEFI

That's a firmware API. I'm asking about the hardware.

[rdos]

My GOP buffer (Intel with HDMI) is behaving very strangely. Sometimes it is really fast and sometimes really slow. I'm starting to suspect that it might be slow (or fast) on BSP and the other way with AP cores. Since the stress test app is switched between cores that could explain this anomaly.

Maybe EFI sets up some MTRRs on BSP and then they are not setup on AP cores? I do setup the pages to WC, but there seems to be something else going on too. I also use double buffering. With AMD, GOP LFB is a lot faster, even on dated hardware.

[Octocontrabass]

Maybe EFI sets up some MTRRs on BSP and then they are not setup on AP cores?

That's possible. Firmware is supposed to set things up the same on every core, but sometimes that doesn't happen.

[nullplan]

Maybe EFI sets up some MTRRs on BSP and then they are not setup on AP cores?

On one hand I want to say that this would be pretty poor firmware. On the other hand, there certainly are examples of poor firmware in the world. Can you dump the MTRRs on the different cores?

I typically assume the firmware set up the MTRRs correctly. If that is false, then I actually need code to replicate the MTRRs to the APs. OK, that would be simple, as I already have code to replicate other MSRs to the APs, but I thought this was one area where it wasn't needed. I also found code to set up the MTRRs on all of the APs in Coreboot, and I thought if the Coreboot people thought of it, the professionals must have also. But maybe not.

[rdos]

I think I discovered the problem. I reprogram the MSR_PAT on BSP so entry 1 is WC, but then I fail to do the same on AP processors. I think that might explain this anomaly. I probably assumed the MSRs were global, but they apparently are not.

[Octocontrabass]

MSRs are never global, although per-socket MSRs are effectively global on single-socket systems. It's best to assume MSRs are unique for each logical thread unless the manuals specify otherwise.