Page Fault and General Protection Fault

[22OsC]

Can somebody explain why I get Page Fault and/or General Protection Fault after I wrote like 1k lines of code? This is happening only when I map the memory. The linker and everything are the same as here https://github.com/stivale/stivale2-barebones. Is about memory? linking? the kernel is not in a canonical address? I have a local .git and if I remove those lines everything is normal. And those functions are called way before the memory and heap initialization.

[Octocontrabass]

What kind of debugging have you done so far? I see you're using QEMU, so you can attach a debugger like GDB or use one of QEMU's logging options like "-d int" to get more information about the exceptions.

[22OsC]

What kind of debugging have you done so far? I see you're using QEMU, so you can attach a debugger like GDB or use one of QEMU's logging options like "-d int" to get more information about the exceptions.

the "-d int" is not working at all on my Linux system but with VScode I set a breakpoint when the mapping is happening and I used "step into" to check the next thing that is going to be executed randomly gets redirected to another random function. not only that but if I change the main.c function some of the code inside of it is not executed. And this is happening only if I add more code to the kernel. I added "-O0" to GGC because I want to debug it ofc if it's useful. Oh, I almost forgot, on VMware, I get General Protection Fault every time when it's mapping, but in QEMU randomly throws General Protection Fault and Page Fault and on VirtualBox only Page Fault.

[Octocontrabass]

the "-d int" is not working

Try disabling hardware acceleration. I think "-M q35,accel=tcg" will do it.

Oh, I almost forgot, on VMware, I get General Protection Fault every time when it's mapping, but in QEMU randomly throws General Protection Fault and Page Fault and on VirtualBox only Page Fault.

It sounds like your page tables are messed up and the resulting exception varies depending on when the TLB attempts to read your broken page tables. The QEMU monitor has "info mem" and "info tlb" commands that might help you see the problem. (Always try both; they don't report the same information.)

[22OsC]

It sounds like your page tables are messed up and the resulting exception varies depending on when the TLB attempts to read your broken page tables.

Maybe it's because I read the cr3, map the addresses and then I write the same cr3 with mapped memory instead of creating a blank cr3 and then mapping it?
I remember that I tried this in the past and it was a direct Triple Fault, I struggled to fix that for almost a week and I managed to workaround the "solution" I mentioned earlier.

[Octocontrabass]

Maybe it's because I read the cr3, map the addresses and then I write the same cr3 with mapped memory instead of creating a blank cr3 and then mapping it?

There's no such thing as a "blank CR3". Did you mean a blank PML4?

Modifying the page tables while they're in use is fine, as long as you take care to invalidate the TLB properly, but it's not a good idea with the bootloader page tables since you have no idea how the bootloader decided to set those up.

I remember that I tried this in the past and it was a direct Triple Fault, I struggled to fix that for almost a week and I managed to workaround the "solution" I mentioned earlier.

It sounds like the page tables you were creating weren't set up correctly. QEMU can tell you more about the faults leading up to the triple fault with "-d int" (as long as you turn off hardware acceleration).

[22OsC]

Maybe it's because I read the cr3, map the addresses and then I write the same cr3 with mapped memory instead of creating a blank cr3 and then mapping it?

There's no such thing as a "blank CR3". Did you mean a blank PML4?

Modifying the page tables while they're in use is fine, as long as you take care to invalidate the TLB properly, but it's not a good idea with the bootloader page tables since you have no idea how the bootloader decided to set those up.

I remember that I tried this in the past and it was a direct Triple Fault, I struggled to fix that for almost a week and I managed to workaround the "solution" I mentioned earlier.

It sounds like the page tables you were creating weren't set up correctly. QEMU can tell you more about the faults leading up to the triple fault with "-d int" (as long as you turn off hardware acceleration).

Well... I disabled the hardware acceleration and now I see that I can debug everything. Now I tried to create a blank PML4 and map the memory. It's a problem as I said earlier; the Triple Fault, good thing now that i can see the memory mapping and other stuff but still.. i don't get it why it's so broken. I will pase here the Triple Fault, mem info & mem tlb

Last stuff before the Triple Fault:

Code: Select all

  1517: v=68 e=0000 i=0 cpl=0 IP=0038:00000000be678413 pc=00000000be678413 SP=0030:00000000bfe6b480 env->regs[R_EAX]=00000000b9c40000
RAX=00000000b9c40000 RBX=0000000000000003 RCX=00000001b1000000 RDX=0000000000000001
RSI=0000000000000006 RDI=0000000000000188 RBP=00000000bfe6b4f0 RSP=00000000bfe6b480
R8 =00000001b1000083 R9 =0000000000000001 R10=00000000000a0000 R11=000000000000006c
R12=00000000b9c57000 R13=0000800000000000 R14=0000000000000004 R15=ffff8001b1200000
RIP=00000000be678413 RFL=00000206 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0038 0000000000000000 ffffffff 00af9a00 DPL=0 CS64 [-R-]
SS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT=     00000000bfbeea98 00000047
IDT=     00000000bf2d0018 00000fff
CR0=80010033 CR2=0000000000000000 CR3=00000000bfc01000 CR4=00000668
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000004 CCD=00000000bfe6b448 CCO=EFLAGS  
EFER=0000000000000d00
check_exception old: 0xffffffff new 0xe
  1518: v=0e e=0010 i=0 cpl=0 IP=0008:ffffffff8002d9a5 pc=ffffffff8002d9a5 SP=0010:ffffffff8029ff38 CR2=ffffffff8002d9a5
RAX=0000000000100000 RBX=0000000000000000 RCX=000f8000c0384003 RDX=ffff8000c0000000
RSI=0000000000000001 RDI=0000000000100000 RBP=ffffffff8029ff40 RSP=ffffffff8029ff38
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000030 R11=0000000000000010
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff8002d9a5 RFL=00000092 [--S-A--] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 00000000 00209a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 00000000 00009200 DPL=0 DS   [-W-]
DS =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
FS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
GS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT=     ffffffff80043940 00000037
IDT=     ffffffff80055980 00000fff
CR0=80000013 CR2=ffffffff8002d9a5 CR3=0000000000100000 CR4=00000620
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000008 CCD=ffffffff8029ff38 CCO=SUBQ    
EFER=0000000000000d00
check_exception old: 0xe new 0xe
  1519: v=08 e=0000 i=0 cpl=0 IP=0008:ffffffff8002d9a5 pc=ffffffff8002d9a5 SP=0010:ffffffff8029ff38 env->regs[R_EAX]=0000000000100000
RAX=0000000000100000 RBX=0000000000000000 RCX=000f8000c0384003 RDX=ffff8000c0000000
RSI=0000000000000001 RDI=0000000000100000 RBP=ffffffff8029ff40 RSP=ffffffff8029ff38
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000030 R11=0000000000000010
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff8002d9a5 RFL=00000092 [--S-A--] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 00000000 00209a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 00000000 00009200 DPL=0 DS   [-W-]
DS =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
FS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
GS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT=     ffffffff80043940 00000037
IDT=     ffffffff80055980 00000fff
CR0=80000013 CR2=ffffffff80055a60 CR3=0000000000100000 CR4=00000620
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000008 CCD=ffffffff8029ff38 CCO=SUBQ    
EFER=0000000000000d00
check_exception old: 0x8 new 0xe

info mem & info tlb:




And in function writecr3 vscode throws this error

Exception has occurred.
Quit

[Octocontrabass]

Code: Select all

check_exception old: 0xffffffff new 0xe
  1518: v=0e e=0010 i=0 cpl=0 IP=0008:ffffffff8002d9a5 pc=ffffffff8002d9a5 SP=0010:ffffffff8029ff38 CR2=ffffffff8002d9a5

The first exception is a page fault. The error code 0x0010 indicates an instruction fetch from a page that is not present. CR2 indicates the address of the instruction.

info mem & info tlb:

QEMU says your page tables don't map anything to that address! How are you setting up these page tables, anyway?

[22OsC]

CR2 indicates the address of the instruction.

i found the instruction:

Code: Select all

ffffffff8002d992 <_ZL8writecr3m>:
ffffffff8002d992:	55                   	push   %rbp
ffffffff8002d993:	48 89 e5             	mov    %rsp,%rbp
ffffffff8002d996:	48 83 ec 08          	sub    $0x8,%rsp
ffffffff8002d99a:	48 89 7d f8          	mov    %rdi,-0x8(%rbp)
ffffffff8002d99e:	48 8b 45 f8          	mov    -0x8(%rbp),%rax
ffffffff8002d9a2:	0f 22 d8             	mov    %rax,%cr3
ffffffff8002d9a5:	90                   	nop       << here is where CR2 indicate the address
ffffffff8002d9a6:	c9                   	leaveq 
ffffffff8002d9a7:	c3                   	retq   

QEMU says your page tables don't map anything to that address! How are you setting up these page tables, anyway?

Well... I used https://github.com/Absurdponcho/PonchoO ... Convention as inspiration and mapping is done like this:

Code: Select all

#define NORMAL_VMA_OFFSET 0xFFFF800000000000
#define P2V(addr) ((uint64_t)(addr) + NORMAL_VMA_OFFSET)
GlobalPageTableManager.MapMemory((void *)P2V(t), (void *)t);

nothing is really changed only in

Code: Select all

GlobalAllocator.ReadMemoryMap();

and it's instead of getting UEFI conventional memory, is getting STIVALE2 Usable Memory.

for creating a new PML4 I created this function:

Code: Select all

PageTable *KernelPML4;

void *CreateNewPML4()
{
    if (KernelPML4 == NULL)
    {
        // KernelPML4 = (PageTable *)readcr3();
        KernelPML4 = (PageTable *)GlobalAllocator.RequestPage();
        memset(KernelPML4, 0, 0x1000);
        return KernelPML4;
    }
    PageTable *NewPML4 = (PageTable *)GlobalAllocator.RequestPage();
    memset(NewPML4, 0, 0x1000);
    // for (size_t i = 0; i < 512; i++)
        // NewPML4->Entries[i] = KernelPML4->Entries[i];
    return NewPML4;
}

void initialize_vmm()
{
    KernelPML4 = (PageTable *)CreateNewPML4();
    GlobalPageTableManager = PageTableManager(KernelPML4);
    for (uint64_t t = 0; t < bootparams.mem.Size; t += 0x1000)
    {
        GlobalPageTableManager.MapMemory((void *)P2V(t), (void *)t);
    }
    for (uint64_t t = (uint64_t)bootparams.Framebuffer->BaseAddress; t < (uint64_t)bootparams.Framebuffer->BaseAddress + (bootparams.Framebuffer->BufferSize + 0x1000); t += 0x1000)
    {
        GlobalPageTableManager.MapMemory((void *)P2V(t), (void *)t);
    }
    writecr3((uint64_t)KernelPML4);
}

and also i changed the page map indexer to:

Code: Select all

PageMapIndexer::PageMapIndexer(uint64_t VirtualAddress)
{
    PDP_i = (VirtualAddress & ((uint64_t)0x1FF << 39)) >> 39;
    PD_i = (VirtualAddress & ((uint64_t)0x1FF << 30)) >> 30;
    PT_i = (VirtualAddress & ((uint64_t)0x1FF << 21)) >> 21;
    P_i = (VirtualAddress & ((uint64_t)0x1FF << 12)) >> 12;
}

and that's all, everything is the same, only a few variable names changed which is not affecting anything

is something wrong with poncho's implementation?

[Octocontrabass]

Code: Select all

#define NORMAL_VMA_OFFSET 0xFFFF800000000000

That's fine, but your kernel is at a different address.

instead of getting UEFI conventional memory, is getting STIVALE2 Usable Memory.

That's fine, but your kernel is not in usable memory.

It looks like the problem is that you're not mapping your kernel at all.

If you didn't request fully virtual mappings for your kernel, you need to map kernel memory (instead of usable memory) with an offset of 0xFFFFFFFF80000000 (instead of 0xFFFF800000000000). You can continue to map the other memory however you like.

If you did request fully virtual mappings, you won't be able to use a fixed offset - you'll have to work out the correct physical and virtual addresses some other way.

[Ethin]

Couldn't you just use the HHDM structure that Stivale2 gives you?

[22OsC]

Code: Select all

#define NORMAL_VMA_OFFSET 0xFFFF800000000000

That's fine, but your kernel is at a different address.

instead of getting UEFI conventional memory, is getting STIVALE2 Usable Memory.

That's fine, but your kernel is not in usable memory.

It looks like the problem is that you're not mapping your kernel at all.

If you didn't request fully virtual mappings for your kernel, you need to map kernel memory (instead of usable memory) with an offset of 0xFFFFFFFF80000000 (instead of 0xFFFF800000000000). You can continue to map the other memory however you like.

If you did request fully virtual mappings, you won't be able to use a fixed offset - you'll have to work out the correct physical and virtual addresses some other way.

Well... I tried to remake the mapping with Stivale2 HHDM, PMRs, and also stivale2's base addresses, and still no success. I came up with an idea that I did way before posting this but I didn't know about QEMU Monitor and I mapped the memory like this:

Code: Select all

#define KERNEL_VMA_OFFSET 0xFFFFFFFF80000000
        ...
        KernelPML4 = (PageTable *)GlobalAllocator.RequestPage();
        memset(KernelPML4, 0, 0x1000);
        GlobalPageTableManager = PageTableManager(KernelPML4);
        for (uint64_t t = 0; t < bootparams.mem.Size; t += 0x1000)
        {
            GlobalPageTableManager.MapMemory((void *)P2V(t), (void *)t);
        }
        for (uint64_t *ptr = &_kernel_start; ptr < (void *)ALIGN_UP(&_kernel_end, 0x1000); ptr += 0x1000)
        {
            GlobalPageTableManager.MapMemory(ptr, ptr - KERNEL_VMA_OFFSET);
        }
        ...

The "info tlb" reports that the higher half address where the kernel is mapped (I think) but still not working


and also the interrupt:

Code: Select all

check_exception old: 0xffffffff new 0xe
  1506: v=0e e=0010 i=0 cpl=0 IP=0008:ffffffff8002e3ea pc=ffffffff8002e3ea SP=0010:ffffffff802a9910 CR2=ffffffff8002e3ea
RAX=0000000000100000 RBX=0000000000000000 RCX=00000003802a8003 RDX=00000000000000a8
RSI=0000000000000001 RDI=ffffffff802a9880 RBP=ffffffff802a9910 RSP=ffffffff802a9910
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000030 R11=0000000000000010
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff8002e3ea RFL=00000006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 00000000 00209a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 00000000 00009200 DPL=0 DS   [-W-]
DS =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
FS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
GS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT=     ffffffff80043b60 00000037
IDT=     ffffffff80055b00 00000fff
CR0=80000013 CR2=ffffffff8002e3ea CR3=0000000000100000 CR4=00000620
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=ffffffff802ab000 CCD=0000000000005000 CCO=SUBQ    
EFER=0000000000000d00
check_exception old: 0xe new 0xe
  1507: v=08 e=0000 i=0 cpl=0 IP=0008:ffffffff8002e3ea pc=ffffffff8002e3ea SP=0010:ffffffff802a9910 env->regs[R_EAX]=0000000000100000
RAX=0000000000100000 RBX=0000000000000000 RCX=00000003802a8003 RDX=00000000000000a8
RSI=0000000000000001 RDI=ffffffff802a9880 RBP=ffffffff802a9910 RSP=ffffffff802a9910
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000030 R11=0000000000000010
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff8002e3ea RFL=00000006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 00000000 00209a00 DPL=0 CS64 [-R-]
SS =0010 0000000000000000 00000000 00009200 DPL=0 DS   [-W-]
DS =0010 0000000000000000 00000000 00009300 DPL=0 DS   [-WA]
FS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
GS =001b 0000000000000000 00000000 0000f300 DPL=3 DS   [-WA]
LDT=0000 0000000000000000 00000000 00008200 DPL=0 LDT
TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy
GDT=     ffffffff80043b60 00000037
IDT=     ffffffff80055b00 00000fff
CR0=80000013 CR2=ffffffff80055be0 CR3=0000000000100000 CR4=00000620
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=ffffffff802ab000 CCD=0000000000005000 CCO=SUBQ    
EFER=0000000000000d00
check_exception old: 0x8 new 0xe

[Octocontrabass]

The "info tlb" reports that the higher half address where the kernel is mapped (I think)

Are you sure? Last I checked, pages are 0x1000 bytes apart, not 0x8000.

Code: Select all

for (uint64_t *ptr = &_kernel_start; ptr < (void *)ALIGN_UP(&_kernel_end, 0x1000); ptr += 0x1000)

I think this line is the problem. (Hint: C pointer arithmetic.)

[22OsC]

The "info tlb" reports that the higher half address where the kernel is mapped (I think)

Are you sure? Last I checked, pages are 0x1000 bytes apart, not 0x8000.

Code: Select all

for (uint64_t *ptr = &_kernel_start; ptr < (void *)ALIGN_UP(&_kernel_end, 0x1000); ptr += 0x1000)

I think this line is the problem. (Hint: C pointer arithmetic.)

Well... I fixed that, my mistake but it would be too great to work!
I see that now pages are 0x1000 how they should be from the beginning, but it's something new. page fault with e=0000...

This is the current TLB (ignore that the normal map starts at 0xFFFF8000002ab000, i changed after the screenshots):





in this gif there is ----A---W, not sure what A stands for (accessed?) in the kernel mapping but not sure if this can break something

and the new code:

Code: Select all

        uint64_t normal_map = 0xFFFF800000000000;
        for (uint64_t t = (uint64_t)ALIGN_UP(&_kernel_end, 0x1000) - KERNEL_VMA_OFFSET;
             t < bootparams.mem.Size; t += 0x1000)
        {
            GlobalPageTableManager.MapMemory((void *)normal_map, (void *)t);
            normal_map += 0x1000;
        }
        for (uint64_t k = (uint64_t)&_kernel_start; k < (uint64_t)ALIGN_UP(&_kernel_end, 0x1000); k += 0x1000)
        {
            GlobalPageTableManager.MapMemory((void *)k, (void *)k - KERNEL_VMA_OFFSET);
        }

[Octocontrabass]

page fault with e=0000...

That means a data read from a not-present page (instead of an instruction fetch). I can't tell you what's wrong without more information, though.

in this gif there is ----A---W, not sure what A stands for (accessed?) in the kernel mapping but not sure if this can break something

It means "accessed". It won't break anything.