computing the page directory and page table entries

[Schol-R-LEA]

Ah, I wasn't aware that both the kernel and stack needed to be identity mapped, and am somewhat perplexed, given that I wasn't identity mapping them in the bootloader, either.

Nothing needs to be identity-mapped unless you're relying on it being identity-mapped. As far as I can tell, you've correctly jumped to the kernel in the higher half and set up a stack in the higher half, so you don't need either of those to be identity-mapped anymore.

That's what I thought... am I misunderstanding what Nullplan meant?

[Schol-R-LEA]

Oops, missed this earlier.

Code: Select all

IP=0008:00000000c000166b
CR2=0000000000a00000

I would guess one of your loops is iterating far beyond the end of an array. Try addr2line or objdump to track down which one.

I've never heard of addr2line before, and I've only used objdump for dumping the headers and disassembly of an ELF file. I'm not sure how I would use either of them to determine where a loop was overrunning. I'll have to check the documentation on both.

Meanwhile, I have added some sanity checks to my code, though none of them have caught anything yet.

Code: Select all

void set_page_directory_entry(uint32_t index,
                              size_t pte_address,
                              bool page_size, bool rw,
                              bool user, bool write_thru, bool no_caching)
{
    // first, clear the directory entry
    // memset(&page_directory[index], 0, sizeof(union Page_Directory_Entry));

    // SANITY CHECK - does this overrun the table?
    if (index > PD_ENTRY_COUNT)
    {
        kprintf("Invalid directory entry index: %x\n", index);
        panic();
    }

    if (page_size)
    {

        page_directory[index].mpage_entry.present = true;
        page_directory[index].mpage_entry.read_write = rw;
        page_directory[index].mpage_entry.user = user;
        page_directory[index].mpage_entry.write_thru = write_thru;
        page_directory[index].mpage_entry.cache_disable = no_caching;
        page_directory[index].mpage_entry.accessed = false;
        page_directory[index].mpage_entry.dirty = false;
        page_directory[index].mpage_entry.page_size = true;
        page_directory[index].mpage_entry.global = false;
        page_directory[index].mpage_entry.available = 0;
        page_directory[index].mpage_entry.page_attribute_table = false;
        page_directory[index].mpage_entry.address_hi = (pte_address >> 20) & 0x1FF;
        page_directory[index].mpage_entry.reserved = false;
        page_directory[index].mpage_entry.address_lo = pte_address & 0xFF;
    }
    else
    {
        page_directory[index].kpage_entry.present = true;
        page_directory[index].kpage_entry.read_write = rw;
        page_directory[index].kpage_entry.user = user;
        page_directory[index].kpage_entry.write_thru = write_thru;
        page_directory[index].kpage_entry.cache_disable = no_caching;
        page_directory[index].kpage_entry.accessed = false;
        page_directory[index].kpage_entry.dirty = false;
        page_directory[index].kpage_entry.page_size = false;
        page_directory[index].kpage_entry.available = 0;
        page_directory[index].kpage_entry.address = pte_address & 0xFFFFF;
    }
}


void set_page_table_entry(uint32_t de,
                          uint32_t te,
                          size_t address,
                          bool page_size, bool rw,
                          bool user, bool write_thru,
                          bool no_caching)
{
    uint32_t index = te + (de * PT_ENTRY_COUNT);
    // kprintf("Page dir:table = %x:%x -> index %x\n", de, te, index);

    // SANITY CHECK - does this overrun the table?
    if (de > PD_ENTRY_COUNT)
    {
        kprintf("Invalid directory entry: %x\n", de);
        panic();
    }

    if (index > (PT_ENTRY_COUNT * PD_ENTRY_COUNT))
    {
        kprintf("Invalid table entry index: %x\n", index);
        panic();
    }

    page_tables[index].fields.present = true;
    page_tables[index].fields.read_write = rw;
    page_tables[index].fields.user = user;
    page_tables[index].fields.write_thru = write_thru;
    page_tables[index].fields.cache_disable = no_caching;
    page_tables[index].fields.accessed = false;
    page_tables[index].fields.dirty = false;
    page_tables[index].fields.page_size = page_size;
    page_tables[index].fields.page_attribute_table = false;
    page_tables[index].fields.global = false;
    page_tables[index].fields.available = 0;
    page_tables[index].fields.address = address;
}

Code: Select all

/* set a block of page directory and page table entries matching a block of memory */
void set_page_block(uint32_t phys_address,
                    uint32_t virt_address,
                    uint32_t block_size,
                    bool page_size, bool rw,
                    bool user, bool write_thru,
                    bool no_caching)
{
    struct Page_Directory_Frame frame;

    get_frame(&frame, virt_address, block_size);

    // SANITY CHECKS - make sure that the calculated values are sound
    if (frame.dir_start > PD_ENTRY_COUNT)
    {
        kprintf("Invalid directory start: %x\n", frame.dir_start);
        panic();
    }

    if (frame.dir_end > PD_ENTRY_COUNT)
    {
        kprintf("Invalid directory endpoint: %x\n", frame.dir_end);
        panic();
    }

    if (frame.page_start > PT_ENTRY_COUNT)
    {
        kprintf("Invalid page table entry start: %x\n", frame.page_start);
        panic();
    }

    if (frame.page_end > PT_ENTRY_COUNT)
    {
        kprintf("Invalid page table entry endpoint: %x\n", frame.page_end);
        panic();
    }

    // initialize the iteration variables here rather than in the for statement,
    // as the values need to carry over fro one iteration to the next
    uint32_t pd_entry = frame.dir_start;
    uint32_t pt_entry;
    size_t addr = phys_address;

    for (bool first_entry = true; pd_entry <= frame.dir_end; pd_entry++, first_entry = false)
    {
        // if this is the first iteration of the loop, use the computed page entry location, 
        // otherwise start from the beginning of the page entry
        pt_entry = first_entry ? frame.page_start : 0;

        // if this is the final iteration of the loop, use the computed page end location,
        // otherwise fill the whole page entry
        uint32_t pt_current_end = (pd_entry == frame.dir_end) ? (frame.page_end + 1) : PT_ENTRY_COUNT;

        // SANITY CHECK - does this overrun the table?
        if (pt_current_end > PT_ENTRY_COUNT)
        {
            kprintf("Invalid local page table entry endpoint: %x\n", pt_current_end);
            panic();
        }

        set_page_directory_entry(pd_entry,
                                 (size_t) &page_tables[pt_entry],
                                 page_size, rw,
                                 user, write_thru,
                                 no_caching);


        for (; pt_entry < pt_current_end; pt_entry++, addr += PAGE_SPAN)
        {
            set_page_table_entry(pd_entry,
                                 pt_entry,
                                 addr,
                                 page_size, rw,
                                 user, write_thru,
                                 no_caching);
        }
    }
}

[Schol-R-LEA]

I forgot to add the current interrupt log:

Code: Select all

check_exception old: 0xffffffff new 0xe
     0: v=0e e=0000 i=0 cpl=0 IP=0008:00000000c0001bd7 pc=00000000c0001bd7 SP=0010:00000000c1003f90 CR2=00000000c0001bd7
EAX=01400000 EBX=00040c10 ECX=08000000 EDX=08000003
ESI=0000b482 EDI=00000004 EBP=c1003fc8 ESP=c1003f90
EIP=c0001bd7 EFL=00000086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     c0102000 00007fff
IDT=     00000000 000003ff
CR0=80000011 CR2=c0001bd7 CR3=01400000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=00000020 CCD=c1003f90 CCO=ADDL
EFER=0000000000000000
check_exception old: 0xe new 0xe
     1: v=08 e=0000 i=0 cpl=0 IP=0008:00000000c0001bd7 pc=00000000c0001bd7 SP=0010:00000000c1003f90 env->regs[R_EAX]=0000000001400000
EAX=01400000 EBX=00040c10 ECX=08000000 EDX=08000003
ESI=0000b482 EDI=00000004 EBP=c1003fc8 ESP=c1003f90
EIP=c0001bd7 EFL=00000086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     c0102000 00007fff
IDT=     00000000 000003ff
CR0=80000011 CR2=00000070 CR3=01400000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=00000020 CCD=c1003f90 CCO=ADDL
EFER=0000000000000000
check_exception old: 0x8 new 0xe

As you can see, it is now page faulting on a line of code here:

Code: Select all

IP=0008:00000000c0001bd7 CR2=00000000c0001bd7

This seems to be a different problem than that which was coming up earlier. Why it is faulting on an address (0xc0001bd7) that ought to be in the region mapped for the kernel isn't clear to me.

EDIT: Never mind, I just looked at the `info mem` and `info tlb` dumps in the QEMU monitor. It is clear that the mapping itself is failing, and that the address must be the first instruction following the mov %0, %%cr3 instruction.

[Schol-R-LEA]

Quick update: I forgot to set the page directory and page table structures to packed. Adding that fixes the corrupted page table issue.

Code: Select all

0000000000000000-0000000001800000 0000000001800000 -r-
00000000c0000000-00000000c0800000 0000000000800000 -r-
00000000c1000000-00000000c1010000 0000000000010000 -r-

It is still triple faulting, but the interrupt dump is now:

Code: Select all

0: v=0e e=0002 i=0 cpl=0 IP=0008:00000000c0000b06 pc=00000000c0000b06 SP=0010:00000000c1003e64 CR2=0000000001800000

I think I know where this problem lies, but I'll need to check a few things.

[nullplan]

Nothing needs to be identity-mapped unless you're relying on it being identity-mapped. As far as I can tell, you've correctly jumped to the kernel in the higher half and set up a stack in the higher half, so you don't need either of those to be identity-mapped anymore.

That's what I thought... am I misunderstanding what Nullplan meant?

I will admit that I didn't entirely follow your odyssey up to this point. The code you keep posting has the bit in it where you set CR0.PG, which made me think that it is not set before (else, why would it be there?). In order to enable paging, you must have at least the page of code containing the "mov cr0, eax" instruction and the following instruction identity mapped. Easiest way to do that is to identity map the entire kernel. This is because more generally, whenever you change paging, the code you are currently executing on must translate to the same physical address before and after the change (so that counts for both modifying CR0.PG and CR3). And before setting CR0.PG, everything is identity mapped, so the code must be identity mapped.

Note that the above are the instructions for the 486 and later. For the 386, they said it would be possible to just have an indirect jump instruction directly after the "mov cr0" instruction, and that would work without identity mapping the code. The idea being that the jump could be executed from the instruction queue. This no longer works.

The function you keep posting ends after setting CR0.PG. Since with that setting, the interpretation of all addresses changes, and you want your "ret" to keep working, ESP must continue to refer to the same memory after setting CR0.PG. This is possible by identity mapping it. Else, if the stack is in some linear mapped memory, you can adjust ESP. But note that all the pointers on the stack will continue to refer to the old locations from before the switch, so this approach is fraught with peril.

If I misunderstood your intentions, and CR0.PG is actually already set, then you might want to get rid of that snippet, since it is misleading.

[Schol-R-LEA]

If I misunderstood your intentions, and CR0.PG is actually already set, then you might want to get rid of that snippet, since it is misleading.

It is already set; I set it in the boot loader when I make the temporary initial mappings. I'll remove that part you mention as being redundant.

In other news, I was wrong about using `packed` as fixing the corrupted map; it was actually crashing before it even go there due to a previously missed bug. I am going to keep the structures as `packed` for now, as it may come up later, but I am not sure if it is really needed.

Code: Select all

struct Page_Directory_Entry_kilobyte
{
    bool present:1;
    bool read_write:1;
    bool user:1;
    bool write_thru:1;
    bool cache_disable:1;
    bool accessed:1;
    bool dirty:1;
    bool page_size:1;
    uint8_t available:4;
    uint32_t address:20;
}__attribute__((packed));


struct Page_Directory_Entry_megabyte
{
    bool present:1;
    bool read_write:1;
    bool user:1;
    bool write_thru:1;
    bool cache_disable:1;
    bool accessed:1;
    bool dirty:1;
    bool page_size:1;
    bool global:1;
    uint8_t available:3;
    bool page_attribute_table:1;
    uint8_t address_hi:8;
    bool reserved:1;
    uint16_t address_lo:10;
}__attribute__((packed));


union Page_Directory_Entry
{
    uint32_t raw_entry;
    struct Page_Directory_Entry_kilobyte kpage_entry;
    struct Page_Directory_Entry_megabyte mpage_entry;
} __attribute__((aligned(0x1000), packed));


struct Page_Table_Entry_fields
{
        bool present:1;
        bool read_write:1;
        bool user:1;
        bool write_thru:1;
        bool cache_disable:1;
        bool accessed:1;
        bool dirty:1;
        bool page_size:1;
        bool page_attribute_table:1;
        bool global:1;
        uint8_t available:3;
        uint32_t address:19;
}__attribute__((packed));


union Page_Table_Entry
{
    uint32_t raw_entry;
    struct Page_Table_Entry_fields fields;
} __attribute__((aligned(0x1000), packed)) ;

[Schol-R-LEA]

Here is the `info mem` dump at the point when the OS crashes. I can tell that there is a pattern to the corruption but I can't quite figure it out.

Code: Select all

0000000000000000-0000000000405000 0000000000405000 -rw
0000000000406000-0000000000408000 0000000000002000 -rw
0000000000408000-0000000000409000 0000000000001000 -r-
0000000000409000-000000000040a000 0000000000001000 -rw
000000000040e000-000000000040f000 0000000000001000 -rw
0000000000410000-0000000000413000 0000000000003000 -r-
0000000000414000-0000000000416000 0000000000002000 -r-
0000000000418000-0000000000419000 0000000000001000 -rw
000000000041b000-000000000041e000 0000000000003000 -rw
0000000000420000-0000000000440000 0000000000020000 -rw
0000000000440000-0000000000441000 0000000000001000 -r-
0000000000441000-0000000000443000 0000000000002000 -rw
0000000000444000-0000000000460000 000000000001c000 -rw
0000000000467000-0000000000474000 000000000000d000 -rw
0000000000475000-0000000000476000 0000000000001000 -r-
0000000000477000-0000000000479000 0000000000002000 -rw
000000000047a000-0000000000500000 0000000000086000 -rw
0000000000504000-0000000000505000 0000000000001000 -rw
0000000000510000-0000000000511000 0000000000001000 -r-
0000000000518000-000000000051a000 0000000000002000 -rw
0000000000522000-0000000000523000 0000000000001000 -r-
0000000000524000-0000000000526000 0000000000002000 -rw
0000000000800000-0000000000805000 0000000000005000 -rw
0000000000806000-0000000000808000 0000000000002000 -rw
0000000000808000-0000000000809000 0000000000001000 -r-
0000000000809000-000000000080a000 0000000000001000 -rw
000000000080e000-000000000080f000 0000000000001000 -rw
0000000000810000-0000000000813000 0000000000003000 -r-
0000000000814000-0000000000816000 0000000000002000 -r-
0000000000818000-0000000000819000 0000000000001000 -rw
000000000081b000-000000000081e000 0000000000003000 -rw
0000000000820000-0000000000840000 0000000000020000 -rw
0000000000840000-0000000000841000 0000000000001000 -r-
0000000000841000-0000000000843000 0000000000002000 -rw
0000000000844000-0000000000860000 000000000001c000 -rw
0000000000867000-0000000000874000 000000000000d000 -rw
0000000000875000-0000000000876000 0000000000001000 -r-
0000000000877000-0000000000879000 0000000000002000 -rw
000000000087a000-0000000000900000 0000000000086000 -rw
0000000000904000-0000000000905000 0000000000001000 -rw
0000000000910000-0000000000911000 0000000000001000 -r-
0000000000918000-000000000091a000 0000000000002000 -rw
0000000000922000-0000000000923000 0000000000001000 -r-
0000000000924000-0000000000926000 0000000000002000 -rw
0000000000c00000-0000000000c05000 0000000000005000 -rw
0000000000c06000-0000000000c08000 0000000000002000 -rw
0000000000c08000-0000000000c09000 0000000000001000 -r-
0000000000c09000-0000000000c0a000 0000000000001000 -rw
0000000000c0e000-0000000000c0f000 0000000000001000 -rw
0000000000c10000-0000000000c13000 0000000000003000 -r-
0000000000c14000-0000000000c16000 0000000000002000 -r-
0000000000c18000-0000000000c19000 0000000000001000 -rw
0000000000c1b000-0000000000c1e000 0000000000003000 -rw
0000000000c20000-0000000000c40000 0000000000020000 -rw
0000000000c40000-0000000000c41000 0000000000001000 -r-
0000000000c41000-0000000000c43000 0000000000002000 -rw
0000000000c44000-0000000000c60000 000000000001c000 -rw
0000000000c67000-0000000000c74000 000000000000d000 -rw
0000000000c75000-0000000000c76000 0000000000001000 -r-
0000000000c77000-0000000000c79000 0000000000002000 -rw
0000000000c7a000-0000000000d00000 0000000000086000 -rw
0000000000d04000-0000000000d05000 0000000000001000 -rw
0000000000d10000-0000000000d11000 0000000000001000 -r-
0000000000d18000-0000000000d1a000 0000000000002000 -rw
0000000000d22000-0000000000d23000 0000000000001000 -r-
0000000000d24000-0000000000d26000 0000000000002000 -rw
0000000001000000-0000000001005000 0000000000005000 -rw
0000000001006000-0000000001008000 0000000000002000 -rw
0000000001008000-0000000001009000 0000000000001000 -r-
0000000001009000-000000000100a000 0000000000001000 -rw
000000000100e000-000000000100f000 0000000000001000 -rw
0000000001010000-0000000001013000 0000000000003000 -r-
0000000001014000-0000000001016000 0000000000002000 -r-
0000000001018000-0000000001019000 0000000000001000 -rw
000000000101b000-000000000101e000 0000000000003000 -rw
0000000001020000-0000000001040000 0000000000020000 -rw
0000000001040000-0000000001041000 0000000000001000 -r-
0000000001041000-0000000001043000 0000000000002000 -rw
0000000001044000-0000000001060000 000000000001c000 -rw
0000000001067000-0000000001074000 000000000000d000 -rw
0000000001075000-0000000001076000 0000000000001000 -r-
0000000001077000-0000000001079000 0000000000002000 -rw
000000000107a000-0000000001100000 0000000000086000 -rw
0000000001104000-0000000001105000 0000000000001000 -rw
0000000001110000-0000000001111000 0000000000001000 -r-
0000000001118000-000000000111a000 0000000000002000 -rw
0000000001122000-0000000001123000 0000000000001000 -r-
0000000001124000-0000000001126000 0000000000002000 -rw
0000000001800000-0000000001805000 0000000000005000 -rw
0000000001806000-0000000001808000 0000000000002000 -rw
0000000001808000-0000000001809000 0000000000001000 -r-
0000000001809000-000000000180a000 0000000000001000 -rw
000000000180e000-000000000180f000 0000000000001000 -rw
0000000001810000-0000000001813000 0000000000003000 -r-
0000000001814000-0000000001816000 0000000000002000 -r-
0000000001818000-0000000001819000 0000000000001000 -rw
000000000181b000-000000000181e000 0000000000003000 -rw
0000000001820000-0000000001840000 0000000000020000 -rw
0000000001840000-0000000001841000 0000000000001000 -r-
0000000001841000-0000000001843000 0000000000002000 -rw
0000000001844000-0000000001860000 000000000001c000 -rw
0000000001867000-0000000001874000 000000000000d000 -rw
0000000001875000-0000000001876000 0000000000001000 -r-
0000000001877000-0000000001879000 0000000000002000 -rw
000000000187a000-0000000001900000 0000000000086000 -rw
0000000001904000-0000000001905000 0000000000001000 -rw
0000000001910000-0000000001911000 0000000000001000 -r-
0000000001918000-000000000191a000 0000000000002000 -rw
0000000001922000-0000000001923000 0000000000001000 -r-
0000000001924000-0000000001926000 0000000000002000 -rw
00000000c0000000-00000000c0405000 0000000000405000 -rw
00000000c0406000-00000000c0408000 0000000000002000 -rw
00000000c0408000-00000000c0409000 0000000000001000 -r-
00000000c0409000-00000000c040a000 0000000000001000 -rw
00000000c040e000-00000000c040f000 0000000000001000 -rw
00000000c0410000-00000000c0413000 0000000000003000 -r-
00000000c0414000-00000000c0416000 0000000000002000 -r-
00000000c0418000-00000000c0419000 0000000000001000 -rw
00000000c041b000-00000000c041e000 0000000000003000 -rw
00000000c0420000-00000000c0440000 0000000000020000 -rw
00000000c0440000-00000000c0441000 0000000000001000 -r-
00000000c0441000-00000000c0443000 0000000000002000 -rw
00000000c0444000-00000000c0460000 000000000001c000 -rw
00000000c0467000-00000000c0474000 000000000000d000 -rw
00000000c0475000-00000000c0476000 0000000000001000 -r-
00000000c0477000-00000000c0479000 0000000000002000 -rw
00000000c047a000-00000000c0500000 0000000000086000 -rw
00000000c0504000-00000000c0505000 0000000000001000 -rw
00000000c0510000-00000000c0511000 0000000000001000 -r-
00000000c0518000-00000000c051a000 0000000000002000 -rw
00000000c0522000-00000000c0523000 0000000000001000 -r-
00000000c0524000-00000000c0526000 0000000000002000 -rw
00000000c0800000-00000000c0805000 0000000000005000 -rw
00000000c0806000-00000000c0808000 0000000000002000 -rw
00000000c0808000-00000000c0809000 0000000000001000 -r-
00000000c0809000-00000000c080a000 0000000000001000 -rw
00000000c080e000-00000000c080f000 0000000000001000 -rw
00000000c0810000-00000000c0813000 0000000000003000 -r-
00000000c0814000-00000000c0816000 0000000000002000 -r-
00000000c0818000-00000000c0819000 0000000000001000 -rw
00000000c081b000-00000000c081e000 0000000000003000 -rw
00000000c0820000-00000000c0840000 0000000000020000 -rw
00000000c0840000-00000000c0841000 0000000000001000 -r-
00000000c0841000-00000000c0843000 0000000000002000 -rw
00000000c0844000-00000000c0860000 000000000001c000 -rw
00000000c0867000-00000000c0874000 000000000000d000 -rw
00000000c0875000-00000000c0876000 0000000000001000 -r-
00000000c0877000-00000000c0879000 0000000000002000 -rw
00000000c087a000-00000000c0900000 0000000000086000 -rw
00000000c0904000-00000000c0905000 0000000000001000 -rw
00000000c0910000-00000000c0911000 0000000000001000 -r-
00000000c0918000-00000000c091a000 0000000000002000 -rw
00000000c0922000-00000000c0923000 0000000000001000 -r-
00000000c0924000-00000000c0926000 0000000000002000 -rw
00000000c1000000-00000000c1005000 0000000000005000 -rw
00000000c1006000-00000000c1008000 0000000000002000 -rw
00000000c1008000-00000000c1009000 0000000000001000 -r-
00000000c1009000-00000000c100a000 0000000000001000 -rw
00000000c100e000-00000000c100f000 0000000000001000 -rw
00000000c1010000-00000000c1013000 0000000000003000 -r-
00000000c1014000-00000000c1016000 0000000000002000 -r-
00000000c1018000-00000000c1019000 0000000000001000 -rw
00000000c101b000-00000000c101e000 0000000000003000 -rw
00000000c1020000-00000000c1040000 0000000000020000 -rw
00000000c1040000-00000000c1041000 0000000000001000 -r-
00000000c1041000-00000000c1043000 0000000000002000 -rw
00000000c1044000-00000000c1060000 000000000001c000 -rw
00000000c1067000-00000000c1074000 000000000000d000 -rw
00000000c1075000-00000000c1076000 0000000000001000 -r-
00000000c1077000-00000000c1079000 0000000000002000 -rw
00000000c107a000-00000000c1100000 0000000000086000 -rw
00000000c1104000-00000000c1105000 0000000000001000 -rw
00000000c1110000-00000000c1111000 0000000000001000 -r-
00000000c1118000-00000000c111a000 0000000000002000 -rw
00000000c1122000-00000000c1123000 0000000000001000 -r-
00000000c1124000-00000000c1126000 0000000000002000 -rw

[Octocontrabass]

In other news, I was wrong about using `packed` as fixing the corrupted map; it was actually crashing before it even go there due to a previously missed bug. I am going to keep the structures as `packed` for now, as it may come up later, but I am not sure if it is really needed.

I like to use Compiler Explorer to check things like this. For example, it looks like you're accidentally padding some of your types to 4kB instead of the intended 4 bytes.

I notice you have definitions for 4MB pages, but you don't have any code to check whether the CPU supports 4MB pages or enable them in case they are supported.

[Schol-R-LEA]

I was inserting the addresses into the directory and table fields incorrectly the whole time.

OK, so the good news is, the memory is getting mapped correctly now:

Code: Select all

0000000000000000-0000000001c00000 0000000001c00000 -rw
00000000c0000000-00000000c0c00000 0000000000c00000 -rw
00000000c1000000-00000000c1400000 0000000000400000 -rw

The bad news is, it is still getting a triple fault immediately after CR3 is set. This time the error is caused by, of all things, an Invalid Opcode exception.

Code: Select all

check_exception old: 0xffffffff new 0x6
     0: v=06 e=0000 i=0 cpl=0 IP=0008:00000000c0003751 pc=00000000c0003751 SP=0010:00000000c1003fa0 env->regs[R_EAX]=0000000001400000
EAX=01400000 EBX=00040c10 ECX=00000000 EDX=00000050
ESI=0000b482 EDI=00000004 EBP=c1003fc8 ESP=c1003fa0
EIP=c0003751 EFL=00000046 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 00000000 ffffffff 00cf9a00 DPL=0 CS32 [-R-]
SS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 00000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS32-busy
GDT=     c0102000 00007fff
IDT=     00000000 000003ff
CR0=80000011 CR2=00000000 CR3=01400000 CR4=00000000
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=00000000 CCD=00000000 CCO=ADDB
EFER=0000000000000000

I'm going to check this in as an interim stage, and then go to bed; I will take this new problem on in the morning.

[Octocontrabass]

OK, so the good news is, the memory is getting mapped correctly now:

Without seeing the physical addresses, how do you know it's getting mapped correctly?

[rdos]

I notice you have definitions for 4MB pages, but you don't have any code to check whether the CPU supports 4MB pages or enable them in case they are supported.

Using 4MB pages is only supported with protected mode paging. For PAE and long mode, the page size on the next level is 2MB. I only support 2MB pages, and it's use would require PAE paging.

[Schol-R-LEA]

I notice you have definitions for 4MB pages, but you don't have any code to check whether the CPU supports 4MB pages or enable them in case they are supported.

Using 4MB pages is only supported with protected mode paging. For PAE and long mode, the page size on the next level is 2MB. I only support 2MB pages, and it's use would require PAE paging.

I ended up removing the support for 4MiB pages last night, as I don't expect to ever use them and I suspected that there was an error in the code meant to support them. I had included it in the off-chance that I would, but for this project at least, I think it would be rather pointless.

[Schol-R-LEA]

OK, so the good news is, the memory is getting mapped correctly now:

Without seeing the physical addresses, how do you know it's getting mapped correctly?

Fair point, I can also go over the TLB mappings.

Code: Select all

0000000000000000: 0000000000000000 ---DA---W
0000000000001000: 0000000000002000 ----A---W
0000000000002000: 0000000000004000 ----A---W
0000000000003000: 0000000000006000 ----A---W
0000000000004000: 0000000000008000 --------W
0000000000005000: 000000000000a000 --------W
0000000000006000: 000000000000c000 --------W

...

0000000000100000: 0000000000200000 --------W
0000000000101000: 0000000000202000 --------W
0000000000102000: 0000000000204000 --------W
0000000000103000: 0000000000206000 --------W
0000000000104000: 0000000000208000 --------W
0000000000105000: 000000000020a000 --------W

...

0000000000300000: 0000000000600000 --------W
0000000000301000: 0000000000602000 --------W
0000000000302000: 0000000000604000 --------W
0000000000303000: 0000000000606000 --------W
0000000000304000: 0000000000608000 --------W
0000000000305000: 000000000060a000 --------W

...

000000000003ff000: 00000000007fe000 --------W
0000000000400000: 0000000000000000 ---DA---W
0000000000401000: 0000000000002000 ----A---W
0000000000402000: 0000000000004000 ----A---W
0000000000403000: 0000000000006000 ----A---W
0000000000404000: 0000000000008000 --------W

...

0000000001000000: 0000000000000000 ---DA---W
0000000001001000: 0000000000002000 ----A---W
0000000001002000: 0000000000004000 ----A---W
0000000001003000: 0000000000006000 ----A---W
0000000001004000: 0000000000008000 --------W

...

00000000013ff000: 00000000007fe000 --------W
0000000001400000: 0000000000000000 ---DA---W
0000000001401000: 0000000000002000 ----A---W
0000000001402000: 0000000000004000 ----A---W
0000000001403000: 0000000000006000 ----A---W
0000000001404000: 0000000000008000 --------W

...

00000000017ff000: 00000000007fe000 --------W
0000000001800000: 0000000000000000 ---DA---W
0000000001801000: 0000000000002000 ----A---W
0000000001802000: 0000000000004000 ----A---W
0000000001803000: 0000000000006000 ----A---W

...

0000000001bff000: 00000000007fe000 --------W
00000000c0000000: 0000000000000000 ---DA---W
00000000c0001000: 0000000000002000 ----A---W
00000000c0002000: 0000000000004000 ----A---W
00000000c0003000: 0000000000006000 ----A---W
00000000c0004000: 0000000000008000 --------W

...

00000000c03ff000: 00000000007fe000 --------W
00000000c0400000: 0000000000000000 ---DA---W
00000000c0401000: 0000000000002000 ----A---W
00000000c0402000: 0000000000004000 ----A---W
00000000c0403000: 0000000000006000 ----A---W

...

00000000c0bff000: 00000000007fe000 --------W
00000000c1000000: 0000000000000000 ---DA---W
00000000c1001000: 0000000000002000 ----A---W
00000000c1002000: 0000000000004000 ----A---W
00000000c1003000: 0000000000006000 ----A---W
00000000c1004000: 0000000000008000 --------W

So, what I am seeing - assuming I am reading this correctly - is that each mapped section is getting mapped to the beginning of memory. It is also incrementing the mapping by x2 rather than a consistent +0x1000.

It is clear that the function which iterates through the page table entries is always starting at zero, rather than at the intended physical address.

Code: Select all

/* set a block of page directory and page table entries matching a block of memory */
void set_page_block(uint32_t phys_address,
                    uint32_t virt_address,
                    uint32_t block_size,
                    bool rw, bool user,
                    bool write_thru,
                    bool no_caching)
{
    struct Page_Directory_Frame frame;

    get_frame(&frame, virt_address, block_size);

    // SANITY CHECKS - make sure that the calculated values are sound
    if (frame.dir_start > PD_ENTRY_COUNT)
    {
        kprintf("Invalid directory start: %x\n", frame.dir_start);
        panic();
    }

    if (frame.dir_end > PD_ENTRY_COUNT)
    {
        kprintf("Invalid directory endpoint: %x\n", frame.dir_end);
        panic();
    }

    if (frame.page_start > PT_ENTRY_COUNT)
    {
        kprintf("Invalid page table entry start: %x\n", frame.page_start);
        panic();
    }

    if (frame.page_end > PT_ENTRY_COUNT)
    {
        kprintf("Invalid page table entry endpoint: %x\n", frame.page_end);
        panic();
    }

    // initialize the iteration variables here rather than in the for statement,
    // as the values need to carry over fro one iteration to the next
    uint32_t pd_entry = frame.dir_start;
    uint32_t pt_entry;
    size_t addr = phys_address;

    for (bool first_entry = true; pd_entry <= frame.dir_end; pd_entry++, first_entry = false)
    {
        // if this is the first iteration of the loop, use the computed page entry location, 
        // otherwise start from the beginning of the page entry
        pt_entry = first_entry ? frame.page_start : 0;

        // if this is the final iteration of the loop, use the computed page end location,
        // otherwise fill the whole page entry
        uint32_t pt_current_end = (pd_entry == frame.dir_end) ? frame.page_end + 1 : PT_ENTRY_COUNT;


        // SANITY CHECK - does this overrun the table?
        if (pt_current_end > PT_ENTRY_COUNT)
        {
            kprintf("Invalid local page table entry endpoint: %x\n", pt_current_end);
            panic();
        }

        set_page_directory_entry(pd_entry,
                                 (size_t) &page_tables[pt_entry],
                                 rw, user,
                                 write_thru,
                                 no_caching);


        for (; pt_entry < pt_current_end; pt_entry++, addr += PAGE_SPAN)
        {
            set_page_table_entry(pd_entry,
                                 pt_entry,
                                 addr,
                                 rw, user,
                                 write_thru,
                                 no_caching);
        }
    }
}

I'll look into this now.

EDIT: Part of the problem appears to be in how I am extracting the upper 12 bits of the address when inserting it into the address field.

Code: Select all

    page_tables[index].fields.address = address >> 12;

This is causing it to overshoot the target address by a factor of 2. However, shifting 13 bits instead causes it to alternate rather than smoothly increment:

Code: Select all

0000000000000000: 0000000000000000 ---DA---W
0000000000001000: 0000000000000000 ----A---W
0000000000002000: 0000000000002000 ----A---W
0000000000003000: 0000000000002000 ----A---W
0000000000004000: 0000000000004000 ----A---W
0000000000005000: 0000000000004000 ----A---W
0000000000006000: 0000000000006000 ----A---W
0000000000007000: 0000000000006000 --------W

I'll keep at it with this to see how I can fix this.

[Octocontrabass]

Code: Select all

        uint32_t address:19;

Well, you can't fit the upper 20 bits of an address in only 19 bits. I'd say this struct is wrong.

[Schol-R-LEA]

Code: Select all

        uint32_t address:19;

Well, you can't fit the upper 20 bits of an address in only 19 bits. I'd say this struct is wrong.

You are right, of course, but it's worse than that - I accidentally added a page_size field to that struct, which is what led me to think that it should be 19 bits rather than 20. Removing that field and changing the address field to 20 bits does fix that particular issue.

So now I need to tackle the question of why it keeps setting the translations for each block to the start of memory. (Actually, it does in fact identity-map the kernel section, which is annoying because, as Octocontrabass pointed out, that mapping isn't actually necessary.)