Higher Half Nightmare
Posted: Sat May 03, 2008 4:27 pm
This isn't really an implementation issue, just a horrible process that I've just endured.
I recently added some code to my x86-64 kernel port to invalidate the mappings of my code in lower memory (since the kernel actually runs at 0xFFFFFFFFC0100000
). It's just one line of code since I cheated and used a large 2MB page for identity mapping 
. This was also to avoid any potential problems my kernel could have once it gets mapped on to user processes.
And surprise, surprise, a page-fault occurred.
I quickly hunted for every possible bug in my paging code. Here's the process I went through:
- Realized my kcalloc function was returning physical addresses (it's supposed to return virtual) so all newly created page structure pointers had pointed to physical memory addresses
- A tiny calculation bug in my memory bitmap code
- I hadn't accounted for virtual memory addressing in the code that allocated pages for the kernel heap
- And finally, a page-fault that occurred on a retq instruction. CR2 was set to 100110. So, I did an objdump and saw that it pointed to my GDT. I'd loaded GDTR originally with a physical address, which ironically does not work for pointing to this memory-segmenting structure (as the identity mapping had been removed). Replaced that with a virtual address, et voila.
Everything's working properly...for now...
I recently added some code to my x86-64 kernel port to invalidate the mappings of my code in lower memory (since the kernel actually runs at 0xFFFFFFFFC0100000
). It's just one line of code since I cheated and used a large 2MB page for identity mapping . This was also to avoid any potential problems my kernel could have once it gets mapped on to user processes.And surprise, surprise, a page-fault occurred.
I quickly hunted for every possible bug in my paging code. Here's the process I went through:
- Realized my kcalloc function was returning physical addresses (it's supposed to return virtual) so all newly created page structure pointers had pointed to physical memory addresses
- A tiny calculation bug in my memory bitmap code
- I hadn't accounted for virtual memory addressing in the code that allocated pages for the kernel heap
- And finally, a page-fault that occurred on a retq instruction. CR2 was set to 100110. So, I did an objdump and saw that it pointed to my GDT. I'd loaded GDTR originally with a physical address, which ironically does not work for pointing to this memory-segmenting structure (as the identity mapping had been removed). Replaced that with a virtual address, et voila.
Everything's working properly...for now...