Clarifying higher half kernel approach & the GDT trick
Posted: Sat Jun 20, 2020 11:54 am
I have been trying to figure out the right approach for implementing a higher-half kernel with the GDT/IDT properly set and paging enabled. My current implementation works, but I wanted to clear a few things up...and hopefully provide an answer for someone googling this question in the future 
Just to preface - this is a lot of questions, and I'm definitely not expecting one person to provide some huge in-depth explanation. I've done a fair amount of reading the wiki and going over relevant Intel docs for these operations so I have a rough idea of how this works, but it's a lot to take in. What I'm trying to accomplish here is just get a slightly more holistic view of memory setup.
The way I've set up my paging (a-la barebones higher half) is to:
Here is what I have been trying to figure out:
(1) GDT & IDT Addressing
Before setting up paging, I had already implemented the GDT/IDT setup in my kernel, which at the time used linear addresses. I was under the impression that the GDTR needs a linear address (at least, that's what it seems like from the Intel Manual description, and online guides I've read say the same thing). But after I implemented paging, I wanted to see what would happen with my current setup, since I don't set the GDT and IDT until my kernel main routine -- which occurs after paging. I booted the kernel in Bochs, and sure enough, I'm loading the GDTR/IDTR with virtual addresses (e.g. for GDT, the address is 0xc0105020). But it seems to work; here is my GDT after setting the segment registers and performing a long jump:
This surprised me a bit, but I'm not going to complain when things work! However, I wanted to see what the general opinion is on the "right" thing to do - even though this works for now, I don't want to get into trouble down the line, and in general I'm curious to see what's going on here. I think I know the answers to some of these, but as I said, for completeness sake I'd like to try and map out the whole picture.
(A) Does order matter in terms of setup? Should I be setting GDT and/or IDT first? Currently I setup my GDT and IDT in C (for the most part), calling them from kernel main after paging. I don't set up my stack until paging is enabled, and I don't think doing so earlier would accomplish anything -- since I link at 0xC0000000, the stack top value is 0xc0109070. I assume that calling C functions with that invalid linear stack address before enabling paging would crash. So I think moving those GDT/IDT setup calls to occur before paging could be tricky. I could re-implement my GDT setup in assembly without too much trouble, but the IDT would be tougher.
(B) After I enable paging, I suppose I could still get the physical address of the GDT/IDT if I wanted to by taking their virtual address and manually extracting PD entry --> PT entry --> physical address. If I keep my current setup order, is it better to perform this bit-shifting and use the physical address to load the registers, even though the virtual address seems to be working properly?
(C) If I do set up GDT/IDT before paging, using their linear addresses, do I need to manage them in any way after enabling paging? I should note that after I enable paging, I un-map the identity page table (PD entry 0), so that 1:1 mapping is gone -- virtual address 0x00100000 is no longer mapped to that physical memory. Will segment operations ignore paging and just go straight to the physical address loaded in the GDTR? I would think so, but currently there's a virtual address in there and that is working...
.
(D) On that note, since my current setup DOES work - how is the GDTR working with a virtual address? Or does the lgdt/lidt instruction itself perform the translation from virtual to physical, and just load the physical address?
(2) Does the GDT trick accomplish anything not satisfied by the approach described above?
While I'm on the subject of setting up the GDT for higher half kernels, I wanted to clear up a few thoughts on the GDT trick.Through my own searches on the subject of higher-half kernels, I keep running into references to this approach (though it seems like the actual HigherHalfGDT page, from ~2005, is gone. Is this now seen as a bad approach?).
From what I gather, the idea is that by setting the GDT base of the kernel code/data segments to 0x40000000 instead of 0x0, any higher-half virtual memory references above 0xC0000000 will wrap back around because of 32-bit overflow (e.g. 0x40000000:0xC0100000 --> 0x00100000, which gives us the physical address of our kernel code). This way, we just have to do the identity-mapping of the kernel, and don't have to bother setting the PD[768] entry / going through my approach above. I just wanted to make sure I had the right idea, and see if there is any other benefit. Several people refer to it as a "hack", the page is gone - and it doesn't seem like it saves that much work/complexity - so I just wanted to see what the general consensus was in 2020.
Just to re-iterate -- I know this is a lot of questions! Just want to try and hash this out a bit. Thanks in advance, and I promise I am working on answering these myself.
Just to preface - this is a lot of questions, and I'm definitely not expecting one person to provide some huge in-depth explanation. I've done a fair amount of reading the wiki and going over relevant Intel docs for these operations so I have a rough idea of how this works, but it's a lot to take in. What I'm trying to accomplish here is just get a slightly more holistic view of memory setup.
The way I've set up my paging (a-la barebones higher half) is to:
- * Link the kernel at 0xC0000000
* Set up a page directory,
* Set up the first page table
* Identity-map the kernel to that page table
* Use that table set the PD entry at both the first and 768th PD entries (768 representing address 0xC0000000).
* Enable Paging
* Reset the first page directory entry to 0, now that EIP is using virtual addresses
Here is what I have been trying to figure out:
(1) GDT & IDT Addressing
Before setting up paging, I had already implemented the GDT/IDT setup in my kernel, which at the time used linear addresses. I was under the impression that the GDTR needs a linear address (at least, that's what it seems like from the Intel Manual description, and online guides I've read say the same thing). But after I implemented paging, I wanted to see what would happen with my current setup, since I don't set the GDT and IDT until my kernel main routine -- which occurs after paging. I booted the kernel in Bochs, and sure enough, I'm loading the GDTR/IDTR with virtual addresses (e.g. for GDT, the address is 0xc0105020). But it seems to work; here is my GDT after setting the segment registers and performing a long jump:
Code: Select all
Global Descriptor Table (base=0x00000000c0105040, limit=39):
GDT[0x0000]=??? descriptor hi=0x00000000, lo=0x00000000
GDT[0x0008]=Code segment, base=0x00000000, limit=0xffffffff, Execute/Read, Non-Conforming, 32-bit
GDT[0x0010]=Data segment, base=0x00000000, limit=0xffffffff, Read/Write
GDT[0x0018]=Code segment, base=0x00000000, limit=0xffffffff, Execute/Read, Non-Conforming, 32-bit
GDT[0x0020]=Data segment, base=0x00000000, limit=0xffffffff, Read/Write
(A) Does order matter in terms of setup? Should I be setting GDT and/or IDT first? Currently I setup my GDT and IDT in C (for the most part), calling them from kernel main after paging. I don't set up my stack until paging is enabled, and I don't think doing so earlier would accomplish anything -- since I link at 0xC0000000, the stack top value is 0xc0109070. I assume that calling C functions with that invalid linear stack address before enabling paging would crash. So I think moving those GDT/IDT setup calls to occur before paging could be tricky. I could re-implement my GDT setup in assembly without too much trouble, but the IDT would be tougher.
(B) After I enable paging, I suppose I could still get the physical address of the GDT/IDT if I wanted to by taking their virtual address and manually extracting PD entry --> PT entry --> physical address. If I keep my current setup order, is it better to perform this bit-shifting and use the physical address to load the registers, even though the virtual address seems to be working properly?
(C) If I do set up GDT/IDT before paging, using their linear addresses, do I need to manage them in any way after enabling paging? I should note that after I enable paging, I un-map the identity page table (PD entry 0), so that 1:1 mapping is gone -- virtual address 0x00100000 is no longer mapped to that physical memory. Will segment operations ignore paging and just go straight to the physical address loaded in the GDTR? I would think so, but currently there's a virtual address in there and that is working...
.
(D) On that note, since my current setup DOES work - how is the GDTR working with a virtual address? Or does the lgdt/lidt instruction itself perform the translation from virtual to physical, and just load the physical address?
(2) Does the GDT trick accomplish anything not satisfied by the approach described above?
While I'm on the subject of setting up the GDT for higher half kernels, I wanted to clear up a few thoughts on the GDT trick.Through my own searches on the subject of higher-half kernels, I keep running into references to this approach (though it seems like the actual HigherHalfGDT page, from ~2005, is gone. Is this now seen as a bad approach?).
From what I gather, the idea is that by setting the GDT base of the kernel code/data segments to 0x40000000 instead of 0x0, any higher-half virtual memory references above 0xC0000000 will wrap back around because of 32-bit overflow (e.g. 0x40000000:0xC0100000 --> 0x00100000, which gives us the physical address of our kernel code). This way, we just have to do the identity-mapping of the kernel, and don't have to bother setting the PD[768] entry / going through my approach above. I just wanted to make sure I had the right idea, and see if there is any other benefit. Several people refer to it as a "hack", the page is gone - and it doesn't seem like it saves that much work/complexity - so I just wanted to see what the general consensus was in 2020.
Just to re-iterate -- I know this is a lot of questions! Just want to try and hash this out a bit. Thanks in advance, and I promise I am working on answering these myself.