Higher half kernels and a few paging questions

[McZ]

I have been looking at the two different Higher half with grub example kernels in the FAQ

HigherHalfBareBones
HigherHalfWithGdt

I have tested HigherHalfWithGdt.

Her is how it setup paging.

Code: Select all

// Declare the page directory and a page table, both 4kb-aligned
unsigned long kernelpagedir[1024] __attribute__ ((aligned (4096)));
unsigned long lowpagetable[1024] __attribute__ ((aligned (4096)));

// This function fills the page directory and the page table,
// then enables paging by putting the address of the page directory
// into the CR3 register and setting the 31st bit into the CR0 one
void init_paging()
{
        // Pointers to the page directory and the page table
        void *kernelpagedirPtr = 0;
        void *lowpagetablePtr = 0;
        int k = 0;

        kernelpagedirPtr = (char *)kernelpagedir + 0x40000000;  // Translate the page directory from
                                                                // virtual address to physical address
        lowpagetablePtr = (char *)lowpagetable + 0x40000000;    // Same for the page table

        // Counts from 0 to 1023 to...
        for (k = 0; k < 1024; k++)
        {
                lowpagetable[k] = (k * 4096) | 0x3;     // ...map the first 4MB of memory into the page table...
                kernelpagedir[k] = 0;                   // ...and clear the page directory entries
        }

        // Fills the addresses 0...4MB and 3072MB...3076MB of the page directory
        // with the same page table

        kernelpagedir[0] = (unsigned long)lowpagetablePtr | 0x3;
        kernelpagedir[768] = (unsigned long)lowpagetablePtr | 0x3;

        // Copies the address of the page directory into the CR3 register and, finally, enables paging!

        asm volatile (  "mov %0, %%eax\n"
                        "mov %%eax, %%cr3\n"
                        "mov %%cr0, %%eax\n"
                        "orl $0x80000000, %%eax\n"
                        "mov %%eax, %%cr0\n" :: "m" (kernelpagedirPtr));
}

it says it maps the first 4mb in the loop well that I undrestand 1024 * 4kb pages would be 4mb. But how does the mapping really work? I have commented a bit of the paging C code above to what I have understand each part of the code does.

At the beginning of the file this two is decleared, this is the real storage place for the pagedir and pagetable (or have I missed something?) this two would then have a vaddr (virtual address) I gess in the higher half where the kernel is mapped

Code: Select all

unsigned long kernelpagedir[1024] __attribute__ ((aligned (4096)));
unsigned long lowpagetable[1024] __attribute__ ((aligned (4096)));

Then this two pointers will be pointing to the paddr (physical address) where it actually are stored in memory.

Code: Select all

kernelpagedirPtr = (char *)kernelpagedir + 0x40000000;
lowpagetablePtr = (char *)lowpagetable + 0x40000000;

This is from the loop, so what I have understand it will put the value (k * 4096) | 0x3; into each "slot" of the page table, well that is self explaining though. so the (k * 4096) will be the paddr and 0x3 is a the flags for the page ( 0x3 is RW and Present bit )

Code: Select all

lowpagetable[k] = (k * 4096) | 0x3; 

Self explaining pice of code, will clear the entire page dir.

Code: Select all

kernelpagedir[k] = 0;

Then after the loop this two are set into the page dir, this I don't understand yet. but the lowpagetablePtr is the paddr for the above pagetable and then 0x3 is the same flags as above. so this will actually map an entire 4mb page that has 4kb subpages mapped above.

Code: Select all

kernelpagedir[0] = (unsigned long)lowpagetablePtr | 0x3;
kernelpagedir[768] = (unsigned long)lowpagetablePtr | 0x3;

but why 0 and 768? the code comments says it is 0-4mb and 3072-3076MB, and why put the same info in both of them?

Then there is the ASM code that actually do the magic.

So now paging is enabled, and if I use any address after this that address would be a virtual address and the MMU will look into the pagedir/pagetables to find the physical address and then read/write to that address even though the application thinks it writing to another place.. valid virtual addresses would be addresses below 4mb and between 3072 and 3076mb altough they actually are the same place in the physical memory.

if I have understand everything ok why can't I remove the line where I map the first 4mb and leave that part unmapped?

edit: or if I can what can't I do if I remove that line?

edit 2: I can access videomem at both 0xB8000 and 0xC00B8000 isn't that a result of the mapping above that these two vaddr points to the same paddr.

[JAAman]

but why 0 and 768? the code comments says it is 0-4mb and 3072-3076MB, and why put the same info in both of them?

i don't think you understand what its doing:

if I have understand everything ok why can't I remove the line where I map the first 4mb and leave that part unmapped?

when you enable paging, your code is running in the low block (the first 4MB), after you enable paging...your still running in the first 4MB, so you must map the first 4MB, or you will get a #PF! so that is why you cannot remove the line that maps the first 4MB

after you have paging set up, you jump to the higher address (which was also set up to point to the same physical address -- so that doesn't actually change where the code is executing, but rather changes the virtual address where the CPU thinks its executing -- so the upper part must also be mapped to the same page

once you are executing at the higher address, you can unmap the lower address (or reasign it to another section of code or data), but until the jump it must be mapped, therefore you cannot remove the code mapping it, or you will recieve a #PF (and a tripple-fault if you don't have your IDT setup yet or if you still have it mapped to lower addresses)

[McZ]

Ahhh thank you

So then I need to jump to the high address, I thought the jumping part was already written in the example code but now when I look at the inline assembly code there is no jumping anywhere.

But if I don't understand wrong it is still running in the lower part of the memory because there is no jump to the higher part of memory. so actually the example code in the FAQ is incomplete.

EDIT:
my inline assembly is no good so how can I make the far jump in inline asm in this codeblock, or where can I read more about inline asm in GCC

Code: Select all

asm volatile (  "mov %0, %%eax\n"
         "mov %%eax, %%cr3\n"
         "mov %%cr0, %%eax\n"
         "orl $0x80000000, %%eax\n"
         "mov %%eax, %%cr0\n" :: "m" (kernelpagedirPtr));

[McZ]

no one can help me with jumping in inline asm to make my code to really run at the higher half?

Code: Select all

asm volatile (  "mov %0, %%eax\n"
        "mov %%eax, %%cr3\n"
        "mov %%cr0, %%eax\n"
        "orl $0x80000000, %%eax\n"
        "mov %%eax, %%cr0\n" :: "m" (kernelpagedirPtr));

[Pype.Clicker]

honnestly, i'd discourage anyone to switch operating modes with inline assembly. You should rather do that in a dedicated .s (or .asm) file ...

[McZ]

I just began to write an enable_paging function in a separate .asm file instead so I can remove the inline asm.

how can I make sure that I have successfully enabled paging? and how can I make sure that my code really is running in the higer half?

Edit :

This is my new asm function in NASM

Code: Select all

[BITS 32]            ; 32 bit code
[global enable_paging]   ; make 'enable_paging' accessible from C code

enable_paging:
   ; Write cr3
   push ebp
   mov ebp, esp
   mov eax, [ebp+8]
   mov cr3, eax
   
   ; read cr0
   mov eax, cr0
   
   ; write cr0
   or eax, 0x80000000
   mov cr0, eax
   pop ebp

        ; Need to jump here

flush2:
        ret

how can I make the jump, in for example the bootsector I do I jump like this

Code: Select all

jmp 0x7c00:start
:start
    ....

I assume I will need to do something like that, but whit what code do I have to replace the 0x7c00 with? the address where I have the kernel 0xC0000000 doesn't work

[Pype.Clicker]

you may want to read about selectors and friends in the FAQ.

Your 'jmp <segment>:<offset>' is still allowed, but instead of having the "physical address / 16" as segment value, it's now the value of a selector pointing to a proper descriptor in the GDT where the base address is.

E.g. if you want to jump to offset 0xC0000000 in a segment that starts at physical address 0x40100000, you need a descriptor with base=0x40100000 (e.g. in entry #1 of the GDT) and then you do

Code: Select all

jmp 0x8:0xc0000000

how can I make sure that I have successfully enabled paging?

by running code ... for instance if you wrote 0xcafe at physical address X and expect that address to be mapped at Y once paging is enabled (make sure X!=Y for a proper tests), read memory at Y and see if '0xcafe' shows up

[McZ]

I have been playing around with the
HigherHalfWithGdt
example code from the FAQ. And in that code the GDT is setup AFTER paging is setup.

looks something like this:

Code: Select all

/* FIRST enable paging and THEN load the real GDT! */
init_paging();
init_gdt();

do I have to initialize paging before the GDT because GRUB enters pmode for me?

[Pype.Clicker]

look closer at the "ASM" part -- which is what's invoked first. You'll notice that GRUB's GDT is replaced with the so-called "trickGDT" quite early. Then the paging is enabled, and then only the "definitive" GDT is defined.

[McZ]

I forgot about the trick gdt.

why does the trick gdt has the base at 0x4000000 not 0x40100000 or why do I have to have 0x40100000 as base and not 0x40000000 as it already is in the code?

[Pype.Clicker]

you're referring at

E.g. if you want to jump to offset 0xC0000000 in a segment that starts at physical address 0x40100000, you need a descriptor with base=0x40100000 (e.g. in entry #1 of the GDT) and then you do

?

Don't take my words as if they were a prophecy here: i've never toyed with higher half myself so i can only highlight stuffs to make the concepts clearer, but i cannot tell you what precise values you should use to get the magic working ...

[McZ]

is there a way to check where my code is executing or how I should explain it, so I can see that I'm really working in the higher half of the memory.

[Pype.Clicker]

you may want to check that using the bochs. E.g. in Clicker, all my user programs starts running with CS=F and EIP=0.
so

Code: Select all

00000000000i[     ] reset of 'speaker' plugin device by virtual method
00000000000i[     ] set SIGINT handler to bx_debug_ctrlc_handler
Next at t=0
(0) [0xfffffff0] f000:fff0 (unk. ctxt): jmp far f000:e05b         ; ea5be000f0
<bochs:1> vbreak 0xf:0
<bochs:2> c
00000004163i[BIOS ]  rombios.c,v 1.130 2005/02/13 08:47:30 vruppert Exp $

and then i get

Code: Select all

(0) Breakpoint 1, 0x1000000 (0xf:0x0)
Next at t=861750045
(0) [0x010ab000] 000f:00000000 (unk. ctxt): nop                       ; 90
<bochs:3> 

in the first user-level program. it is running with CS:EIP=f:0 and on physical page 0x010ab000.

Now the second program is

Code: Select all

(0) Breakpoint 1, 0x1000000 (0xf:0x0)
Next at t=863862841
(0) [0x010f3000] 000f:00000000 (unk. ctxt): push ebp                  ; 55
<bochs:4> 

same virtual address, other physical address. In your case, you should see the code stepping on contiguous physical addresses (unless you jump to some other page, but usually people just far jump to the next byte ;D) while the CS:EIP location will switch from e.g. "0x0018:40100123" to "0x0008:C010012A" ...

it's not _required_ to change selector, but it may be helpful to quickly figure out what's going on (e.g. maintaining one selector with the base-trick and another one with 0-base that you use after paging is enabled may make things clearer than rewriting and reloading segments).

[JoeKayzA]

is there a way to check where my code is executing or how I should explain it, so I can see that I'm really working in the higher half of the memory.

Using the emulator's debugging mechanism is probably better, but on real hardware, you could also do a 'call' to the next instruction (just put a dummy label at the next instruction), pop the return address off the stack and then do what you want with it (I recommend printing it ). You get the linear address of the next instruction after the call then.

cheers Joe

[McZ]

how does the bochs debugger work? do I set a vbreak to seg:off which is the virtual address where the break should be, is this address an address that where the code will execute?

for example

if the code would look something like this and I set vbreak to 0xc02 and the jump on the line before 0xc02 would jump over 0xc02 so then my break will not trigger?

Code: Select all

0xc01 : jmp 0xc05
0xc02 : ... *** break point
...
0xc05 : ...