questions on high half kernel II

[asmboozer]

I could not understand why pagedir 0 should be set in the os FAQhttp://www.osdev.org/osfaq2/index.php/HigherHalfBareBones

Code: Select all

; loader.asm

global _loader                          ; Make entry point visible to linker.
extern _main                            ; _main is defined elsewhere

; setting up the Multiboot header - see GRUB docs for details
MODULEALIGN equ  1<<0             ; align loaded modules on page boundaries
MEMINFO     equ  1<<1             ; provide memory map
FLAGS       equ  MODULEALIGN | MEMINFO  ; this is the Multiboot 'flag' field
MAGIC       equ    0x1BADB002     ; 'magic number' lets bootloader find the header
CHECKSUM    equ -(MAGIC + FLAGS)  ; checksum required

; This is the virtual base address of kernel space. It must be used to convert virtual
; addresses into physical addresses until paging is enabled. Note that this is not
; the virtual address where the kernel image itself is loaded -- just the amount that must
; be subtracted from a virtual address to get a physical address.
KERNEL_VIRTUAL_BASE equ 0xC0000000                  ; 3GB
KERNEL_PAGE_NUMBER equ (KERNEL_VIRTUAL_BASE >> 22)  ; Page directory index of kernel's 4MB PTE.


section .data
align 0x1000
BootPageDirectory:
    ; This page directory entry identity-maps the first 4MB of the 32-bit physical address space.
    ; All bits are clear except the following:
    ; bit 7: PS The kernel page is 4MB.
    ; bit 1: RW The kernel page is read/write.
    ; bit 0: P  The kernel page is present.
    ; This entry must be here -- otherwise the kernel will crash immediately after paging is
    ; enabled because it can't fetch the next instruction! It's ok to unmap this page later.

I could not understand why it can not fetch the next instruction! so I change the code below

Code: Select all

    dd 0x00000083

to

Code: Select all

     dd  0

Code: Select all

    times (KERNEL_PAGE_NUMBER - 1) dd 0                 ; Pages before kernel space.
    ; This page directory entry defines a 4MB page containing the kernel.
    dd 0x00000083
    times (1024 - KERNEL_PAGE_NUMBER - 1) dd 0  ; Pages after the kernel image.

the run test got the result

Code: Select all

<bochs:8>
Next at t=11464457
(0) [0x001000b0] 0008:0x001000b0 (unk. ctxt): mov ecx, cr0              ; 0f20c1

<bochs:9>
Next at t=11464458
(0) [0x001000b3] 0008:0x001000b3 (unk. ctxt): or ecx, 0x80000000        ; 81c900
000080
<bochs:10>
Next at t=11464459
(0) [0x001000b9] 0008:0x001000b9 (unk. ctxt): mov cr0, ecx              ; 0f22c1

<bochs:11>
Next at t=11464460
(0).[11464460] ??? (physical address not available)
<bochs:12>
bx_dbg_step_over_command:: Invalid physical address
<bochs:13>
bx_dbg_step_over_command:: Invalid physical address
<bochs:14>
bx_dbg_step_over_command:: Invalid physical address

so I have two questions here,
1. why it can not fetch the next instruction after paging enabled if page dir 0 is not set? when & why page dir 0 could be unmapped?

2.
why the test result is as the above??


thanks.

[Colonel Kernel]

The last instruction before paging is enabled is a move into CR0. The address of that instruction is in EIP before it is executed, and that address is physical and somewhere in the lower half (let's say 0x00100004 for the purposes of this example). After the move is executed, the address in EIP is incremented by the size of the move instruction, so it's now 0x00100008 (again, I'm making up numbers for the sake of the example).

The problem is, after you enable paging, the address in EIP is no longer a physical address -- it is a virtual address. This means that if you unmap the page that EIP points to, then you'll get a page fault on the next instruction fetch. In other words, when enabling paging, the processor does not magically translate the contents of EIP into something sensible. This is why you need to keep that lower-half page mapped until after you've jumped to the higher half (i.e. -- loaded EIP with the proper virtual address, something like 0xC0100008 for example).

Does this make sense now?

[asmboozer]

The last instruction before paging is enabled is a move into CR0. The address of that instruction is in EIP before it is executed, and that address is physical and somewhere in the lower half (let's say 0x00100004 for the purposes of this example). After the move is executed, the address in EIP is incremented by the size of the move instruction, so it's now 0x00100008 (again, I'm making up numbers for the sake of the example).

thanks,
i have also tried another paging tutorial.http://www.osdev.org/osfaq2/index.php/HigherHalfWithGdt

i modified the init_paging function.

commented line

Code: Select all

// kernelpagedir[0] = (unsigned long)lowpagetablePtr | 0x3;

and modified the kmain function
commented lines.

Code: Select all

 
// We clear the screen and print our welcome message
        //cls();
        //helloworld();

        // Hang up the computer
        //for (;;);

the test result is below

Next at t=195583592
(0) [0x001000a5] 0008:0xc01000a5 (unk. ctxt): mov cr0, eax ; 0f22c0

<bochs:10>
Next at t=195583593
(0).[195583593] ??? (physical address not available)
<bochs:11>

this code falles in init_paging function.

Code: Select all

c0100020 <init_paging>:
c0100020:       55                      push   %ebp
c0100021:       89 e5                   mov    %esp,%ebp
c0100023:       83 ec 10                sub    $0x10,%esp
c0100026:       c7 45 f4 00 00 00 00    movl   $0x0,0xfffffff4(%ebp)
c010002d:       c7 45 f8 00 00 00 00    movl   $0x0,0xfffffff8(%ebp)
c0100034:       c7 45 fc 00 00 00 00    movl   $0x0,0xfffffffc(%ebp)
c010003b:       b8 00 20 10 c0          mov    $0xc0102000,%eax
c0100040:       05 00 00 00 40          add    $0x40000000,%eax
c0100045:       89 45 f4                mov    %eax,0xfffffff4(%ebp)
c0100048:       b8 00 30 10 c0          mov    $0xc0103000,%eax
c010004d:       05 00 00 00 40          add    $0x40000000,%eax
c0100052:       89 45 f8                mov    %eax,0xfffffff8(%ebp)
c0100055:       c7 45 fc 00 00 00 00    movl   $0x0,0xfffffffc(%ebp)
c010005c:       eb 25                   jmp    c0100083 <init_paging+0x63>
c010005e:       8b 55 fc                mov    0xfffffffc(%ebp),%edx
c0100061:       8b 45 fc                mov    0xfffffffc(%ebp),%eax
c0100064:       c1 e0 0c                shl    $0xc,%eax
c0100067:       83 c8 03                or     $0x3,%eax
c010006a:       89 04 95 00 30 10 c0    mov    %eax,0xc0103000(,%edx,4)
c0100071:       8b 45 fc                mov    0xfffffffc(%ebp),%eax
c0100074:       c7 04 85 00 20 10 c0    movl   $0x0,0xc0102000(,%eax,4)
c010007b:       00 00 00 00 
c010007f:       83 45 fc 01             addl   $0x1,0xfffffffc(%ebp)
c0100083:       81 7d fc ff 03 00 00    cmpl   $0x3ff,0xfffffffc(%ebp)
c010008a:       7e d2                   jle    c010005e <init_paging+0x3e>
c010008c:       8b 45 f8                mov    0xfffffff8(%ebp),%eax
c010008f:       83 c8 03                or     $0x3,%eax
c0100092:       a3 00 2c 10 c0          mov    %eax,0xc0102c00
c0100097:       8b 45 f4                mov    0xfffffff4(%ebp),%eax
c010009a:       0f 22 d8                mov    %eax,%cr3
c010009d:       0f 20 c0                mov    %cr0,%eax
c01000a0:       0d 00 00 00 80          or     $0x80000000,%eax
c01000a5:       0f 22 c0                mov    %eax,%cr0
c01000a8:       c9                      leave  
c01000a9:       c3                      ret    

i also check the eip value

<bochs:7>
Next at t=195583590
(0) [0x0010009d] 0008:0xc010009d (unk. ctxt): mov eax, cr0 ; 0f20c0

<bochs:8>
Next at t=195583591
(0) [0x001000a0] 0008:0xc01000a0 (unk. ctxt): or eax, 0x80000000 ; 0d0000
0080
<bochs:9>
Next at t=195583592
(0) [0x001000a5] 0008:0xc01000a5 (unk. ctxt): mov cr0, eax ; 0f22c0

<bochs:10>
Next at t=195583593
(0).[195583593] ??? (physical address not available)
<bochs:11> info cpu
eax:0x80000011, ebx:0x0002bd20, ecx:0x0001e200, edx:0x000003ff
ebp:0xc0105008, esp:0xc0104ff8, esi:0x0002be3e, edi:0x0002be3f
eip:0xc01000a8, eflags:0x00000086, inhibit_mask:0
cs:s=0x0008, dl=0x0000ffff, dh=0x40cf9a00, valid=1
ss:s=0x0010, dl=0x0000ffff, dh=0x40cf9300, valid=7
ds:s=0x0010, dl=0x0000ffff, dh=0x40cf9200, valid=5
es:s=0x0010, dl=0x0000ffff, dh=0x40cf9300, valid=1
fs:s=0x0010, dl=0x0000ffff, dh=0x40cf9300, valid=1
gs:s=0x0010, dl=0x0000ffff, dh=0x40cf9300, valid=1
ldtr:s=0x0000, dl=0x0000ffff, dh=0x00008200, valid=1
tr:s=0x0000, dl=0x0000ffff, dh=0x00008300, valid=1
gdtr:base=0x00100006, limit=0x17
idtr:base=0x00000000, limit=0xffff
dr0:0x00000000, dr1:0x00000000, dr2:0x00000000
dr3:0x00000000, dr6:0xffff0ff0, dr7:0x00000400
cr0:0x80000011, cr1:0x00000000, cr2:0x00000000
cr3:0x00102000, cr4:0x00000000
done
<bochs:12>

who can tell me why bochs doesn't work at init_paging()function? thanks

[Solar]

???

Code: Select all

// kernelpagedir[0] = (unsigned long)lowpagetablePtr | 0x3;

Did I get this right that you have again removed the mappings?

[asmboozer]

???

Code: Select all

// kernelpagedir[0] = (unsigned long)lowpagetablePtr | 0x3;

Did I get this right that you have again removed the mappings?

yes, i have removed the mapping.
i understand the first tutorial as Colonel Kernel said, but not this one.

[JAAman]

if i understand you correctly, you tried to remove the line which maps the kernel into the lower-half paging tables? this cannot work:

when paging is first enabled, you must have the lower mapping -- even with the GDT trick, you are still running in lower half when paging is enabled

this is one reason i dont like the GDT trick, as it is much more confusing than the other way because its easy to mistakenly think that your running in higher half already when you really arnt

when you use the GDT trick, there are 2 jumps:
one before you enable paging, this makes your code think its running in the upper half by modifying your GDT to alter the translation between logical and linear

and one after you enable paging, to reset your CS.base to 0 (which automatically changes your code to run in the upper-half)

this is why you must have the kernel code loaded into both the upper and lower half when paging is enabled

[asmboozer]

if i understand you correctly, you tried to remove the line which maps the kernel into the lower-half paging tables? this cannot work:

when paging is first enabled, you must have the lower mapping -- even with the GDT trick, you are still running in lower half when paging is enabled

this is one reason i dont like the GDT trick, as it is much more confusing than the other way because its easy to mistakenly think that your running in higher half already when you really arnt

when you use the GDT trick, there are 2 jumps:
one before you enable paging, this makes your code think its running in the upper half by modifying your GDT to alter the translation between logical and linear

and one after you enable paging, to reset your CS.base to 0 (which automatically changes your code to run in the upper-half)

this is why you must have the kernel code loaded into both the upper and lower half when paging is enabled

thansk for your replies,
1,
can I think if paging enabled, but no gdt updated with the gdt tricks, one can not use jmp CS:EIP for the sake of wrong cs:base is selected etc.?


2, if ignore the all the address of concept, they last point to the same physical memory address? if so I do not care what the address type it is. only where is the physical address I care about.

3,as the info cpu dumps the eip register,
the value of it is 0xc0xxyyzz when the exception occurs.

i think the cpu would translate the address into right physical address because now the paging is enabled. and the page directory is rightly set.

would you explains the exception under the context? if you feel too few the context it is, i would give you more.

also if you done so, I would understand you more easier.

thanks!!!

[Colonel Kernel]

this is one reason i dont like the GDT trick, as it is much more confusing than the other way because its easy to mistakenly think that your running in higher half already when you really arnt

I agree whole-heartedly -- that's why I wrote HigherHalfBareBones in the first place. Also, the technique used in HigherHalfBareBones should work just as well on architectures with no segmentation.

also if you done so, I would understand you more easier.

I for one am having a pretty difficult time understanding you. Are you trying to understand how HigherHalfBareBones works? Is there some aspect of it that conflicts with your own design or implementation constraints? What exactly are you trying to accomplish...?

[asmboozer]

this is one reason i dont like the GDT trick, as it is much more confusing than the other way because its easy to mistakenly think that your running in higher half already when you really arnt

I agree whole-heartedly -- that's why I wrote HigherHalfBareBones in the first place. Also, the technique used in HigherHalfBareBones should work just as well on architectures with no segmentation.

also if you done so, I would understand you more easier.

I for one am having a pretty difficult time understanding you. Are you trying to understand how HigherHalfBareBones works? Is there some aspect of it that conflicts with your own design or implementation constraints? What exactly are you trying to accomplish...?

thanks your first post, it let me understand my first question.

I am trying to understand the codes used in the
Higher-half kernel with GDThttp://www.osdev.org/osfaq2/index.php/HigherHalfWithGdt

I don't know why page dir[0] must be set in this tutorial.
so I modified the code,
but the modified code doesn't work, and I don't know the cause.
the boches only complains at init_paging function just when it finished set paging enabled.

you can see my second post,
i already give details on my question,

<bochs:7>
Next at t=195583590
(0) [0x0010009d] 0008:0xc010009d (unk. ctxt): mov eax, cr0 ; 0f20c0

<bochs:8>
Next at t=195583591
(0) [0x001000a0] 0008:0xc01000a0 (unk. ctxt): or eax, 0x80000000 ; 0d0000
0080
<bochs:9>
Next at t=195583592
(0) [0x001000a5] 0008:0xc01000a5 (unk. ctxt): mov cr0, eax ; 0f22c0

notice the line below

<bochs:10>
Next at t=195583593
(0).[195583593] (physical address not available)

the last line is the cause of failure, but I could not understand it.

[asmboozer]

this is one reason i dont like the GDT trick, as it is much more confusing than the other way because its easy to mistakenly think that your running in higher half already when you really arnt

I agree whole-heartedly -- that's why I wrote HigherHalfBareBones in the first place. Also, the technique used in HigherHalfBareBones should work just as well on architectures with no segmentation.

also if you done so, I would understand you more easier.

I for one am having a pretty difficult time understanding you. Are you trying to understand how HigherHalfBareBones works? Is there some aspect of it that conflicts with your own design or implementation constraints? What exactly are you trying to accomplish...?

after you first post, I have understand why page dir[ 0] should be set.
thank u!
now I try to understand why http://www.osdev.org/osfaq2/index.php/HigherHalfWithGdt also needs to set page dir[ 0].

I do my test to understand it.but I fail to yet.

[asmboozer]

what's the problem?

no one can understand me?

or no one knows the answer of my question?

[asmboozer]

this is why you must have the kernel code loaded into both the upper and lower half when paging is enabled

I can not see it is the why you said that the code should have two page dir to be valid(0,0xC0 00 00 00 >> 22).

would you give me the answer according to the context,which can explain why the error appears there, why it is that error message?

thanks.

[Solar]

I indeed have problems understanding your English. Also, I find your way of responding to others confusing - I am never sure whether you really understood the other person. You usually continue attackin the problem from a slightly different angle, without giving feedback on the helpfulness of the previous post, and I can't tell whether the initial problem has been solved (and you're asking something new) or you're still trying to figure out the very same thing.

No offense intended!

That being said, I'm not the right person to ask, anyway - I yet have to write real / protected mode conversion code myself.

[asmboozer]

I indeed have problems understanding your English. Also, I find your way of responding to others confusing - I am never sure whether you really understood the other person. You usually continue attackin the problem from a slightly different angle, without giving feedback on the helpfulness of the previous post, and I can't tell whether the initial problem has been solved (and you're asking something new) or you're still trying to figure out the very same thing.

No offense intended!

That being said, I'm not the right person to ask, anyway - I yet have to write real / protected mode conversion code myself.

maybe my english is too poor to express myself,

so I have listed the problem here, and I think I have detailed my problems in this post.

I said after Colonel Kernel answered my first post, I understand the problem of the first post.

in the 3rd post, i posted my test on higher half with gdt.
my intent is to understand why I have to set page dir 0 in both higher half tutorials.

all other posts are about the problem with gdt.

why I respond you all in this way, I think it's my poor english.

you can not understand me, me either.

I think I have give the details of my problem, but I feel your answer is nothing related to problem.

I do like the answer of Colonel Kernel's first answer.

I am eager to know the answer of problem with gdt.

thanks anyway.

hope this time, you all can understand my meaning.

[JAAman]

i'm having trouble understanding your responses as well, but i will try again:

2, if ignore the all the address of concept, they last point to the same physical memory address? if so I do not care what the address type it is. only where is the physical address I care about.

3,as the info cpu dumps the eip register,
the value of it is 0xc0xxyyzz when the exception occurs.

i think the cpu would translate the address into right physical address because now the paging is enabled. and the page directory is rightly set.

as i thought... this shows a fundumental misunderstanding of the difference -- let me try to explain it better:

2, if ignore the all the address of concept, they last point to the same physical memory address? if so I do not care what the address type it is. only where is the physical address I care about.

while the addresses eventually do become physical addresses, you can never ignore the different types, because they are generated differently, and if you are not translating them correctly, they will not point to the same addresses

i think the cpu would translate the address into right physical address because now the paging is enabled. and the page directory is rightly set.

no, the CPU doesnt care what the physical address is, it will give an error only if the page directory is wrong -- and in this case the page directory is not rightly set, because you have set the wrong page directory:

3,as the info cpu dumps the eip register,
the value of it is 0xc0xxyyzz when the exception occurs.

no -- EIP contains a logical address, which is never seen by the page tables --

first:
EIP -- contains -- 0xc012 3456

modified by GDT entry:
0xc012 3456 + CS.base = 0x0012 3456

lookup in page tables: 0x0012 3456 -- this is the address which must be mapped into the page tables, and you dont have it mapped -- the high address never reaches the page tables

the entry in the page table must then map the address 0x0012 3456 to the physical address 0x0012 3456 where your code must be



later after you modify the GDT to alter your CS.base, and reload it with jmp cs:here, then it will be like this:

first:
EIP -- contains -- 0xc012 3456 -- same as before

modified by GDT entry: -- cs.base has changed, and is now 0
0xc012 3456 + CS.base = 0xc012 3456

lookup in page tables: 0xc012 3456 -- this is the address which must be mapped into the page tables

the entry in the page table must then map the address 0xc012 3456 to the physical address 0x0012 3456 where your code must be

only now are you using the higher half page tables



hopfully this will help -- try to follow what i said on paper, and look at figure 3-1 in the intel volume 3 (tried to find an equivelant in AMD for a different perspective, but they didnt seem to have one)