Conflicting information about paging

[Caleb1994]

Hello, I have been struggling for the past couple days trying to work out exactly how to setup paging. According to http://wiki.osdev.org/Paging , the "frame" fields in the page directory entry and page table entry hold a 4KB alligned physical address to either the page table, or the block of memory a page manages (respectively of course). I assume that when they say, "4kb aligned" that it means that 0x00000 refers to address 0x00000000 and 0x00001 refers to address 0x00001000. All other bits are used for flags, and such.

But, the tutorial that this page suggests: http://wiki.osdev.org/Tutorials (JamesM's tutorial to be exact)
defines a page directory like so:

Code: Select all

typedef struct page_directory
{
   /**
      Array of pointers to pagetables.
   **/
   page_table_t *tables[1024];
   /**
      Array of pointers to the pagetables above, but gives their *physical*
      location, for loading into the CR3 register.
   **/
   u32int tablesPhysical[1024];
   /**
      The physical address of tablesPhysical. This comes into play
      when we get our kernel heap allocated and the directory
      may be in a different location in virtual memory.
   **/
   u32int physicalAddr;
} page_directory_t;

ignoring the extra stuff he adds (which doesn't matter), the page_table_t* array doesn't make sense. Shouldn't this (if the OSDev page was correct) be a page_directory_entry (not defined there, but this would be the 32bit value that the paging page given above talks about). At first I thought maybe those flags weren't needed, but they are! One is the present flag, which, without that the cpu will throw a page fault.

Also, that frame address should be "4kb aligned" as talked about earlier. Here it's just a regular address. I am thoroughly confused, since this seems to be one of the two tutorials that I've seen pop up the most (this and this one: http://www.brokenthorn.com/Resources/OSDev18.html ) and I would assume with users who follow it (and a link from this wiki) that it would be correct.

Now, granted, I have been staring at this for 2 days, but I think I'm reading it correctly. lol

If I have just been starring at it too long, smack me, turn me in the right direction, and disregard this post lol

[bluemoon]

I assume that when they say, "4kb aligned" that it means that 0x00000 refers to address 0x00000000 and 0x00001 refers to address 0x00001000. All other bits are used for flags, and such.

When it is about system programming, never assumes, check manuals.
By 4kb aligned it means the lowest 12 bits in the page entry is not used for addressing, but as flags.
whatever you put in the entry, only the high bits are used for addressing thus it can only refer to aligned block.

[Yargh]

JamesM's tutorial is a bit strange in how it handles paging/usermode. If I remember correctly, that tutorial allocates a page-aligned chunk of memory after the kernel and places a PDE struct there. Since the fields that differ between the PTE and PDE are only used for special features of the CPU, they aren't needed in the given tutorial which focuses on 32-bit. If you looked at the code in the tutorial, it sets the required fields in the PDE, as they are in the same bit location (in the dword that is the entry) as the ones in the PTE. If the PDE is page-aligned, the page tables are also all aligned as they are 4kb (size of 1 page) each and start on a page boundary.

Correct me if I'm wrong, as I'm sorta in the same situation.

[Caleb1994]

Berkus 1: As I said, this field wasn't important to my question. It's not part of what the CPU uses in the Page Directory. It's extra stuff that he added which is apperantly useful during cloning of a page (if I remember correctly. I don't have the tutorial in front of me)
Berkus 2: I did read the tutorial. He said that array is an array of addresses to the page tables, but never said anything about the flags in each of those entries that need to be set (namely the present flag).

Yargh: Could you point me to the code (or maybe just the function would be helpful) where he initializes these bits? I don't see it. I download the source tarball, and looked through the tutorial on the website, can all I saw was him assigning the allocated addresses and using memset(..., 0, ....) for initialization (where "..." are the other variables he used for size and address). Yes it seems I had the wrong idea of what "page aligned" meant. sorry. lol

bluemoon: so was my idea wrong? I couldn't tell if you were just giving me general advice (which was good advice btw) or if you also meant that my assumption was wrong. lol If it was, how do these addresses fully refer to the memory? You can't refer to all 4GB of memory with 20 bits (you can refer to (2^19 * 2 - 1 = 1048575) bytes, which is not 4GB).

[bluemoon]

You can actually refer to the full 32-bit addressing space with 20 most significant bits plus 12 zeros,
If it helps with example, think about this:

In the traditional 32bit 2 level page structures,
each page directory account for a 4M chunk in the 4G addressing space (4G divided by 1024 entry = 4M each)
for address 0xdeadbeef (11011110 10101101 10111110 11101111)
it corresponding to the 890'th entry (starting from zero)
So, the MMU look-up PDE[890] and fetch a 32bit number,
for such 32-bit number:
- the lowest bits define the flag and attributes, eg is it a chunk of 4M memory or it reference to a 2nd level page table
- the high 20 bits denote the target address, say, 0x12345XXX means 0x12345000, which is 4K aligned address.

The page table works in similar manner.
So the MMU will eventually get a 4KB aligned address, (say, 0x12345000)

Finally, 0xdeadbeef translated to physical 0x12345000 | (0xdeadbeef & 0xFFF) = 0x12345eef

[Caleb1994]

Oh! I was dropping the last 3 hex digits off, which makes it a totally different number! 0xFFFFF is 1048575 and 0xFFFFF000 is 4294963200 :O Okay. I see what you mean, but could you direct me to the code where he initializes those flags? I have some functions that I can do it with, and I can figure out where to put it, but I would like to see where he does it, and his implementation, since I"m following his tutorial xD oh and "He/His" is refering to the JamesM tutorial xP Thanks, again!

[bluemoon]

A quick look on the tutorial I see the code:

Code: Select all

dir->tablesPhysical[table_idx] = tmp | 0x7; // PRESENT, RW, US.

If you follow the tutorial and read carefully, you should be able to locate it.

The reason you not recognize the 0x7 I think you have not fully read the intel manuals.
Remember, tutorials are supplement, no tutorial can or intent to replace the manual.

[Caleb1994]

I assume you haven't read the code enough or did not put any thought to it.

Read the code? Yes. Read the code ENOUGH? Apparently not... Totally my fault. Sorry lol

If you follow the tutorial and read carefully, you should be able to locate it.

Yes, I should have been able to... I deserve whatever criticism you wish

I think I had looked at multiple page table allocation functions, and my brain just took a break at that part . Essentially, I thought too much into it. I do that sometimes. I make things much harder then they need to be. lol

The reason you not recognize the 0x7 I think you have not fully read the intel manuals.
Remember, tutorials are supplement, no tutorial can or intent to replace the manual.

I have not read them fully. I should probably do that. Ha Thanks for the advice!


EDIT:

Well, now I feel really stupid. I switched to the tab with the tutorial in it (I hadn't closed it) and right there, in front of my face was this sentence:

The physical location gets stored in 'tablesPhysical' (after several bits have been set telling the CPU that it is present, writeable, and user-accessible), and the virtual location is stored in 'tables'.

...

[Caleb1994]

The manuals do help a lot guys I got it figured out. I am now in paging mode! I had a little struggle at the beginning (My addresses were mapping fine until 0xFF, and after that the frame addresses were being reset to 0x0! :O, read on...) but with the help of GDB I found my problem!

I was shifting the frame addresses in my PTE/PDE (Page * Entry) functions. I was doing it like so (from my days before the manuals ):

Code: Select all

frame = (frame << 12) & PTE_FRAME;
pte |= frame;

I took out the shifting, and my "vmm_get_phys" returns the correct physical address for a given virtual one (after I map it) and no more Triple Faults! A screen shot of my small kernel is attached. It shows an example of allocating physical blocks, and accessing them without page faults (which means my map function works correctly )

Thanks again, this forum has been a life saver!