Paging Question

[Abunada]

Hi

I think my kernel has really come a long way, I implemented drivers for the keyboard, mouse and the VGA. Unfortunately my kernel still runs with a flat memory model with no paging. I have implemented processes but they have some serious flaws that just can't be solved without paging. I would like to enable paging and give each process its virtual address space. I read the Intel manuals, and found it pretty easy; However, when I looked at James Molley's Tutorial, I saw some code that confused me!!!

Code: Select all

typedef struct page_directory
{
   page_table_t *tables[1024];
   u32int tablesPhysical[1024];
   u32int physicalAddr;
} page_directory_t;

This is the way he represented his page directory, As far as I know the page directory is an array of PTE which is a descriptor not merely "pointers to page tables". I mean I am quoting from the Intel manual volume 3 section 3.6.2:

Page directory — An array of 32-bit page-directory entries (PDEs) contained in a 4-KByte page. Up to 1024 page-directory entries can be held in a page directory.

Does any body have an explanation.
Thanks
Cheers Abdullah

[Abunada]

Anyone

[manonthemoon]

Anyone

Have patience, you haven't even waited 3 hours...

As far as I know the page directory is an array of PTE which is a descriptor not merely "pointers to page tables"

You are correct in your interpretation of the Intel manuals.

What JamesM is doing is creating a structure that contains extra data to be used by his memory manager. The "tablesPhysical" is the actual page directory entry containing the descriptors explained in the Intel Manuals. The array of pointers gives the virtual addresses of those page tables, so that you can directly modify them. When loading a page directory into CR3, you would use the physicalAddr, which points to tablesPhysical (the actual page directory for the processor). The "pointers to page tables" are just for convenience in the kernel.

EDIT: This is because the CPU needs physical addresses, but the kernel can only access memory using virtual addresses. There are other workarounds for this, but this is the one JamesM chose.

[Creature]

The entries are allocated by his placement address memory allocator (kmalloc_ap). That way the address is automatically page-aligned and the physical address is returned. The physical address is then stored inside the 'tablesPhysical' array with the page flags it needs. This probably isn't the only way of doing it, and I believe it has been suggested before that some of the code is totally unnecessary. But the point is, it works and it's much more easily understandable than the Intel manuals, and that's what counts for (mostly) beginners that read his tutorials.

[Abunada]

Finally I understand, Thanks for the replies and *phew* there aren't any contradictions. I am absolutely sorry guys, I should have read the tutorial before posting, since the only thing I do is look at code all over, and apparently his code has passed by. However, I just wanted to ask whether a good workaround is to map the kernel code one-to-one with physical memory. this way the virtual and physical addresses will be the same, therefore the kernel can access its structures easily.

Or am I missing something here!
Thanks Again
Cheers, Abdullah

[Abunada]

Oh, and one more thing I want you guys to clarify is the bit order in the page structure, I am assuming this is a PTE, but the ordering I read in the Intel manuals was different, here is how he defined his structure:

Code: Select all

typedef struct page
{
   u32int present    : 1;   // Page present in memory
   u32int rw         : 1;   // Read-only if clear, readwrite if set
   u32int user       : 1;   // Supervisor level only if clear
   u32int accessed   : 1;   // Has the page been accessed since last refresh?
   u32int dirty      : 1;   // Has the page been written to since last refresh?
   u32int unused     : 7;   // Amalgamation of unused and reserved bits
   u32int frame      : 20;  // Frame address (shifted right 12 bits)
} page_t;

What is written in Intel's manual volume 3A Section 3.7.6 is that there is two bits between the user and accessed bits (namely PWT, PCD). Apperently he combined them with the unused bits; doesn't bit ordering count?

Thanks again guys, really appreciate your help!
Cheers, Abdullah.

[Creature]

James doesn't include all the members in his structure. Instead, the values you'll probably be missing are in the 'unused' field. For example, here's my paging structure. I'm using the same setup as JamesM in his tutorial, except the missing bits should be defined:

Code: Select all

/* Structure defining a page. */
struct Page
{
	unsigned		Present			: 1;	/* Indicates whether this Page is present. */
	unsigned		ReadWrite		: 1;	/* In what ways is this Page accessible? Set = Read-Write, Unset = Read-Only. */
	unsigned		UserMode		: 1;	/* Indicates whether the Page is supervisor mode only (Cleared) or not (Set). */
	unsigned		WriteThrough	: 1;	/* Controls whether write-through caching is enabled (set to 0 for write-back caching). */
	unsigned		DisableCache	: 1;	/* Can the page be cached? */
	unsigned		Accessed		: 1;	/* Indicates whether the Page has been accessed (since the last refresh). */
	unsigned		Dirty			: 1;	/* Indicates whether the Page is dirty (has been written to [since the last refresh]). */
	unsigned		Size			: 1;	/* Size of the page. If 0, the size is 4 kB (PAGE_SIZE). */
	unsigned		Global			: 1;	/* Is this page global (can be used to prevent flushing when i.e. task-switching occurs). */
	unsigned		Unused			: 3;	/* 3 Unused bits, can be used for other purposes. */
	unsigned		FrameAddress	: 20;	/* 20 Bits representing the frame address of the page (shifted 12 bits to the right). */
} PACKED;

Or it could just be wrong, I don't really remember. I do remember that someone posted about it a while ago and James replied to it, it should be somewhere on the forums.

[Abunada]

I know that the bits that are missing are in the unused portion of the structure, however, what I am talking about is the actual ordering of the bits. For example your page structure has the bits layout just as the intel manual says, i.e. you have the WriteThrough (PWT) and DisableCache (PCD) bits in the right place (between the UserMode Bit and the Accessed Bit).

Anyway Thanks For The Reply
Abdullah

[manonthemoon]

... but the ordering I read in the Intel manuals was different...doesn't bit ordering count?

Yes. Follow the Intel Manuals--they are always correct.

However, I just wanted to ask whether a good workaround is to map the kernel code one-to-one with physical memory. this way the virtual and physical addresses will be the same, therefore the kernel can access its structures easily.

No. That is a workaround, but it's not the best way to do it. If you want a Higher Half Kernel, then it's not possible. I strongly suggest "fractal mapping", where you map a PDE to point to the page directory itself. It is briefly explained in the JamesM tutorial:

One solution to this problem is to never access your page tables directly, but to map one page table to point back to the page directory, so that by accessing memory at a certain address you can see all your page tables as if they were pages, and all your page table entries as if they were normal integers.

However, it incorrect says that it wastes 256 MB of virtual memory space (only 4MB, actually).

[egos]

However, I just wanted to ask whether a good workaround is to map the kernel code one-to-one with physical memory. this way the virtual and physical addresses will be the same, therefore the kernel can access its structures easily.

Only at startup time, before the moving/mapping kernel into virt addr space at high adresses.

I strongly suggest "fractal mapping", where you map a PDE to point to the page directory itself.

Yes, you're right, it's professional solution. But one should remember that this technology requires a kernel protection on PDE level. Else user space will be invalid. For example, I have self link in last PDE.