OSDev.org

This should go on the quick links but I'll post it here because it's relevent to this topic:

Wikipedia is actually getting quite good now. For what you need, search for "Virtual Memory", "Paging" and "Demand Paging" at http://www.wikipedia.org. Each gives a different article. I have only skimmed over them but I'm in the middle of designing this part of my OS so I'll read them properly soon.

Alternatively, heres a direct link to the Virtual Memory category, currently listing 16 topics: http://en.wikipedia.org/wiki/Category:Virtual_memory.

Doing a wikipedia search on any subject you don't understand is well worth a shot.

HTH,
Paul Barker

Thank you all for your help. I hope I'm not too notorious (is this the right expression?).

So, when we're talking of allocating/freeing pages in the pmm, we're only talking about allocating blocks of 4k and not of the pages stored in page directories, etc.? And then, I enabled paging in the vmm setup? I'm really lost, please forgive me...

Candamir wrote: So, when we're talking of allocating/freeing pages in the pmm, we're only talking about allocating blocks of 4k and not of the pages stored in page directories, etc.?

But yes! I mean, somewhere you have to store the information that this 4k block, which happens to be virtual 0x00100000 for process X, actually refers to physical 0xdead000. That's the entry in the page directory.

And then, I enabled paging in the vmm setup?

If you want to. The MMU works on pages, but whether you use the MMU, or provide some way for pages to be swapped out to disk, is up to you.

Quote from: Candamir on 22-Mar-06, 01:54PM
So, when we're talking of allocating/freeing pages in the pmm, we're only talking about allocating blocks of 4k and not of the pages stored in page directories, etc.?

But yes! I mean, somewhere you have to store the information that this 4k block, which happens to be virtual 0x00100000 for process X, actually refers to physical 0xdead000. That's the entry in the page directory.

heh, sry to say that, just some posts before you differentiated all aspects involved very nicely. But now your mixing it up yourself.

Your definetely right that different people often use the same keywords but mean quite different things, but if this shouldn't end up in Candamir being totally confused you should stick to the terms you introduced yourself.

You distinguished between physical memory and virtual memory.
Well, if he asks for the physical memory manager and one follows your definitions and hints you gave before, then (t)his manager has nothing to do with the page tables/ dirs except one thing.
Provided that (you're right, it's all up to him and his design) he uses paging in his kernel, the phys memory manager provides physical addresses of pages and keeps track of their usage (free/used). That should be all.
In case of no paging support this manager this would of course have more responsibilities.

Now if paging is enabled you can't use memory by it's physical address directly anymore (ok there's identity mapping, but that are still virtual addresses being used there).
Because of that you're in need of something that manages the virtual adresses / the addressspace and makes pages you allocated accessable.
Therefore the functionality of arranging / manipulating the page tables/ page dirs should fall under this category.
One may call this a virtual memory manager or not, but it may be distinguished from the physical part by it's basic functionality:
This one does the mapping, as the other one is providing the memory to be mapped.

When you allocate a page in the pmm we should be talking about retrieving a physical address of a page and mark it as used in somekind of book keeping information. That's it for the phys mem part. This address is definetely going to end up in a page dir/ page tab, may be even only temporarily, but that shouldn't be anything the pmm has to do with more than providing the address itself.

And then, I enabled paging in the vmm setup?

Yes, provided you wanted to after this . You would need to have mapped the memory that contains your kernel at a certain address where it is found when you enable paging. (is up to you - look into the faq - identity mapping - higher half kernel)

hope it helps

OZ wrote: heh, sry to say that, just some posts before you differentiated all aspects involved very nicely. But now your mixing it up yourself.

Don't think so. When we are talking about allocating and freeing pages, we are talking about pages in the page directories. The purely physical, completely un-virtual memory management I described as "Step 1" would not bother with pages, but simply record start-of-block and size-of-block.

So, when implementing the pmm, can I just enable paging (page directory and page tables, etc.) and use the physical address stored in bits 31 to 12 of every page table entry? Could I still use them only as references to physical memory and not using virtual addresses in a loop when creating the stack? Or do I have to implement my own stack and just adding 4096 to the addresses? Or do I have to enable paging until later, in the vmm?

BTW, with your help, I finally understood the two virtuals (addressing and memory on harddisk or elsewhere). Thank you.

PS: When enabling paging, do I have to start from physical address 0x0 or from the end of kernel (would make more sense to me). And in all cases, I need the size of RAM in order to not map more pages than my RAM can hold? And then, how can I map any physical page to a virtual address?

Oops... checked this thread only after the other one...

Candamir wrote: So, when implementing the pmm, can I just enable paging (page directory and page tables, etc.) and use the physical address stored in bits 31 to 12 of every page table entry?

You can do whatever you like. But when you enable paging, you're using virtual addressing, whether these addresses happen to match the physical ones or not.

Or do I have to enable paging until later, in the vmm?

You can do whatever you feel like. The three steps I outlined above are for understanding, not necessarily for implementing.

PS: When enabling paging, do I have to start from physical address 0x0 or from the end of kernel (would make more sense to me).

Again, up to you. Most systems use high addresses from the kernel, so applications can start at 0x0 instead of some arbitrary address.

And in all cases, I need the size of RAM in order to not map more pages than my RAM can hold?

You need the size and addresses of RAM so you know where to map your pages to. RAM isn't necessarily consecutive...

And then, how can I map any physical page to a virtual address?

...with the page tables??

Honestly, have you read the Intel docs? It sounds like you're going by tutorials only (A Bad Thing(tm) )...

PS: When enabling paging, do I have to start from physical address 0x0 or from the end of kernel (would make more sense to me).

As Solar stated the design of your addressspace is up to you, but
you have to start from 0x0 anyway by the means of mapping theses pages. Again you got to distinguish between physical locations of pages and their actual virtual mapping.

The memory starting from 0x0 definetely needs to be mapped at a place where you want it to be in your addressspace. As you will lateron need to access device registers etc. It doesn't matter though if you identity map it - accessing it at the same addresses virtual as their physical pendant or use a higher half scheme - all up to you.

The same for the kernel, those pages that hold it need to be mapped in, otherwise you will get a pagefault resulting in a double fault at once when activating paging (as the interrupt handler can't be reached though and that's another page fault). Therefore you need to map all the memory in you use till now, where you map it is totally up to you.

And then, how can I map any physical page to a virtual address?

Have a look at how a virtual address is structured. eg. 0x100000 (1 mb) That would be the address in binary. Now look at how it is split -
10 bit - 10 bit - 12 bit. Think of the maximum values these patterns can hold. 2^10 = 1024 / 2^12 = 4096.

Now if the MMU needs to translate the 0x100000 virtual to the appropriate physical address it does the following.

CR3 holds the physical address of the page that contains the page directory entries. These entries (PDE) are 4 byte in size.
Now back to our address - the PDE part was 10 bit, therefore it can address every PDE that is in a page directory. (the page dir) 4096 / 4 bytes per PDE = 1024 PDEs per page directory).

Natively a 32bit cpu can address 4 gb of memory.
If you calculate how much pages that are (2^32 / 4096) you will get 1048576 pages. The PDEs page frame part is 20 bits.
Now calculate 2^20 and guess what's the page frame part of the PDE for. You will see that this part can address any page of the 1048576. The attributes are for managing access rights and stuff like that.

Now think of the PDE's page frame part as the physical address of a page that is used as a page table. A page table consists of page table entries (PTEs) which got the same format like PDEs.
Therefore there are again 1024 PTEs per page table. And again do these have a page frame part that can address any page of those 1048576 that could be there.
Now back to the virtual address - look at the PTE part - it's 10 bit wide - similiar purpose it's indexing the appropriate PTE.

Using the page frame address of that indexed PTE the MMU now knows the base address of the page that is addressed by that virtual address. Now the last part is the offset part of the virtual address. This is just the offset into this page. 2^12 = 4096 therefore every byte of the page that was pointed to before by the PTE, can be accessed.

As this whole construct works on the physical addresses of pages the virtual addressing is realized through simply putting the physical page that is addressed by 0x100000 in another PTE than that which gets accessed by this mechanism if you access the virtual address 0x100000.

Just as a small calculation example to demonstrate this:
Again the 0x100000 virtual address in binary The MMU then would look up the very first PDE as the index is zero. Then it would take the 256th PTE [0100000000] of that page table.
Now if this would be identity mapping this PTE would contain Which would access the physical page that is addressed at 1 mb.
But as it's totally up to at which virtual address you map which physical page you could also map the page that has 1 gb as physical address here to be accessed at 1 mb virtual.

The other way round is therefore also possible as you can map the 0x100000 phys to be accessed at 1 gb virtual.
Then you would need to calculate which page table and the appropriate index for insertion. I hope you get the point and I didn't tell crap.