OSDev.org

Hi,
I feel akward when ask you guys those questions: can anyone give me a brielf explaination about paging? How does it work on higher half? Why user proccess can use up to 4 GiB while we only have 1 GiB RAM? How does mapping process actually do in Jose's tutorial, chapter 14, section "Proccess Page Table Setup", initproc_init() implementation, line 26, code: Why he map his kernel up to 128 MiB while his kernel only 8 MiB in size? And in every user proccess, what is kernel stack use for and how to map it? I have read a lot of wikis, articles, documents, books and forums, even examine a lot of hobby OSes's code and still not understand.
Best Regards,

Don’t read Jose’s (or anyone else’s) tutorial. Read the documentation from Intel and AMD.

Or just read this Wiki: https://wiki.osdev.org/Paging

Hi,

I have read a lot of wikis, articles, documents, books and forums, even examine a lot of hobby OSes's code and still not understand.

Rather then looking at code, it is more important to know the theory behind it: step away from the computer and work it out. What do you already know and what have you learned? Given you have read a lot of documentation already, what is your best way to describe what paging is and how it works? i.e. if you wanted to map the physical frame at 1MB to the virtual page at 3GB, what do you think needs to happen?

Perhaps if we can pinpoint where the confusion might be we can provide a more through response.

How does it work on higher half? Why user proccess can use up to 4 GiB while we only have 1 GiB RAM?

There are 3 layers here: RAM, the physical address space, and virtual address spaces. Processes have their own separate virtual address space. Paging is to allow us to map virtual > physical (which may not necessarily be RAM).

neon wrote:Rather then looking at code, it is more important to know the theory behind it: step away from the computer and work it out. What do you already know and what have you learned? Given you have read a lot of documentation already, what is your best way to describe what paging is and how it works? i.e. if you wanted to map the physical frame at 1MB to the virtual page at 3GB, what do you think needs to happen?

Hi,
As far as I know, paging is the most important memory management method for morden OSes. Any address that user process sees is all virtual, protect them from conflict with another process's address space and prevent them from reading or writing to another process's address space. With paging, you can use any address you like (i.e 0xC0000000) even your physical memory is lower than that. It is also a good memory protection method (compare to segmentation). In your example, when map physical frame at 1MB to the virtual page at 3GB, the corresponding page directory entry will contain the page table frame number (which is the physical address shift right by 12) and the corresponding page table entry will contain the physical frame number (which is shift right by 12 too).

neon wrote:There are 3 layers here: RAM, the physical address space, and virtual address spaces. Processes have their own separate virtual address space. Paging is to allow us to map virtual > physical (which may not necessarily be RAM).

Thanks for the reply!

Edited: Here is my code on PAE paging: https://github.com/NeonLightions/Amore-OS/tree/master When I enable paging (vmm_init() in src/system/memory/vmm.c), it causes triple fault. Can anyone help me fix this code?

Hi,

Sounds like you got the idea behind it, but will add a few more details.

Why user process can use up to 4 GiB while we only have 1 GiB RAM?: The present bit tells us if a page is "in memory" or not. When this bit is clear, we can use the remainder fields however way we want to by introducing Software PTE types - such as to implement page swapping which allows us to do this.

Assumptions... I am going to assume for now that all that code is at 1MB and that we only need to set up Identity Space, in which case you will probably be fine with just mapping the first page directory entry (first 4MB). Don't worry about mapping anything else. Even if you want a higher half kernel (we can discuss that later), you'll still want the first 4MB as Identity Space anyways (with perhaps an exception for the Zero Page.) From what I can see, I believe this should be fine for your system.

Note on mapping all of memory... The code you posted appears to be trying to map all of the address space -- which isn't what you want to do. The goal is to set up Identity Space and Kernel Space (and possibly others). Nothing else needs to be mapped to memory.

For reference, here is our code for setting up Identity Space. I edited out the irrelevant parts for what you should focus on: That's it. Your first initial paging code shouldn't be any more complex (you would just need to add an extra set of tables for the PDPT.)

Note on vmm_alloc_page... Your vmm_alloc_page calls kheap_temp. This is backwards. When setting up paging, you only need free Page Frame Numbers (PFN)'s from the frame allocator, nothing else. The heap allocator should call the VMM AllocPage if needed and AllocPage should call the PMM GetFreeFrame if it needs to map something into memory. Until the VMM is set up, you should only use the PMM FreeFrame.

Note on bit fields... Compilers can handle bit fields in different ways. Be absolutely sure the compiler is treating bit fields as you expect them too.

Thanks for your advices. Can you explain about page swapping and how to do that?

Hi,

Just create a software PTE (present bit set to 0) that stores the location of the page on disk to some dedicated partition or page file. When the current software attempts to read or write it, a page fault is triggered, the OS discovers its location from the PTE and loads it into memory. If memory starts running low, the system can select pages to move out of memory (thus freeing the selected page.)

But, I wouldn't worry about page swapping for now.

neon wrote:Hi,

Just create a software PTE (present bit set to 0) that stores the location of the page on disk to some dedicated partition or page file. When the current software attempts to read or write it, a page fault is triggered, the OS discovers its location from the PTE and loads it into memory. If memory starts running low, the system can select pages to move out of memory (thus freeing the selected page.)

But, I wouldn't worry about page swapping for now.

Hi,
Thank you for a lot of useful information. I will keep learning. Don't close this thread yet, cause i may return in future. Cya!

Hi,

Sure, there are a lot of tricks to reduce the memory footprint that can only be done with paging -- shared memory (with copy-on-write), page swapping, and demand loading. Even for large applications, you can greatly limit memory by mapping only what is needed and keeping the rest of it on disk. Copy on write and page swapping make use of Software PTE's and shared memory comes naturally with how page tables work. Demand loading makes use of the page fault handler to quickly find and map pages on-demand.

I do tend to consider these somewhat advanced topics though since you should have a good stable OS first.