Virtual Address Space for each process

[XCHG]

I now have two privilege levels in my Operating System. One of course is the PVL/DPL/CPL of 0 for the kernel and another is PVL/DPL/CPL 3 for user processes. These processes along with the kernel are working together fine right now. What I want to do is to start coding a memory manager using paging. I am comfortable with paging and know what I should do with it but the problem is that if I want to implement the feature X for example, then I will need feature Y which itself needs X and Z and it becomes a whole mess of dependency chains.

Now I want to know if anyone can, please, tell me where to start from? Should I start coding hard drive procedures/functions or should I go straight to the memory manager. Is virtual memory effective without page files? Could you guys please give me some pointers of where to start, please?

[AJ]

Hi,

Personally, I think getting the memory manager reliable must be one of the top priorites. If you get in to disk access, you will need memory buffers. If you don't have a memory manager, where do you put these?

In my first kernel attempt, I had a kmalloc routine for my kernel and a malloc() routine for user processes. Now, I have a single malloc() routine which works for both kernel and user processes - I have not yet had problems with this, as I assign user/supervisor pages based on the CPL. Malloc() is part of my libc - the kernel only deals with frame allocation.

It is very easy to get in a muddle with the memory manager. My basic idea to start with was to have a pair of 'every case' allocation and freeing routines which could take a number of flags, such as privilege levels, whether the entire routine should be paged in straight away etc... etc.... kmalloc() and friends were then just inline functions which referenced this big function with varying parameters. This whole thing got quite untidy.

The main thing seems to be: *design* a simple interface initially. Whatever your memory manager does, all the user and kernel modules should only see the basic interface. This helps keep dependencies to an absolute minimum.

Certainly paging works well without a page file. I only plan to worry about page files when my OS and associated tasks get bigger than physical RAM. That's not going to happen at least until I get round to a GUI (months and months away...).

If you have nice device driver and memory management interfaces, adding page file support to your MM shouldn't be too bad (I won't go so far as to say it's trivial...).

Of course, this is probably one of those questions you can answer with 'Well, how do you want to do it?...'

Cheers,
Adam

[jnc100]

The way I do it is as follows:

I have two memory managers - one for allocating various sized chunks of physical memory from 4kB to 4MB using a buddy system and the other just divides up areas of a virtual address space from 0x0 to 0xffffffff. Note that I don't alter any page tables at this stage.

When I wish to create a process, I ask the virtual memory manager for various chunks, for the .text, .data and .bss sections, an arbritrary sized heap and a stack. Still no changing of page tables.

I then write in the entry for the virtual memory manager where to get the information for the .text and .data sections (e.g. a disk location or physical memory location if loaded as a module by grub).

When a page fault is triggered (as it will be when I try to jump to the entry point of the process), the following algorithm is used:

Code: Select all

If user bit of error code is set:
 - if trying to access outside any sections defined for the process, kill the process
 - else: if vmm reports section has bits defined:
       - if on disk:
              - allocate one page of physical memory
              - load from disk to that physical memory location
              - map to requested section
       - else (not on disk, but in memory)
              - map to requested section
    - else (if no bits (.bss, heap or stack))
              - allocate one page of physical memory and zero
              - map to requested section
Else (if kernel access)
 - if outside mapped sections for kernel or user process
    - panic
 - else
    - allocate a page of physical memory
    - map

The physical memory allocator then does the following:

Code: Select all

 - if memory free, return it
 - else: if swap_to_disk not enabled, return NULL
    - else (swap_to_disk available)
             - swap some pages to disk that haven't been used for a while
             - blank the pages
             - free all but one, return that one

The final part is the user space malloc function. I use dlmalloc and implement sbrk as a system call which just increases the size of the section the vmm allocates for the user heap.

Regards,
John.

[XCHG]

Thank you guys. I just finished writing the virtual memory manager for my kernel. Now I have a few questions that I'd appreciate if someone could answer for me:

1) Do I have to create 1024 PDEs and 1048576 PTEs for each process that I create in order to map the whole virtual memory for that process?

2) How should I take advantage of paging without a hard disk driver in order to create virtual address spaces for processes?

3) Is using a byte-map beneficial for assigning protection to each PTE?

4) When exactly does the Page Fault Exception become handy? What kinds of processing should be done on a PFE?


Thank you guys in advance. I appreciate your time.

[INF1n1t]

Hey

I can't answer the first three questions, but the number 4 is mine

4) When exactly does the Page Fault Exception become handy? What kinds of processing should be done on a PFE?

AFAIK, #PF occurs when the processor want to access a page in the memory, but the page isn't there. For example, the operating system wants to free some space temporary (you have a working multitasking operating system with several processes, but the user doesn't use one of them for some time), so some pages will go to a page file (Windows for example does this). When #PF occurs, Windows will try to load the page file from the page file (on the hard drive) back into the memory.

And a little info for the second question:

2) How should I take advantage of paging without a hard disk driver in order to create virtual address spaces for processes?

Hm, paging gives you the opportunity to map virtual address space to physical address space. You could identically map the second 2 GBs of each process (that is, map the process' virtual address space), so they can share the kernel code
Also, you can protect your operating system on a page basis

[Aali]

and i can answer question #1

the short answer is no, you dont

the long answer is, because of the fact that if you mark anything as non-present (clear the present bit) the other bits are ignored, which means you dont have to allocate anything at all until you're actually mapping something to that address

[jnc100]

1) Do I have to create 1024 PDEs and 1048576 PTEs for each process that I create in order to map the whole virtual memory for that process?

In terms of tables, at least for 32bit, that translates to 4 kB for the page directory and 4 MB for the page tables. If a process wants to access all 4 GB of address space, then you really do need 4 MB + 4 KB for paging structures. However, it makes sense to use the same page tables for the kernel in each address space. I allocate 1 GB of space to the kernel, and therefore create 1 MB of page tables on startup which are mapped into the same place in each address space, so when kernel data changes, it changes in each process automatically.

2) How should I take advantage of paging without a hard disk driver in order to create virtual address spaces for processes?

I think you are confusing paging with swapping. Paging (at least how I see it) is mapping physical memory to virtual memory. This doesn't need hard disk access. Swapping is when you take a physical page that isn't used often and save it to disk to free up physical memory for other processes. This is usually accomplished by clearing the present bit in the page tables of the process that uses the particular page you are saving to disk, setting some bit saying its on the disk and using the other bits to determine where in the swap file it is.

3) Is using a byte-map beneficial for assigning protection to each PTE?

I don't quite follow what you mean. Could you provide an example?

4) When exactly does the Page Fault Exception become handy? What kinds of processing should be done on a PFE?

See my description above, with the added check that if the page is not present and has the 'on disk' bit set, then load it from disk to a free page of physical memory (you may have to swap out something else) and map it to where the application expects it to be.

Regards,
John.