OSDev.org

Let's say I get a page fault in a user process and it has to grab the memory region from the kernel's heap, but it isn't mapped in the process because I only map the first 2 MB of the kernel's process, that means, no heap.
How do I solve this? I can't come up with anything :/

Hi,

ohboy wrote: Let's say I get a page fault in a user process and it has to grab the memory region from the kernel's heap, but it isn't mapped in the process because I only map the first 2 MB of the kernel's process, that means, no heap.
How do I solve this? I can't come up with anything :/

I think you're confusing completely different things here. Normally an OS has 3 seperate memory managers, one that handles physical memory, another for linear memory and the last for "heap management" (malloc/free).

The physical memory manager handles raw RAM pages, is only really used by the linear memory manager, and is typically implemented within the kernel. There's a variety of different ways a physical memory manager can be implemented...

The linear memory manager handles mapping physical pages into virtual memory, and may include swapping data to/from disk, memory mapped files, etc. In C the linear memory manager is accessed via "mmap" and associated functions. The linear memory manager works with 4 Kb pages (and possibly 2 Mb or 4 Mb pages) of memory only. Typically the linear memory manager is also implemented within the kernel.

The "heap manager" allows areas of the address space to be allocated and freed, in any size and possibly with any alignment. In C this is done via "malloc" and associated functions. The "malloc" function finds and allocates a suitable area in the address space, and then might use the linear memory manager to allocate any pages into that area. On some OS's "malloc" does not allocate any actual memory - "allocation on demand" is used so that memory is only allocated when it is accessed (e.g. via page fault handler). Also some OS's may mark the allocated area as "in use", and only allow allocation on demand to allocate pages for parts that have been marked as "in use" (so that an incorrect pointer results in a page fault instead of allocating a page). Often the heap manager is implemented in a library (not within the kernel).


Now it seems to me you need a physical memory manager, and because you haven't got one you're using the linear memory manager to shift pages from the kernel to user-space instead.


Cheers,

Brendan

I've got a physical memory manager alright, a stack-based page allocator.

My problem is, I keep one heap per process, and the list of memory regions is kept in a linked list on the kernel's heap, and it's only visible to the kernel.
What I want is to access that list without having to switch process. (switching address space on a page fault would be too heavy I think)

That thing you said about allocation on demand interests me, but how would you implement it? Put an "allocation ID" in the free to use bits of the page?

well, a possible solution consist of having 3 heaps:
- userlevel heap (for malloc's)
- kernel-level system-wide heap (for process structures, for instance)
- kernel-level process-specific heap (for the memory regions list, among other things).

Trick is you only need the memory region of the *current* process, and a fault is always started by the current process ...

How would you map a kernel-level heap in a process? Dedicate a virtual memory area just to be used by that heap, and then map on page fault?

ohboy wrote: How would you map a kernel-level heap in a process? Dedicate a virtual memory area just to be used by that heap, and then map on page fault?

something like this, yes ...

Hi,

I've mixed 2 posts here...

ohboy wrote: My problem is, I keep one heap per process, and the list of memory regions is kept in a linked list on the kernel's heap, and it's only visible to the kernel.
What I want is to access that list without having to switch process. (switching address space on a page fault would be too heavy I think)

One solution would be to either limit the kernel's heap to the visible 2 Mb, or expand the visible area. For e.g. my OS has 1 Gb of "kernel space" that is mapped into every address space, so that the kernel can access all of it's data regardless of which process/thread the current address space belongs to.

You could also split the kernel's heap into 2 seperate heaps, where one is visible in all address spaces and the other isn't.

ohboy wrote: How would you map a kernel-level heap in a process? Dedicate a virtual memory area just to be used by that heap, and then map on page fault?

Dedicate an area in each address space for "process-specific" use by the kernel. This isn't much different from dedicating an area in each address space for use by the process (except for the page attributes). For e.g. allocate some memory at the top of the process's part of the address space, and mark it as "system" so that only CPL=0 code can use it. This is probably what I'd do (and I assume probably what Pype.Clicker would do, as he suggested it), but it depends if the data stored in it ever needs to be accessed by the kernel from a different address space.

ohboy wrote: That thing you said about allocation on demand interests me, but how would you implement it? Put an "allocation ID" in the free to use bits of the page?

Almost, but not quite. For "not present" pages the kernel can use 31 bits of the page table entry or 63 bits if it's using PAE (everything except the present bit).


Cheers,

Brendan

How should I do shared virtual memory? Have all processes share the same page tables? Or should the page fault handler figure it out by looking at the kernel's mappings?