OSDev.org

My kernel is loaded at 0xC0000000 virtual (Higher Half Kernel). The kernel heap comes directly after the kernel, and extends up to 0xF0000000 virtual. The kernel heap is used through the kmalloc() function which allocates blocks of memory with a minimum size of 32 bytes.

The current page directory is always at 0xFFBFF000 virtual. The video memory is mapped to 0xFFBFA000. And there are other various things around there such as the GDT and memory bitmap for physical memory manager.

And I use recursive paging so the page tables are mapped to 0xFFC00000 - 0xFFFFFFFF

That is fine, I am quite pleased with how that works.

Now I am implementing multitasking, and am working on setting up page directories for new processes, and I need the physical address of the page table, so I use my physical memory manager to allocate a page for the page directory. I am now intending to temporarily map the page directory to a virtual address such as 0xF0000000 so I can set it up. This made me think about my kernel memory - should I manage the area above the heap better than just statically allocating addresses for specific purposes as I have done up until now?

Would it be worth the time to create a manager for this block of memory above the heap? I was thinking of extending my kernel memory allocator to have a function such as kmalloc_page() which hands out page-aligned pages of virtual address space above the heap.

How does everyone else manage this part of memory? Is static hard-coded allocation a good idea?

Also another question, does the kernel have its own page directory, or does it just reside in the virtual address space of every process without having its own entire address space?

BMW wrote:Would it be worth the time to create a manager for this block of memory above the heap? I was thinking of extending my kernel memory allocator to have a function such as kmalloc_page() which hands out page-aligned pages of virtual address space above the heap.

I started to do that very thing the other day, and thought, "well, why can't my heap have an optional alignment argument?". Your heap needs to dynamically expand anyhow, so I thought it was best to simply take advantage of that. So that's what I'm doing. It definitely makes kalloc a bit more complicated. We'll see how it goes.

EDIT: responding to your other question...

BMW wrote:Also another question, does the kernel have its own page directory, or does it just reside in the virtual address space of every process without having its own entire address space?

The kernel does have it's own page directory. CR3 only switches entering and exiting user<->kernel space. The kernel paging is basically just a generic map of all physical ram (or however you design... simply that it's a map of directly accessible ram with a purpose for the kernel).

mrstobbe wrote:The kernel does have it's own page directory. CR3 only switches entering and exiting user<->kernel space.

Perhaps I misunderstood but if having "Higher Half Kernel", the CR3 switches are not needed when switching between user mode and kernel mode. I think that is the whole point of the design.

Antti wrote:

mrstobbe wrote:The kernel does have it's own page directory. CR3 only switches entering and exiting user<->kernel space.

Perhaps I misunderstood but if having "Higher Half Kernel", the CR3 switches are not needed when switching between user mode and kernel mode. I think that is the whole point of the design.

That entirely depends on if that's the purpose of the higher-half allocation. Splitting allocation brings two fold benefit if the kernel paging is setup see through: physical address + virtual address map all point to the same place. I wasn't saying that his kernel was going to CR3 switch on a context switch, I was saying (or would have rather if I thought it were relevant) that it allows the kernel to drop into lower-half CR3 without worrying about virtual address not mapping where they should. I was just pointing out that the kernel does need it's own paging structure, and that CR3 switches can be necessary at times.

EDIT: clarification.

mrstobbe wrote:Your heap needs to dynamically expand anyhow

Couldn't I just reserve all memory above the kernel (excluding the reserved addresses such as those used by recursive paging and page dir) for the heap?

EDIT: Another question: If each process has the kernel mapped to the higher part of its address space, what happens when the kernel address space is changed (e.g. a new page table is added to page directory)? Do I have to copy the changes to each process's page directory (and disable interrupts in the mean time to ensure no context switch while updating page directories)? Or... wait this is a good idea! Whenever a context switch is performed, I copy the kernel part of page directory of the current process to the next process... would that work?

The simplest way of propagating changes to kernel tables is to allocate the tables at boot (<256 pages worth) and insert their addresses into every process.

thepowersgang wrote:The simplest way of propagating changes to kernel tables is to allocate the tables at boot (<256 pages worth) and insert their addresses into every process.

Well I have a higher half kernel (kernel address space loaded into each process) so the kernel page table addresses are in every process.

So you are saying just commit all of the kernel pages at boot and leave them committed so the kernel's part of the page directory won't change?

Yo:

Nice topic.

@mrstobbe: Higher half kernels are placed into the higher half of every process for the reason that Antti identified, and they do not require address space switches when transitioning from kernel-mode to user-mode or vice-versa.

BMW wrote: Couldn't I just reserve all memory above the kernel (excluding the reserved addresses such as those used by recursive paging and page dir) for the heap?

Yes, you very well can. I do the same: my kernel doesn't have any virtual address space "map" as such. There is a fixed area for the kernel executable, which is architecture dependent, and a fixed area for mapping in the page-tables for the mapping API, which is also arch-dependent. After that, the rest of the kernel's address space is generically allocatable.

Yes, tPG is suggesting that you fill-in all of the kernel's top-level paging structure entries at boot. This ensures that you don't have to propagate changes to all processes when a top-level change is made to the kernel's address space. You can employ other approaches, all of which are just varying degrees of compromise between "map all top-level tables at boot" and "demand map as needed".

--Peace out,
gravaera

gravaera wrote:Yes, tPG is suggesting that you fill-in all of the kernel's top-level paging structure entries at boot. This ensures that you don't have to propagate changes to all processes when a top-level change is made to the kernel's address space. You can employ other approaches, all of which are just varying degrees of compromise between "map all top-level tables at boot" and "demand map as needed".

Would my idea of copying the current kernel page directory part over at each context switch work? That would also make adding processes easier - I wouldn't have to copy it then.

BMW wrote:

gravaera wrote:Yes, tPG is suggesting that you fill-in all of the kernel's top-level paging structure entries at boot. This ensures that you don't have to propagate changes to all processes when a top-level change is made to the kernel's address space. You can employ other approaches, all of which are just varying degrees of compromise between "map all top-level tables at boot" and "demand map as needed".

Would my idea of copying the current kernel page directory part over at each context switch work? That would also make adding processes easier - I wouldn't have to copy it then.

It would work. A nice improvement is to add a 'version number' to your address spaces. Whenever the kernel address space is modified increase the version number and store that number in the process structure. When that process runs next time, if the version number matches there is no need to copy the tables.

gerryg400 wrote:A nice improvement is to add a 'version number' to your address spaces. Whenever the kernel address space is modified increase the version number and store that number in the process structure. When that process runs next time, if the version number matches there is no need to copy the tables.

Nice idea, will do that. Thanks.