Kernel Virtual Address Space Management
Kernel Virtual Address Space Management
My kernel is loaded at 0xC0000000 virtual (Higher Half Kernel). The kernel heap comes directly after the kernel, and extends up to 0xF0000000 virtual. The kernel heap is used through the kmalloc() function which allocates blocks of memory with a minimum size of 32 bytes.
The current page directory is always at 0xFFBFF000 virtual. The video memory is mapped to 0xFFBFA000. And there are other various things around there such as the GDT and memory bitmap for physical memory manager.
And I use recursive paging so the page tables are mapped to 0xFFC00000 - 0xFFFFFFFF
That is fine, I am quite pleased with how that works.
Now I am implementing multitasking, and am working on setting up page directories for new processes, and I need the physical address of the page table, so I use my physical memory manager to allocate a page for the page directory. I am now intending to temporarily map the page directory to a virtual address such as 0xF0000000 so I can set it up. This made me think about my kernel memory - should I manage the area above the heap better than just statically allocating addresses for specific purposes as I have done up until now?
Would it be worth the time to create a manager for this block of memory above the heap? I was thinking of extending my kernel memory allocator to have a function such as kmalloc_page() which hands out page-aligned pages of virtual address space above the heap.
How does everyone else manage this part of memory? Is static hard-coded allocation a good idea?
Also another question, does the kernel have its own page directory, or does it just reside in the virtual address space of every process without having its own entire address space?
The current page directory is always at 0xFFBFF000 virtual. The video memory is mapped to 0xFFBFA000. And there are other various things around there such as the GDT and memory bitmap for physical memory manager.
And I use recursive paging so the page tables are mapped to 0xFFC00000 - 0xFFFFFFFF
That is fine, I am quite pleased with how that works.
Now I am implementing multitasking, and am working on setting up page directories for new processes, and I need the physical address of the page table, so I use my physical memory manager to allocate a page for the page directory. I am now intending to temporarily map the page directory to a virtual address such as 0xF0000000 so I can set it up. This made me think about my kernel memory - should I manage the area above the heap better than just statically allocating addresses for specific purposes as I have done up until now?
Would it be worth the time to create a manager for this block of memory above the heap? I was thinking of extending my kernel memory allocator to have a function such as kmalloc_page() which hands out page-aligned pages of virtual address space above the heap.
How does everyone else manage this part of memory? Is static hard-coded allocation a good idea?
Also another question, does the kernel have its own page directory, or does it just reside in the virtual address space of every process without having its own entire address space?
Currently developing Lithium OS (LiOS).
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
Re: Kernel Virtual Address Space Management
I started to do that very thing the other day, and thought, "well, why can't my heap have an optional alignment argument?". Your heap needs to dynamically expand anyhow, so I thought it was best to simply take advantage of that. So that's what I'm doing. It definitely makes kalloc a bit more complicated. We'll see how it goes.BMW wrote:Would it be worth the time to create a manager for this block of memory above the heap? I was thinking of extending my kernel memory allocator to have a function such as kmalloc_page() which hands out page-aligned pages of virtual address space above the heap.
EDIT: responding to your other question...
The kernel does have it's own page directory. CR3 only switches entering and exiting user<->kernel space. The kernel paging is basically just a generic map of all physical ram (or however you design... simply that it's a map of directly accessible ram with a purpose for the kernel).BMW wrote:Also another question, does the kernel have its own page directory, or does it just reside in the virtual address space of every process without having its own entire address space?
Re: Kernel Virtual Address Space Management
Perhaps I misunderstood but if having "Higher Half Kernel", the CR3 switches are not needed when switching between user mode and kernel mode. I think that is the whole point of the design.mrstobbe wrote:The kernel does have it's own page directory. CR3 only switches entering and exiting user<->kernel space.
Re: Kernel Virtual Address Space Management
That entirely depends on if that's the purpose of the higher-half allocation. Splitting allocation brings two fold benefit if the kernel paging is setup see through: physical address + virtual address map all point to the same place. I wasn't saying that his kernel was going to CR3 switch on a context switch, I was saying (or would have rather if I thought it were relevant) that it allows the kernel to drop into lower-half CR3 without worrying about virtual address not mapping where they should. I was just pointing out that the kernel does need it's own paging structure, and that CR3 switches can be necessary at times.Antti wrote:Perhaps I misunderstood but if having "Higher Half Kernel", the CR3 switches are not needed when switching between user mode and kernel mode. I think that is the whole point of the design.mrstobbe wrote:The kernel does have it's own page directory. CR3 only switches entering and exiting user<->kernel space.
EDIT: clarification.
Re: Kernel Virtual Address Space Management
Couldn't I just reserve all memory above the kernel (excluding the reserved addresses such as those used by recursive paging and page dir) for the heap?mrstobbe wrote:Your heap needs to dynamically expand anyhow
EDIT: Another question: If each process has the kernel mapped to the higher part of its address space, what happens when the kernel address space is changed (e.g. a new page table is added to page directory)? Do I have to copy the changes to each process's page directory (and disable interrupts in the mean time to ensure no context switch while updating page directories)? Or... wait this is a good idea! Whenever a context switch is performed, I copy the kernel part of page directory of the current process to the next process... would that work?
Currently developing Lithium OS (LiOS).
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
- thepowersgang
- Member
- Posts: 734
- Joined: Tue Dec 25, 2007 6:03 am
- Libera.chat IRC: thePowersGang
- Location: Perth, Western Australia
- Contact:
Re: Kernel Virtual Address Space Management
The simplest way of propagating changes to kernel tables is to allocate the tables at boot (<256 pages worth) and insert their addresses into every process.
Kernel Development, It's the brain surgery of programming.
Acess2 OS (c) | Tifflin OS (rust) | mrustc - Rust compiler
Currently Working on: mrustc
Acess2 OS (c) | Tifflin OS (rust) | mrustc - Rust compiler
Currently Working on: mrustc
Re: Kernel Virtual Address Space Management
Well I have a higher half kernel (kernel address space loaded into each process) so the kernel page table addresses are in every process.thepowersgang wrote:The simplest way of propagating changes to kernel tables is to allocate the tables at boot (<256 pages worth) and insert their addresses into every process.
So you are saying just commit all of the kernel pages at boot and leave them committed so the kernel's part of the page directory won't change?
Currently developing Lithium OS (LiOS).
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
- gravaera
- Member
- Posts: 737
- Joined: Tue Jun 02, 2009 4:35 pm
- Location: Supporting the cause: Use \tabs to indent code. NOT \x20 spaces.
Re: Kernel Virtual Address Space Management
Yo:
Nice topic.
@mrstobbe: Higher half kernels are placed into the higher half of every process for the reason that Antti identified, and they do not require address space switches when transitioning from kernel-mode to user-mode or vice-versa.
Yes, tPG is suggesting that you fill-in all of the kernel's top-level paging structure entries at boot. This ensures that you don't have to propagate changes to all processes when a top-level change is made to the kernel's address space. You can employ other approaches, all of which are just varying degrees of compromise between "map all top-level tables at boot" and "demand map as needed".
--Peace out,
gravaera
Nice topic.
@mrstobbe: Higher half kernels are placed into the higher half of every process for the reason that Antti identified, and they do not require address space switches when transitioning from kernel-mode to user-mode or vice-versa.
Yes, you very well can. I do the same: my kernel doesn't have any virtual address space "map" as such. There is a fixed area for the kernel executable, which is architecture dependent, and a fixed area for mapping in the page-tables for the mapping API, which is also arch-dependent. After that, the rest of the kernel's address space is generically allocatable.BMW wrote: Couldn't I just reserve all memory above the kernel (excluding the reserved addresses such as those used by recursive paging and page dir) for the heap?
Yes, tPG is suggesting that you fill-in all of the kernel's top-level paging structure entries at boot. This ensures that you don't have to propagate changes to all processes when a top-level change is made to the kernel's address space. You can employ other approaches, all of which are just varying degrees of compromise between "map all top-level tables at boot" and "demand map as needed".
--Peace out,
gravaera
17:56 < sortie> Paging is called paging because you need to draw it on pages in your notebook to succeed at it.
Re: Kernel Virtual Address Space Management
Would my idea of copying the current kernel page directory part over at each context switch work? That would also make adding processes easier - I wouldn't have to copy it then.gravaera wrote:Yes, tPG is suggesting that you fill-in all of the kernel's top-level paging structure entries at boot. This ensures that you don't have to propagate changes to all processes when a top-level change is made to the kernel's address space. You can employ other approaches, all of which are just varying degrees of compromise between "map all top-level tables at boot" and "demand map as needed".
Currently developing Lithium OS (LiOS).
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
Re: Kernel Virtual Address Space Management
It would work. A nice improvement is to add a 'version number' to your address spaces. Whenever the kernel address space is modified increase the version number and store that number in the process structure. When that process runs next time, if the version number matches there is no need to copy the tables.BMW wrote:Would my idea of copying the current kernel page directory part over at each context switch work? That would also make adding processes easier - I wouldn't have to copy it then.gravaera wrote:Yes, tPG is suggesting that you fill-in all of the kernel's top-level paging structure entries at boot. This ensures that you don't have to propagate changes to all processes when a top-level change is made to the kernel's address space. You can employ other approaches, all of which are just varying degrees of compromise between "map all top-level tables at boot" and "demand map as needed".
If a trainstation is where trains stop, what is a workstation ?
Re: Kernel Virtual Address Space Management
Nice idea, will do that. Thanks.gerryg400 wrote:A nice improvement is to add a 'version number' to your address spaces. Whenever the kernel address space is modified increase the version number and store that number in the process structure. When that process runs next time, if the version number matches there is no need to copy the tables.
Currently developing Lithium OS (LiOS).
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."
Recursive paging saves lives.
"I want to change the world, but they won't give me the source code."