OSDev.org

Hi, I have confusions. While creating an user process I create a new address space for that process and copy all higher half mappings from kernel address space and lower half is filled with 0. If the process needs to expand its heap it uses syscall and allocate some physical memory but the lower half of process address space is filled with 0, how do you allocate physical memory if needed, from process address space? Because physical memories are identity mapped and it belongs to the lower half of address space.

Currently, I copy the lower half of kernel address space to process address space for physical memory allocation within process address space.

Again, another problem I have is I can't write to 0x10000000 in virtual box with 2gb of ram size. Is it vbox bug? Or I am doing wrong?

Thanks

Kamal123 wrote:Because physical memories are identity mapped and it belongs to the lower half of address space.

No it isn't, or at least shouldn't be. Yes, when you first initialize your kernel, you need to set it up like that. Then you need to map the kernel (code, data, and stack) into the higher half and move all pointers up there as well. And then clear the lower half. Then the lower half is free for user space to use it.

The best way to ensure no pointers remain on the low half is to make the function that enables paging a noreturn function. It enables paging, reloads the stack pointer, and jumps somewhere completely unrelated. And then that place basically has to start afresh.

Also, when you initialize a new user process, you don't just clear out the lower half, you also initialize at least the OS-facing parts to contain the file mappings. I find it quite interesting that you can model all of user address space as file mappings, though some files are anonymous. Then obviously more memory can be allocated using a system call, traditionally mmap(). You do have syscalls, right?

Kamal123 wrote:Again, another problem I have is I can't write to 0x10000000 in virtual box with 2gb of ram size. Is it vbox bug? Or I am doing wrong?

Does the memory map provided by the platform give that address as RAM? If so, is it included in your page map?

Hi thanks for your reply, If all the lower half is cleared after kernel initialisation, how do you allocate physical memory if required?

And 0x10000000 is not present in the memory map provided by vbox uefi..

Kamal123 wrote:Hi thanks for your reply, If all the lower half is cleared after kernel initialisation, how do you allocate physical memory if required?

You have a data structure to keep track of free memory, right? Map it somewhere in the higher half. When you need to allocate physical memory, you'll use that structure to find free memory.

Kamal123 wrote:And 0x10000000 is not present in the memory map provided by vbox uefi..

That means it isn't RAM. It's unusual for a gap to be there if you have 2GB assigned to your VM, though. If you add up all the usable regions in the memory map, is there around 2GB of usable RAM in total?

Thanks for clearing my confusion..

No it's not 2gb in total...it's less than 2gb.

Kamal123 wrote: If all the lower half is cleared after kernel initialisation, how do you allocate physical memory if required?

Those don't have much to do with each other. Physical memory addresses in a higher-half kernel are merely numbers. That is all. They are not addresses you can access directly. You can access them indirectly, but that is where things get complicated.

The physical allocator has to somehow keep track of which physical RAM is available and what RAM is reserved. After that, it returns the addresses of blocks fitting the criteria given, while marking those blocks as reserved themselves. There are ways to do this without touching the memory yourself. I have a list of available blocks, and a list reserved blocks, and 128 of each are builtin to the static memory area.

But if you are using a linked list, for example, then the question becomes how to access physical RAM. I have a 64-bit kernel, so I have enough address space to allow for a linear mapping at the beginning of kernel memory space. If you are building a 32-bit kernel, however, things get more complicated. You can use Linux' approach of just mapping memory linearly to its address space and switching out the high memory mappings. If you would like to give the boot environment more freedom, you can also map your kernel logically and reserve a page to swap out of temporary access. However, I have heard that has terrible performance.

However you do it, you need to start thinking about how your memory space is supposed to work. I can't do that for you.

Hi,
After some experiments I can finally map some virtual address to physical addresses and than clear the lower half of kernel address space, after clearing I can write to those virtual addresses which were mapped before clearing the lower half, but after clearing the lower half if I need to map a physical to virtual address I can map just by allocating a physical page number. It's just a physical page number, but while mapping if I need to setup the page tables entries for that virtual address like pml4 entry, pml2 entry, pml3 entry..etc..it triple faults, because those physical page mappings were cleared from lower half and cannot be written into it... How do you map virtual address after clearing the lower half? And when do you clear the lower half mapping in kernel initialisation phase?..

Thanks in advance..

Do I write in Chinese or what? Yeah, you need to figure out how you want to access physical memory. So now we have established you are writing a 64-bit kernel, the easiest way to do this is to map all physical address space (such as is necessary) to the start of the high half. The kernel is mapped to -2GB anyway (so you can use -mcmodel=kernel). So after creating that mapping, you can access any physical address by just adding the offset and treating the result as a pointer. Or to say it in code That way, writing the paging code is trivial. But the important thing is, you must establish that linear mapping first.

Thank you so much for help,

My confusions are cleared. Now I can allocate & map physical address to any virtual address from higher half while lower half is cleared.

Manas Kamal