Page 1 of 1

A good way for storing&accessing paging structure

Posted: Sun Dec 15, 2013 12:51 pm
by lexected
Hello

During the summer summer I jumped from writing C programs to lower level and learnt many things about assembly and operating systems. Now I want to get back to it and I'm decided to write a microkernel.

I have used GRUB2 for loading my ELF file, which is planned to be a higher half kernel. Its entry point maps the whole kernel to the highest gigabyte of memory and tries to create more address spaces for other processes. Physical memory management code uses memory map provided by GRUB to fill a bitmap, which covers whole possible 4GBs of memory. My kernel uses page directory (set up by entry assembly) located after kernels code, data and uninitialized data, together with a single page table which has been enough so far.

Whenever I want to create a new page table or page directory, I need to allocate it somewhere. I can get address of first free physical page from the bitmap I have, but I need to access it somehow, as it's not paged. So, I can use a special address within my kernel´s page table to write PDEies pointing on page tables that are global (kernel) and then use it for different PDE, or I can identity-page its physical location. If I choose to do the first, I need to map every single page directory or page table I'm going to work with to my special page and execute invlpg each time I manipulate with it. The second option requires creating lots of page tables for only holding another page tables and directories for different address spaces to be accessible for the kernel every time its necessary, but it can save me some time. I would choose to create a special physical location that would be identity-paged in kernel´s space, but the problem is that I don't know how many page tables and directories am I going to have. And if I wanted to manage somthing like "blocks of paging entities", I would need to hold bitmap for each of them + have some array or list in kernel.

How can I hold page tables and page directories to be quickly accessible when system needs it, without limiting quantity of address spaces?
How do hobby operating systems usually solve this problem?

Thanks,
Filip.

Re: A good way for storing&accessing paging structure

Posted: Sun Dec 15, 2013 3:25 pm
by BMW
Thankfully, there is a very elegant solution to this problem.

I'll give you a clue: what would happen if you made the last page directory entry point to the physical address of the page directory?

See if you can work it out. :) If not, I'll explain a bit more.

Re: A good way for storing&accessing paging structure

Posted: Sun Dec 15, 2013 4:15 pm
by bluemoon
BMW wrote:Thankfully, there is a very elegant solution to this problem.
I'll give you a clue: what would happen if you made the last page directory entry point to the physical address of the page directory?
See if you can work it out. :) If not, I'll explain a bit more.
Note that it is not a must to use the last entry for recursive paging, you can indeed use any entry in the first level of page structure.

Re: A good way for storing&accessing paging structure

Posted: Sun Dec 15, 2013 5:25 pm
by BMW
bluemoon wrote:Note that it is not a must to use the last entry for recursive paging, you can indeed use any entry in the first level of page structure.
Indeed, of course. I just said the last entry as it is what I use, and because he has a higher half kernel.

Re: A good way for storing&accessing paging structure

Posted: Mon Dec 16, 2013 11:26 am
by lexected
That's interesting, thank you all!

I have one more question:
As I want to create a microkernel, I want to place everything that has something to do with filesystems or storage devices into user processes, and I wonder how can I load for example ATA driver process code, when I don't have anything that could load the process itself. Do I need to include this "basic" processes to kernel file and place them to physical memory by editing linker script? Is there any more elegant way for "loading" these processes?

Re: A good way for storing&accessing paging structure

Posted: Mon Dec 16, 2013 3:23 pm
by BMW
filipadamer wrote:That's interesting, thank you all!

I have one more question:
As I want to create a microkernel, I want to place everything that has something to do with filesystems or storage devices into user processes, and I wonder how can I load for example ATA driver process code, when I don't have anything that could load the process itself. Do I need to include this "basic" processes to kernel file and place them to physical memory by editing linker script? Is there any more elegant way for "loading" these processes?
Would it be possible to get your bootloader to load it?

Re: A good way for storing&accessing paging structure

Posted: Tue Dec 17, 2013 1:41 am
by lexected
I thought that I could use "initrd" from GRUB, but it's only possible after using "linux" command (I use multiboot). I don't know.

Re: A good way for storing&accessing paging structure

Posted: Tue Dec 17, 2013 4:01 am
by Combuster

Re: A good way for storing&accessing paging structure

Posted: Wed Dec 18, 2013 10:54 am
by lexected
Thanks for the link on modules.
In GRUB2 manual I found only insmod command which loads relocatable object for GRUB´s kernel. Can I load my user space programs on arbitrary addresses (by customizing linker output), or it's necessary to write my own module?

Re: A good way for storing&accessing paging structure

Posted: Wed Dec 18, 2013 12:14 pm
by mnovotny
GRUB2 has "module" command too, but it's part of the multiboot/multiboot2 (GRUB) module. This is how I boot my OS in grub.cfg (this is in menuentry):

multiboot /boot/bootem
module /boot/kernelem
boot

(Yes, I load kernel as a module.)

You could load all the critical drivers as modules and GRUB will tell you where they are loaded via the multiboot structure. However the modules are not parsed by GRUB, they are just copied raw, so if you're using elf or some other format you will have to parse it yourself.

Re: A good way for storing&accessing paging structure

Posted: Wed Dec 18, 2013 12:50 pm
by lexected
Thank you, I found it in GRUB sourcecode. (BTW SK :D )