problem understanding memory management in JamesM tutorial

[vjain20]

Hi,

I am following JamesM OS development tutorial and I am stuck at the Paging section.
I don't understand how the memory is being managed. Following are the queries :

- How are the bits in the bitset set on allocation? The kmalloc function just modifies the placement
address and does not set the corresponding bits in bitset which seems counter-intuitive.

- How is the identity mapping being done. I know that the following code is doing it but it is quite confusing:

Code: Select all

int i = 0;
    while (i < placement_address)
    {
        // Kernel code is readable but not writeable from userspace.
        alloc_frame( get_page(i, 1, kernel_directory), 0, 0);
        i += 0x1000;
    }

It would be great if someone could explain this.

-Thanks.

[Nessphoro]

Basically it takes the virtual address, from 0 to placement_address and maps it to the same location in the physical memory.
Because the physical page allocator has not been used, every page allocated will be at the start.

[JamesM]

- How is the identity mapping being done. I know that the following code is doing it but it is quite confusing:

You're right, it's written slightly counterintuitively. Why not just have a for() loop and call alloc_frame on each iteration from 0 to placement_address?

The answer is that get_page() can modify placement_address! If it needs to map a new page table, it'll call alloc_frame itself and placement_address will change. So that's why we use the while loop, to make it *obvious* that placement_address could have changed from loop entry to loop exit.

Note that the equivalent for loop would have the same semantics, but for loops are usually written expecting loop-invariant start and end conditions so I decided to write it differently to show that.

[vjain20]

Thanks for the explanation! I have a few more queries to ask. It would be great if you could answer these :

1. As I understand the code that I provided above, it does two things :
   1. mark all the memory in use as occupied in the bitset
   2. create page tables and identity-mappings
   Am I right ?
2. In the function below what is the use of having a parameter for physical address
   when it is set equal to the return value.
   

   Code: Select all

   u32int kmalloc(u32int sz, int align, u32int *phys)
   {
     if (align == 1 && (placement_address & 0xFFFFF000)) // If the address is not already page-aligned
     {
       // Align it.
       placement_address &= 0xFFFFF000;
       placement_address += 0x1000;
     }
     if (phys)
     {
       *phys = placement_address;
     }
     u32int tmp = placement_address;
     placement_address += sz;
     return tmp;
   }

3. How does the code ensure that virtual address in a mapping are equal to the physical address in when the alloc_frame()
   method always calls the first_frame() instead of directly assigning physical address ?
4. It seems that the page tables will contain mappings of the pages which are occupied by
   directory and tables themselves. Am I right ?

-Thanks
Vaibhav Jain

[JamesM]

Thanks for the explanation! I have a few more queries to ask. It would be great if you could answer these :

1. As I understand the code that I provided above, it does two things :
   1. mark all the memory in use as occupied in the bitset
   2. create page tables and identity-mappings
   Am I right ?

IIRC, yes.

[*] In the function below what is the use of having a parameter for physical address
when it is set equal to the return value.

Code: Select all

u32int kmalloc(u32int sz, int align, u32int *phys)
{
  if (align == 1 && (placement_address & 0xFFFFF000)) // If the address is not already page-aligned
  {
    // Align it.
    placement_address &= 0xFFFFF000;
    placement_address += 0x1000;
  }
  if (phys)
  {
    *phys = placement_address;
  }
  u32int tmp = placement_address;
  placement_address += sz;
  return tmp;
}

The function you mention, kmalloc, has two versions. The version you posted is simply the version used when there is no kernel heap, and so trivially the physical address of allocated memory is the same as the virtual address. In a later chapter when a proper heap is implemented, it becomes more complex.

[*] How does the code ensure that virtual address in a mapping are equal to the physical address in when the alloc_frame()
method always calls the first_frame() instead of directly assigning physical address ?

It doesn't ensure that a virtual address is equal to a physical address. There is no such assumption.

[*] It seems that the page tables will contain mappings of the pages which are occupied by
directory and tables themselves. Am I right ?

[/quote][/quote]

Not in my implementation. I think you're referring to the recursive page directory trick, which those tutorials don't use.

[vjain20]

Hi,
Thanks for replying!

It doesn't ensure that a virtual address is equal to a physical address. There is no such assumption.

I am referring to identity-mapping here. As mentioned in the tutorial - " initialise_paging() firstly creates the frames bitset, and sets everything to zero using memset. Then it allocates space (which is page-aligned) for a page directory. After that, it allocates frames such that any page access will map to the frame with the same linear address, called identity-mapping. This is done for a small section of the address space, so the kernel code can continue to run as normal."

Not in my implementation. I think you're referring to the recursive page directory trick, which those tutorials don't use.

In the while loop below i goes from 0 to placement address i.e. from the frame 0 to the placement address. Since the
page directory and page tables lie within this memory range , this memory will also get mapped in the page directory and tables.
What I mean to say is that if the Page directory starts from an address X in physical memory there will be a mapping in
a page table mapping X to some virtual address (= X in case of identity-mapping). Is my understanding wrong ?

Code: Select all

int i = 0;
    while (i < placement_address)
    {
        // Kernel code is readable but not writeable from userspace.
        alloc_frame( get_page(i, 1, kernel_directory), 0, 0);
        i += 0x1000;
    }

[JamesM]

Your understanding appears to be correct on both counts.

[JamesM]

James, did you get to posting the new version of the tutorials?

I didn't like them (again) and started again. I'm actively working on the new ones - the code is mostly complete and the textual crap is coming next.

It's a different ethos than the last one. https://github.com/jmolloy/JMTK

[JamesM]

James, did you get to posting the new version of the tutorials?

I didn't like them (again) and started again. I'm actively working on the new ones - the code is mostly complete and the textual crap is coming next.

It's a different ethos than the last one. https://github.com/jmolloy/JMTK

Cool, as usual - if you need help reviewing them you know where to look.

Awesome, thanks