Need ideas on paging in long mode

[NeonLightions]

Hi,
I need someone to consult, fix and comment on paging in long mode. I have implemented some functions here: https://github.com/NeonLightions/Amore-OS-x64, but it's quite confused. And can you please suggest me some resources or articles that are helpful in 4-level paging? Thank you very much!

[iansjack]

I would suggest that your question is a little too open ended.

You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?

As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.

In summary, what exactly is your problem?

[NeonLightions]

I would suggest that your question is a little too open ended.

You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?

As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.

In summary, what exactly is your problem?

Hi, i got triple fault when enable paging. I have used debugger and realized that i got error at walking on pml4 to look for pdpt. I think the problem is my macros. Can you help me? Also is there any topic or thread that show how to walk with 4-level paging?

[NeonLightions]

I would suggest that your question is a little too open ended.

You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?

As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.

In summary, what exactly is your problem?

Hi, i got triple fault when enable paging. I have used debugger and realized that i got error at walking on pml4 to look for pdpt. I think the problem is my macros. Can you help me? Also is there any topic or thread that show how to walk with 4-level paging?

[nullplan]

You are using code for 32-bit PAE paging. This will not work in 64-bit mode. In 32-bit PAE paging, there are indeed 32-byte page directory pointer tables, but in 64-bit mode, all layers of the paging tree are naturally aligned pages.

[NeonLightions]

You are using code for 32-bit PAE paging. This will not work in 64-bit mode. In 32-bit PAE paging, there are indeed 32-byte page directory pointer tables, but in 64-bit mode, all layers of the paging tree are naturally aligned pages.

I see, but that is just temporary solution . Now things I need are: Is there anything wrong with my index calculator macros? If so, what should I need to fix? How to calculate index in 4-level paging?

[nullplan]

I see, but that is just temporary solution

It is not any solution. The CPU will look at that misaligned address in CR3 and throw it back into your face in the form of a tripple fault. By setting CR3 to a misaligned address, or any address that is not a proper PML4, you make it impossible to resolve any addresses once paging is enabled. So the next instruction address cannot be resolved, nor can the address of the IDT, that of the GDT, of the TSS, or any stack in the TSS. That is why you get a tripple fault.

Now things I need are: Is there anything wrong with my index calculator macros?

No, those are fine, albeit written weirdly. All those leading zeroes can be omitted, they serve absolutely no purpose, except to take up space and hinder readability.

There is ADDR_PAGE_ROUND_UP, which as written always returns 0. As used, I should think you meant

Code: Select all

#define ADDR_PAGE_ROUND_UP(addr) (((addr) + 0xfff) & -0x1000)
/* or possibly: */
#define ADDR_PAGE_ROUND_UP(addr) ADDR_PAGE_ROUND_DN((addr) + 0xfff)

Besides, __paging_init() simply cannot work, because it is writing virtual addresses into paging structures and into CR3, but those all require physical addresses. Unless you are currently identity-mapped, you should not be doing that.

[NeonLightions]

I see, but that is just temporary solution

It is not any solution. The CPU will look at that misaligned address in CR3 and throw it back into your face in the form of a tripple fault. By setting CR3 to a misaligned address, or any address that is not a proper PML4, you make it impossible to resolve any addresses once paging is enabled. So the next instruction address cannot be resolved, nor can the address of the IDT, that of the GDT, of the TSS, or any stack in the TSS. That is why you get a tripple fault.

Now things I need are: Is there anything wrong with my index calculator macros?

No, those are fine, albeit written weirdly. All those leading zeroes can be omitted, they serve absolutely no purpose, except to take up space and hinder readability.

There is ADDR_PAGE_ROUND_UP, which as written always returns 0. As used, I should think you meant

Code: Select all

#define ADDR_PAGE_ROUND_UP(addr) (((addr) + 0xfff) & -0x1000)
/* or possibly: */
#define ADDR_PAGE_ROUND_UP(addr) ADDR_PAGE_ROUND_DN((addr) + 0xfff)

Besides, __paging_init() simply cannot work, because it is writing virtual addresses into paging structures and into CR3, but those all require physical addresses. Unless you are currently identity-mapped, you should not be doing that.

Okay, that solution has been removed (just not post to repo). Is there a way to make the calculation faster and neater? So, the address of those page structures are naturally aligned so I don't need to align it?

[nullplan]

Is there a way to make the calculation faster and neater? So, the address of those page structures are naturally aligned so I don't need to align it?

I wouldn't worry about speed at this juncture. No, this is basically about design. I have a physical allocator that can by its very nature only return pages. Therefore I don't need to align the addresses that fall out of the allocator. But that means I have to make the general purpose allocator more complicated. I have a "kmalloc()" for allocating miscellaneous small things, but its memory pool can only grow, so for large things, directly calling the physical allocator is encouraged. Later on, I might experiment with demand paging, and then things get interesting.

[NeonLightions]

Hi,
I have looked through this tutorial: https://github.com/josehu07/hux-kernel/ ... ging-Setup and I found this line:

Code: Select all

/** Helper macro on getting the pointed-to address stored in an entry. */
#define ENTRY_FRAME_ADDR(entry) ((uint32_t) (entry).frame << 12)

What's that line mean?

[thewrongchristian]

Hi,
I have looked through this tutorial: https://github.com/josehu07/hux-kernel/ ... ging-Setup and I found this line:

Code: Select all

/** Helper macro on getting the pointed-to address stored in an entry. */
#define ENTRY_FRAME_ADDR(entry) ((uint32_t) (entry).frame << 12)

What's that line mean?

(entry).frame is the physical page number (though, not physical address). So the first physical page would be page number 0, the next page would be page number 1 etc.

To get the physical address, we shift the page number left by 12 bits, which leaves the lower 12 bits as 0, and therefore leaving the resulting address aligned on a 4096 byte boundary (page size).

So, for page number 1:

• Page address in binary: 0000 0000 0000 0000 0001 0000 0000 0000
• Page address in hex: 0x00001000

[NeonLightions]

Hi,
I have looked through this tutorial: https://github.com/josehu07/hux-kernel/ ... ging-Setup and I found this line:

Code: Select all

/** Helper macro on getting the pointed-to address stored in an entry. */
#define ENTRY_FRAME_ADDR(entry) ((uint32_t) (entry).frame << 12)

What's that line mean?

(entry).frame is the physical page number (though, not physical address). So the first physical page would be page number 0, the next page would be page number 1 etc.

To get the physical address, we shift the page number left by 12 bits, which leaves the lower 12 bits as 0, and therefore leaving the resulting address aligned on a 4096 byte boundary (page size).

So, for page number 1:

• Page address in binary: 0000 0000 0000 0000 0001 0000 0000 0000
• Page address in hex: 0x00001000

Oh, I get it. But will it work on 4-level paging? Cause that tutorial is for 2 level paging.

[nullplan]

Oh, I get it. But will it work on 4-level paging? Cause that tutorial is for 2 level paging.

And this is where you need to know the big difference between non-PAE and PAE paging: The size of the page table entries. In non-PAE (2-level) paging, the entries are 32 bits, and in PAE and 4-level paging, they are 64 bits. So if you defined your page table entry type correctly, the macro would almost work, except that the result would have to be a 64-bit number, not a 32-bit one.

I don't use bit fields. Ever. It has so far never been worthwhile for me to do so. In case of page table entries, each entry simply contains the physical address of either the next layer or the page itself, with some flags ORed in. This avoids the issue of shifting just to get back to where you were.

[NeonLightions]

Ok, just 1 more question: In Intel 64 and IA-32 Architectures Software Developer's Manual, Volume 3, Chapter 2, Section 2.1.5.1, it said:

Code: Select all

 With 4-level paging, there is only one PML4 table and its base physical address is stored in CR3

. Is it said only 1 PML4 table per process or the entire OS use only 1 PML4 table?

[iansjack]

One per process.