Page 1 of 1
Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 2:53 am
by NeonLightions
Hi,
I need someone to consult, fix and comment on paging in long mode. I have implemented some functions here:
https://github.com/NeonLightions/Amore-OS-x64, but it's quite confused. And can you please suggest me some resources or articles that are helpful in 4-level paging? Thank you very much!
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 4:11 am
by iansjack
I would suggest that your question is a little too open ended.
You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?
As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.
In summary, what exactly is your problem?
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 4:56 am
by NeonLightions
iansjack wrote:I would suggest that your question is a little too open ended.
You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?
As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.
In summary, what exactly is your problem?
Hi, i got triple fault when enable paging. I have used debugger and realized that i got error at walking on pml4 to look for pdpt. I think the problem is my macros. Can you help me? Also is there any topic or thread that show how to walk with 4-level paging?
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 4:56 am
by NeonLightions
iansjack wrote:I would suggest that your question is a little too open ended.
You have implemented paging functions. So, do they work? If not, what errors are you getting? If you are getting errors, what steps have you taken towards debugging these errors?
As for resources - the AMD and Intel manuals provide detailed explantations of the paging mechanism.
In summary, what exactly is your problem?
Hi, i got triple fault when enable paging. I have used debugger and realized that i got error at walking on pml4 to look for pdpt. I think the problem is my macros. Can you help me? Also is there any topic or thread that show how to walk with 4-level paging?
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 5:07 am
by nullplan
You are using code for 32-bit PAE paging. This will not work in 64-bit mode. In 32-bit PAE paging, there are indeed 32-byte page directory pointer tables, but in 64-bit mode, all layers of the paging tree are naturally aligned pages.
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 5:21 am
by NeonLightions
nullplan wrote:You are using code for 32-bit PAE paging. This will not work in 64-bit mode. In 32-bit PAE paging, there are indeed 32-byte page directory pointer tables, but in 64-bit mode, all layers of the paging tree are naturally aligned pages.
I see, but that is just temporary solution
. Now things I need are: Is there anything wrong with my index calculator macros? If so, what should I need to fix? How to calculate index in 4-level paging?
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 6:29 am
by nullplan
NeonLightions wrote:I see, but that is just temporary solution
It is not any solution. The CPU will look at that misaligned address in CR3 and throw it back into your face in the form of a tripple fault. By setting CR3 to a misaligned address, or any address that is not a proper PML4, you make it impossible to resolve any addresses once paging is enabled. So the next instruction address cannot be resolved, nor can the address of the IDT, that of the GDT, of the TSS, or any stack in the TSS. That is why you get a tripple fault.
NeonLightions wrote:Now things I need are: Is there anything wrong with my index calculator macros?
No, those are fine, albeit written weirdly. All those leading zeroes can be omitted, they serve absolutely no purpose, except to take up space and hinder readability.
There is ADDR_PAGE_ROUND_UP, which as written always returns 0. As used, I should think you meant
Code: Select all
#define ADDR_PAGE_ROUND_UP(addr) (((addr) + 0xfff) & -0x1000)
/* or possibly: */
#define ADDR_PAGE_ROUND_UP(addr) ADDR_PAGE_ROUND_DN((addr) + 0xfff)
Besides, __paging_init() simply cannot work, because it is writing virtual addresses into paging structures and into CR3, but those all require physical addresses. Unless you are currently identity-mapped, you should not be doing that.
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 6:51 am
by NeonLightions
nullplan wrote:NeonLightions wrote:I see, but that is just temporary solution
It is not any solution. The CPU will look at that misaligned address in CR3 and throw it back into your face in the form of a tripple fault. By setting CR3 to a misaligned address, or any address that is not a proper PML4, you make it impossible to resolve any addresses once paging is enabled. So the next instruction address cannot be resolved, nor can the address of the IDT, that of the GDT, of the TSS, or any stack in the TSS. That is why you get a tripple fault.
NeonLightions wrote:Now things I need are: Is there anything wrong with my index calculator macros?
No, those are fine, albeit written weirdly. All those leading zeroes can be omitted, they serve absolutely no purpose, except to take up space and hinder readability.
There is ADDR_PAGE_ROUND_UP, which as written always returns 0. As used, I should think you meant
Code: Select all
#define ADDR_PAGE_ROUND_UP(addr) (((addr) + 0xfff) & -0x1000)
/* or possibly: */
#define ADDR_PAGE_ROUND_UP(addr) ADDR_PAGE_ROUND_DN((addr) + 0xfff)
Besides, __paging_init() simply cannot work, because it is writing virtual addresses into paging structures and into CR3, but those all require physical addresses. Unless you are currently identity-mapped, you should not be doing that.
Okay, that solution has been removed (just not post to repo). Is there a way to make the calculation faster and neater? So, the address of those page structures are naturally aligned so I don't need to align it?
Re: Need ideas on paging in long mode
Posted: Sun Dec 26, 2021 8:08 am
by nullplan
NeonLightions wrote: Is there a way to make the calculation faster and neater? So, the address of those page structures are naturally aligned so I don't need to align it?
I wouldn't worry about speed at this juncture. No, this is basically about design. I have a physical allocator that can by its very nature only return pages. Therefore I don't need to align the addresses that fall out of the allocator. But that means I have to make the general purpose allocator more complicated. I have a "kmalloc()" for allocating miscellaneous small things, but its memory pool can only grow, so for large things, directly calling the physical allocator is encouraged. Later on, I might experiment with demand paging, and then things get interesting.
Re: Need ideas on paging in long mode
Posted: Mon Dec 27, 2021 6:54 am
by NeonLightions
Hi,
I have looked through this tutorial:
https://github.com/josehu07/hux-kernel/ ... ging-Setup and I found this line:
Code: Select all
/** Helper macro on getting the pointed-to address stored in an entry. */
#define ENTRY_FRAME_ADDR(entry) ((uint32_t) (entry).frame << 12)
What's that line mean?
Re: Need ideas on paging in long mode
Posted: Mon Dec 27, 2021 11:15 am
by thewrongchristian
(entry).frame is the physical page number (though, not physical address). So the first physical page would be page number 0, the next page would be page number 1 etc.
To get the physical address, we shift the page number left by 12 bits, which leaves the lower 12 bits as 0, and therefore leaving the resulting address aligned on a 4096 byte boundary (page size).
So, for page number 1:
- Page address in binary: 0000 0000 0000 0000 0001 0000 0000 0000
- Page address in hex: 0x00001000
Re: Need ideas on paging in long mode
Posted: Mon Dec 27, 2021 5:33 pm
by NeonLightions
thewrongchristian wrote:
(entry).frame is the physical page number (though, not physical address). So the first physical page would be page number 0, the next page would be page number 1 etc.
To get the physical address, we shift the page number left by 12 bits, which leaves the lower 12 bits as 0, and therefore leaving the resulting address aligned on a 4096 byte boundary (page size).
So, for page number 1:
- Page address in binary: 0000 0000 0000 0000 0001 0000 0000 0000
- Page address in hex: 0x00001000
Oh, I get it. But will it work on 4-level paging? Cause that tutorial is for 2 level paging.
Re: Need ideas on paging in long mode
Posted: Tue Dec 28, 2021 12:47 am
by nullplan
NeonLightions wrote:Oh, I get it. But will it work on 4-level paging? Cause that tutorial is for 2 level paging.
And this is where you need to know the big difference between non-PAE and PAE paging: The size of the page table entries. In non-PAE (2-level) paging, the entries are 32 bits, and in PAE and 4-level paging, they are 64 bits. So if you defined your page table entry type correctly, the macro would almost work, except that the result would have to be a 64-bit number, not a 32-bit one.
I don't use bit fields. Ever. It has so far never been worthwhile for me to do so. In case of page table entries, each entry simply contains the physical address of either the next layer or the page itself, with some flags ORed in. This avoids the issue of shifting just to get back to where you were.
Re: Need ideas on paging in long mode
Posted: Tue Dec 28, 2021 4:23 am
by NeonLightions
Ok, just 1 more question: In Intel 64 and IA-32 Architectures Software Developer's Manual, Volume 3, Chapter 2, Section 2.1.5.1, it said:
Code: Select all
With 4-level paging, there is only one PML4 table and its base physical address is stored in CR3
. Is it said only 1 PML4 table per process or the entire OS use only 1 PML4 table?
Re: Need ideas on paging in long mode
Posted: Tue Dec 28, 2021 5:44 am
by iansjack
One per process.