Moving to PAE-paging

[OdinVex]

Ah wow, way off and glad, much happier with those numbers. But I though it took 512 8b entries to map 2MB, and that would fill an entire PTE....oooh wait, * 512....ah right, missed that part. No wonder my other divisions came up with .25, 4 to 1 etc. Thanks combuster.

[rdos]

Using 2MB pages is not a good idea, so I only think the idea has merit as a theoretical possibility.

[Combuster]

Using 2MB pages is not a good idea, so I only think the idea has merit as a theoretical possibility.

Proof wanted. After all the intel manuals explicitly contradict you by suggesting the use of PSE for performance.

[rdos]

Using 2MB pages is not a good idea, so I only think the idea has merit as a theoretical possibility.

Proof wanted. After all the intel manuals explicitly contradict you by suggesting the use of PSE for performance.

Every hardware vendor want developers to use their new, fancy functions. After all, that is why they were implemented, and why the manual describes them in the first place.

The "proof" is if any major OS uses 2MB pages (preferentially exclusively).

Disadvantages of 2MB pages:
1. More internal fragmentation.
2. Page guard techniques consumes far more physical memory since now there is a need to waste up to 2MB instead of 4k in order to add guard pages.

If the OS uses both 2M and 4k pages, this will complicate the code quite a lot, and I doubt it is justifiable. The code for allocating pages in a mixed-page size environment might slow down the OS more than what can be gained in terms of performance for TLB translations.

[bluemoon]

2. Page guard techniques consumes far more physical memory since now there is a need to waste up to 2MB instead of 4k in order to add guard pages.

Why you need to allocate memory for guard page? I would just mark it as not present.

[Combuster]

The "proof" is if any major OS uses 2MB pages

Stock linux?

cat /proc/meminfo
(...)
DirectMap4k: 297980 kB
DirectMap2M: 14364672 kB
DirectMap1G: 2097152 kB

And per illustration of how simple it can be: the two 1G pages probably map to the 2GB video ram on my geforce card. Guaranteed no waste.

I hope you realize you've been rambling nonsense yet again. (waits for the incoming denials and fallacies - they always do)

[AJ]

Hi,

Using 2MB pages is not a good idea, so I only think the idea has merit as a theoretical possibility.

I realise that RDOS has a few points of design which wouldn't necessarily be considered "mainstream", but this statement is simply false. One of the significant advantages is apparent when you look at global pages for kernel space - a global page only requires a single TLB entry per 2MiB rather than per 4KiB. Then there's the Intel Architecture Optimumization Reference which explains about software prefetches not needing to stop at 4KiB boundaries.

The "proof" is if any major OS uses 2MB pages (preferentially exclusively).

Why does a major OS have to use 2MiB pages exclusively for the technique to be valid? Combuster has shown Linux making use of them and Windows Server appears to allow configuration of large pages on a per application level (at least - as far as SQL server is concerned). Yes, this link does suggest pitfalls too, but then you'd only use large pages when appropriate, wouldn't you?

At a very minimum, being able to allocate large pages for system space at boot time and then using small pages for everything else once your main allocator comes online would probably be worth considering. If you have a design that's flexible enough.

Cheers,
Adam

[OdinVex]

Is it possible to use 4K and 2M Page Translation at the same time? I might be misunderstanding some documentation.

[Combuster]

Is it possible to use 4K and 2M Page Translation at the same time?

DirectMap4k: 297980 kB
DirectMap2M: 14364672 kB

That was a bit too obvious...


The only thing PSE really does is add a bit to page directory entries that simply says "this is not an address of a page table, but of actual memory to map". Page lookups just repeatedly take off the top bits of the address and use them as an index in a table. That table provides either a pointer to the next table, or is marked as "last in the sequence". When either the "end-of-pagetables" large bit is found or the bottom level of page table is reached, all the bits from the address that haven't been used yet are attached to the address that is stored in the page table.
It basically means that you can technically go through anything between 1 and 4 levels of page tables and at each level decide if you need another lookup in the next level of tables or not. Just see it as navigating a tree where not all branches lead as high as all the others.

For practical considerations, you need to be aware of is that not all processors have the logic to break out on any level, and that such logic needs to be enabled - but you have the manuals for those details.

[OdinVex]

Oh I meant using both paging sizes simultaneously, as in a first PTE entry in be a 4KB, second and third 2MB, etc, mix and match.

[Combuster]

Nonononono. Not like that. You mix 2M pages and references to page tables.

Code: Select all

           |
           |
   +----------------+
   | page directory |
   +----------------+
      |L=0    |   |L=1
      |       |   |
+----------+  2M  2M
|page table|
+----------+
 |  |  |  |
 4k 4k 4k 4k  <-- add these together and you also get 2MB

[rdos]

I realise that RDOS has a few points of design which wouldn't necessarily be considered "mainstream", but this statement is simply false. One of the significant advantages is apparent when you look at global pages for kernel space - a global page only requires a single TLB entry per 2MiB rather than per 4KiB. Then there's the Intel Architecture Optimumization Reference which explains about software prefetches not needing to stop at 4KiB boundaries.

OK. By locating the boot image on a 2MB (possibly 4MB) boundrary, I could use 2/4MB pages for the kernel image, with no other effects. That will make context switches between processes faster as fewer TLBs are needed by kernel.

Adding functionality for allocating physical memory in 2/4MB chunks would be easy as the physical memory manager uses bitmaps that are aligned on larger boundaries than that. Allocating 2/4MB aligned linear address space could be implemented quite easily as well. With those functions I could use larger pages for hardware devices, and for DMA buffers, possibly also for file system buffers.

[OdinVex]

Can anyone provide some advice (and hopefully some post-thought poc) on what size paging I should use? 4KB or 2MB? I know it depends on a wide variety of things but hm, I kinda want to stick to one type alone and use that. At 2MB I'd have to implement process separation which in its own would have overhead but would it outweigh 4KB paging without process 'active' separation. Young, theorizing still.

[rdos]

Can anyone provide some advice (and hopefully some post-thought poc) on what size paging I should use? 4KB or 2MB?

I'd use 4k as a standard even if I started from scratch. Using 2M pages for everything doesn't seem practical. It would defeat the purpose of protection when the granularity is increased from 4k to 2M, and it would waste memory (especially when using guard pages).

[OdinVex]

Thought so, just seems wasteful to use a GB to map 512GB. :/