Is it fair to expect PAE to be present on the system?

[AlfaOmega08]

Time has come for my kernel to handle long mode. Reading TFM I learnt that there are three tipes of paging:
* "Standard" 32-bit paging: PageDirectoy => PageTable => Frame. PD and PT with 1024 entries each.
* PAE paging: PDPT => PageDirectory => PageTable => Frame. PDPT with 4 entries, PD and PT with 512 entries.
* Long mode (IA-32e) paging: PML4 => PDPT => PageDirectory => PageTable => Frame. All structures made of 512 entries.

Now. I tought at first that it would have been nice to create two kernels as many do here:
* A kernel for quite outdated machines (x86 archs)
* A kernel to use all the potential of your pc and run at full speed (x86_64)

I tought to implement both standard paging and pae paging into the x86 version and only the IA-32e paging in the x86_64 one (which is required).

It is easy to go from PAE to IA-32e: just add a new structure (PML4) and make the PDPT containing 512 entries instead of just 4. Set some flags and it's all right. But the transistion from 32-bit paging and PAE is not so obvious. It's not matter of adding another structure. You basically need this (eg):

Code: Select all

struct PageTable
{
    uintptr_t frames[NumberOfEntries];
}

where NumberOfEntries shall be 512 if you detect using CPUID that the pc supports PAE, 1024 otherwise. This can't be achieved with C actually as it needs to know the count at compile time.

Since I wouldn't like to plan triple-version (x86 no PAE, x86 PAE, x86_64) releases, here's my question: is it fair to expect PAE to be present on the system?
PAE was introduced in 1995 by Intel with the Pentium Pro, and it's available on all of them processors since then (P2, P3, P4, all cores and Celerons) with the only exception of the Pentium M at 400 Mhz. It has been supported by AMD since 1999 with the Athlon K7. So you would only get stuck on an old AMD K6 (which, twist of fate, I own), or pre-PentiumPro CPUs. How many use such a processor at home? Or at work? Are them still so widely used?

Thanks

[Solar]

Are them still so widely used?

Rephrase that question in your mind:

Will they still be so widely used by the time your OS reaches a point where it would make a difference to anybody not being on your core development team anyway?

[AlfaOmega08]

Are them still so widely used?

Rephrase that question in your mind:

Will they still be so widely used by the time your OS reaches a point where it would make a difference to anybody not being on your core development team anyway?

For the first time, Solar, your hilarity made me laugh. Good answer anyway

[OSwhatever]

It depends what your OS is aiming for. If it is only going to be used in x86 desktops, then I would assume that most of them are new enough supporting PAE.

In embedded devices, there are a bunch x86 clones based on lesser x86 models like the old 486 which doesn't have PAE. Then you have to use the old page table structure instead.

Most kernels are written so that it supports any type of translation and any page sizes.

[a5498828]

good os should have a module with exported functions that manage paging. So if any newer paging comes up, you just swap module and there is no diffrence.
OS should consist of modules only in fact.

[Tosi]

There are ways of detecting PAE. Check the sections on paging and CPUID in the Intel manuals.

[AlfaOmega08]

good os should have a module with exported functions that manage paging. So if any newer paging comes up, you just swap module and there is no diffrence.
OS should consist of modules only in fact.

And how would you load a module from the hd when all the drivers are still uninitialized? Paging should be enabled, as the phys memory manager, asap. Grub would be a good solution, but there's a thing I don' like about grub modules: what's the point of splitting your kernel in modules if grub will anyway load them all. There isn't a selective module loading...

[Brendan]

Hi,

good os should have a module with exported functions that manage paging. So if any newer paging comes up, you just swap module and there is no diffrence.
OS should consist of modules only in fact.

And how would you load a module from the hd when all the drivers are still uninitialized? Paging should be enabled, as the phys memory manager, asap. Grub would be a good solution, but there's a thing I don' like about grub modules: what's the point of splitting your kernel in modules if grub will anyway load them all. There isn't a selective module loading...

For me, boot code loads a "boot image" which contains any modules that might be needed, so everything needed during boot is in RAM before the kernel is started. For something like a generic boot CD lots of modules might be needed so lots of modules are in the boot image. When the OS is being installed (e.g. after it's been booted from something like a generic boot CD and knows which modules actually are needed), the OS can generate a minimised boot image that doesn't include unnecessary modules and install that.


Cheers,

Brendan

[Kevin]

Will they still be so widely used by the time your OS reaches a point where it would make a difference to anybody not being on your core development team anyway?

Why do you think that the core development team isn't important? I would assume that most OSes are never used by anybody outside the "core development team", so just taking their requirements seems to be a reasonable approach.

[Combuster]

Counterquestion: have you considered dropping PAE paging over legacy paging? How many actual computers do you think have more than 3G/4G ram and do not have long mode, and what do you otherwise gain by PAE paging over legacy paging?

[AlfaOmega08]

Counterquestion: have you considered dropping PAE paging over legacy paging? How many actual computers do you think have more than 3G/4G ram and do not have long mode, and what do you otherwise gain by PAE paging over legacy paging?

It would make the transition to IA-32e paging a lot easier.

[gravaera]

Counterquestion: have you considered dropping PAE paging over legacy paging? How many actual computers do you think have more than 3G/4G ram and do not have long mode, and what do you otherwise gain by PAE paging over legacy paging?

It would make the transition to IA-32e paging a lot easier.

If it does, then there's something direly wrong with your abstractions :/

[Owen]

If it does, then there's something direly wrong with your abstractions :/

No, it means you share one paging implementation between IA-32 and AMD64...

[blobmiester]

Counterquestion: have you considered dropping PAE paging over legacy paging? How many actual computers do you think have more than 3G/4G ram and do not have long mode, and what do you otherwise gain by PAE paging over legacy paging?

What about disabling execution of the page? Bit #63 (XD) of both the page directory entry and page table entry, while in PAE, allow (0) or disallow (1) instruction fetching from that 2 MiB or 4 KiB region of memory.

From section 4.1.3 of the latest intel manuals:

IA32_EFER.NXE enables execute-disable access rights for PAE paging and IA-32e
paging. If IA32_EFER.NXE = 0, software may fetch instructions from any linear
address that paging allows the software to read; if IA32_EFER.NXE = 1, instructions
fetches can be prevented from specified linear addresses (even if data reads from the
addresses are allowed). Section 4.6 explains how access rights are determined. (32-
bit paging always allows software to fetch instructions from any linear address that
may be read; IA32_EFER.NXE has no effect with 32-bit paging. Software that wants
to limit instruction fetches from readable pages must use either PAE paging or IA-32e
paging.)

EDIT: Oops. Didn't catch the 'and not long mode' part of your post Combuster. This still partially applies as my computers (Pentium II and Pentium IV) have both PAE paging and the NXE bit. I like the idea of disabling execution from data pages of processes.

[Combuster]

Pentium 2 does not support NX. Many P4s don't either, and the P4s who do are likely to have long mode as well (if there is a XD-LM feature gap at all, Wikipedia is not explicit about it). I suggest you check your facts again.