Moving to PAE-paging

[rdos]

I searched the current code-base for dependency on page-table structures. Essentially, all of these are referenced with 4 selectors, 2 for the system page tables and 2 for the current process page tables. One selector is for the page directory, and one is for the page tables. It turns out that a large majority of the references are to the process page table selector (in total about 70 references), and those are in many modules, including some device-drivers. The other 3 selectors are only referenced in a few relevant kernel modules.

An additional problem is that the API to manipulate physical memory should be changed to use 64-bits (two 32-bit registers, like EBX:EAX). There are about 150 references to these functions. Some of these allocations must return a physical address below 4G, while most can handle any address (page table manipulations).

I think a good idea is to provide new functions to manipulate the process page table selector (and thus phase-out the direct usage of process page table). Then I would provide new physical memory manipulation functions and migrate to them. After this is done, it is fairly easy to just change a few functions in order to switch to PAE mode. And best of all, if the CPU doesn't support PAE, it works seemlessly with typical paging as well.

But which mappings would long mode / PAE use? Long mode uses 4 levels of page tables, so does that mean 4 mappings would be created (page tables, page dir, directory ptr and PML4)? The full page-table alias for 48-bit addresses would use too much memory (512G if I've calculated it correctly), so that would not be possible. The 32-bit code would probably only map the first 4G in 3 mappings. (page tables, page dir, and 4bit of directory pointer). The full mappings would require long mode to access.

[bluemoon]

I think you can reduce the use of selector by the idea of recursive page directory (this should work with segment model as well)
The benefit is that you no longer need to access page directory except startup code and process creation - any page fault you only need to access page table entry, including #PF of the memory holding 4 levels of page directories - this give an interesting side effect: The PF handler for 32-bit and 64-bit OS is almost identical:
Take mine as example:

Code: Select all

// -------------------------------------------------
MMU_PADDR MMU_alloc(void) {
    MMU_PADDR addr = 0;
    _INT_DISABLE();
    _SPIN_LOCK(&MMU_lock);
    if ( MMU_sp==0 ) goto fail;
    MMU_sp--;
    addr = MMU_stack[MMU_sp];
fail:
    _SPIN_UNLOCK(&MMU_lock);
    _INT_RESTORE();
    return addr;
}

// -------------------------------------------------
void INT_0E(long code, uint32_t addr, unsigned long ip) {
    uint32_t  page, prot, pd_index;
    uint32_t* pt;

    pt = MMU_PT(addr);
    if ( (code&1) == 0 ) {  // Page Not Present
        if (( pt[MMU_PT_INDEX(addr)] & MMU_K_ONDEMAND ) == 0 ) {
            kprintf ("  INT0E : #PF Page Fault Exception. IP:%X CODE:%d ADDR:%X\n"
                     "        : PD[%d] PT[%d]\n", ip, code, addr, MMU_PD_INDEX(addr), MMU_PT_INDEX(addr)); 
            kprintf ("    #PF : Access to unallocated memory.\n");
            kprintf ("        : ADDR: %X PT[%d]: %X\n", addr, MMU_PT_INDEX(addr), pt[MMU_PT_INDEX(addr)]);
            __asm volatile ("cli; hlt");
        }
    
        page = MMU_alloc();
        prot = pt[MMU_PT_INDEX(addr)] & MMU_PROT_MASK;
        pt[MMU_PT_INDEX(addr)] = page | prot | MMU_PROT_PRESENT;
        _INVLPG(addr);
        memset ((void*)((addr>>12)<<12), 0, 4096);
        
        if ( (pd_index=MMU_PD_INDEX(addr))>=512 ) {
            uint32_t* pd = MMU_PD(addr);
            k_PD[pd_index] = pd[pd_index];
            MMU_version++;
        }
    } else {
        kprintf ("  INT0E : #PF Page Fault Exception. IP:%X CODE:%d ADDR:%X\n"
                 "        : PD[%d] PT[%d]\n", ip, code, addr, MMU_PD_INDEX(addr), MMU_PT_INDEX(addr)); 
        kprintf ("      #PF : Access to protected memory.\n");
        kprintf ("          : ADDR: %X PTE[%d]: %X\n", addr, MMU_PT_INDEX(addr), pt[MMU_PT_INDEX(addr)]);
        __asm volatile ("cli; hlt");
    }
}

The 64-bit version only differ by the datatype change to uint64_t.



To access the full 4 levels of page directory in 32-bit mode (but why?), you may temporally map such node into the PDTE and travel one level.

[Brendan]

Hi,

To access the full 4 levels of page directory in 32-bit mode (but why?), you may temporally map such node into the PDTE and travel one level.

I think I know why: 64-bit applications, using a 32-bit kernel that can only access the first 4 GiB of the full 48-bit virtual address space.

There is (at least?) one alternative though - don't let 64-bit applications use the entire 48-bit virtual address space that the hardware is capable of.

For example, if you do a "recursive page directory pointer table trick" (instead of a "recursive PML4 trick") then 64-bit applications would be restricted to 512 GiB. In this case the mapping created by the trick would cost 1 GiB of the 4 GiB virtual address space that the kernel can access.

The next step down would be a "recursive page directory table trick", which would be mostly pointless on its own (64-bit applications would be restricted to 1 GiB). However, you could have 64 individual mappings (so that 64-bit applications are limited to 64 GiB of virtual address space, each mapping costs 2 MiB, and all 64 of the mappings cost a total of 128 MiB).

Of course if you're using multiple mappings, you could "bank switch" between them. For example, you could have 64 of these 1 GiB mappings where only one of them can be accessed by the 32-bit kernel at a time (and 64-bit applications would be limited to 32 TiB of space). It just means that to switch from one mapping to another you have modify a PDPT entry and then flush the daylights out of the TLBs (to make sure you don't accidentally access the old/stale mapping). For SMP (e.g. where several CPUs may be trying to modify different areas in different "banks" of the same virtual address space at the same time) you should be able to get by with a "per process" lock.


Cheers,

Brendan

[rdos]

I think I know why: 64-bit applications, using a 32-bit kernel that can only access the first 4 GiB of the full 48-bit virtual address space.

Wrong. The 32-bit kernel could:
1. Temporarily switch to long mode (by setting L bit), and use the full page-table mapping
2. Map the page directory pointer into a new area
3. Use long mode page fault handlers

There is (at least?) one alternative though - don't let 64-bit applications use the entire 48-bit virtual address space that the hardware is capable of.

No, there isn't. That's not an alternative.

For example, if you do a "recursive page directory pointer table trick" (instead of a "recursive PML4 trick") then 64-bit applications would be restricted to 512 GiB. In this case the mapping created by the trick would cost 1 GiB of the 4 GiB virtual address space that the kernel can access.

A better alternative is to map PML4 and page directory ptr like the page tables are mapped, and then take one page directory ptr entry and temporarily map it.

But none of this makes any sense since the pagefault handler active when 64-bit applications run is the long mode version.

[rdos]

I think you can reduce the use of selector by the idea of recursive page directory (this should work with segment model as well)

The selectors are mapped to a recursive page directory. They keep the aspects of this hidden by setting up the proper linear addresses.

To access the full 4 levels of page directory in 32-bit mode (but why?), you may temporally map such node into the PDTE and travel one level.

There probably is no reason as the code that maps buffers can be run before switching to 32-bit mode.

[Combuster]

I think I know why: 64-bit applications, using a 32-bit kernel that can only access the first 4 GiB of the full 48-bit virtual address space.

Wrong. The 32-bit kernel could:
1. Temporarily switch to long mode (by setting L bit), and use the full page-table mapping
2. Map the page directory pointer into a new area
3. Use long mode page fault handlers

And thus it ceases to be a 32-bit OS.

[rdos]

I think I know why: 64-bit applications, using a 32-bit kernel that can only access the first 4 GiB of the full 48-bit virtual address space.

Wrong. The 32-bit kernel could:
1. Temporarily switch to long mode (by setting L bit), and use the full page-table mapping
2. Map the page directory pointer into a new area
3. Use long mode page fault handlers

And thus it ceases to be a 32-bit OS.

It cannot cease to be something it never was. RDOS today already is a mixed-bitness OS, and adding long mode bitness would only complement that picture. The goal is to resuse existing code while still supporting 64-bit applications, and possibly device-drivers. There will be no "write a clean 64-bit OS" that doesn't support 32-bit, 16-bit or V86.

[rdos]

I think the optimal solution is to always map system-wide pages below 4GB, so they can be shared between 32-bit and 64-bit. The system wide page tables is what the system page dir and page table selectors currently map. Thus, the new design needs new selectors that maps the first 4GB page tables, page directory, and 2-bits of the directory pointer. 64-bit code can reference the page-structures either by calculating their linear addresses, or with FS/GS selector which still can contain a non-zero base. Alternatively, it can call a 32-bit handler to read system wide page contents. The full page tables are only needed for current process mappings for 64-bit applications, and may not need to be referenced by 32-bit code (other than as a debug-tool). The 32-bit kernel instead keeps process page tables that only maps 4G. The full page tables can be mapped as the last entry in PML4, and thus can be referenced at top of memory. 64-bit applications probably map the first PML4 entry (512G) of linear memory as "not present" or "kernel only" in order for unsafe flat code not to be able to access it in userland. That would mean 1T byte of 64-bit memory would be unavailable.

[Brendan]

Hi,

There will be no "write a clean 64-bit OS" that doesn't support 32-bit, 16-bit or V86.

There will also be no "write a clean 64-bit OS, then add support for 32-bit and 16-bit in the cleanest possible way".

Let's be honest here; every time you try to add anything to the OS the existing design makes it a pain in the neck (e.g. random pieces of code directly accessing page tables because there was no clean abstraction, making it hard to add support for PAE without changing lots of different pieces of code). I eventually suggest fixing the cause of the problem once and for all; and you respond by working around symptoms, and making your existing strange mess more strange and more messy.

Let's try something. Find a version of your source code from last year, and use "diff" to see how much source code has been rewritten/replaced/modified in the last 12 months. I'm guessing it's going to be close to 50% of the OS. Now think about the next 12 months - I'm guessing that you'll be rewriting/replacing/modifying another 50% to add 64-bit support. Ironically, 50 + 50 = 100. With the same amount of work that you're spending in a 24 month period, you could've created something modern, clean and maintainable to use for the 10+ years. That sounds like a good investment to me.

Of course a new/clean 64-bit version of the OS can still support the old (32-bit and 16-bit) applications (even the "skanky beyond belief" segmented stuff), if supporting these old applications is actually necessary. To be honest I doubt it is actually necessary - it's not a desktop OS, and for embedded systems, once they're deployed they're rarely touched or updated. It's not like ATM machines are downloading updates from "rdos.com" and auto-installing them every second Wednesday or anything like that.


Cheers,

Brendan

[rdos]

Let's be honest here; every time you try to add anything to the OS the existing design makes it a pain in the neck (e.g. random pieces of code directly accessing page tables because there was no clean abstraction, making it hard to add support for PAE without changing lots of different pieces of code). I eventually suggest fixing the cause of the problem once and for all; and you respond by working around symptoms, and making your existing strange mess more strange and more messy.

Doing direct page-table manipulations never was considered a problem, so no abstraction interface have been created. This is what I'll create now in order to be able to switch to PAE. This will make the design clean because no code would be allowed to directly access the page tables, and this can be garanteed by for instance renaming the selectors and the unportable access functions for physical memory. Thus, the design will not become a mess, but clean.

Let's try something. Find a version of your source code from last year, and use "diff" to see how much source code has been rewritten/replaced/modified in the last 12 months.

I just did. 1/3d of the files in the kernel haven't been modified at all since december 2011. Of those that are modified, the changes are in most cases small.

Maybe if you replace 12 months with 12 years it's more interesting, but looking at a source distribution from 2000, I still find many of the same modules, but they have been heavily modified. In fact, a lot of the code is more than 10 years old.

I'm guessing it's going to be close to 50% of the OS. Now think about the next 12 months - I'm guessing that you'll be rewriting/replacing/modifying another 50% to add 64-bit support. Ironically, 50 + 50 = 100. With the same amount of work that you're spending in a 24 month period, you could've created something modern, clean and maintainable to use for the 10+ years. That sounds like a good investment to me.

Multiply the figures with 10 and your guesses becomes more realistic.

I think the 64-bit specific code will be much less than 10% of the code-base, and concentrated to one or a few new device-drivers running in 64-bit mode. In the long run, it could be a larger part, especially if the file system is ported to userland using IPC. But I won't implement any real 64-bit devices in kernel, it will all be microkernel-based in userland. Because that is the only way to isolate it from critical parts of the kernel.

Of course a new/clean 64-bit version of the OS can still support the old (32-bit and 16-bit) applications (even the "skanky beyond belief" segmented stuff), if supporting these old applications is actually necessary. To be honest I doubt it is actually necessary - it's not a desktop OS, and for embedded systems, once they're deployed they're rarely touched or updated. It's not like ATM machines are downloading updates from "rdos.com" and auto-installing them every second Wednesday or anything like that.

We do regular updates currently about once a month. Some of these also replace parts of the OS kernel.

[Brendan]

Hi,

Let's be honest here; every time you try to add anything to the OS the existing design makes it a pain in the neck (e.g. random pieces of code directly accessing page tables because there was no clean abstraction, making it hard to add support for PAE without changing lots of different pieces of code). I eventually suggest fixing the cause of the problem once and for all; and you respond by working around symptoms, and making your existing strange mess more strange and more messy.

Doing direct page-table manipulations never was considered a problem, so no abstraction interface have been created. This is what I'll create now in order to be able to switch to PAE. This will make the design clean because no code would be allowed to directly access the page tables, and this can be garanteed by for instance renaming the selectors and the unportable access functions for physical memory. Thus, the design will not become a mess, but clean.

Doing direct page-table manipulations never was considered a problem (which highlights the poor quality of "engineering" that went into the original design of the OS), so no abstraction interface has been created (despite Intel introducing PAE about 20 years ago and AMD introducing long mode about 10 years ago). This will make the overall design a little cleaner, in the same way that polishing a hub cap will make a filthy car a little cleaner, but "a little cleaner" isn't the same as "clean".

Let's try something. Find a version of your source code from last year, and use "diff" to see how much source code has been rewritten/replaced/modified in the last 12 months.

I just did. 1/3d of the files in the kernel haven't been modified at all since december 2011. Of those that are modified, the changes are in most cases small.

I would've expected that adding SMP support (if it was done properly - e.g. with attention to scalability) would've caused over 1/3rd of the kernel's files to have been modified on its own.

Of course I'm not too sure exactly what is in your kernel - maybe there's a massive web browser or something built into it, and even though most of the scheduler and memory management has been rewritten a lot of "other" files didn't need to be touched.

Maybe if you replace 12 months with 12 years it's more interesting, but looking at a source distribution from 2000, I still find many of the same modules, but they have been heavily modified. In fact, a lot of the code is more than 10 years old.

If a lot has been heavily modified (e.g. 75%) and a lot is more than 10 years old (e.g. 75%); does that mean the project is 150% of an OS?

I'm guessing it's going to be close to 50% of the OS. Now think about the next 12 months - I'm guessing that you'll be rewriting/replacing/modifying another 50% to add 64-bit support. Ironically, 50 + 50 = 100. With the same amount of work that you're spending in a 24 month period, you could've created something modern, clean and maintainable to use for the 10+ years. That sounds like a good investment to me.

Multiply the figures with 10 and your guesses becomes more realistic.

I'm used to micro-kernels. To me, "the OS" is boot code, physical and virtual memory management, scheduler, IPC. It doesn't really include applications, GUI, file systems or drivers (even though it technically should); as I'm used to all of that being "user land" rather than a core part of the OS.

Despite this, I still think you're exaggerating. Write down each piece of your OS and how long you estimate it'd take to port to 64-bit or convert to "flat 32-bit" - it'd be fun to see if it adds up to anything close to 20 years (and possibly even more fun to figure out why).

I'll start (feel free to edit where necessary):

• 1 week for any changes to boot code
• 2 weeks to completely re-implement physical memory management as 64-bit
• 2 weeks to completely re-implement virtual memory management as 64-bit
• 2 weeks to completely re-implement the scheduler as 64-bit
• 1 week to completely re-implement IPC as 64-bit
• 1 week to add a "backward compatibility" API for flat 32-bit processes to use
• 1 week to get ACPICA working again
• 1 week to port the VFS to 64-bit
• 3 weeks (at 1 file system per week) to port 3 file systems to 64-bit
• 2 weeks to port the TCP/IP stack to 64-bit
• 50 weeks (at 2 device drivers per week) to update 100 existing device drivers
• 9 weeks for "miscellaneous"

This works out to 75 weeks. Maybe my "24 months" initial guess was too much.

Note 1: These estimates are based on someone who's done it all before (no time added for learning or research), and for parts that are "re-implemented" it assumes that existing code can be used as a rough guide. Basically, the estimate above wouldn't be appropriate for a beginner starting their first OS.

Note 2: I am assuming "8 hours per day, 5 days per week". If this assumption is wrong it could easily lead to a bad estimate. For example, if you spend 3 hours per week on the OS then the list above would come out at 1000 weeks (instead of 75 weeks), which would be about 20 years.


Cheers,

Brendan

[rdos]

Doing direct page-table manipulations never was considered a problem, so no abstraction interface have been created. This is what I'll create now in order to be able to switch to PAE. This will make the design clean because no code would be allowed to directly access the page tables, and this can be garanteed by for instance renaming the selectors and the unportable access functions for physical memory. Thus, the design will not become a mess, but clean.

Most of the code is well-behaved and uses 6 kernel APIs to allocate physical memory and page-tables. That accounts for perhaps 80% of the occurences. The only problem here is that physical addresses needs to be extended to 64-bits, and a second general register needs to be used. Some device-drivers also cannot tolerate above 4G addresses, so there needs to be two different calls for allocating physical memory: One for any address, and one for an address below 4G (there is also an historic call for AT DMA which has even fewer bits).

I estimate to fix this issue in a few hours.

I would've expected that adding SMP support (if it was done properly - e.g. with attention to scalability) would've caused over 1/3rd of the kernel's files to have been modified on its own.

Most of the changes are not related to SMP-support. Most of the SMP-related issues were related to using cli/sti instead of spinlocks. Code that used synchronization primitives had no issues at all.

Despite this, I still think you're exaggerating. Write down each piece of your OS and how long you estimate it'd take to port to 64-bit or convert to "flat 32-bit" - it'd be fun to see if it adds up to anything close to 20 years (and possibly even more fun to figure out why).

I'll start (feel free to edit where necessary):

• 1 week for any changes to boot code
• 2 weeks to completely re-implement physical memory management as 64-bit
• 2 weeks to completely re-implement virtual memory management as 64-bit
• 2 weeks to completely re-implement the scheduler as 64-bit
• 1 week to completely re-implement IPC as 64-bit
• 1 week to add a "backward compatibility" API for flat 32-bit processes to use
• 1 week to get ACPICA working again
• 1 week to port the VFS to 64-bit
• 3 weeks (at 1 file system per week) to port 3 file systems to 64-bit
• 2 weeks to port the TCP/IP stack to 64-bit
• 50 weeks (at 2 device drivers per week) to update 100 existing device drivers
• 9 weeks for "miscellaneous"

This works out to 75 weeks. Maybe my "24 months" initial guess was too much.

Note 1: These estimates are based on someone who's done it all before (no time added for learning or research), and for parts that are "re-implemented" it assumes that existing code can be used as a rough guide. Basically, the estimate above wouldn't be appropriate for a beginner starting their first OS.

Note 2: I am assuming "8 hours per day, 5 days per week". If this assumption is wrong it could easily lead to a bad estimate. For example, if you spend 3 hours per week on the OS then the list above would come out at 1000 weeks (instead of 75 weeks), which would be about 20 years.

Use something in between (like 8 hours per week), and add many more issues to your list, and you are still close to 10-20 years.

Some more issues:

• Sound
• USB
• Graphics
• Executable loaders
• Com-ports

[Brendan]

Hi,

I would've expected that adding SMP support (if it was done properly - e.g. with attention to scalability) would've caused over 1/3rd of the kernel's files to have been modified on its own.

Most of the changes are not related to SMP-support. Most of the SMP-related issues were related to using cli/sti instead of spinlocks. Code that used synchronization primitives had no issues at all.

So you're saying that to add SMP support, all you did was replace CLI/STI with spinlocks; and you didn't redesign anything for scalability (e.g. change algorithms and data structures, to avoiding lock contention, reducing cache effects, etc)?

I would've expected that adding SMP support (if it was done properly - e.g. with attention to scalability) would've caused over 1/3rd of the kernel's files to have been modified on its own (but I guess you didn't do it properly, with no attention to scalability at all, and this is why so few files were changed).

Note 2: I am assuming "8 hours per day, 5 days per week". If this assumption is wrong it could easily lead to a bad estimate. For example, if you spend 3 hours per week on the OS then the list above would come out at 1000 weeks (instead of 75 weeks), which would be about 20 years.

Use something in between (like 8 hours per week), and add many more issues to your list, and you are still close to 10-20 years.

Ok - at 8 hours per week my estimate would become about 375 weeks, or 7.2 years.

Some more issues:

• Sound
• USB
• Graphics
• Executable loaders
• Com-ports

I counted all of that under "50 weeks (at 2 device drivers per week) to update 100 existing device drivers"; unless you're saying there's 100 drivers plus those extra 5.


Cheers,

Brendan

[rdos]

About some 8 hours of work, and the whole kernel now works with 64-bit physical addresses. There are a few things I need to look closer at later. The AHCI driver uses filesystem buffers that might have above 4G physical addresses, so I need to address that issue. The same goes for USB drivers, that use buffers that are allocated, and thus could have above 4G addresses. The other devices (like network, sound, floppy, IDE) have no such issues and either don't use physical addresses, or allocates the buffers themselves below 4G.

The issues with some code using the page-tables directly are also left to solve. The bulk of this could should be possible to port to the existing page-table API.

I also really need a computer with more than 4G of memory to test this out, which I currently don't have.

[Owen]

Hi,

I would've expected that adding SMP support (if it was done properly - e.g. with attention to scalability) would've caused over 1/3rd of the kernel's files to have been modified on its own.

Most of the changes are not related to SMP-support. Most of the SMP-related issues were related to using cli/sti instead of spinlocks. Code that used synchronization primitives had no issues at all.

So you're saying that to add SMP support, all you did was replace CLI/STI with spinlocks; and you didn't redesign anything for scalability (e.g. change algorithms and data structures, to avoiding lock contention, reducing cache effects, etc)?

I would've expected that adding SMP support (if it was done properly - e.g. with attention to scalability) would've caused over 1/3rd of the kernel's files to have been modified on its own (but I guess you didn't do it properly, with no attention to scalability at all, and this is why so few files were changed).

Theres a saying - great is the enemy of good.

Or - If I had infinite time, I'd implement a brilliantly scalable scheduler, have finely tuned lock granularity, and be using the optimal algorithms everywhere.

But I don't - and I mostly would like things to work. Therefore, I have a memory manager which is single threaded, my scheduler doesn't even attempt to minimise context switches, and I could probably make my memory barriers more precise and hence more efficient. But you know what? I'm fine with that. I've tried to keep my interfaces as clean as possible, but I expect them to change. I know there are parts of my memory management system which will change; and that there are parts of my system which are more tightly coupled than ideal.

However, "Great is the enemy of good" is, in my view, an important policy. It is, in many ways, a generalisation of "Premature optimisation is the root of all evil."

I don't agree with all of the design decisions rdos makes - It's been pretty clear for years that x86 segmentation is a deprecated feature, and that its performance and support going forward is going to decrease. If I had a OS which was built upon segmentation and assembly, I'd be working towards

• Making the assembly portions able to run in Long mode's compatibility submode. This would primarily involve removal of use of call gates and other system segment descriptors removed in long mode.
• Migrating the whole thing, over time, to C, or C++, or some other language in which things like pointer size don't get encoded into all my source code

(I'd evaluate - on a case by case basis - which modules would be relatively straightforward to move to compatibility submode vs those which it would be simpler, or as simple, to just rewrite now, so that given a bit of time, I would eventually have an OS which could run natively on AMD64)

In the end, I'd want an OS which ran on at least IA-32, AMD64, ARMv7 and ARMv8 AArch64.

But a complete rewrite? Rewrites are almost always completely destined to fail and often suffer from the second system effect.

I think by now most of us - especially rdos - know that you think he should rewrite his OS from scratch.

In the past, some have expressed complaints at rdos pushing segmentation all over the forum, and he seems to have toned it down. However much you or I disagree with his choices - obviously you more than I - the continuous pushing of the opposite also becomes annoying.

Theres a time and a place for segmented vs unsegmented design discussion - and thats design discussion - not a thread about the implementation of a new paging method.

I enjoy reading rdos' posts - they provide an interesting change of pace. In many ways they remind me of Raymond Chen's posts on The Old New Thing, or the commentary from the original Mac team, or the documentation for Mac OS Classic or AmigaOS - they describe a system which is radically different, and a design team working under interesting and unusual design constraints.

Phew. That came out long. Sorry about that. </rant>