Hi,
bewing wrote:Brendan wrote:32-bit 80x86 Apple computers use EFI already.
Well, then either it's going to be a standard, or Apple has jumped the gun again, and screwed itself again. I haven't looked through the EFI spec completely, but it uses too much space for partitions, etc., and I'm really scared that it will do a really stupid job of not passing me enough hardware specific data about the machine. And may not support dual-booting (multi-booting) and alternative OSes properly or prettily.
EFI is a standard, but it's a complex one - it's almost like there's a "mini-OS" being used to boot the real OS, with full (optional) command line interface, it's own device drivers, it's own bootable applications, etc. Part of the idea is that people can write OSs in high level languages and recompile them for different platforms with minimal changes to the boot code, etc (where "different platforms" means some 80x86 machines and all Itanium machines). Support for dual-booting isn't a problem though...
bewing wrote:Brendan wrote:The main problem isn't competition between manufacturers, but the addition of new features.
Exactly. So you always have to pick a minimum level of chip to support, and then deny support of new features after that point.
If you say the minimum requirement is a P6 you can't assume a P6 (and all the features "P6" includes) are present - you have to test and make sure, because otherwise someone will try to run it on a Pentium and complain that it crashed. This isn't necessarily as easy as it sounds - for e.g. if you get the CPU family from CPUID and make sure it's 6 or higher, but then the CPU could be anything from a dodgy Cyrix chip (which lacks some of the features you'd expect in an Intel 80686) to the latest Core Duo or Opteron.
From a "commercial product" perspective it then becomes difficult to get people to understand exactly what is required. If you claim the OS works on any Intel Pentium Pro or later CPU, then does that include a Cyrix MediaGX or AMD K6? If you explicitly state each feature that the OS requires (PAE, PSE, MMX/SSE1, etc) then people will just get confused (ask the average computer user what sort of CPU they've got and they'll probably say something like "I don't know, but it says 'Dell' on the front" - this is the main reason for Microsoft's "Vista Ready" marketting you'll see if you're looking to buy a new computer).
I guess this mostly depends on the target market for your OS and the OS's design. From my perspective, anyone trying to get into the desktop/server market is going to have a hard time - it'd be much easier to get your OS used in (for e.g.) embedded systems, where 80486 is still being used. Because of this I set 80486 as my minimum requirement and hope that my code will be usable for both embedded systems and desktop/server, so that it could gain users in embedded and then grow into desktop/server from there (rather than starting with direct competition with Microsoft and Linux - a strategy I'd call "suicide").
So, with 80486 as my minimum requirement can I realistically expect a desktop or server user to run an OS that doesn't support SMP, PAE, SSE, write-combining and other newer features? Quite frankly, no (after many years of work I wouldn't be able to recommend my own OS without being dishonest).
With this in mind you end up setting the minimum CPU required *and* the maximum CPU supported properly. For example, an OS could be designed for P6 to Pentium 4, where it'll take advantage of new features introduced in Pentium 4 (and will actually run on the a brand new CoreDuo, but won't take advantage of new features introduced with CoreDuo).
In general, limiting the range of properly supported CPUs is similar to limiting the quality of the OS. It makes the OS easier to design and implement, but makes it harder to find anyone willing to actually use the OS (i.e. it limits the target market); and to be perfectly honest, except for educational purposes, there's no point wasting your time writing an OS if you don't at least hope that someone might actually use it one day. To me this means that once you go beyond "educational purposes" you really can't afford to limit the OS if it's not necessary, and therefore can't afford to deny support of new features if it's not necessary.
Now if you understand my rambling (even if you don't necessarily agree), then it's time for what I'd consider the single most important thing I've learnt about developing OSs in the last ten years or more:
the code is worthless, the design is everything.
What I mean by this is that it's possible to create an OS design that is capable of being extended, and that failing to allow for extending the OS is the worst possible thing you can do.
Basically the OS must be designed to handle everything conceivable (including new CPU features, new hardware, new ways of doing things, different ways of booting, etc) while the OS's implementation will only support a limited subset of these things at any given point in time. The opposite would be an OS that is designed for a limited subset of things that will probably need to be (at least partially) redesigned and rewritten every time you need to support something new.
bewing wrote:You picked the 1st gen of 64 bit chips as your minimum level -- I picked the P6 generation of the older series. But neither one of us is going to be supporting the NEXT set of new features after this. So it's the same deal, except you support one design level higher than me. If you keep trying to support the newest features, that means your OS will never be finalized.
No.
My OS is (hopefully) designed to handle everything conceivable, and I'm supporting everything from 80486 to stuff that hasn't been invented yet. There are exceptions to this - inserting and removing hot plug CPUs, and the removal (but not insertion) of hot-plug RAM. My justification for this is that the OS is designed for redundancy, so turning a computer off to change CPUs or remove RAM shouldn't be a problem (even for "mission critical" servers).
If I keep trying to support the newest features then my OS may never be finalized, but if I fail to try there's no point finalizing it (it'll be worthless/obsolete anyway), unless I start again from scratch or do major redesign & changes (which takes longer than trying to support the newest features).
bewing wrote:Brendan wrote:In the last 7 years I've progressed a lot - I learnt how crappy my OS was ....
I think it would be
extremely educational if you were to create a posting on "advanced kernel design" that specified what issues you found in your 1st version that were crappy and needed a redo. I saw your post on the other thread about your IPC message buffers being too big, for one thing.
My biggest mistake would be taking too long to realise that the OS must be designed to handle everything conceivable.
My initial designs were mostly "self-education", but the last in the series could be described as "single CPU, single user, designed for 80386 only". I decided I needed to support PAE and the code wasn't extendable enough and there were other things I didn't quite like, so I rewrote.
The next was "single CPU, single user, designed for 80386 to Pentium II". I decided I needed to support multi-user and the code wasn't extendable enough and there were other things I didn't quite like, so I rewrote.
The next was "single CPU, multi-user, designed for 80386 to Pentium III". I decided I needed to support SMP and the code wasn't extendable enough and there were other things I didn't quite like, so I rewrote.
The next was "multi-CPU, multi-user, designed for 80386 to (32-bit) Pentium 4". I decided I needed to change the scheduler and the code wasn't extendable enough and there were other things I didn't quite like, so I rewrote.
The next was "modular, multi-CPU, multi-user, designed for 80486 to (64-bit) Pentium 4". It was mostly an experiment that wasn't anywhere near as complete as the others, but I decided I needed to support headless systems and diskless systems and the (small) amount of code I had wasn't extendable enough and there were other things I didn't quite like, so I rewrote.
Of course there were a lot of smaller things I didn't like (but they all could've been fixed if the OS was designed to be extendable).
One of them was "variable frequency scheduling". The idea is that all tasks get the same length time slices, but higher priority tasks get more of them - for example a task might get one 2 ms time slice and another task with twice as much priority might get 2 time slices at 2 mS each. This worked very well for my earlier OSs and gave very smooth and fair scheduling, but the problem is deciding who gets the CPU next. The best algorithm I could come up with (worst case) involved scanning a table of "ready to run" tasks twice - an "O(2*N)" algorithm, which sucked when there's a large number of tasks that are ready to run. I still really like "variable frequency scheduling" though (but I won't be using it again either).
Another was letting user-level code access (read-only) data structures in kernel space. It sounds nice and easy - for e.g. an application calls the kernel API and the kernel returns a pointer to FOO, or the OS is designed so that an application can directly read FOO without calling the kernel API (saves some cycles). It's a bad idea though - as soon as you change the address of FOO or try to get 32-bit applications to run on a 64-bit kernel it breaks.
Assuming that the OS can always have direct access to video and keyboard is another bad idea. Previous versions of my OS had full graphical menu driven boot code where (during installation) the OS expected the user to make decisions before the kernel was started. If the OS is running on a machine with a touch screen and no keyboard, or it's a headless machine (no video and keyboard) then you end up with an OS than can't boot.
Supporting SMP is another one. Now and then I hear people say that it's easy to add SMP to an existing (single-CPU) OS - don't believe them (they haven't tried it). What you end up with is huge re-entrancy problems (deadlocks) and/or bad performance due to lock contention, and you'll need to redesign just about everything to fix it.
Then there's little annoyances - some memory management code that could be optimized a bit, a scheduler that doesn't handle something just right, some messy and badly documented boot code, etc. By themselves they're all fixable, but if you don't fix them as soon as possible they start to add up. Before long "Yay, my nice clean OS" starts to become "Hmm, that OS thing again" and you start looking closer and start finding more little annoyances. That's when you realise there's a larger problem somewhere, or when Intel and AMD introduce some new feature (like virtualization extensions).
Anyway, my newest design similar to the last, just more modular - e.g. at least 7 different/seperate modules before it leaves real mode (boot loader, boot manager, user interface, CPU detection, physical address space manager, domain manager, kernel setup code), a micro-kernel made from at least 7 different/seperate modules (CPU manager, physical memory manager, linear memory manager, scheduler, IPC, I/O manager, critical error handler), with explicit "specifications" for the interfaces between each module and no hidden dependancies that'll be forgotten about a month later (e.g. IPC code that directly changes page table entries so everything breaks when you change something in the linear memory manager).
I guess you could say I've finally learnt to make the OS extendable. The idea is that anyone will be able to replace any module without looking at any of the code for any other modules, just by using the specifications and documentation.
For example, currently there's 3 different boot loaders that can be used (one for booting from floppy, one for booting from a network via. PXE, one for booting from ROM) where none of the other code cares which is used. Then there's 2 seperate user interface modules (one for keyboard/video and one for serial/dumb terminal). There's also a "pretend" boot manager that actually decompresses the boot image and starts the real boot manager, so that I can (recursively) have compressed boot images where none of the other code cares if the boot image is/was compressed or not.
If someone wants to write a new user interface module that uses the sound card (speakers and microphone) and speech recognition, then that won't be a problem. If someone wants to boot the OS from thier home made PCI card that's easy enough too. They will need to write the code for it, but they won't need to learn all the messy details of every other piece of code before they do.
This will continue - I'll have several linear memory managers, several schedulers, several IPC modules, etc. If someone writes a much more efficient IPC module or wants to do research on different scheduling algorithms, then they can just put their module into a standard boot image.
More importantly, if I stuff something up or find out something has performance problems (or discover a better/faster way to do something) I can replace that part without caring about the rest of the OS, if AMD invent a new way of doing paging in long mode it'll be easy for me to support it, and when Intel release "super extended hyper-media enhancement technology" I'll be detecting it and using it before most people figure out what it actually is.
bewing wrote:Brendan wrote:I doubt anyone buying a new 80x86 desktop/server will get a 32-bit CPU.
LOL! E-Machines is a
huge moneymaker, selling 32bit machines for $400 each. I would bet money that 75 to 80% of machines sold in '07 will be cheap 32 bit, until 64bit and multi-core machines come down below $600.
You might want to double check that.
For an example, have a look at
this $500 computer from eMachines that comes with
this Intel Celeron D 356 (which is actually a 65nm "Pentium 4" class CPU with a 512 KB L2 cache running at 3.33 GHz with a 533 MHz front side bus that supports 64-bit and "No eXecute")....
bewing wrote:RE: web based computers
Once upon a time (a long time ago) people used mainframes with dumb terminals. They hated it, but it was economical because computers were expensive and took up several rooms. Since then every 10 years or so some smart twit decides to re-invent something similar (IIRC the last twit was Sun trying to push "thin-clients", although to be fair it's a little different when you own both the servers and the clients).
In general it doesn't work, just like convincing people to use public transport doesn't work - it's not "theirs". As the price of hardware keeps dropping the viability of a "network OS" for the general public will keep decreasing (excluding temporary fads).
The technology to do it has been around for about 30 years - you only need an X client (or VNC client) on each computer, large enough server/s and enough bandwidth. Of course the idea of using a large bloated layer of HTML is new, but I can't see how worse technology will make a bad idea better (excluding temporary fads).
Cheers,
Brendan
](./images/smilies/eusa_wall.gif "Brick wall"){kind=small}
, what file sys do they use ?.
, but if you code a good browser, you will have a long list of user program 
.
grow up..