An interesting operating system design

[thre3dee]

Hi everyone,

A mate of mine (who's done some 'toy' bootstrapping before) and I have been toying with the idea of designing and developing a 'research' OS based almost entirely on objects. Btw, you don't need to warn us about the development of everything as we know how much work goes into even a simple kernel (and we hope we get even that far!) because its more of a practical, fun exercise in trying to design an operating system that deviates from the files, folders, square window UIs style generic modern operating system.

Our basic premise is that eveything in the system is an object of some sort. In code any hardware, file or runtime (three main types) object inherits from a SystemObject class which consequently inherits from a code-wide Object base class. A system object may represent a file, a process, a network device, a game controller (and any other type of system object).

Not only is the system represented by an object but so is the filesystem. In fact, the storage functionality more resembles a heirarchical file/folder system than actually is one.

For example, you may have two partitions on your drive for use with the OS (we've named it Objective-OS for the moment) so each is represented by a VolumeObject under the 'Volumes' collection. Under that are FileObject's. The Object file heirarchy resembles a folder heirarchy however with one key distinction; folders are files themselves and contain child file objects.

For example, your system might contain such runtime objects (linked to their respective hardware and system objects such as a filesystem or process):

Code: Select all

+ Volumes as VolumeObject collection
	+ System as VolumeObject equivalent to C:\ or Macintosh HD
	+ Backup as VolumeObject
		+ Media as generic FileObject containing no data
			- MySong as AudioFileObject of sub audio type "MP3"
		+ Docs as generic FileObject containing no data
			+ index as HTMLFileObject
				- topics as XMLFileObject

+ Devices
	+ USBDevices as USBDeviceObject collection
		- GameControllers as GameControllerObject collection

+ Programs
	+ Shell as ProgramObject containing children
		- Thread1 as ThreadObject

Such a model would make developing applications simpler and quicker. A program would be an assembly with child file objects that contain any resources it might use during execution.

Think of "MyApplication.Images.WindowBackground" as "program.exe/images/windowbackground.jpg" in a familiar environment.

MyApplication is an executable with child resource file objects. You run the application and its working object is itself so it may simply do a OpenRelative("Images.WindowBackground") call in code.

For example, in C++ you might do this:

Code: Select all

OS::ImageFileObject *image =
	OS::OpenRelative("Images.WindowBackground")->As <OS::ImageFileObject>();
if (image) {
	image->SetPixel(100, 23);
	image->Save();
} else {
	// create some random image buffer and create image file object
	image = new OS::ImageFileObject(myImageData, 100, 100, OS::ImageFileTypeJPEG);

	// attach to our current program object
	OS::GetCurrentFileObject()->AttachChild(image);

	// save image (write object to the filesystem)
	image->Save();
}

// Release image object (and file handle)
image->Release();

Upon the OpenRelative call, the system framework first finds the object (which turns out to be a file object), checks some metadata and finds that it is a JPEG Image file, checks against a number of file loaders with this information and the ImageFileOpener interface loads it into an ImageFileObject. When the programmer calls the SetPixel function, the image data is loaded (lazy loading) and the pixel set.


This isn't all of our thoughts on the design but it'd great to hear your thoughts about such a model and the pros and cons. Like I said, we probably might never get far given operating systems are a monstrous amount of work no matter what you do, but this is a research OS design so we can design it however we want

[AJ]

Hi and Welcome!

I am programming my boot loader and kernel in C++ and use much the same system for internal use (each object in the kernel inherits from Object, so for debugging purposes each object can be assumed to implement certain methods). So far, my design does not extend this in to userland. Ideally, I want the majority of userland code in C#.

This brings me to a suggestion. Have you considered making more use of managed code? It's not for everyone and could actually make you more work, but some of your framework stuff already sounds a little remenicent of C#.

One thing I struggled with to start with was my VFS. I had an Object of type Node which was inherited by Mountpoint, File, Directory, Link, etc... Sorting out in my head the "call chain" when you called a "write" property on one of these nodes got a little too convoluted and I am currently redesigning. There's not really a moral to this except "be careful when planning a new type of VFS"!

In fact, if you are planning anything "new", remember to put a lot of effort in to your design document and try to stick with it!

Good luck and I'll watch with interest.

Cheers,
Adam

[Love4Boobies]

Don't forget to check out http://research.microsoft.com/os/Singularity/ while you're at it .

[thre3dee]

Hi and Welcome!

I am programming my boot loader and kernel in C++ and use much the same system for internal use (each object in the kernel inherits from Object, so for debugging purposes each object can be assumed to implement certain methods). So far, my design does not extend this in to userland. Ideally, I want the majority of userland code in C#.

This brings me to a suggestion. Have you considered making more use of managed code? It's not for everyone and could actually make you more work, but some of your framework stuff already sounds a little remenicent of C#.

One thing I struggled with to start with was my VFS. I had an Object of type Node which was inherited by Mountpoint, File, Directory, Link, etc... Sorting out in my head the "call chain" when you called a "write" property on one of these nodes got a little too convoluted and I am currently redesigning. There's not really a moral to this except "be careful when planning a new type of VFS"!

In fact, if you are planning anything "new", remember to put a lot of effort in to your design document and try to stick with it!

Good luck and I'll watch with interest.

Cheers,
Adam

Thanks for your comments. Yeah, there'll definitely be a lot of design going into the system Object abstraction. Essentially, the biggest reason for this design choice is programmability. To create even more work for ourselves, I'm actually designing a natively compiled C++/C# based programming language called N (possibly after 'Native') that will be the user programming language. The language fixes some syntax annoyances with C++, and incorporates some C# features such as properties.

Also, our dynamic library design will resemble .Net assemblies; that is, native compiled funcions and classes with public code meta information embedded in the binary. Essentially no including header files. But even though it sounds a lot like C#, the main difference is that the language restrictions and simple background runtime checks for things like array indexing means that it can be compiled to native fast machine code. It's like a safe version of C++ without virtualizing it.

[thre3dee]

Here's an early sample of the language:

Code: Select all

module OS
{
	#summary Thrown when an Object contains an invalid reference count (i.e. 0)
	public exception ObjectReferenceException
	{
		#summary The object that caused the exception.
		Object ref Instance;

		#summary
		public constructor (Object ref instance)
		{
			reason = "Object at 0x" + addressof(Instance).ToString("%p") + " has an invalid reference count";
		}
	}


	#summary Base class for all subclasses
	public superclass Object
	{
		#summary The object reference count.
		covert dword ReferenceCount;

		#summary Default constructor
		public constructor ()
			[ReferenceCount = 1]
		{
		}

		#summary Destructor
		public virtual destructor
		{
		}

		#summary Retains an instance of the object.
		public Retain ()
		{
			if (ReferenceCount == 0)
				throw(ObjectReferenceException(self));
			else
				++ReferenceCount;
		}

		#summary Releases an instance, reclaiming the object when none are left
		public Release ()
		{
			if (ReferenceCount == 0)
				throw(ObjectReferenceException(self));
			else if (--ReferenceCount == 0) {
				reclaim self;
			}
		}

		#summary Returns a string representation of the object.
		public virtual String ToString ()
		{
			return "OS.Object";
		}
	}
}

module Containers
{
	public open subclass Array <T> of Object
	{
		public T ref[] Buffer;
		public dword Size;
	}
}

As you can see its sort of a hybrid of C++ and C# however there's some useful additions and differences.

• Forward declaration is present as the compiler will gather all code information into its tables first.
• A ref is merely a safer pointer. The pointer is either null or pointing to a valid object (in its lifetime) and cannot have arithmetic applied to it.
• A simple ref only points to one object (as opposed to C++ pointers which have knowledge of their data).
• A ref [] can point to an array of objects and has an embedded count in the memory block. For example: char ref[] str = new char[5]; looks like this in memory: dword[5] { '\0', '\0', '\0', '\0', '\0' }
• Given the above array model, the countof(array) keyword finds this array count which allows native foreach(char c in char[]) as the foreach merely refers to this embedded count number.
• Superclasses cannot be instantiated (such as a pure virtual base class.
• A template is an open class and requires less syntax to declare the types etc.
• get/set C# properties are in the language.
• An exception type is now present. The reason keyword is a string which is displayed either in a debugger or in an exception window. Its simply an Exception.String reason;
• A new covert keyword is in. Basically means private however no record of it is entered into the assembly. Its basically a hidden class member.

Those are the main differences I think.

[Walling]

Your project sounds really interesting. I'll keep an eye on it! I'm planning to implement my own language, but for now I'm just playing around with treetop, a nice grammar parser for Ruby.

[thre3dee]

oh and while normal comments are the same as C++, it also supports various documentation comments beginning with # such as #summary, #param and #remarks which are built into the assembly for both generating documentation and also code sense in IDEs.

[Walling]

Can you parse anything yet? Compile anything? I'll look forward to hear about it and eventually try it myself if you release it to public. (open source?)

[thre3dee]

Well we only started the whole project about two weeks ago and we're still getting to grips with what is required in even a basic kernel. Our first goal will be to make the bootloader run a simple C main loop then we can go from there. I've done a bit of script language development (never finished one) so I have an idea what is required but I'll be fully designing and documenting the programming language before I attempt to make a compiler.

What I hope to have in the end (prolly in a thousand years time due to the amount of stuff I want in it) is the N programming language and the Native Code Framework for N that will be the basis of all applications on the system. Essentially we want the user experience systems (such as the shell) built in N with the NCFramework as well and have only the core kernel and drivers written in C/C++.

Once I've got the N Reference documentation written I'll post it on here if you're interested in checking out the language.

[mbluett]

Interesting concepts.

Have you looked at Oberon (and/or AoS). These are completely object oriented O/S's.

As to the filesystem, I have been thinking of using a database instead of a traditional filesystem. In this way modifications to the filesystem can be done very easily. Essentially there is just one physical file, but all the O/S knows about is a database.

Performance wise there should not be any problems: Database accesses (with indexing) are very fast.

Another issue that has arisen in my work in helping people with their computing complications is that the current O/S designs are way to complicated. I realize the O/S has been designed to provide use for a range of user types (i.e., novices to experts) and as a result is complex. But no O/S vendor yet has created a Novice mode whereby a lot of the complexity is reduced.

There are other issues the user indirectly comes in contact with. For example, driver problems.

Windows has always been plagued with driver problems sometimes crashing the O/S. This should never occur. If a driver is at fault the most it should do is cause problems with access to a particular device or set of devices.

There should be no reason why the O/S can't run in Ring 0 and the drivers run in Ring 1. This would prevent a driver from killing the O/S. It would also prevent the annoying reloading of the O/S to remove and/or add a new driver. By the way I am aware that Linux can load and unload drivers dynamically. However, I believe bad Linux drivers can still kill the O/S just as they can under Windows.

Back to the GUI: Most users do not understand how to use windows. In fact, mostly they will use one application in full screen mode and once finished with it they close it and move onto another application.

In my opinion, there should be an O/S that does away with the concept of windows (at least for these type of users), thus simplifying their computer use experience.

As to the programming libraries of a specific O/S, there are just too many and a lot of redundant functionality causing great confusion. Would it not be better to have one set of functions that the whole O/S and user applications depend on?

What I was thinking would be an object-oriented O/S that after run would present the user with a GUI with the typical user functions they require (e.g., email function, Internet Browser function, etc). From one function to the next there might be some layout changes and/or some menu changes, but that's it. Each function would be full screen. As well, each function would be making use of the same callable modules (object methods) that would either be in memory or would be loaded when required. Each user function would be described by a database meta-object which would list what function modules it uses.

Because there is only one class library on the system, programmers would be spending their time improving what already exists or adding new functionality rather than constantly re-inventing the wheel.

Also with regards to programming, it is way too complex for the average general user functions that are required. I would envision a situation where a user function programmer would create a meta-object which describes a set of processes and the O/S takes care of selecting the appropriate classes to use and appropriately configuring them. Of course the set of classes have to be created for use by the system. This would be the more complex programming, but it is done once and that is it.

The time a user function programmer would spend, would be in the layout of the user function (e.g., columns, function groupings, screen areas, etc) and associated interactions. This would eliminate the complexities of using pointers, variable declarations, the use of templates, the use of generics, ArrayLists as opposed to ordinary arrays, syntax issues, etc. etc. etc.

Comments?

[DeletedAccount]

Hi ,
Your design looks interesting , I think , I am seeing a well thought out design after a long time . I will look forward to it .

Regards
Thomas

[01000101]

This reminds me a lot of the AS/400 OS. It's files, 'folders', and libraries are all objects and so are programs and objects that compile them.

If you go the route of objects, please look at that OS and do 'almost' everything aesthetically different! It takes the complexity of a command line, and increases it 10-fold due to the odd sentence-like commands and object-based everything.

[JAAman]

Interesting concepts.
Windows has always been plagued with driver problems sometimes crashing the O/S. This should never occur. If a driver is at fault the most it should do is cause problems with access to a particular device or set of devices.

There should be no reason why the O/S can't run in Ring 0 and the drivers run in Ring 1. This would prevent a driver from killing the O/S.

actually, windows already does this (and has since at least win95, iirc) unfortunately most driver writers have refused to comply with this, and instead use the old ring0 driver system -- one of the biggest problems people have had with vista (also true of 64bit x86 versions of both vista and xp) is that it forces almost all drivers to run in ring3 (windows avoids use of ring1/2) which breaks compatibility with drivers that have refused to run in ring3 despite MS asking all driver writers to do so for the last 10-15 years... (although note that all drivers that came on the windows XP disc, and probably win95/98 also are all ring3 drivers)

It would also prevent the annoying reloading of the O/S to remove and/or add a new driver.

windows already does this, and actually does not require a reboot to load/unload ring3 drivers (at least under both XP and vista) -- what does require a reboot is the driver installation software -- and there really isnt any good reason most of the time -- and vista can recover from a hard driver crash without any problems or rebooting required (Runescape-fullscreen tends to crash my video drivers on occasion, when it happens, i can simply terminate the program, and windows will restart the drivers, and notifies me that it has done so)

windows vista is actually specifically designed to never shutdown, for any reason, though i still recommend restarting at least once every couple months -- or whenever it wants to restart for updates (firefox tends to cause a lot of proplems if its not shut down... so at least restart that if not the whole computer) -- although restarting for updates isnt actually neccessary anymore either since vista sp1 can install patches without restarting the system, but for some reason MS has chosen not to use this feature very much (or disable it by default) which means its probably still not 100% reliable

by default, 'shutting down' is actually a suspend state, which, according to my meter readings, takes exactly the same amount of power as completely shutting off the system, but allows you to restart in less than 2 seconds (although for some reason it takes a bit to reestablish internet connection -- at least on my system -- though that might be firefox, rather than windows, having that problem), and doesnt close programs or change anything -- sometimes i will actually highlight part of a web page before "shutting down" the system, so that when it comes back, i can remember exactly where on the page a was reading when i left it (no, this is not the same as hibernate -- which is also available -- and will be used automatically if you suspend and then loose power or unplug the system)

[mbluett]

actually, windows already does this (and has since at least win95, iirc) unfortunately most driver writers have refused to comply with this, and instead use the old ring0 driver system...

If I was designing an O/S that my reputation stands on, I would force the driver vendors to do it the way I wanted.

As to Vista and from what I have read, they still have the same driver policy as with XP (i.e., they strongly suggest drivers be written for user-mode, but don't force the issue).

windows already does this, and actually does not require a reboot to load/unload ring3 drivers (at least under both XP and vista) -- what does require a reboot is the driver installation software -- ...

I have seen conditions where a driver does not get removed until after a reboot. I have de-installed something like Norton (which uses drivers). If you subsequently look at the drivers after the de-install and before a reboot, you will see Norton drivers still in operation. After the reboot they are gone. These drivers could be Ring 0 drivers (I don't know). All I know is this kind of thing happens frequently.

I don't know whether this same condition occurs under Vista as I have had no need to upgrade to it. I would rather keep my $130 in the bank (or use it for something else). There is just not enough advantage in Vista to justify the cost.

firefox tends to cause a lot of proplems if its not shut down...

As to firefox, I am surprised you have so many problems. I have firefox open with multiple windows and tabs for weeks without problems (at least since 3.x.x) and there does not seem to be any memory leaks as with 2.x.x.

[mbluett]

In addition to what I wrote previously regarding O/S design:

This design would preclude virus'. Wouldn't that be nice.

There is no way a virus could enter the system for the following reasons:

1. No programs are ever installed (they come as part of the O/S or are added via Meta-descriptions).

2. As to files being downloaded via browser nothing can ever be run via a download (in any case, If there was a requirement to run something, the browser would be run in a VM). The same applies to email.

In the case of email, textual files, document files and pictures would be saved in the database. In the case of documents, they would be viewed with a VM'ed viewer or in the case where they are required for editing, they would be converted to the XML standards that are being used by the latest document processors.

The O/S would be distributed from well known sites only and all developers would have to be "known" before being allowed to make any changes to the O/S. Thus, at least, reducing the possibility of someone doing something nefarious as this is the only point of virus ingress.

The source code for the O/S could be made publicly available, but no one would trust it unless it came from one of the "well known sites".

Remember, this system would be designed mostly for Novices. Most of them have minimal application requirements. As the O/S would be free (or at most minimal cost perhaps to maintain the distribution sites), there is no reason why most of the general public would not be using it.

Linux is a great O/S, but it is still way too complicated for the general public. Everything is fine as long as nothing unusual occurs. The problem is unusual things occur all the time due to the operations that "can" be performed: Operations that the general public don't understand.