OSDev.org

Admittedly, I have a large learning curve ahead of me even though I have been working in this industry for more than 20 years.

Over that period of time I have watched operating systems (compilers, the Internet, etc) become increasingly complex. So much so that it is resulting in confusion and frustration for the general user and excessive time spent by programmers in coming up with results.

First an historical reference and some comments.

One thing Apple used to have (in its proprietary method) is that it could insure greater stability. Having all the code proprietary at least for the Kernel and drivers, certainly controls things: Certainly much better than the chaos within MS O/S's. I believe everything ran in Ring 0 which is not typical of the O/S's today. In a controlled coding scenario this is probably not a problem as any upgrades that introduced bugs would hopefully get resolved quickly as evidenced by the development of Linux and many other Open Source projects.

With control it is likely that some flexibility would be lost although it would be questionable if this would be an issue.

I propose a more simplified approach and to this end I would like feedback as to why such a proposal would or would not be successful.

Keep in mind this O/S would be for the general user (not necessarily for sophisticated users). The objective is to try and reduce the complicated environment the average user has to contend with.

O/S Design:

1. No memory segmentation
2. No paging ??
3. No traditional filesystem (instead use a single file which is a database containing various entities)
These entities would not resemble the current files we are used to. However, the database would contain all the
descriptive information (and more) that these files currently contain.
4. One large function library that all applications and the kernel use.
5. Library functions are loaded on demand and based on this system design would not necessarily have to be unloaded.
With everything loaded there is a very good chance it would not exhaust all available memory (with the exception of
machines that have under 1 Gig of Ram which this O/S would not support anyway).
6. No windowing GUI
7. All applications are built (programmed) using Meta-data: Basically the application and interactions are "described"
rather than coded in the traditional sense.
8. No runnable applications ever installed: Meta-data would be the only thing ever downloaded.
9. Drivers should run at a level so as to never prevent the Kernel or a user application from running.The most that would
occur is access to a particular device (or set of devices would be compromised). If a failure of this type occurs, the
driver would automatically be reloaded. The assumption is that most driver problems would be of intermittent nature.
Hopefully, any bad problems would be caught while the O/S is in development/testing.


Advantages:

1. Very simple memory management and as a result reduced programming complexity due to lack of "paging"
requirements.
2. Likely, no requirement for virtual memory as the amounts of memory these days would be able to house all user
application requirements.
3. Likely no chance of memory fragmentation.
4. No possibility of virus invasion (i.e., nothing downloaded can ever run).
5. For web/email compatibility, sand-boxing would be used thus preventing viral infections.
6. No lengthy programming and debugging processes (other than for the people that program the Kernel, drivers and
function libraries): User application programming is strictly meta-data definitions.
7. Non-windowing GUI decreases programming complexity, reduces user confusion.
8. No extra software to be purchased (although I suppose Meta-data could incur a cost as this could be produced by
anyone).
9. Single library eliminates redundant programming, prevents virus', reduces user application creation time, drastically
reduces programming learning curve.
10. Greater O/S stability.
11. The requirement to reboot would hopefully be gone.
12. Demand loaded libraries would reduce boot-up time.
13. Database would result in easier modifications to the way objects are handled. The typical O/S cannot easily have it's
filesystem altered as there are requirements for backward compatibility. This would not be an issue with this O/S.

I know a lot of this sounds too good to be true. But, it might just be possible.

Possibly open source code, but trusted sites would insure stability and prevent malicious content. All programmers working on the O/S would have to be "known" in order to prevent malicious code entering the trusted environment. This might occur but only once where a particular individual was involved. Or, perhaps a scenario where code submitted by one individual must be scrutinized by others before it's acceptance into the mainstream code base. In any case, the Linux distributions and many other projects seem to be devoid of such occurrences.

What would really be great is to get a group project going to develop such an O/S. For one person, this would be a monumental task although do-able.

Comments are welcome.

I don't have time to comment on all of them, but...


I don't like that because that file would have to be loaded at each read/write per applications, resulting in possible conflicts and decreased stability. Plus, thats much slower.


With no possibility for additions? That makes the OS less usable, and that library would be HUGE.


Ah, that explains #4. Loaded on demand? So, while the application is executing? Doesn't seem that efficient...


How are that simpler? Command line? Few users would be OK with that.


That makes the applications very inflexible. How would that work? No coding?


#4: Couldn't the virus be programmed as meta-data?
#6: Makes the application less usable and less useful, more difficult to program.
#7: How would that reduce confusion when most are used to it? What would be in it's place?
#8: Again, less usable and less versatile OS.
#9: But drastically reduces speed and stability. And how does that prevent viruses?
#10: I disagree.
#11: Linux rarely has to reboot, from my personal experience.
#12: How? I would think that because the program runs slower the boot-up time would be longer.

One big problem for me is that nearly no applications would fit into this system without rewriting everything. There are some good ideas/intentions, but I don't like the solutions (IMHO. But others may like it).
I do like the idea of a better GUI, but I'm confused on what you mean.

-JL

sounds like a textbook K.I.S.S. definition for an OS.

That is indeed an interesting idea. I think simplicity is really the way to go when designing... anything. The more complex products become, the more bugs the produce and usually the hardware requirements become higher. Although complex software brings overly-functional tools, I still believe that an efficient and simple base architecture can be just as functional (if designed with some extensibility in mind).

Your idea of meta-data software, is great for virus alleviation, but horrible for extensibility. Honestly, I think being able to provide a good stable and easy-to-use API would be an excellent thing to have instead of a limiting development platform.

Like most projects that require a group of people, it will be very hard to find people to develop for this. I personally believe that an OS design with simplicity in mind would be a great project and could even potentially go somewhere, but getting people to develop it will be near-impossible. I also think that having a community project would be a godsend for this forum, but unfortunately, I don't see it happening.

Why would the database have to be accessed at each read/write of an application?

The only reason the database needs to be accessed is when an application needs information to carry on with it's functioning. This would only occur at the time the application starts and if some object within the database is loaded by the application.

I am surprised to hear you say that it would be much slower as database accesses are very fast these days given the proper engine. I would like to hear others opinions on this method.



Of course there would be possibility for additions. Requests would be made to the development community that are responsible for the O/S development. This is the same case with all other open source projects. The library would not be huge as there would be a lot less redundancy involved. Consider the .NET library. It isn't that big, but does a lot of things and is continuously being added to.

My biggest problem with the current way of doing things is that there are a huge number of libraries out there. For a programmer to become proficient in making use of all of the diversity demands a huge amount of time and at the initial stages of that programmers learning curve it is very likely the coding will not be efficient (i.e., fraught with bugs, etc). Also, think of the amount of resources that have been involved in "re-inventing the wheel" so to speak. There is so much redundant code from library to library. What a waste of time and resources.



I am surprised at this comment as that is exactly what happens with DLLs under all versions of Windows and with dynamic libraries under Linux.



I was not clear enough. There would be a GUI it just would not allow multiple windows to be created. Each application would be full screen. Just that alone makes it easier for the user. Also keep in mind: I am not intending this O/S for technical types that easily know ther way around an O/S. This is intended for people who have very little understanding of computer usage.



The only limitations that would exist would be from the limitations of the current library. But that would only be for a short time until the next O/S update was available. The O/S could be patched between reboots if required. The Meta-data would describe what library calls to make and how to make use of the calls. This kind of paradigm already is in use via XAML and XML and other assundry mechanisms.

For example, the meta-data would describe how many frames exist. It would describe the buttons and other widgets that are in use and what those buttons and widgets do.



I suppose there is a possibility; however, there is no way to hide such things as currently can be done. There also is no way to infect some other executable. All one would need do is de-install the associated meta-data, for the application that was installed, to resolve the problem. This severely limits the nasty behavior of such entities.



Remember this important point: This O/S is not intended for experts. It is intended for the general public. The general public does not have the requirements that you might have.

The applications would have as much functionality as the general public requires. This would be mainly governed by the library in use at any particular point in time.



Note: I said "non-windowing GUI". This means that there is a GUI. Windows result in confusion. For example, think of the last time when a window popped behind another window and you didn't realize it had happened. Can you imagine how that would complicate the life of the general public? This is only one of the many difficulties that can occur.



I don't see how it would reduce speed and stability.

Virus' are prevented purely from the fact that the only people doing any "real" programming are a trusted group and the actual applications are created from meta-data that makes use of the libraries code (not code that anyone can put together).



How can you not agree that there would more stability? Could you elaborate?



I would agree whole heartedly. The problem is that the general public does not use Linux.



Demand loaded libraries would improve bootup speeds simply from the point that they don't all have to be loaded at bootup time.

How would "the program" run slower?



You are absolutely right: Existing programs would not fit into this system. It demands a complete new approach that leaves all existing software behind (with the exception that pieces of code could be re-used in building the O/S and it's libraries and drivers).

-JL[/quote]

I don't have time for a full critique, but I wanted to give my $0.02 about this idea of downloadable "meta-data". In fact, what you describe is just code in disguise. This is a common mistake when dealing with XML -- it looks different than code, but if you treat it as code, it's code.

Imagine that someone downloads meta-data that describes making library calls that do this:

• Reads the user's contact list.
• Sends a hard-coded spam payload to every e-mail address in that list.

Simplicity of implementation does not give you security, and as long as computers talk to each other over the Internet, you will need security.

I think you're trying to simplify the world based on your own narrow view of it, when in fact the world is complicated and how people use computers to interact with the world and each other is equally complicated. That's not to say that the current state of affairs is perfect, but taking OS technology more or less back to the early 1980's isn't going to help.

I think it would be slower because the programs wouldn't run according to your design (which makes little sense to me, but w/e). Also, it depends on what you mean by on-demand loading. Thats a very unclear definition. Should I be taking it literally, very literally or not?


OK, I like the idea of removing redundancies. But what if I want to write my own library my own way without submitting it to the community to program?

One big problem I have with your design is that it's not very portable seeming, nor expendable personally. Of course, you say that it's for the general public, but then again, those who do program it would have to be able to deal with it too.
And I believe that several (or many) smaller parts is bigger than one large thing. Seems that your design makes the OS be too single, IMHO.


...to retrieve the data? i think that many smaller files would be better than one large database. Look at the Windows registry.

And I was asking weather the library was loaded per each function call, or at the beginning of execution. Obviously the latter is good. But I was wondering about on-demand loading.

I think (from what I guess) that the application would run slower because it wouldn't be directly executing, but the OS would be decoding it (also decreasing boot-time).

I like the idea of a simple GUI for beginning users, but for the large percentage of the population that can deal with computers, they would have a hard time with it.

Overall, I like the idea, but I don't like the design of several aspects.

-JL

It does not need to be treated as code. I realize how XML can be used, but this O/S does not have to follow the XML guidelines.



I can see your point; however, I am sure there are ways around such conditions. It maybe that the meta-data downloads have to be regulated just as the O/S would be. Transaction logging could track such occurrences with time-stamping. This is something downloadable meta-data could not turn off. There could be some innovative method of detecting conditions like this and suggesting to the user that this action is occurring. Also, keep in mind, applications would only run on this system at the users request. I did not detail this before but, they cannot be scheduled. And applications don't run in the background (at least not user applications). Maybe these are possibilities to get around the problems you are suggesting.

Even if it is not possible to prevent a condition as you have suggested, eradicating a problem like this from your system would be much easier. The only way to be absolutely sure a virus has been removed from current systems is to re-install (applications and all) consuming a large amount of time and proving to be a difficult task for most people. In the case, of this O/S, the meta-data just has to be un-installed and that's it.



This is true. However, security could be had by sand-boxing any application that communicates to the outside world. Would this not be adequate?


[/quote]

I don't have a narrow view of the world: The fact that I see how complex it is precludes that.

All I am professing is simplicity and proficiency. Just because the world is complicated, doesn't mean it can't be simplified (at least in part). There is no reason, I have seen so far, to not pursue such a dream.

I don't understand your statement: Of course the programs would run according to my design. How can you say they won't?

As to "on-demand loading", all I mean is that when an application needs a function it is loaded on demand just as functions are loaded on demand from DLLs in current MS operating systems. The only difference might be that they may not need to be unloaded to free up memory. The idea is to design it so that there is no need to have multiple copies of the same function in memory.



I guess I will have to repeat myself again: This O/S is not intended for people like yourself who are technically competent. It is intended for the general public: People who do not write their own libraries.



It could be ported to any hardware platform that it needs to be ported to. Why not? All that it would take is programming effort on the O/S developer's part.

It is also expandable even by yourself as remember it would be open source.

If I was developing such a system I could easily use it.

There are things that could be done to get around not having access to multiple windows. Or, maybe have an advanced mode that provides access to certain flexibility that the general public would not care to use.

The point is, if you look hard enough, there are always solutions.



You are right that an application would have to retrieve data periodically, but having a lot of database experience I can tell you this would be very fast: As fast as opening a file, reading it and then closing it again. In fact it might even be faster as a result of indexing and also that the database is always open.



No, the O/S would never be decoding anything except in the case where a VM was required to sand-box an Internet style application.



The problem is the larger percentage of the computing population cannot deal effectively with computers: Rather it is a relatively small percentage that can do this effectively. We are talking millions (perhaps billions) of general users as compared to thousands of technically competent users. A considerable population difference.

Ok, that makes more sense. Sorry if these comments seem stupid, I'm pretty sick.



Seems to me that you design does not include runable programs, you say so yourself. Thats why I can say that they wont run.


How can it be used if no one develops for it!? If it's not to be used by developers, nothing will be developed for it. Unless you was a specific team of people to do so, but how are you going to write all the programs the general public needs? Consider the logistics.
And no you don't have to "repeat yourself again", I got it the first time. I just don't get how you expect a system like that to go anywhere.

And I meant porting as in 'porting other applications and drivers to it'.


True, but if I have a filesystem tree in memory, I can read the index and read the file quickly as well. I don't know, I don't have database experience. But it seems to me that it would be less efficient. I don't know, maybe I'm wrong.

And yes, there are more incompetent users out there. But is designing a new interface for them better than effectively simplifying the current one and teaching them how to use it?

So then, I'm confused. Explain to me how a program runs.

-JL

Hi,



Usually people who say they want to use a database instead of a file system only really want to change the terminology.

For example, on the computer I'm using there's a database called "/dev/sda1" that contains "entries". There's also a "virtual database" called "/" and you can embed other databases into this virtual database by doing something like "mount -t ext2 /dev/sda1" (so that entries in the embedded database look like they're actually in the virtual database). There isn't any file systems (they're all databases) and there isn't any files (they're all entries in the database). Honest...

I think what Mbluett wants is a normal file system with extended file attributes and perhaps some indexing (and different terminology so people don't realize that almost every modern OS already does the same thing).

Of course I should probably mention that nobody really uses extended attributes, because (for e.g.) users have better things to do than make up thousands of descriptions for thousands of pictures that their wife doesn't know about. Basically extended attributes sound like a great idea until you try to figure out where the data for these extended attributes is meant to come from.


Cheers,

Brendan

I think what Mbluett wants is a clean filesystem with a layout like a database.
(Users, Programs, etc all nicely sorted.)

As for the meta-data approach I have one word to say: XUL.

**IDEA** OS with FireFox/Gecko as it's GUI.

Extended attributes, as you call them, are a great idea. The major problem with them is that not everything supports them. Let's say, for instance, an album of photos exists on somebody's computer, and they have them all beautifully tagged with location, date, subject matter, etc. As soon as they upload them to an Internet website (Flickr for example, though it could just be a plain old HTTP directory dump), they lose all this information, and anybody who downloads the album JUST gets the photos and -maybe- a descriptive name, nothing more. Once this problem is fixed, attributes become a lot more useful.

Ok, now I'm pretty sure you don't understand XML.

The XML "guidelines" specify syntax, nothing more. You said that your OS will interpret the XML to determine "what library calls to make and how to make use of them". This is about semantics, not syntax; This is treating XML as an interpreted scripting language. Therefore, your OS would treat downloaded XML as code.



Yes, there are, but they're complicated. Using XML has three good properties though:

1. It's effectively a scripting language, so it's memory-safe and limited to do what your API allows it to do.
2. It's easier to parse and analyze than a typical programming language, so you could scan it for malicious intent. This is still difficult, though.

You mean "regulated" like, for example, Apple's iPhone AppStore? That could work, but having one organization of humans check code for bugs isn't scalable. As others have said, this would limit the popularity of your OS because it would limit the number of new capabilities that could be added to it.



Have you ever seen the UAC prompts in Windows Vista? Or any other nagging dialogs like "Are you SURE you want to open this attachment that claims to be a celebrity sex video?" Users become trained very quickly to just ignore the warnings and click OK.



Yes, but then the apps become mostly useless since they can't access any local resources on the machine.

Which brings up another question I have... How would a person browse the web on your OS? Would Javascript be allowed? If not, your idea is already stillborn since everyone and their dog does almost everything online these days.

Heh... True, most of the time. With WinFS, MS actually tried to implement a database instead of a file system. This is what a lot of people think of when discussing this idea, IMO.



Bingo. Those things add real value for end-users. That's why we have Spotlight on OS X, Windows Desktop Search, etc.



I like your examples! In practice, people are more patient than you think. Even so, it helps when the source of metadata can be automated somehow. For example, CDDB can be used to automatically download metadata for music. I'm sure a similar database service exists for movies. I believe that a real "killer app" would be an AI that is able to take a few example photos and auto-tag the rest by learning what people and places look like, what clues in the picture suggest an approximate date (e.g. -- snow, leaves, holiday decorations, sunny beaches, etc.) It wouldn't have to be exact, just good enough to save people from doing most of the grunt work themselves.



Fortunately, a lot of file formats have attributes embedded in them already (e.g. -- ID3 in mp3; attributes like resolution, description, date taken in jpeg; etc.) All that's needed is a plug-in architecture that can present these "embedded attributes" in a standard way to the file system, or at least to the indexing service. This is how Spotlight works on OS X.

Sorry to hear that. Hope you are well soon.



Ah, sorry. Further clarification required. What I meant by this was that no applications would have to be installed once the O/S is running. In other words most of what is required is provided with the O/S. However, it would be possible for metadata to be downloaded which could provide extra functionality.



What I am hoping for is that a simplified system for the general public could be developed so as to make their lives easier. The O/S's that people use today are in most cases too complicated for the general public. I would like to see something developed to help these people. I think the that is adequate motivation in having a system like this go somewhere.

The existing O/S's are OK for the rest of us as we know how to repair things when they break. For most people simple things cost them money to repair. And I mean simple things: I went to clients house the other day to help resolve a problem. All that had happened was there were no longer any icons on the desktop and he did not know how to get them back. It was a simple matter of right-clicking on the desktop, selecting "Arrange icons by..." and selecting "Show Desktop icons".

This is just one example of a simple thing that causes frustration and inconvenience for the general public.



Well this can be done, but the development paradigm is different. An applications functionality can be ported, not the actual code. And you would be right in saying this would be re-inventing the wheel. However, it would be the last time the wheel would be re-invented. The only thing that would occur in future would be improvements to the way that application functions.



Simplifying an existing interface is one approach. However, that does not get rid of the complicated underpinnings. For example, how do we get rid of problems where drivers can hang a system? How do we simplify the programming effort that is involved with the current O/S infrastructure? How do we make it so that a user does not have to know anything of how to traverse a filesystem?

In order to accomplish things like this, in some cases, the very underpinnings of the O/S have to be changed.



Metadata would be used to describe what functions make up an application. It would also describe how those functions interact. It would also provide an entry point for the application. Haven't really thought all of this through thoroughly yet. My objective was to get the creative juices flowing to come up with some potentially novel solutions for accomplishing this.

What I would like to see is a way to be able to decrease the amount of development time involved in creating applications for the general public. I am not talking about sophisticated video processing applications (for example). What I am talking about is a word processor, an email app, a browser, a desktop clock, an address book: The things the general public use on a day-to-day basis.

Another objective is to reduce the amount of buggy software. I have installed so many main stream apps that are riddled with bugs. Can you imagine how frustrating bugs would be for the Novice who would automatically think "What did I do wrong!": They didn't do anything wrong, in fact it isn't even anything they could prevent from happening.

If the programming paradigm was improved, this would help to reduce these problems considerably.

Currently, the programming environments we have are so complex it is very easy to create bugs and not know it. Hence the reason why there have been attempts to build applications that will scrutinize code looking for obvious problems. Unfortunately, these applications are not 100% successful and even if they were, most of us wouldn't be able to afford them unless we wrote them ourselves (more redundancy).

If the real programming is reserved for development of libraries, the O/S and drivers, then over time the bugs would become fewer and fewer. As time went on the changes required to this core would probably be reduced as well.

At the user programming level the programming is much easier and there is no chance of introducing a bug because all one is doing is making use of the existing library functions (in a building block scenario). A similar approach is development with Java Beans.

-JL[/quote]