OSDev.org

Hello everyone. In the past I've made a few posts on this board regarding an OS I've been working on, on and off. However, as of now I have really only done design work, and have implemented nothing. I have prototyped a lot of my designs in Ruby to test their feasibility, but now I'm at the point where I actually need to code this damn thing, and I've come to realize the magnitude of this project is well beyond my capability as an individual. Even with dozens of people working on it, my OS could take years to finish. And considering the fact I've never actually implemented an OS in practice before, I'd estimate the practical completion date, if I started right now, to be somewhere around the year 2050. But then again, I usually severely underestimate the time it takes me to code things.

So I quite obviously need some help . I would have asked much sooner, but I was too busy debating with myself exactly what the best approach to getting help would be, and I only recently came to the conclusion that I'd be better off just making my OS open source from the start. That being said, I'm not ready to go public just yet. Before I do that, I need help with two things.

The first thing is, a reality check. Yes, I need someone who is much more experienced than me to help me go through my design and make sure that everything I'm planning to do is as feasible as I think it is. I'm a fairly experienced programmer, and I have a pretty good level of familiarity with low level computer architecture, but having never actually tried to implement anything for my project yet, I can easily imagine there may be unforeseen difficulties that may need to be resolved before my design is finalized.

The second thing is, a toolchain. My OS does not use a conventional binary format. There are almost no existing tools and compilers that I could use that would be suitable for my needs. I could use an assembler and a system of macros, but then my whole OS would be stuck in platform dependent assembly, which is not something I want. At the very least I need a complier, preferably written in a scripting language such as Ruby, to bootstrap this project.

If you can help with either of these things, please let me know.

But why should you help you ask, when I have told you virtually nothing of my project? Well okay, here is what I can say about what my OS is, and what I'm trying to accomplish.

There is a famous quote by Albert Einstein, "Make everything as simple as possible, but not simpler." In a nutshell, that is the goal of my operating system. Modern operating systems, and modern software design in general, suck. That's because nothing is ever as simple as it should be. When you give two people the task of solving the same problem by writing software, the solutions they end up with are invariably different. When you have an environment of programmers working independently, with differing ideas about correct design, and then at the end of the day you have to make their software work together, you get a mess.

NodeOS is the solution to this problem. NodeOS is designed so that all code is organized into a class based functional hierarchy, where there is no more than one class for any given type of object. For example, the class for a specific type of image (JPEG for example), must always be a subclass of the basic Image class, and must provide an interface that is compatible with that of the Image class. The goal would be to have no more than one JPEGImage class on the entire operating system, that all other code on the system would use. Additionally, in cases where it would not prove to be a significant detriment to performance, all code that wishes to use an image could preferably use the common Image class's interface, as opposed to dealing with the object as a JPEGImage directly. In this manner, at a certain cost in performance, you can guarantee that any code that you write for the operating system, can deal with any format of image data, without having to be specifically coded to handle it. And of course, the common interface is designed to be as simple as it can be made, and as consistent with interfaces for similar types of object (such as video data) as possible.

Now I'll admit, it's fairly easy to apply that kind of approach to something like an image class, and significantly less easy to apply it to most other types of classes. Yet so far, I have not been able to think of any situation where there isn't a way to do it. Maybe you can. If you think so, let me know. But in my experience, no matter how elusive it may seem at first, there is always an approach to designing a class in NodeOS, that is essentially definitive. In other words, if there is a concept that you can encapsulate in an object, you can code it in such a way that it needs to be written once, and only once.

I don't really think this is a horribly revolutionary idea. Why it hasn't been tried before, I'm not entirely sure. But I nevertheless think it would be a major step forward to have an OS that actually worked like this. And that's certainly not all there is too it. The class hierarchy is one of the big things in NodeOS, but there's so much more. I think that at least potentially, on the other side of NodeOS's development, there is an environment where getting computers to actually do the things you want is a privilege no longer restricted to programmers. Perhaps also an end to forced incompatibility by the use of closed standards. And dare I say it? There may even be an end to the drudgery of coding, by way of a very drastic decrease in the amount of time and effort it takes to go from concept to implementation for virtually anything you can imagine coding. All with a level of simplicity and consistency that will make it a joy to work with. Think Minecraft, only as an operating system.

So, are you interested now? I really hope you are, because without your help it's not going to happen. Please send me a PM if you can assist with the two things I mentioned, namely the "reality check" and the toolchain. If you have any general comments or questions regarding the project, or would like to accuse me of having unrealistic expectations or foolhardy optimism, please leave a comment below.

I disagree. It is feasible. I thought similarly to you at first, until I tried to prove it wasn't feasible and was unable to do so. There are actually a lot of design choices that you can make to get around the issues you mentioned.

Corruption resilience - Shouldn't be part of the JPEG class to begin with. Implement it with a "layer" (which is a concept unique to my OS that I have honestly never seen before anywhere else, and would be happy to explain if you'd like to discuss it in PM).

Deadline - Don't have one. Classes within NodeOS are written with the idea of getting them right, not getting them done fast. If a class is rushed and suboptimal, it will be replaced sooner or later. There are actually provisions in the OS design so that if a commonly used class is deemed suboptimal, it can be replaced with a class that has a separate interface without breaking backward compatibility with previously written classes that depend on it (with the exception of subclasses).

GPGPU - As long as it can use the same public interface as any other image, this poses no issues. This is actually more of an issue of providing multiple implementations depending on the system's hardware and capabilities, which is accounted for in the design.

Keep 'em coming, I could do this all day.

Err... well, I mean, I looked at your OS but it doesn't seem... that... good? If I had the toolchain I'd like, I could've eliminated about 90% of your code, and made it easier to read and understand. I mean, I'm not trying to put you down or anything, but you have to understand that what I'm doing is way off of traditional OS design, so you can't apply the same expectations. Once again, I'd be happy to discuss specifics in a PM. Maybe you could help me with my design since you have the experience.

Xzyx987X wrote: if I started right now, to be somewhere around the year 2050.

You would also accomplish being the best life procrastinator ill have ever known. I may even give you a metal..

And I would graciously accept that "medal", as I have done plenty of things in my life to earn it. For instance, just today, I started work on painting some chairs I'd been putting off for the past three years... But in all seriousness (although that last remark was entirely serious), it's more like this is just plain not going to get done if I can't get at least a couple of other people working on this project. I just plain don't have the time and energy to do it all myself. And I work much better if I can play off of other people's thoughts and ideas, as they help keep me grounded and rational.

berkus wrote:It gets curioser and curioser, so now you want to implement a toolchain, too?

I don't want to, I need to. My OS does not use a conventional structure for compiled code, and as such, cannot utilize existing toolchains. Hell, I'm not sure if it can utilize existing languages. Short of using an assembler and macros for every damn thing in the OS, I'd need something like a hybrid of Ruby and C to implement my design.

berkus wrote:Which leads me to believe that you didn't even study the field thoroughly.

I actually came by it from a different angle. I originally wanted to write an emulator for a certain classic video game platform, but I dropped it when I decided the potential results did not justify the amount of effort it would take (I was a pretty immature programmer at the time). I never came back to it, but I learned a lot about low level software design in the process. I also took classes in assembly and electronics in college, and I've read a few books that cover OS design fundamentals. But other than that, I guess you're right. Hey, nobody's perfect. That's why I'm looking for someone to help me iron out the kinks in my design.

berkus wrote:O RLY? What does it use then? "Non-conventional structure" does not count as an answer.

It's... complicated. Most of the differences come about from the way I handle memory management. Others are for the sake of compartmentalization. You never have large monolithic binaries in my OS, every class is it's own independent binary. And even the methods within that binary are not static immutable things at runtime. I'd be happy to explain in detail via PM, but only if you agree to actually help me.

berkus wrote:So you agree you didn't study the field.

I think I've studied it less than some people who've written operating systems, but also more than others. I basically know what I'm doing, but I lack practical experience in actually implementing it.

berkus wrote:Good luck.

Yes, I'll be needing plenty of that...

Xzyx987X wrote:There is a famous quote by Albert Einstein, "Make everything as simple as possible, but not simpler." In a nutshell, that is the goal of my operating system.

I think you're misinterpreting that quote (I just so happened to set it as my sig a few days ago, so I was thinking about it.) The important part is not the "simple" but the "possible": the point is that the best you can do is the simplest thing that is also correct and complete. If you think about the things that Einstein actually did, this makes a lot of sense: special relativity, for example, is an ingenious way of looking at electricity, magnetism, and light in a way that fits all known phenomena; however, it is clearly more complex to work with then classical mechanics, which is a very good approximation of reality at low speeds. The correctness here is more important than the simplicity, but the genius is in how simple it is despite the correctness (if you understand hyperbolic geometry, of course.)

The moral is, "simplicity" is not a justification for a design. Not only is it difficult to subjectively achieve, but it also encourages the cutting of corners. Also, everyone wants it anyway, so thinking that you'll do it best and first is unreasonable. Here's a better quote for you: "Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it." (Alan Perlis)

As far as your design goes, it seems relatively complex to actually implement, and, as berkus pointed out, is lacking in a few areas. It also is a very general design: it doesn't specify any particular interfaces, just that there will be "simple" interfaces. Basically you're saying that you will come up with the most ingenious interfaces later, because your system doesn't prevent you from doing so (which is very backward logic.)

Xzyx987X wrote:The goal would be to have no more than one JPEGImage class on the entire operating system, that all other code on the system would use. Additionally, in cases where it would not prove to be a significant detriment to performance, all code that wishes to use an image could preferably use the common Image class's interface, as opposed to dealing with the object as a JPEGImage directly.

How is this different than a collection of shared libraries written in an object oriented language? Because that's basically how modern operating systems work already...

NickJohnson wrote:I think you're misinterpreting that quote (I just so happened to set it as my sig a few days ago, so I was thinking about it.) The important part is not the "simple" but the "possible": the point is that the best you can do is the simplest thing that is also correct and complete. If you think about the things that Einstein actually did, this makes a lot of sense: special relativity, for example, is an ingenious way of looking at electricity, magnetism, and light in a way that fits all known phenomena; however, it is clearly more complex to work with then classical mechanics, which is a very good approximation of reality at low speeds. The correctness here is more important than the simplicity, but the genius is in how simple it is despite the correctness (if you understand hyperbolic geometry, of course.)

The moral is, "simplicity" is not a justification for a design. Not only is it difficult to subjectively achieve, but it also encourages the cutting of corners. Also, everyone wants it anyway, so thinking that you'll do it best and first is unreasonable. Here's a better quote for you: "Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it." (Alan Perlis)

As far as your design goes, it seems relatively complex to actually implement, and, as berkus pointed out, is lacking in a few areas. It also is a very general design: it doesn't specify any particular interfaces, just that there will be "simple" interfaces. Basically you're saying that you will come up with the most ingenious interfaces later, because your system doesn't prevent you from doing so (which is very backward logic.)

Well, they key to achieving this is that my system makes it so you can always go back and make things more simple later, rather than holding on to unnecessary complexity for legacy compatibility reasons. Actually though, I already have a very good idea what my interfaces will look like. I've designed a lot of the core interfaces already, and based many more off of classes from the Ruby programming language. I really like the Ruby String class for instance, because it's both simple, self-apparent, and has virtually all the capability you need built into a relatively short list of methods. That should give you somewhat of an idea what my other interfaces look like.

As for your remarks on simplicity, I pretty much agree. My OS has some relatively complex elements, for the sake of providing a very simple environment for end users and developers. However, in my OS, I prioritize as follows:

1. Consistency
2. Simplicity
3. Transparency
4. Speed
5. Functionality

The reason for this order is, that while I hold simplicity with very high regard, I consider consistency to be a higher form of simplicity. I consider it to be in fact, a systemic form of simplicity that is more important than the simplicity of any one component.

NickJohnson wrote:How is this different than a collection of shared libraries written in an object oriented language? Because that's basically how modern operating systems work already...

A lot of reasons really. In my OS the JPEGImage class is an object that the OS handles directly. This allows for it to be a standard interface that is used by all programs on the system, written in all languages. This is important for a few reasons. First of all, you get an unheard of level of consistency. No matter what piece of code on the OS you look at, if it deals with image manipulation it's going to look pretty much the same. Second, regardless of the language you're working with, you can share objects across the system, and even across memory spaces. It also allows for security to occur on the object level, since objects are the primary component of the operating system. This dramatically simplifies the security system, since all data of the system uses the same scheme for security, which is additionally extremely simple. The list goes on...

berkus wrote:One thing that I tried to highlight in my first two posts is Beginner Mistakes page on the wiki. You either did not read it or deliberately keep on breaking some rules outlined there.

While this might be good for having a discussion, this part you also avoid by saying you will provide details in a PM. This is a useless and non-productive game. You might not've understood my hints previously, so here it is in plain text.

Right, I understand what you are saying. I come here with no codebase (actually, I have a codebase, but it's only a prototype), provide as few details on what I'm working on as humanly possible, and yet I expect people to jump on board and give me everything I ask for. I would be skeptical too if I were you. The bottom line is, all I'm asking for is the opportunity to straighten out my design a bit with the help of someone from the community with experience in actual OS development before I go public with my design. Asking for developers for my toolchain was more of an interest check than anything. So really, what I'm asking for in the here and now, is no more than for one person in the community to give me a few hours of their time to help me out with the early stages of my project. Is that really such an unreasonable thing to ask? In fact, with the time you've spent debating me on why what I'm doing is wrong, you could have made significant progress in actually assisting me with what I asked for in the first place.

Ok, great! Care to at least PM me your Skype handle? I checked your profile, and the pages linked in your signature but it didn't seem to be there.

The reality is that no one's going to help you with it, so don't waste time in discussions like this one. At the moment you don't appear to have anything other than some general ideas which probably tie in with what everyone else is doing with their OS. What makes you a little bit different is that you want to use Ruby, but even people who work purely with assembler or machine code still tend to aim for as much simplicity as possible, having a single piece of code which does a particular thing and which is universally called by any program in the machine that needs to use that function. I program directly in machine code, but I build objects and classes in a similar way to using an object oriented programming language, so we're all doing much the same thing, only stopping from time to time to argue about trivial differences in ideology.

Most of us want to produce an OS which isn't bloated and which doesn't take ages to do something which it ought to be able to do in an instant so that we don't keep having to wait for the OS to catch up with what we want to do. My netbook takes five minutes to boot before I can start work, Firefox takes two minutes to start up, Windows keeps getting tied up in knots, and the cursor is periodically stolen away from me for ten seconds at a time for no obvious reason - the cause of all these problems is probably unnecessary complexity which has resulted in programs that are such a mess that they can't be tamed. It shouldn't be like that and we all know that we could do better, if only we had a massive team of people to work with. And yet a massive team doesn't help either, because that's what leads to most of the complexity and mess being generated in the first place.

So, what you need to do is stop theorising, recognise that no one's going to help you build your OS (at least not in the early stages), and get on with building a simple OS from the ground up so that you can gain a better understanding of the actual difficulties that lie ahead. Later on, you might impress someone enough to get them to help develop your OS, but the odds may be a thousand to one against that. If you're lucky enough to find someone to work with, you'll probably then spend most of your work time arguing with them about the right way to do things and your project will grind along for a while before it eventually stops. You need to be realistic about this - you're on your own, and the better for it.

P.S. Berkus will eat you for breakfast.

Well, I more or less understand it's going to be an uphill battle. I'm not expecting everyone to jump on the bandwagon right away. But one thing that tells me that I may have something worth pursuing, is the fact what I have already with my prototype code already looks significantly different from any other OS I've seen. I think my design has a lot of potential. It's not good just because it's simple and clean, and hasn't been tarnished by the hands of numerous unskilled programmers. It's good because it takes a completely different approach than any other OS, that naturally leads to better code. I think once I actually put up my design and short term roadmap for public consumption, I'll get at least a few people interested. But, we'll see what Berkus thinks.

DavidCooper wrote:The reality is that no one's going to help you with it, so don't waste time in discussions like this one. At the moment you don't appear to have anything other than some general ideas which probably tie in with what everyone else is doing with their OS. What makes you a little bit different is that you want to use Ruby, but even people who work purely with assembler or machine code still tend to aim for as much simplicity as possible, having a single piece of code which does a particular thing and which is universally called by any program in the machine that needs to use that function. I program directly in machine code, but I build objects and classes in a similar way to using an object oriented programming language, so we're all doing much the same thing, only stopping from time to time to argue about trivial differences in ideology.

Most of us want to produce an OS which isn't bloated and which doesn't take ages to do something which it ought to be able to do in an instant so that we don't keep having to wait for the OS to catch up with what we want to do. My netbook takes five minutes to boot before I can start work, Firefox takes two minutes to start up, Windows keeps getting tied up in knots, and the cursor is periodically stolen away from me for ten seconds at a time for no obvious reason - the cause of all these problems is probably unnecessary complexity which has resulted in programs that are such a mess that they can't be tamed. It shouldn't be like that and we all know that we could do better, if only we had a massive team of people to work with. And yet a massive team doesn't help either, because that's what leads to most of the complexity and mess being generated in the first place.

So, what you need to do is stop theorising, recognise that no one's going to help you build your OS (at least not in the early stages), and get on with building a simple OS from the ground up so that you can gain a better understanding of the actual difficulties that lie ahead. Later on, you might impress someone enough to get them to help develop your OS, but the odds may be a thousand to one against that. If you're lucky enough to find someone to work with, you'll probably then spend most of your work time arguing with them about the right way to do things and your project will grind along for a while before it eventually stops. You need to be realistic about this - you're on your own, and the better for it.

P.S. Berkus will eat you for breakfast.

Concur.

(Especially the bit about Berkus).

Well, berkus and I had a nice discussion just earlier, and while it was pointed out to me that some of my ideas were perhaps not as novel as I had at first thought, I think the whole of my OS still comes together nicely enough that it will look pretty unlike any OS you've seen before. If not in concept, then at least in elegance of execution. Either way, berkus agreed that my design was on solid enough footing, so I'm going to do my best to get what I have organized enough to post it on Github, and from there on this project will be public.

I've decided the first step in the actual implementation of my OS will be to develop a Ruby derivative language to use for the coding, and implement it as a cross complier for the SH4 platform. Why SH4? Because that's the processor the Dreamcast uses, and the Dreamcast is a great PC-like platform with a simple RISC CPU, that I feel is ideal for a reference implementation. God only knows I don't want to touch x86 with a ten foot pole until I actually have my design working the way I want it. I would of course like to have help, although I do not necessarily expect it. If you have experience with writing compilers and/or SH4 software development, and are interested in this project, let me know. I will of course be posting the address of the Github repository with my documentation and prototype soon, so you can get a better idea of what exactly you'd be working on implementing.