Open questions in OS design?

[Steven]

I've been reading the postings on this forum for little while now and it seems to be a lot of the same:

-non-developers making outlandish, uneducated, and irrelevant comments (Like this guy)

-developers (I'm taking their word for it) who haven't passed basic algebra yet and want to tackle some calc problems (Like this guy)

-os dev hobbyists looking for information on interfacing with some piece of hardware, help with accomplishing some basic kernel feature (memory management, process management, etc), or otherwise seeking help with standard more-or-less-cookie-cutter-os-design implementation details (which I'm assuming is really 1/2 of the board's legitimate purpose).

-os dev hobbyists looking to tweak some standard component of their os to function in a "new" way, but ultimately not really solving any real world problems, making any earth-shattering discoveries, or really doing anything you couldn't accomplish in user land (Like this guy) Which I'm not trying to diminish because I think if he phrased this comments differently, and sold them as hard as MS sold their WinFS pie-in-the-sky idea, he would have gotten more attention.

-and os dev hobyists looking for some completely new concepts in os design (Singularity et al).

I don't get particularly thrilled by emulating existing systems I can't improve upon (not to mention have the time to take to completion), and wouldn't have any interest in actually developing an os if I didn't think I could "bring something to the party" so to speak. And while I can't speak for anyone else, I have a feeling a lot of people come here initially feeling the same way. So with the exception of the people coming here who never have and never will make their own os, I think what we're all really interested in is: what's the next big thing?

So my question is this: what do you think are the major open questions in os design?

Are any major advancements at this point going to have to occur in the hardware arena? Is our job just to keep up with the advancement in hardware design (better support for multi-processor, etc)?

Why does it seem to me like the most important part of an os seems to be largely ignored: the system api?

What do you think about systems with managed memory paradigms? While this seem to me to have a lot of promise for making existing systems more stable and easier to develop for does it really get us anywhere?

Is the blossoming multi-core era our next big challenge?

What criteria would you use to evaluate what an "advancement" is?

Are computers, as they exist, with our ability to control their basic functions (as we do today) sufficient and "good enough" for what we need?

It also seems like the chicken and the egg problem; people didn't really "need" personal computers until they were first developed, and then largely until they became widespread and ubiquitous; at which point people found reasons to need them (to improve their business processes, write term papers without white-out, etc) (or, more accurately, people found good ways to sell them). When the relay was invented no one thought it (and it's decendents) would become the basis of the modern computer, or invisioned what that would mean for society. Will we have to wait for 10 years after the next major breakthrough before we even realize it happened?

I'm interested to see if this will spawn any sort of discussion, and what everyone will have to say about it.

Thanks

[Troy Martin]

You forgot the all important cotton-topics: http://forum.osdev.org/viewtopic.php?f=11&t=18397 - http://forum.osdev.org/viewtopic.php?f=11&t=18404

(Oh, and he goes from liking me to hating me later, it's quite funny!)

[Steven]

You forgot the all important cotton-topics: http://forum.osdev.org/viewtopic.php?f=11&t=18397 - http://forum.osdev.org/viewtopic.php?f=11&t=18404

(Oh, and he goes from liking me to hating me later, it's quite funny!)

EDIT: LOL, yes, well, the trolls are unavoidable.

[Colonel Kernel]

I think the most interesting open problems are at a higher level than OS dev, but will have a strong influence on the directions of future OS research. The move to many-core is a big one, but it's just part of an even bigger move towards parallel and distributed computing. This move will require huge advances in programming languages, run-time environments, and OSes alike. Where all three intersect is the most interesting research area, IMO.

[JackScott]

You forgot to mention the most important interaction of them all: the interaction between (wo)man and machine. I still find it hard to believe that in the last 25-30 years, no new great advances have been made in either the world of GUIs or CLIs. The last really big invention in this area was the mouse, IMHO. And that was a while ago now.

I know there are things like speech recognition. But does anybody ever use it? Nope, because it's worse than the current methods. There must be something better, I'm sure of it.

[Stevo14]

You forgot to mention the most important interaction of them all: the interaction between (wo)man and machine. I still find it hard to believe that in the last 25-30 years, no new great advances have been made in either the world of GUIs or CLIs. The last really big invention in this area was the mouse, IMHO. And that was a while ago now.

I know there are things like speech recognition. But does anybody ever use it? Nope, because it's worse than the current methods. There must be something better, I'm sure of it.

I think the most important advancement happening in this area right now is the touch/multi-touch interfaces that are becoming popular. In a way, it removes the middle-man (the mouse in this case) allowing the user more direct and natural interaction with the computer.

The problem for hobby osdevers is the fact that they are relatively expensive and probably not very well standardized (perhaps someone knows for sure on this last one? Are they standardized at all?).

[01000101]

I see a lot of people coming here to basically re-invent the wheel. I see a lot of Bran's kernel tutorial extenders, and that was not it's purpose (or it shouldn't be). 90% of the OS's here look the same and most likely have a lot of common code between them. While this is not a bad thing at all for someone looking to just learn about how operating systems are programmed, it is monotonous to someone looking to develop something 'non-standard' or unique.

On that note, a lot of the people that come here and preach about some crazy new design and innovative idea are nothing more than a puff of smoke as they lack the skills, require help, or get discouraged within the first 10 posts of criticism.

Personally I'm sick of seeing *tutorial OS's* everywhere. That's fine for learning, but to post it here as an accomplishment and make versions as someone *obviously* is just progressing through a step-by-step tutorial gets very old. It also seems that they all are destined for the same outcome... Linux 0.01 (or whatever the earliest Linux is).

I won't toot my own horn on this one as I don't want to say that people need to break away from developing desktop OS's (as I definitely do not mean that), but my OS is (IMHO) very different from the 'norm'.

I'd really like to see more unique ideas (as I've seen a few good ones) being invested in and to take the plunge into the unknown where there are no tutorials to hold your hand.

Once again, I understand that there are a lot of people here that just want to get a feel for real low-level coding and want to see concepts such as GDT/IDT/TSS in live implementations. This is great! But for the people that are coding an OS to be distributed or that will be showcased in one way or another, really need to think about a few things about uniqueness and what people want to see. There are plenty of *flaws* in modern operating systems that could be addressed, but it takes a lot to identify and solve them, but I think it's worth it. Even a failed attempt at implementing a new idea is better in my book than not taking the risk and coding within the safe.

Maybe I'm just a bit cranky and overtired, but I'm a bit sick of seeing... *the same*.

[elderK]

I agree with you, 45. It gets lame seeing the same old crap time and time again.
I mean, you'd think some of the people who come here would actually read through the forums history...
If they did, they'd find the answer to their issues and more...

That or they could simply learn to read the damned datasheets...

They come, all psyched up to write some hardcore kernel - which in the end, distills into Windows with a purple color scheme (or something...). A day later, they realize how much work just implementing a small amount of their idea would be and lose interest...

That, or they really have no real interest in learning about the guts of anything because, they come here begging for 'pluggable' code...

The best advice we can give ANY of those transient people is simply, "Learn to learn".

As for the next big steps in computing? Personally, I think it'll be with massively distributed systems. Moving away from the idea that a computer is yknow, a single machine. No more thinking about the single-core machine, hell, no more thinking about 4-8 core machine. No, instead thinking of infinite cores - infinite users. Thinking of TRUE reliability, true modularity - along with the security issues all of that would bring.

I mean, everyone and their dog knows that in order to gain performance - we are going to have to go massively parallel. That /alone/, is a big issue because afaik, no one really knows how to best utilize multiple cores in a general purpose way.

I also agree with Colonel Kernel, we /are/ going to need massive improvements in programming languages we use.
We are going to have to leave behind legacy. We are going to have to /think/ differently, /see/ differently!

And all of it? Its all exciting

As for higher level, hell yes. We have to change the architecture! We already KNOW what many issues in the future will be! Hell, we know what they are NOW.

@!#$, If you want compatibility... shove a legacy-cpu on a chip somewhere Why design the entire system around it? Why cripple the future? Why the hell keep slamming against the limitations of the past?

~K

[Steven]

I think the most interesting open problems are at a higher level than OS dev, but will have a strong influence on the directions of future OS research. The move to many-core is a big one, but it's just part of an even bigger move towards parallel and distributed computing. This move will require huge advances in programming languages, run-time environments, and OSes alike. Where all three intersect is the most interesting research area, IMO.

As for the next big steps in computing? Personally, I think it'll be with massively distributed systems. Moving away from the idea that a computer is yknow, a single machine. No more thinking about the single-core machine, hell, no more thinking about 4-8 core machine. No, instead thinking of infinite cores - infinite users.

So for the people talking about parallel computing; are you saying the whole concept of threads as we know it today needs to be replaced in order to build better (?) / more efficient (?) / easier to develop for (?) parallel systems? Parallel LINQ and the Parallel extensions for .NET seem to me to be a fairly compelling advancement in this area. They still require a developer to have an understanding of what's going on under the hood to gain any real performance improvements, but they greatly simplify the whole process of developing parallel applications. What are your thoughts as far as where the advancement will come from? Will any major advancement in parallel design come from a hardware design change? Is there some change you are imagining in the processor world that would make your lives much easier?

[Love4Boobies]

You forgot to mention the most important interaction of them all: the interaction between (wo)man and machine. I still find it hard to believe that in the last 25-30 years, no new great advances have been made in either the world of GUIs or CLIs. The last really big invention in this area was the mouse, IMHO. And that was a while ago now.

I know there are things like speech recognition. But does anybody ever use it? Nope, because it's worse than the current methods. There must be something better, I'm sure of it.

Well, first of all, there was Smalltalk, but it didn't get much attention, mainly because people in the industry are stupid. CLI was meant to be a programming language, not just a way to interact with the OS, although that's what it's usually thought of. The same thing should apply to GUI-based UIs. Smalltalk was a step in the right direction.

As far as distributed and parallel computing go , I agree 100%. Only a little while ago, Intel and Microsoft had some kind of a competition that several universities applied to, in order to find a good way of using many-core architectures. Berkeley and some other university won the challenge and they're getting funds in order to continue their research.

As far as hardware innovations go, there was some fuss some years ago (and some think there still is) in genetic computing. IMO that's error prone and although it's a nice step towards parallel computation, quantum computing is the real future.

[Troy Martin]

I designed my OS with minimalistic hardware intended. The code assembles to the 186 instruction set, you need like 256KB of RAM, a floppy drive, a keyboard, a monitor, and a VGA card. That's it. And it's entirely in assembly language and I don't really plan on going to 286 anytime soon.

I agree with Mr. E, the bkerndev copy and pastes are outrageous. They don't have anything more than the simple code they copied and a "hello world" message. Their authors have no freaking clue what the hell user mode is, let alone how the code works.

[Love4Boobies]

Well if you're designing code for the 80186, then your OS has nothing to do with user mode either

[Colonel Kernel]

So for the people talking about parallel computing; are you saying the whole concept of threads as we know it today needs to be replaced in order to build better (?) / more efficient (?) / easier to develop for (?) parallel systems? Parallel LINQ and the Parallel extensions for .NET seem to me to be a fairly compelling advancement in this area.

They're a good start, but a more comprehensive approach to distributed parallel computing is required. AFAIK most parallel libraries that are coming out now still assume mutable shared memory.

What are your thoughts as far as where the advancement will come from?

I think two major advances that we need are transactional memory (with transactional semantics going all the way up to the programming language level), and much higher-level languages for distributed co-ordination of tasks. For example, take a look at the E programming language. It has simple abstractions to deal with some of the complex issues of distributed computing like partial failures, messages arriving out of order, etc.

In the area that I'm now working in (highly scalable web services), these kinds of abstractions are badly needed. Right now the state of the art seems to be a cobbling together of SQL stored procedures to manage shared mutable state and transactions, XML for interchangeable messages (which provides no encapsulation and a lot of verbosity), and silos of high-level language code running on each machine that still have to deal with the accidental complexity of distributed computing on an ad-hoc basis.

Will any major advancement in parallel design come from a hardware design change? Is there some change you are imagining in the processor world that would make your lives much easier?

Not really... I think hardware changes are making software harder to write, not easier. But things like many-core will certainly be better for performance and scalability in the long run. I think we should be able to write apps in such a way that they would run just as easily on a server farm of hundreds of machines as on a single many-core box.

[Troy Martin]

Well if you're designing code for the 80186, then your OS has nothing to do with user mode either

Ahh, but I grasp concepts about user mode. But real mode calls to me like bananas to monkeys.

[dr_evil]

So my question is this: what do you think are the major open questions in os design?

Traditional OSs are becoming less important with every day. From a user perspective it doesn't even matter what kernel design you choose. Almost everything of importance can be achieved with user-mode software.

Short-term-trends:
- Managed OSs (Singularity & ...)
- Scalability & easier multiprocessing/-threading

Long-term-trends:
- Abstraction & virtualization

I believe the future are intelligent clusters/clouds/farms/.... Self-managing systems with automativ failover and such things. In the end a user will use a browser to run his/her applications. The application will run on clouds. Nobody will care any longer which OS runs underneath.