SPICE: lots of theoretical wankery that may someday be an OS

Discussions on more advanced topics such as monolithic vs micro-kernels, transactional memory models, and paging vs segmentation should go here. Use this forum to expand and improve the wiki!
commodorejohn
Posts: 11
Joined: Sat Jul 05, 2014 9:31 pm
Location: Duluth, MN
Contact:

Re: SPICE: lots of theoretical wankery that may someday be a

Post by commodorejohn »

Rusky wrote:If so many good programmers still let buffer overflows through, you cannot lay the whole blame on them for not seeing the obvious, because it is clearly not obvious or stupid at that point.
Excuse me? We can't blame people for continually making the same basic, well-known mistakes (as in, seriously, people, bounds-check network-facing buffers! There is no excuse for not doing this!) because they must not actually be basic or well-known or people wouldn't be making them? Is that seriously what you're saying here?
You still haven't come up with any specific, real criticisms of Unix.
I've come up with several. You (and multiple other people) have just written them off with the age-old mantras of "that doesn't actually happen," "works for me," "I'd like to see you do it better," and "you probably just need to RTFM."

So screw this, I'm done here. I was thinking this would be a good place to actually talk about my thoughts and goals and get actual constructive criticism, but essentially all that's happened is people getting a burr up their @$$ because I had the audacity to suggest that a mainframe OS from the 1970s might not be the best possible template for a modern desktop OS and lecturing at great length on how stupid I must be for thinking that anything else could possibly be better (while continually protesting that no, no, they really don't think Unix is perfect, honest, they just can't believe that I would ever consider anything else to be better in any way,) and how I'm obviously some know-nothing newbie with delusions of grandeur (despite my repeatedly noting that I'm not even taking credit for any of the core concepts, as they were all laid down years ago by people much smarter than myself, and simply forgotten as the world turned to Unix-worship.) I'm sorry to not get to discuss this with the few actually pleasant and open-minded people here, but I don't need this crap. I've wasted enough time on this thread as it is.

I now yield the floor to the inevitable chorus of "ha ha, obviously the fact that he was driven away by our rudeness means he was only full of hot air!" Later.
Computers: Amiga 1200, DEC VAXStation 4000/60, DEC MicroPDP-11/73
Synthesizers: Roland JX-10/MT-32/D-10, Oberheim Matrix-6, Yamaha DX7/FB-01, Korg MS-20 Mini, Ensoniq Mirage/SQ-80, Sequential Circuits Prophet-600, Hohner String Performer
User avatar
AndrewAPrice
Member
Member
Posts: 2299
Joined: Mon Jun 05, 2006 11:00 pm
Location: USA (and Australia)

Re: SPICE: lots of theoretical wankery that may someday be a

Post by AndrewAPrice »

Commodorejohn - Before you continue posting, it would really help your case if you take the time to think up a better alternative to the problems you see. Complaining only gets people down or offends them unless you try to be constructive by offering a solution.
My OS is Perception.
User avatar
Rusky
Member
Member
Posts: 792
Joined: Wed Jan 06, 2010 7:07 pm

Re: SPICE: lots of theoretical wankery that may someday be a

Post by Rusky »

commodorejohn wrote:Excuse me? We can't blame people for continually making the same basic, well-known mistakes (as in, seriously, people, bounds-check network-facing buffers! There is no excuse for not doing this!) because they must not actually be basic or well-known or people wouldn't be making them? Is that seriously what you're saying here?
What's more productive? Continuing to blame only the developers for "obvious" mistakes, or providing tools to catch their errors as well? Especially when those tools will also catch many subtler errors.
FallenAvatar
Member
Member
Posts: 283
Joined: Mon Jan 03, 2011 6:58 pm

Re: SPICE: lots of theoretical wankery that may someday be a

Post by FallenAvatar »

commodorejohn wrote:... bounds-check network-facing buffers! There is no excuse for not doing this!) ...
Performance.
User avatar
SpyderTL
Member
Member
Posts: 1074
Joined: Sun Sep 19, 2010 10:05 pm

Re: SPICE: lots of theoretical wankery that may someday be a

Post by SpyderTL »

commodorejohn wrote:So screw this, I'm done here. I was thinking this would be a good place to actually talk about my thoughts and goals and get actual constructive criticism, but essentially all that's happened is people getting a burr up their @$$ because I had the audacity to suggest that a mainframe OS from the 1970s might not be the best possible template for a modern desktop OS and lecturing at great length on how stupid I must be for thinking that anything else could possibly be better (while continually protesting that no, no, they really don't think Unix is perfect, honest, they just can't believe that I would ever consider anything else to be better in any way,) and how I'm obviously some know-nothing newbie with delusions of grandeur (despite my repeatedly noting that I'm not even taking credit for any of the core concepts, as they were all laid down years ago by people much smarter than myself, and simply forgotten as the world turned to Unix-worship.) I'm sorry to not get to discuss this with the few actually pleasant and open-minded people here, but I don't need this crap. I've wasted enough time on this thread as it is..
I tried to warn him... :mrgreen:
Project: OZone
Source: GitHub
Current Task: LIB/OBJ file support
"The more they overthink the plumbing, the easier it is to stop up the drain." - Montgomery Scott
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: SPICE: lots of theoretical wankery that may someday be a

Post by Brendan »

Hi,
commodorejohn wrote:
Brendan wrote:The differences are how we are planning to solve the complexity and "mistake detection" problems - commodorejohn seems to be going for "interpreted" (which could solve the complexity problem and should solve most "mistake detection" problems, but implies severe performance penalties).
Actually, I'm looking at interpreted systems more for the purposes of portability than security - beyond a certain level of catching access violations, I really don't think that it's feasible or wise to expect the language or runtime environment to do the developer's job of debugging for them.

(About the performance penalties, ten years ago when I was learning Java in college and marvelling over what a mess it was, I would've agreed completely. Now, though, I really think we've reached a point where a sensibly-designed VM can be quite performant enough for pretty much anything besides heavy number-crunching or high-end gaming.)
Ah - to me "interpreted" means nothing is compiled down to native code at all, and Java does what I'd call "JIT compiling it down to native code". The easiest way to determine the likely performance differences (in a language neutral way) is to look at emulators - e.g. Bochs does "interpreted" and is roughly 100 times slower than native, while others (VMware, VirtualPC, VirtualBox, Qemu, etc; without hardware virtualisation) do JIT and are significantly faster (e.g. between 10% to 95% of native speed, depending on how good the JIT compiler is).

Note that I'm currently planning to do "2 step ahead of time compiling"; where programmers compile source code into a portable format (with lots of sanity checking, optimisation, etc); and then end-users download the portable executable and their OS compiles it to native code the first time it's executed.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
User avatar
AndrewAPrice
Member
Member
Posts: 2299
Joined: Mon Jun 05, 2006 11:00 pm
Location: USA (and Australia)

Re: SPICE: lots of theoretical wankery that may someday be a

Post by AndrewAPrice »

commodorejohn wrote:I was thinking this would be a good place to actually talk about my thoughts and goals and get actual constructive criticism, but essentially all that's happened is people getting a burr up their @$$ because I had the audacity to suggest that a mainframe OS from the 1970s might not be the best possible template for a modern desktop OS and lecturing at great length on how stupid I must be for thinking that anything else could possibly be better (while continually protesting that no, no, they really don't think Unix is perfect, honest, they just can't believe that I would ever consider anything else to be better in any way,) and how I'm obviously some know-nothing newbie with delusions of grandeur (despite my repeatedly noting that I'm not even taking credit for any of the core concepts, as they were all laid down years ago by people much smarter than myself, and simply forgotten as the world turned to Unix-worship.)
We know Unix isn't perfect. And although some of us are making Unix clones, many of us are not - we're doing something completely different from scratch. SkyOS, Syllable, MenuetOS, DexOS, ToAruOS. One day I also hope to have a great non-UNIX clone OS. Most of us are aspiring to innovate. Some of us are crazy and make our own languages, compilers, architectures, UIs, and more.

What I and multiple have been asking is - what is your alternative?

This is not about defending UNIX or any other operating system, it's about not liking people that complain without offering an alternative. That's all we want. Your idea doesn't have to be perfect. We're not attacking you. Just show us how you'd so something better.
My OS is Perception.
FallenAvatar
Member
Member
Posts: 283
Joined: Mon Jan 03, 2011 6:58 pm

Re: SPICE: lots of theoretical wankery that may someday be a

Post by FallenAvatar »

Brendan wrote: Note that I'm currently planning to do "2 step ahead of time compiling"; where programmers compile source code into a portable format (with lots of sanity checking, optimisation, etc); and then end-users download the portable executable and their OS compiles it to native code the first time it's executed.
This is what I am looking at. Using a portable arch-agnostic format means at "install" time you can verify almost 100% (99.9%-ish) of instructions being "legal" greatly reducing bugs from programmers (vs. developers)

- Monk
User avatar
Brendan
Member
Member
Posts: 8561
Joined: Sat Jan 15, 2005 12:00 am
Location: At his keyboard!
Contact:

Re: SPICE: lots of theoretical wankery that may someday be a

Post by Brendan »

Hi,
tjmonk15 wrote:
Brendan wrote: Note that I'm currently planning to do "2 step ahead of time compiling"; where programmers compile source code into a portable format (with lots of sanity checking, optimisation, etc); and then end-users download the portable executable and their OS compiles it to native code the first time it's executed.
This is what I am looking at. Using a portable arch-agnostic format means at "install" time you can verify almost 100% (99.9%-ish) of instructions being "legal" greatly reducing bugs from programmers (vs. developers)
My motives are different - I'm doing the majority of sanity checking during the beginning of "source to portable compiling" (so that developers get told of their mistakes as early as possible). The "portable to native" compiling is done for portability and optimisation (including whole program optimisation and optimising for the end-user's specific CPU rather than just "generic 80x86") and isn't meant to do much sanity checking. I also use hardware protection/isolation to ensure that (e.g.) if someone uses a different OS to directly modify previously compiled native executables then the modified/malicious code is contained.


Cheers,

Brendan
For all things; perfection is, and will always remain, impossible to achieve in practice. However; by striving for perfection we create things that are as perfect as practically possible. Let the pursuit of perfection be our guide.
FallenAvatar
Member
Member
Posts: 283
Joined: Mon Jan 03, 2011 6:58 pm

Re: SPICE: lots of theoretical wankery that may someday be a

Post by FallenAvatar »

Brendan wrote:...
My goal with the install-time compile is to check security, the first compile (same as yours) checks for dev bugs/errors. this is to provide the best possible performance with the best of static checking (hopefully) And I'm aiming at all ring 0 code (even user processes) with the above requirement and code pages being read only, and no on-the-fly code generation.

- Monk
linguofreak
Member
Member
Posts: 510
Joined: Wed Mar 09, 2011 3:55 am

Re: SPICE: lots of theoretical wankery that may someday be a

Post by linguofreak »

Rusky wrote: What interface should X11 have used in 1984 so that those new interfaces wouldn't have been necessary? Compositing?
I am sorely, sorely tempted to say "yes". Of course, machines simply didn't have enough memory back then to make compositing viable for any reasonable display resolution, but one thing that always irks me is that people always point to X as an example of everything wrong with Unix and the Unix design philosophy, when X, despite its long association with Unix, isn't very Unixy (for better or for worse).

A Unixy graphics system would have each application draw to its own frame buffer. The kernel would back each frame buffer with a device file. The windowing system would then composite the various application framebuffers into its own frame buffer (which would generally be the hardware frame buffer for the display hardware, but could in principal be another virtual frame buffer, allowing windowing systems to run nested).
embryo

Re: SPICE: lots of theoretical wankery that may someday be a

Post by embryo »

A bit about solution:
Brendan wrote:It'd be more accurate to say package management helps hide the symptoms of "dependency hell"; and hiding symptoms of a problem isn't something I'll ever be fond of
Here Brendan has identified a "symptom". Also many people here have identified some others "symptoms". And next some people offered some partial "solutions" like this:
Brendan wrote:I'm currently planning to do "2 step ahead of time compiling"; where programmers compile source code into a portable format (with lots of sanity checking, optimisation, etc); and then end-users download the portable executable and their OS compiles it to native code the first time it's executed.
But as whole the ideal OS is still very far away.

Why? For how long?

Maybe forever. Just because there is no cooperation. One man can make some weak OS with just a negligible fraction of applications that Windows or Linux have. And of course such OS won't be an ideal.

But it seems to me, that while there are some people wishing to create a multi-step compiler or memory manager or file system or virtual machine or whatever else, we can move the solution a bit closer than "forever far away".

Why people start doing their personal OS? Because they think it is possible to make it better than existing solutions. But the amount of work is great and until the full amount is done the system will suck. Even if the full amount will be done the system still be sucking, because there are plenty of needs the system is not addressing. And it is the seldom best case variant, because more probably the system will suck just because of lack of good architecture or careful programming.

But if we have an architecture that is accepted by most of the osdevers? It isn't that impossible as many of you are thinking now. And it gives a way for creation of a really good OS (but still not an ideal). But why osdevers have not accepted any architecture yet? Just because they see their personal way for some parts of the proposed architecture. How can we overcome such problem?

My proposal is like this:

For the "really good OS" to be accepted it is required to define a call standard and service standard. And that's all.

First, there are components, that any OS must have. It is bootloader, memory manager, scheduler, hardware manager. What prevents us from making those components to complain with the call standard and service standard? For example a memory manger can implement the call standard and expose functionality like allocate and deallocate. And anybody here can implement it's own memory manager if he wishes. If implemented the manager is easily attached to the "Open OS". But if a person is interested in multistep compilers then it is possible to forget about memory manager (just because there already are such managers) and to concentrate on the compiler development. When a compiler is ready it can be used as another component, complaining to the service/call standard. If somebody prefers his personal variant of bytecode then he just creates personal JIT compiler and integrates the bytecode seamlessly into the "Open OS". And here the person with his own bytecode is not required to make a whole OS, it is enough just to get "Open OS" and include there personal compiler and bytecode. If then some other component of OS is seen as "not very good" then the person just implements his own variant of the component. Then we can have very good efficiency and concentration of efforts on the very task a person prefers. And always any participant have an OS ready to use (at least to debug his component). And anyone can take any preferred component to create his personal "distribution".

And yes, it is the Linux way, but with hope for "better architecture".

And now to have such "Open OS" there is a need in the architecture acceptance. As a starting point in such process I propose the "any OS must have" components.

Do you have a boot-loader? Very well, we can include it as a service with the calls like this:

Code: Select all

configureFor(int addressToPlaceSystemImage, enum filesystemUsed, Location imageLocationData);
Do you have a scheduler? We can include it with calls like this:

Code: Select all

int startProcess(long jupmAddress);
int startThread(int processId, long jupmAddress);
boolean killThread(int threadId);
SomeType anyExtraFunctionalityYourSchedulerHas(ParameterType param1, AnotherParameterType param2);
Or you have a memory manger? Or you have a device manager? Or you have anything else? You can test it with "Open OS" after implementing the call and service standard.

And of course, first releases will be just technology demonstrators, but may be after a dozen or two iterations we will have a really good OS.
User avatar
AndrewAPrice
Member
Member
Posts: 2299
Joined: Mon Jun 05, 2006 11:00 pm
Location: USA (and Australia)

Re: SPICE: lots of theoretical wankery that may someday be a

Post by AndrewAPrice »

embryo wrote:But as whole the ideal OS is still very far away.
I don't think there will be an ideal OS, in the same way there isn't an ideal way of cooking (Japanese, American West, Italian) as everyone has their own preferences. That's not to say we can't work towards creating something ideal, or at least something we love and are proud of and hope someone else also loves. But everyone will always have their preferences.

Windows vs. Linux vs. FreeBSD vs. MacOS vs. Amiga vs. AIX

KDE vs. Gnome vs. XFCE vs. i3 vs. Command Line.

Android vs. Windows Phone vs. iOS vs. Symbian

C vs. Assembly vs. Java

Sometimes it could be ideological preferences that don't actually make difference at the user-level, such as Microkernels vs Monolithic Kernels.

Everyone has their own tastes. Sometimes, similar to food, we like to change things up - even the world's largest spaghetti fan may be in the mood for Chinese food.
My OS is Perception.
User avatar
bluemoon
Member
Member
Posts: 1761
Joined: Wed Dec 01, 2010 3:41 am
Location: Hong Kong

Re: SPICE: lots of theoretical wankery that may someday be a

Post by bluemoon »

embryo wrote:Why people start doing their personal OS? Because they think it is possible to make it better than existing solutions.
You neglect the fact that everyone may has different goal, some are mutual exclusive. The direct consequence is everyone want a different OS.
embryo wrote:But why osdevers have not accepted any architecture yet? Just because they see their personal way for some parts of the proposed architecture. How can we overcome such problem?
Yes and no. There are parts that are standard and reusable, from algorithms to driver interfaces and sometime design models. OS is so complex that you can't really think it consist of only ONE architecture.
embryo wrote:What prevents us from making those components to complain with the call standard and service standard?
xkcd: standards
User avatar
Rusky
Member
Member
Posts: 792
Joined: Wed Jan 06, 2010 7:07 pm

Re: SPICE: lots of theoretical wankery that may someday be a

Post by Rusky »

There will always be variation in taste for software, but there are objective measurements too, wherever software hits reality. How many HTTP requests can you handle for a given price? How (in)frequently are vulnerabilities discovered and how bad are they? What framerate is your UI capable of? How quickly can IT set up your system for their particular purpose? Can your system do X?

The objective measurements are what determine success anyway, regardless of the ideology behind them and how it's intended to help the objective measurements. Programming languages with the most extensive library support are used the most, regardless of how safe/fast/easy they are. Operating systems with the most applications are used the most, regardless of how secure/user-friendly/free they are.

The only time the ideology really matters is if it directly affects someone's bottom line, whether that is one of those objective measurements or whether it's how fun you think it is to play with.
Post Reply