Philosophical need for a new OS / language

[AlexHully]

Hi,

Creating an OS is a huge task.

Maybe some of us -like me- took the shortcut of tinkering over freebsd/linux/anything else.

Now, the problem seems to lie far deeper.

You have your OS. Good.

Now, if -like me, again- you need to have proper drivers working, doing low level stuff (machines), it becomes increasingly difficult to compete with established OSes.

You may have, mostly, to rewrite large portions of the driver or, if you are not lucky, have to deal with another hardware because the driver's source is not available.

So, I would like to know, how many of us can find a professional use for their OS in the real world.

The same applies for a language : when doing low level stuffs, even if the language is a system one, it has to have its ecosystem to work properly. And you will still need to use C/C++, because everything low level uses C/C++.

Can we get away from C? (you can answer in one word, I guess).

The underlying: is all that work worth it, for business purposes (so, except for learning how things work)?

Isn't just modifying an existing OS the only realistic thing to do ?

[iansjack]

So, I would like to know, how many of us can find a professional use for their OS in the real world.

I suspect that very few here look for such a use. Personally, I just do it for fun.

And you will still need to use C/C++, because everything low level uses C/C++.

Incorrect.

Can we get away from C? (you can answer in one word, I guess).

Yes, we can. (Three words, I'm afraid.)

Isn't just modifying an existing OS the only realistic thing to do ?

That's up to you. "Yes" if you are seeking a professional use for your OS. And even then you will have to be very lucky to do better than what is already out there. "No" as far as I am concened.

[AlexHully]

You are right, I forgot Assembly and machine code, sorry. Coding in very low level with something other than those languages to communicate with machine isn't the biggest part of the pie, right?

What is your proposal to get away from C/C++?
Rewriting all the drivers out there? (to me it seems like the only option).

[Ready4Dis]

You are missing other languages that have been used as well, like pascal, basic, D, Objective-C, and others?

Why do you want to get away from C/C++? What is the downside? What do you feel is a better language for system development? Is there a better language, or can you design a better one?

And if you don't keep compatibility with any other OS, then yes, your only options are to either re-write all the drivers, or somehow convince the manufacturer to support your OS (doubtful unless you get pretty large, even linux still has a hard time with support).

My OS is for fun and learning, I don't intend it to be used professionally.

I would think, unless you have a specific need that isn't being filled, you are wasting your time if you're not doing it for learning/fun. Just designing an OS that isn't any better at anything than the existing OS's, why would anyone switch? No or few drivers, buggy (new OS hasn't been tried on 5+ million PC's), no clue how long it'll be around (business' like something supported over a longer period of time), etc. Like I said, unless you have something to fill a void, it's purely academic.

[AlexHully]

Thanks for your opinion Ready4Dis

Yes I want to do six things. And I will do them on top of freebsd.

1) a language simpler than c with OOP (and no visible pointer. Pass everything by value and the compiler will do the reference for you. Granted to work). I code in c++ already but I dislike that language. And C.. I kind of like it but.. Actually that language will ressemble C a lot because most of the design works and to me, it seems pure.
2) a language that enables massive partnership with assembly (no more of that inline assembly crap that always bites you in the back). A proper inlined assembly the compiler can optimize.
3) jitted (LOT of modifications but I did most of it already)
4) no runtime and a different import system
5) usable immediatly (copy paste of many bsd libraries). Bottom line : hardware makes the rule anyway (cpu, devices..). There is very few possibilities to innovate if the hardware doesn't allow what you mean.

6) put into life an innovation and see how it compares with the current system (it touches almost every part of the kernel).


The latter is actually the why I went through all this. But I feel discouraged, and will opt for just some -important- modifications over something already existing and already tackling all the hardware problems for me. That is too much otherwise.

And I will be able to come with a professional grade OS pretty quick (weeks).

[Kazinsal]

I don't think you'll ever encounter a language suitable for systems programming that both doesn't expose pointers and can handle data structures without ridiculous amounts of glue written by the programmer to transparently handle the pointers. Look at the ACPI tables -- the root table has to be manually searched for, and it's composed entirely of pointers to other data structures that are in turn full of pointers.

Now, if -like me, again- you need to have proper drivers working, doing low level stuff (machines), it becomes increasingly difficult to compete with established OSes.

Most people don't end up developing an OS anywhere near able to compete with even other "established" hobby OSes. Getting to the forefront of your peers in the OS development community is already a pretty impressive and difficult feat.

So, I would like to know, how many of us can find a professional use for their OS in the real world.

The brutal truth is it's near zero. I know a couple here have had some of their work used in professional environments, though, so it's not impossible. I'm personally content with having someone use it for the sake of using it (or because they think it's a neat OS or whatever). Regardless of what a hobby OS is used for, in order for it to graduate from "hobby" to "project that people use", you need to have something either unique or promoting it.

Can we get away from C? (you can answer in one word, I guess).

Sure. You can use all sorts of languages for systems development; check out Rust.

The underlying: is all that work worth it, for business purposes (so, except for learning how things work)?

Jumping into OS dev for business purposes is a terrible idea. Doing it because you want to learn how to do it is much more realistic.

Isn't just modifying an existing OS the only realistic thing to do ?

Depends on what you want to do with it. I wouldn't recommend writing a whole new kernel and TCP/IP stack for a firewall when you can just use pfSense (or pf on its native OpenBSD), for example.

And I will be able to come with a professional grade OS pretty quick (weeks).

Beginner Mistakes

[Antti]

So, I would like to know, how many of us can find a professional use for their OS in the real world.

For me it sounds very pointless to have limitations. If my OS ended up being good enough, I would definitely welcome professional users. It does not need to compete against general purpose operating systems but be good at something. It is hard to define what is professional and what is not. If I used my OS and wrote a best-selling book, would it be a professional use?

There are a few non-technical details that could make it different from pure hobby projects. First of all, there are no fun error messages, colloquial language, or anything that are usually seen in hobby projects. Another thing is to hide typical hobby OS look-and-feel details, e.g. printing messages like "Setting up IDT... [OK]" and the like. The target audience is not just other hobby OS developers so good splash screens and other small details are highly important. The last thing is the reasonable approach to freedom but I know this is a highly flammable topic so please skip it.

[AlexHully]

Ok you ended my reflexion on all this and confirmed my thoughts.

A good night helped too.
I will use what I already created, letting all the other stuffs managed.

Yes, i will bring something different, but perfectly fitting the current business. That is a realistic (professional) approach.

As far as tcp/ip stack (+ webserver) is involved, we already made it much faster (x3-5).

Maybe it will get traction. Or it will just be used inside my startup. i don't really care because I know i can make my mates much more productive. And the ratio money/time is severely important.

It is not done just to be done, there are savings involved. but I wanted to know where I should stop my digging.

And you gave me your experience. i am massively grateful for your honesty and help.
You are invaluable.

Thank you

[AlexHully]

antti,

I agree with you under one condition: you can innovate greatly when you have the funds to litteraly create new hardware. That is a lot of budget.

But I concur, a lot can still be done, more efficiently, given time and money.

Edit : And this OS will only run on specific hardware. That is a very important point.

[alexfru]

Now, if -like me, again- you need to have proper drivers working, doing low level stuff (machines), it becomes increasingly difficult to compete with established OSes.

You don't always use an existing/established OS/kernel/etc. Sometimes you just can't or it is more reasonable to implement a small one for a specific task.

So, I would like to know, how many of us can find a professional use for their OS in the real world.

Define real world.

A few examples from my practice:

- I had a simple device that could perform a specific task, but I wanted to make it perform two independent instances of the task without changing the existing code much. I made a scheduler that would switch the context between those two essentially identical tasks on a timer interrupt. No need for Linux or RTOS.

- I had a project, where I needed some kind of micro kernel and TI's DSP/BIOS was available and it functionally fit the bill, but it didn't fit memory-wise. I ended up coding the following: scheduler, system calls, locks/mutexes. That was all I needed and it was easy to write and had a tiny footprint.

- I supported and extended a dedicated test micro kernel used in testing of Hyper-V. A test VM would run it with test apps. Nothing fancy, light years away from Linux. It was good enough for its special purpose.

If you stop thinking of an OS being a product in and of itself competing with Windows, Linux and such in their game on their turf, then you can find niches for your OS more easily. That may, however, require that you change your view of an OS and how you decide what features to implement and how and what not to implement.

[AlexHully]

alexfru, that is a very good input

You can make what you need by copy-pasting existing code, saving a lot of time, too and arrive to your final OS.

I think I was wrong thinking about the major players.
And more wrong thinking habits won't change.

What would be your take for a small device like a smartphone? or a tablet?
A custom kernel/OS could suffice given specific (and not switchable) hardware.

It could be made much faster, safer, and flexible than anything existing and still fitting the goal.

[alexfru]

What would be your take for a small device like a smartphone? or a tablet?

Size alone means very little these days. Those are pretty much general purpose computers, which just happen to be small. Unless you remove the general purpose part, you need a full OS for it.

It could be made much faster, safer, and flexible than anything existing and still fitting the goal.

I doubt about flexible if your system designed for few and very specific tasks, unless you view e.g. Linux as rigid in the sense of you being unable to tweak it because of it being way too big and complex for you.
Safer/more secure doesn't come for free nor does necessarily follow from smaller code, although, if the bug density (bugs per line of code) is constant, you get fewer bugs in total in a smaller code base.
Same with speed. You can fnck it up as well, even in small code.

But yes, if things are done right and inputs (data or user-installable software) are restricted, you can have better security, reliability and performance in a smaller and simpler system than in a PC. It's easier to reason about and control smaller and simpler systems.

[AlexHully]

Linux is bloated and difficult to tweak. Not because you don't find the right line of code, but because many things can go wrong, even if not obvious or because of a subtle bug.
The mailing list goes crazy with those examples.

So yes, for me and more and more people, it is a problem

[alexfru]

BSD to the rescue?

[AlexHully]

Absolutely. I chose freebsd. The code is much easier to grasp and modify/optimize.

The comments are really good, too.

It is still a beast, but a manageable one (in my opinion)