(Very incomplete) Concept design of an OS

[ignus]

Some programmers (me first) prefer doing before thinking. It's usually a bad idea, so this time I try to do the opposite.
It's quite a huge project and surely I won't implement it completely. I'll start soon, and I'll go on until I want.
Just a note: I'm not an expert. I'm a very newbie. Something I'll wrote will be wrong. Feel free to abuse me.
Another note: It's a really high-view of an OS. Maybe it's not an OS but a group of services. And the implementation it's quite impossibile.
But I would hear what you guys think about it

So let's start (I divided the system into point of views):

DEVICES
Each device/componenet have to be registered in the device manager. At the start-up, each connected device is initialized.
The category are:

• Processor (Cpu, Gpus, microcontrollers, FPGA, dsp)
• Main memory
• Storage (hdd, ssd, flash storage)
• Inputs (keyboard, mouse, webcam, mics, joypad, buttons, VR, scanner, bluetooth receive, ethernet, wifi)
• Outputs (video, speaker, mechanical outputs like vibration, printer)
• Bus (protocol that manages each internal bus: sata, pci, usb, battary management)

Any device have to offer an interface to talk with:

• Processor: each processor have to specify its task, a CPU regster itself as n device (n = nr of core), a GPU register itself as n cores that can do only particular jobs, a random generator (did you see them? sort of USB that you plug in and gives you random numbers) register itself as default processor that generates random number. So if I ask for a random number, I look at the device manager's default RNG, and then I ask him to give me my random bits.
• Memory: read/write. Must support paging. Memory protection is optional: high privilege task (driver) can write and execute, in every page; user tasks can write or execute, not both (protection agains self modifying code, a tecnique that is very interesting but I won't apply to user apps (see ABI and applications)
• Storage: read/write
• Inputs: many subcategories: example VR: track head, eyes, mouth movement; mics: give me a stream of uncompressed audio at xxkbps
• Outputs: many subcategories: video: bring the video memory buffer from the RAM and print it into the buffer; printer: receive the stram of character and print...
• Bus: each bus fall in its category

PROCESSOR and SCHEDULING
When an application starts, it starts each on a different core (eg: 3 cores, thread 1 = core 0, th2 = core1, th3 = core2, th4 = core0...), if the application requires "special" resources eg. read a file from the hdd, without askng the OS it can ask the hdd (that has to supply a standard "storage" interface) the file the application asked. Simple time scheduler, with a priority queue that lets urgent thread to run more often than common ones.
If the application want to process a shader, it asks to the GPU, that process it and (I don't know) write the processed shader in the video buffer.
If the application ask for a particular custom device that do a particulat task (eg cryptography), if it's not available abort the application.

FILESYSTEM
I know nothing about filesystem. Just quote that the OS's partition divides into /core, /drivers, /user

ABI and applications
Binary data divides into APPS and DRIVERS.
A driver is an already compiled executable that have to register its role in the system (video, audio...). It has high privileges and can access and write memory everywhere (dangerous?)
Apps come in an intermediate form (like java's bytecode) that is compiled during the installation (platform and processor indipendent). Apps can write only inside /user directory.
The OS should come with a rich build-in library, be modern (C++ style) and be coherent (all the function uses stdcall, no in-out parameters, not too much templates because it's ugly)

GRAPHICS
Most difficoult part?
The graphic part should be impemented with a buffer, so displays bring data directly form this buffer. Must support different resolution and different color depth. Build-in library have to implements some low level tasks to draw things on the screen

Feel free to comment

[kutkloon7]

I don't want to be rude, but the questions answered in your design are not the most relevant in developing an OS. More important questions are the memory allocation, support of multitasking (and if yes, what kind of scheduler), what functions is the system library going to support (is it POSIX compliant?), what system calls are there gonna be, and maybe most importantly, what type of kernel you are gonna write (micro-, monolithic, hybrid, exo-, nano-, ...).

If you are unfamiliar with some of these topics, you can always read the wiki (both the osdev wiki and wikipedia contain lots of useful information). You can always try posting a question here, but please make sure you do your research first. You really have to learn about a lot of topics before you're ready to make an OS design.

I hope I didn't discourage you too much. In this case, I think it's a good idea to learn while you're making an OS. Your first OS is probably going to be a simple one anyway (that is, for an OS. It is probably one of the bigger programming projects you ever did). Learning while reading a tutorial is an excellent way to learn IMO. Just look for the tutorials section on the wiki. Also, the series by Brokenthorn is really good (just google it), I use that one myself (I'm going to implement a filesystem soon, when I get to it).

Good luck!

[ignus]

I don't want to be rude

You aren't Each comment is welcome.

However, you're right. I swear I have already read quite a lot on the wiki (but I'm still practice with assembly)
In this forum there are lots of technical post, "how to do this and that", but I wanted to know how other expert imagines their OS finished (well, the first release candidate)
I don't think lots of people will spend 40 minute of their lives writing about their project, but maybe can spend 30 second telling me what they would change in this project

[embryo]

Concept design of a human.

It's a really high-view of an OS. Maybe it's not an OS but a group of services.

It's a really high-view of a human. Maybe it's not a human but a group of organisms.

DEVICES

PARTS
Every human has:

• Legs
• Arms
• Eyes
• Ears
• Nose
• Body
• Something else

Any device have to offer an interface to talk with

Any part has to be connected to others.

• Ears: each ear has to register itself as x device (x=y of devices)
• Legs: walk/run. Must support body. Foot protection with shoes is optional.
• Eyes: look/sleep. Support of RR: track objects, register movements, give me a stream of uncompressed video at xxkbps

PROCESSOR and SCHEDULING

BRAIN AND THINKING
When human is born it start thinking. If brain requires additional resources eg. pleasure, it asks human to start crying.
If human want to process a video it asks brain and (I don't know) write the processed stream in the video buffer.

Feel free to comment

The human above, as it seems to me, is a bit incomplete. I hope anyone can see it. But when we talk about an OS, anyone can collect a bit of words and then ask about "how cool my OS is?".

If it is possible to identify eyes, legs and arms, it doesn't mean that it is possible to create a human using those parts. To create something close to a human it is required to understand in depth every aspect of human existence.

Do you have such understanding regarding an OS?

It seems you haven't. Then instead of creating some global plan it is better to create a plan that you really can implement. And if you still prefer to work on the global "plan" then you are warned - you are trying to create a human without deep understanding of everything related to it.

It is possible to create a human if you start from one cell or even from some microbiological mechanics first. But starting with the goal of a human in mind is a bit too big for one's boots.

[SpyderTL]

I don't want to be rude, but the questions answered in your design are not the most relevant in developing an OS. More important questions are the memory allocation, support of multitasking (and if yes, what kind of scheduler), what functions is the system library going to support (is it POSIX compliant?), what system calls are there gonna be, and maybe most importantly, what type of kernel you are gonna write (micro-, monolithic, hybrid, exo-, nano-, ...).

I disagree, to a certain extent...

I think it's just as important, if not more so, to know why you are writing your own Operating System as it is to know how. Both are very important, but starting to write an OS without knowing where you ultimately want to end up will probably just eat up a lot of your free time, and you'll be doing essentially the same thing that everyone else has already done.

I started my OS with a specific target functionality/behavior in mind, and I have worked backwards to figure out the quickest way to get to that goal. You can learn how to set up memory managers and interrupts as you go, but you can't "learn" how to make a new OS paradigm. That takes imagination. And a lot of trial and error

@ignus, and anyone else with a crazy OS idea:

The OSDev site is a fantastic resource for learning how to make hardware work, and the people on this forum are experts in the field of OS development. But convincing them that a new OS design is worth the development effort, and trying to get their approval for your personal "dream" design will probably never happen. Everyone here already knows what the "perfect" OS would look like to them.

Just don't take it personally when they refuse to accept your ideas. You have to take the negative feedback, and pretend like it is constructive, helpful suggestions.

Otherwise, you will end up spending more time on this site defending your ideas than actually writing code...

So, good luck, and let us know if there is anything we can do to help.

[AndrewAPrice]

I don't want to be rude, but the questions answered in your design are not the most relevant in developing an OS. More important questions are the memory allocation, support of multitasking (and if yes, what kind of scheduler), what functions is the system library going to support (is it POSIX compliant?), what system calls are there gonna be, and maybe most importantly, what type of kernel you are gonna write (micro-, monolithic, hybrid, exo-, nano-, ...).

The examples you give are not the most relevant topics in developing an OS. Once my memory manager and scheduler are written, they're written.

Sounds like you'd come up with a bottom level design for a kernel, and will build an OS on top of that. Ignus came up with a top level design of an OS, and will build a kernel to suit it. Live and let live.

[embryo]

Ignus came up with a top level design of an OS, and will build a kernel to suit it.

There is a distance between design and imagination. It is possible to imagine everything, but most of such "designs" will fail. As an design exercise anybody can count legs, arms and eyes and then tell us he has a "top level design". As time passes it is possible to see that the task just doesn't match the designer.

But for the designer to understand it the exercise should be completed. Next the designer would prefer to propose a bit more viable goals. And in the end he will understand the difference between design and imagination.

[iansjack]

There is a distance between design and imagination.

I've got to agree with that. We see a number of one-off posts here about how people are going to write OSs in Jascript or HTML that run executables from Windows, Linux, and OS X systems, have mind-reading capabilities and are going to take the world by storm. The actual implementation of such a pipe-dream is another matter. The latter is probably the more difficult task.

[embryo]

Some programmers (me first) prefer doing before thinking.

When I see posts like the first in this thread there are two opposite emotions:
1. I want to wish some luck to the OP, for he to get some useful experience and implement at least some parts of the proposed system.
2. I want to show the lack of required experience to prevent the OP from spending a lot of time hunting for a blue bird.

But it seems the OP has taken (at least) my words as a stop signal. It is not the result I expected. Then may be ignus can start some elaboration efforts to get his plan a bit closer to the reality. For example (as it was already proposed) he can declare a goal of the OS, then he can write here some steps, required to achieve the goal. And next, other osdevers can help ignus to select proper steps or propose something better.

But it requires ignus to stop worrying and start telling us about his vision.

[ignus]

First of all, thanks for all this reply. I appreciate it very much, really
I hope not to be off thread, it's just an answer to other people that wrote above here

Ok I lack of enough experience to post on this forum, I just realized that
It was a great project in my mind to became a new Bill Gates but after looking around my "bubble of ideas" and after seeing how many problems have to manage really advanced programmers, I decided to change completely my project, to archive a real goal.

I decided to learn about computers and operative systems, before posting other such useless thread

I found one of OSdev's member project, BareMetal OS, which seems perfect for learning how an os works and how to write beautiful code
In addition, I'm still going on with Art of x86 Assembly which is very clear, and starting with Modern Operative System (which is still to hard, I think)

Do you guys think that's a more "realistic" goal? (intelligent more than realistic)

[SpyderTL]

As I said before, if you follow the recommended path of learning OS development, you will learn a lot in a short amount of time. However, you also run the risk of letting others influence your OS design, and ending up writing essentially the same OS that has been written hundreds of times already.

But it's entirely up to you. Either way you go, we'll be here to help if you get stuck.

Have fun.

[iansjack]

I've got to disagree with you. Computer programming is a craft, just like any other, and the basics have to be learnt before one can produce original works.

Just as a carpenter has to learn how to make dovetail joints before he can craft fine furniture so a programmer has to learn the basics of, for example, file systems or memory management, before he can craft a fine OS.

I can visualize stunning creations in wood, but I don't have the skills to create them. It's all very well to vizualize a stunning OS but without learning and practicing the foundations it is never going to happen.

[SpyderTL]

I'll try to give some feedback on the original question that was asked.

Your high level concept described above is similar to mine, in that all of the devices and components would be represented by well-known interfaces that both the hardware and software would support/adhere to.

The first, most obvious problem is that hardware is fixed, and only supports one set of interfaces that are hard wired into the device itself. So, there is no way for the hardware to support your own interface. So the best you can do is define your interface, and then create software that acts as an "adapter" between your interface and the actual hardware.

The next problem is that it is difficult to throw in the CPU with all of the other "devices". This is because the CPU is, for the most part, the only component that is programmable, and that can execute software. When you want to command the sound card to play an audio clip, you have to program the CPU to communicate with the sound card. The same goes for pretty much all of the devices connected to your machine. Even the FPU/math coprocessor is just a device that waits for instructions from the CPU.

So, if you are going to treat the CPU as a device (windows shows CPUs in the device manager, for example), you really have to treat it like two separate entities: the execution context that is used to run programs and control devices, and the physical device representation, which has properties like how many processors are installed, how many cores each processor has, and what features they support.

You also seem to want applications to have direct access to physical devices. This is how ms-dos worked, which gives you great performance, but at the cost of compatibility, and it almost completely prevents you from running multiple applications at the same time.

Each application would have to support a wide variety of devices, and would very likely not work with new devices that are not supported. You need some well-known interface for your device types so that applications can simply use the interface rather than include support for specific devices. This also allows the OS to manage the actual hardware and share it with all of the applications, so that, say, two applications can play audio at the same time.

[ignus]

You also seem to want applications to have direct access to physical devices. This is how ms-dos worked, which gives you great performance, but at the cost of compatibility, and it almost completely prevents you from running multiple applications at the same time.

In your opinion, is it so wrong to give some software full privileges? For example a particular driver that needs to answer to a request in a limited amount of time, it should be easier to archive the goal.

But it's just an impression, never seen/tested/worked on DOS or similar (unfortunatly)

Just as a carpenter has to learn how to make dovetail joints before he can craft fine furniture so a programmer has to learn the basics of, for example, file systems or memory management, before he can craft a fine OS.

Why not? I'm learning tons of things while writing a very very basic bootloader in these days. Or, at least, it won't hurt

[iansjack]

Why not what? Writing a bootloader is part of learning and practicing using the tools needed for OS development. (There's not another good reason for writing a bootloader.) But it's hardly the OS equivalent of a fine piece of furniture.