Real Time OS, long term project, would love some feedback

[Thaorius]

Hi, my first post on this board .

Some background (You may skip ahead)
---
I have an electronics and some embed programming background, I just find it incredibly interesting to build stuff from scratch, be it a simple power adapter, or a 12 articulation, wi-fi capable, mechanic arm. So, over the years I've amassed a bast amount of ideas to develop, all heavy in hardware (lots of boards to design and produce), but none of them went live, mostly due to lack of time and money to buy the myriad of parts and tools; this, however, is about to change since my business plans are starting to show some profit.

Among this projects, there are: A house controlling system (domotics)(high performance, fully automated, etc, etc), an on-board car computer (providing all sorts of interesting stuff), a miniature airplane and submarine, a rather complicated mechanical arm(which, if proven to be good enough, will be the initial stage to build something bigger, who knows).

So, the need for an operative system appeared, for most things it's not gonna be enough to have an AVR chip with an small, limited and boring firmware. So I looked into a Linux based solution, a stripped down kernel, only the essential tools, etc. This model, though workable, presented a few problems, one of them being that Linux is not really a real time OS. Plus, on my own OS I can add all kinds of special features.

I then started thinking, why not, instead of using Linux, I could build my very own operative system, in such a way, every last instruction executed by the on board computers will have been programmed by me, I learn a lot of interesting stuff while doing it, and, of course, I can say "I wrote an operative system, and it works." as an added bonus.

And so the idea began developing in my head, and just yesterday I got a lot of parts, which, though I haven't thoroughly inspected, would suffice to build 3 small computers (all x86), which should be good enough as a starting point, right?

The basic idea
---
The system would share many features of UNIX systems that I find very nice, like, it would deal with everything as if it were a file, among other things (though it will most likely not be anywhere near POSIX).
I want the kernel to be capable of running on multiple architectures, though the initial testing will be purely on x86 (and most the embed systems I'm going to use are x86 anyways. Mostly Via Micro and Nano ITX boards).
I have thought of many features, and I have many more to think about, but some are:
-Real time task scheduler(specially useful for on board flight computer for instance, or emergency response on domotics(say, someone is electrocuting, so I really need to be able to cut off the power on that house sector)).
-Small footprint, monolithic kernel with on-demand loaded modules.
-Customized disk layout and boot loader (I don't quite like the model used currently, you know, MBR, partition table, I find it rather archaic and lacking in some features I would love to have).
-In the future, though this is not the final objective, I'd like to rewrite the whole system, kernel and applications, using a programming language of my own (I'm fascinated by compilers). I do not have any specifics on this, nor am I even sure I'll ever do such a thing, but I like the concept.
-Designed to run from the smallest systems possible(say, an AVR chip with a 20mhz clock and a few megabytes of ram) to the VIA mainboards with good performance, or even larger, x86_64, multi core, multi cpu systems (though not the aim, for non-embed things I would much rather use Linux).
-I'd like the system to load ELF files (I like the format).
-Some features that, though unique to my current understanding, are far too immature to post just yet.

I'd love to hear some feedback from you guys.

Oh, btw, the project codename is "Iodine", and it will most likely stay that way forever, since this is mainly for my personal use.

[jal]

Hi, my first post on this board :).

Welcome!

Real time task scheduler(specially useful for on board flight computer for instance, or emergency response on domotics(say, someone is electrocuting, so I really need to be able to cut off the power on that house sector)).

I take it you have read up on real-time scheduling, and all the complications that go with it. That said, real-time OSes are quite abundant, and I don't think this is a big obstacle for success.

Small footprint, monolithic kernel with on-demand loaded modules.

For embedded systems this makes sense.

Customized disk layout and boot loader (I don't quite like the model used currently, you know, MBR, partition table, I find it rather archaic and lacking in some features I would love to have).

Note however that the PC BIOS still runs the show on start-up, so there are thing you just cannot get passed, unless you use a different type of firmware (EFI, for instance).

In the future, though this is not the final objective, I'd like to rewrite the whole system, kernel and applications, using a programming language of my own (I'm fascinated by compilers). I do not have any specifics on this, nor am I even sure I'll ever do such a thing, but I like the concept.

Sure, why not. I don't have to explain that this adds lots of extra work. I'd advise you to only do it if it adds a real advantage.

Designed to run from the smallest systems possible(say, an AVR chip with a 20mhz clock and a few megabytes of ram) to the VIA mainboards with good performance

Microcontrollers like the AVR line usually are 8 bit, and have no protected memory. OSes for these systems typically have very different approaches to stuff than those running on higher-end systems. It'd be interesting to see if you can merge them.

I'd like the system to load ELF files (I like the format).

Very practical, but forget about doing that on an AVR :).

Anyway, good luck. It seems you got yourself busy for the next 20 years :).


JAL

[AJ]

Hi,

Just as an extra resource, if you're specifically looking at avr, take a look at AVR Freaks, who have forums and tutorials for 8 and 32 bit AVR's. They've also got stuff on AVR software multitasking and I'm sure RTOS's are also available (try the projects links).

Cheers,
Adam

[redoktober]

pretty nice plan you got there, mate.

In the future, though this is not the final objective, I'd like to rewrite the whole system, kernel and applications, using a programming language of my own (I'm fascinated by compilers). I do not have any specifics on this, nor am I even sure I'll ever do such a thing, but I like the concept.

Sure, why not. I don't have to explain that this adds lots of extra work. I'd advise you to only do it if it adds a real advantage.

i second that. i'm starting my own programming language thing as well, and it's definitely *not* a piece of chocolate cake.

PS - you can substitute 'chocolate cake' for whatever you like

[Thaorius]

@jal: I have been rethinking the monolithic kernel idea, take this scenario for example:
The model plain is in the air, both engines are at full throttle, everything looks just fine, and then, out of nowhere, some random driver crashes, taking the whole system with it, this includes the flight commander and all the hardware control(engine, propane valves... kerosene valves... engine cooling... not good), but, if, instead, when the the driver crashes, I can have another "server" looking for "down" systems and pull it back up, or if not essential, just let it die. But at least, all the hardware should still be operational, and most the software, so there is a very high chance the problem can be solved easily, without much risk of loosing the whole device. The performance problems could be overcomed by simply picking an slightly more powerful system configuration and battery.

Concerning the programming language, the reasons are simple, I see many flaws on the ones I use (namely C/C++/PHP), and those that have features I'd love to have (Python/Ruby/etc) are interpreted, there for, making it useless for most my applications(unless I'm doing WEB stuff, where I use a PHP+Zend Framework+Doctrine stack), since I tend to do stuff requiring performance. So, I came up with a model of a high level language, that can compile to native code, without the need of a runtime library, and that can do both, very high level stuff with a great degree of abstraction (which would be translated to overhead-free ASM) and very low level stuff, namely, setting a register. The design I came up with is pretty workable, and to be honest, what better way to make sure your language is good than do a world-class operative system with it? Oh, and all the applications would be programmed only in this platform agnostic language, which would make them portable without changes. I even thought of a way of making it both, interpreted and native, in such a way, that while developing, you run it in interpreted mode(not applicable for an OS, but for user-land applications) so you'll be able to debug in high level, at a decent speed, and not wait for compile+link timings, not to mention, you can use it as a scripting language, since scripts don't really need that much performance.
In fact, I would much rather do the language before the OS, and write the OS directly with it, but one fundamental problem appears, there will be many language bugs at first, many compiler bugs, etc, and there for, how can I know if the OS code is wrong, or the lang code is? or both?.

About the BIOS calling the shots, easy fix. on x86, I just write up an MBR consisting of a bootloader, that would load my "real" bootloader, and a partition table, consisting of a single, primary partition, that takes the whole disk but sector 1. That way, the BIOS loads my "pre" bootloader, this in turn loads the actual bl image, which loads my kernel, which sees my disk layout and file system properly.

AVRs, I believe they are workable, though I'm not sure that a microkernel would be able to run on an AVR, I don't exactly have much performance there.

@AJ: Yeah, I've seen that page before, I have it bookmarked for when I need it.

@redoktober: I'll drop you a line later, would be nice to hear more about your language.

[jal]

@jal: I have been rethinking the monolithic kernel idea, take this scenario for example:
The model plain is in the air, both engines are at full throttle, everything looks just fine, and then, out of nowhere, some random driver crashes, taking the whole system with it, this includes the flight commander and all the hardware control(engine, propane valves... kerosene valves... engine cooling... not good), but, if, instead, when the the driver crashes, I can have another "server" looking for "down" systems and pull it back up, or if not essential, just let it die. But at least, all the hardware should still be operational, and most the software, so there is a very high chance the problem can be solved easily, without much risk of loosing the whole device. The performance problems could be overcomed by simply picking an slightly more powerful system configuration and battery.

Still, in a modal plain, you'd need very specialized, probably custom-build hardware, and you have not that much choice in CPUs. You'd probably go for ARM or Atmel, as x86 would draw way too much power. But in case of a system for any plain, model or not, I'd build it probably with 2 CPUs, one monitoring the other, and being able to intervene and/or take some task over. But that's a whole different subject anyway. I'd go for microkernel myself, but it really depends on the CPU you're aiming at. If it doesn't have memory protection, for example, you're screwed anyway.

AVRs, I believe they are workable, though I'm not sure that a microkernel would be able to run on an AVR, I don't exactly have much performance there.

Indeed. The messaging overhead would probably be large. On the other hand, there's no memory protection on those anyway, so a crash is potentially disastrous for the entire system.


JAL

[Thaorius]

I was actually thinking on running any embed system with a set of 2 nano-itx boards, both running the OS, which would be distributable. Connected through an ethernet link. Since both the systems update each other, in the event of a system failing to respond a "ping", the other system takes over the hardware and spawns all the needed services, and feeds the state from the other system to this programs (in such a way, one of the systems would be nearly dormant), and the working system would issue a power cycle on the malfunctioning system, and wait for it to go back online (now the initially crashed system would be the dormant one).

Speaking of disk layout, I've been looking at OpenBIOS and coreboot, and it seems I could implement my idea without legacy MBR support, I would need to write a payload for coreboot, but I can only imagine the feeling of accomplishment of seeing a system boot, show up your own BIOS (not entirely, coreboot does most the work, but still) booting your own OS, which runs your own applications in your own programming language, right? ^-^ (and so it seems you were right jal, I've gotten myself busy for the next 20 or so years). The only problem with coreboot, is that is doesn't really support the nano-itx boards, so I'm restricted to the mini-itx ones, which are much bigger and power consuming, and that probably kills the idea of "twin" systems to play big brother on each other (or maybe not, take the model plain example, it would really depend on the size of the plain how much computer horse power I can stuff in it, and how much room for battery I have, remember, I only need 1 hour of power, tops, since there is no point on having a running on-board flight computer if the engines run out of fuel, is there? and with my current math I would need 14l of kerosene and a small tank of propane to keep 2 small jet engines running for an hour at near full throttle).

Oh, and yes, I would most likely need relatively powerful system, to store a video feed in real time, keep an accurate log of everything that happens on every sensor and piece of hardware (engines, flaps, altimeter, wing tension, propane pressure, kerosene levels, etc), maintain an RC control link, all while actually driving the aircraft. Maybe not so much CPU but storage space would be required, but it's really the same as small AVR and most ARM systems won't provide sufficient bandwidth to the storage medium of my choice, will they?

[redoktober]

nano itx boards, eh?
sounds pretty cool!
good luck!

[tantrikwizard]

<snip>...Oh, and yes, I would most likely need relatively powerful system, to store a video feed in real time, keep an accurate log of everything that happens on every sensor and piece of hardware (engines, flaps, altimeter, wing tension, propane pressure, kerosene levels, etc)...<snip>

In most cases such as these, a microcontroller is used for obvious reasons. There's no need for an OS on most embedded systems like this and the dual processors is overkill. yC's cost .25 cents (.50 cents if youre a big spender), can be programmed in C and handle most of the mechanical to electrical conversion that youre talking about with only a handful of components. Throwing dual processors at a toy plane is overkill like installing gold plated toilet seat covers in a mobile home.

[Thaorius]

<snip>...Oh, and yes, I would most likely need relatively powerful system, to store a video feed in real time, keep an accurate log of everything that happens on every sensor and piece of hardware (engines, flaps, altimeter, wing tension, propane pressure, kerosene levels, etc)...<snip>

In most cases such as these, a microcontroller is used for obvious reasons. There's no need for an OS on most embedded systems like this and the dual processors is overkill. yC's cost .25 cents (.50 cents if youre a big spender), can be programmed in C and handle most of the mechanical to electrical conversion that youre talking about with only a handful of components. Throwing dual processors at a toy plane is overkill like installing gold plated toilet seat covers in a mobile home.

It isn't when you buy 2 small jet engines, totaling 12000 USD to begin with . Plus, the plane is just an example, the first project will most likely be a very sophisticated domotics system (for which I have written a bunch of specs), which will include, among other things, some "house remotes" (bluetooth based remote controller powered by an ARM board and a touch screen), a lot of wireless "routers" and "access points" to boost connectivity and widespread a ZigBee signal, as well as a bunch of receiving circuits doing the actual work (power breakers, timers, alarms, phones, clocks, movement sensors, etc) with a battery-powered "brain", which would consist of a set of twin systems for fault tolerance (one of which would shutdown when battery power degrades enough).
Oh, and sadly, microcontrollers are rather expensive here (compared to the number you gave me) . An ATMega, of the smallest ones will cost like 10USD, and a Microchip PIC 16F84A retails at something like 5 USD or similar.

And honestly, to do the model plane task on a small microcontroller, I would need it to store a video feed on a hard drive, within a file system, along with log files, and provide a diagnose console, and of course, drive the entire thing while doing it, not to mention, provide a wireless control link and fast response. This looks like ARM/x86 land to me .

[redoktober]

i'd second thaorius.
a simple microcontroller can't do the stuff he wants to do.
i mean...something like ARM would be necessary..

[~]

The Microchip's 16F84X PICs are obsolete and I think that maybe could stop being produced at some time.

For a system like that I think that a solid state memory like flash or CMOS with a backup battery like the BIOS could be better if you were to make a device like a plane. I don't believe that a spinning hard disk wouldn't suffer as a small plane changes direction quickly. Remember that flash memory however gets some bits damaged after several thousand times of being written or accessed.

You could also use one or more of more powerful PICs, if I remember correctly there are some that work like at 100MHz or maybe more. You could then add RAM and some storage media along with other required peripherals. You could even keep part of the data in the RAM of the "plane" (you would need to interface standard DIMM's, SIMM's, etc.) and the operating system in the PIC, and maybe you could send in both directions the most important data remotely to have a virtually unlimited capacity in the central station, in that way you could need less permanent mobile storage, and you could even make some mechanism to send "commands" to the device in the same way as data to control it more intelligently.

I think that remote high-speed transfer remote systems are already possible. You could simplify and remove unneeded tasks and unstability due to system overcomplication on the mobile part of the system.

You could for example require it to request the central station to process the actual files in a filesystem, and the mobile device would only see the actual data to be stored in its RAM or whatever, and would only be worried of commands like "store this data or log" or "give me this data in this file or this log".

In that way the heaviest, routinary and "worthless" tasks would be done by a true 2GHz+ procesor and the decisive mobile decisions would not have overload or loss of time and since it practically would be running only one process the chances of system failures would reduce greatly.

That would also make it even more portable. You could make one only stable version of the mobile system, and the central system could be composed of Windows, Linux, MacOS, etc., or your own operating system in any existing platform (Mac, PC, Xbox).

[Thaorius]

Though what you say makes perfect sense, I see 3 fundamental problems.

1) Sending data, at high speed, via a radio frequency, with a video feed and other data, is incredibly difficult and very dependent on the environment. Whereas, sending a few bytes won't be so much of a problem. Plus, adding a heavy encryption layer to the data pipe with an AVR/PIC? I see it rather difficult.
2) The plane is no longer portable, now I need to carry around either a full blown desktop computer to process the data feed, or a laptop, none of which are really nice options. And building a decent sized remote control, with the radio transceiver (which needs a monumentally big heat sink to keep the power transistors from burning), plus a gigantic battery to power a system, and of course, the system itself, with the hard-drive/flash and the control buttons and the antenna. Whereas, otherwise, I would only be mounting the control buttons and the radio transceiver, and a small battery, since there is no system to power, I can get away with 1-2 cellphone batteries chained together).
3) Any kind of portability is lost, as the plane is one of many, the domotics system for instance needs to be 100% independent, and this includes running an SCADA like interface when needs appears, not to mention, accept input from a keyboard and a mouse, and of course, I would need much more power than an small microcontroller can provide, so I would be forced to keep adding MCUs every time I add a new functionality to the system, and that would actually consume more power (I can run an ARM board with a cellphone battery (3.7v, 1000mAh) for several hours, wereas AVR/PIC will require more power when combined). And then of course, upgrading any part of the system would require a considerable rewrite of low level, hard to debug code.

Oh, and I'm going to use 16/32gb SD cards as storage, since the port seems to be fairly common on ARM boards.

PD: This all goes without accounting for the fact that re-building the PCB to fix the design errors that will come up (since life in the simulator and in the PCB are quite different) will take up to 2 weeks since there is no company around the corner that will build one, I have to have it printed somewhere else; nor the fact that that the final board with several MCUs would end up being bigger, more power consuming, and without all the perks of an operative system (like giving more resources to a particular process) that an ARM board can provide.

However, if you can think of a viable solution to implement your idea without so many problems, I'd love to hear it.

[~]

There would be no problem using the highest capability processor that consumes the less energy, whatever it is.

The feature of transferring the data to a central computer could be optional, so a mobile system could run standalone or monitored/unmonitored by request of the central computer. I think that for that the data could be sent through a ~1GHz digital radio signal, like cell phones do, and still could have a rather normal battery life. But then protocols and implementations of a high speed transceiver should be investigated, and still that could be an optional feature of some devices, and the ones that have it could use it only when requested.

So reading/writing/transferring locally or remotely would also become optional for a handful of "wireless multimedia capable" devices.

A future characteristic of some of the mobile devices could be the ability to somehow run code directly from RAM (or SD), and having an API. In that way pre-debugged code pieces or commands could be sent remotely (or while the device is turned off and loading "program files" into the SD), and such device would need to be designed with the idea of not requiring hardware modifications to run other code.

The normal remote control interfaces will still apply as normal and could be the default ones.