OSDev.org

Hey guys.

Its been a long time since I have logged into this forum (still has the same theme from 10 years ago). Back in 2006 username was B.E, however I can't remember the email address I used to recover my password. So I decided to create a new account.

Anyway I am working on a new project which will require some of this knowledge again. I am going to be working with this Microcontroller (Datasheet and Reference Manual for anyone interested).

I need to run a few complex algorithms, while reading data from sensors, etc.

Considering I have created a x86 bootloader, and terminal program, I do have some knowledge in OS dev, but I am a bit rusty on it.

Has anything changed in the last 10 years in terms of OS design theory, and if so, can you link me to some resources to get me up to speed, and to refresh my memory?

Well, what precisely is the problem you are trying to solve? It wouldn't make sense for us to discuss the latest developments in scheduling or file systems, say, if all you want is to process data from some sensors.

Love4Boobies wrote:Well, what precisely is the problem you are trying to solve? It wouldn't make sense for us to discuss the latest developments in scheduling or file systems, say, if all you want is to process data from some sensors.

Yes, your right.

The device will have some sensors and wifi connectivity and the algorithms will take this, do some analysis and change its environment to achieve a goal.

All sensors are interrupt driven.

Although I can't give you too much information, however, I can say, that it will require a fair bit of scheduling, interrupt handling, and needs to interact with its environment in real-time.

Hi,

ziggyfish2 wrote:The device will have some sensors and wifi connectivity and the algorithms will take this, do some analysis and change its environment to achieve a goal.

All sensors are interrupt driven.

Although I can't give you too much information, however, I can say, that it will require a fair bit of scheduling, interrupt handling, and needs to interact with its environment in real-time.

I'm not convinced that you need scheduling for a single application running on single-CPU. In fact; I'm not convinced this couldn't/shouldn't be a "bootable/bare metal application" that has no need for any OS at all.


Cheers,

Brendan

The device could be doing 4 tasks at any one time, so yeah it isn't just a single simply application running on a single CPU.

There is a need for an OS, because firstly it needs to be able to share a bunch of resources amongst these tasks, (specifically sensor data, and CPU time).

If this MCU doesn't have enough grunt, I will need to off load some of the tasks to a bigger CPU (the A20, or Raspberry Pi), which will run Linux (which will require the 3 tasks to communicate with the main processor and visa versa), which would make it easier to code. However due to power requirements I am hoping to keep it all on one processor (the A20 has low power mode consumption of 170mA, RPi is similar with 160mA). The device will also be receiving signals from 100s of external sensors.

I am also CPU to be in a low power state as much as possible (only running tasks when its needed).

So to manage the resources, and make sure a single task doesn't hog the CPU (one of the tasks has algorithms with O(N^2), I think the best solution is to create a mini OS.

The added benefit of an OS, is I can keep the CPU in a VLPW (0.33 mA) for much longer (the MCU running at HSRUN takes 116mA).

So you need/want to implement to OS yourself ?
Why don't you use freeRTOS ?

Hi,

ziggyfish2 wrote:The device could be doing 4 tasks at any one time, so yeah it isn't just a single simply application running on a single CPU.

For "single-CPU" the CPU can only do one thing at a time. To me it sounds like the best approach may be for the bootable application to have IRQ handlers that interrupt the main work and do whatever is necessary (where the main work includes "do nothing in low power state until an IRQ handler adds more work to a queue"); with no threads or IPC or scheduler or file system or...

ziggyfish2 wrote:There is a need for an OS, because firstly it needs to be able to share a bunch of resources amongst these tasks, (specifically sensor data, and CPU time).

The only case where an OS is strictly necessary (rather than just convenient) is where there are third-party applications installed by the end user (e.g. where resource usage can't be predetermined).

ziggyfish2 wrote:If this MCU doesn't have enough grunt, I will need to off load some of the tasks to a bigger CPU (the A20, or Raspberry Pi), which will run Linux (which will require the 3 tasks to communicate with the main processor and visa versa), which would make it easier to code.

If this MCU doesn't have enough grunt (to handle the unnecessary bloat), you'll find a bigger CPU and add an order of magnitude more unnecessary bloat (Linux) and find out that a bigger CPU still isn't enough, and then find an even bigger CPU that can handle order of magnitude more unnecessary bloat (on top of what you actually need), and then realise that the "cost per unit" has increased so much that it's no longer a viable/competitive product (because some other company will or have produced an embedded system without all the additional expense of all the pointless bloat that does the same thing cheaper and better).

ziggyfish2 wrote:However due to power requirements I am hoping to keep it all on one processor (the A20 has low power mode consumption of 170mA, RPi is similar with 160mA). The device will also be receiving signals from 100s of external sensors.

I am also CPU to be in a low power state as much as possible (only running tasks when its needed).

So to manage the resources, and make sure a single task doesn't hog the CPU (one of the tasks has algorithms with O(N^2), I think the best solution is to create a mini OS.

The added benefit of an OS, is I can keep the CPU in a VLPW (0.33 mA) for much longer (the MCU running at HSRUN takes 116mA).

And because I need to tighten screws I think I need a hammer, because a hammer is the only thing I'm used to and I can't figure out how something that isn't a hammer (e.g. a screw driver) might be more useful.


Cheers,

Brendan

Brendan wrote:Hi,
For "single-CPU" the CPU can only do one thing at a time. To me it sounds like the best approach may be for the bootable application to have IRQ handlers that interrupt the main work and do whatever is necessary (where the main work includes "do nothing in low power state until an IRQ handler adds more work to a queue")

Isn't that what an operating system does? Add tasks to a queue, then runs a PIT to share the CPU amongst tasks, until those tasks need to wait for an interrupt again (in most cases either a mouse event or disk even, in my case its sensor data or a timing event)?

Brendan wrote:The only case where an OS is strictly necessary (rather than just convenient) is where there are third-party applications installed by the end user (e.g. where resource usage can't be predetermined).

I see it as an OS is required when multiple applications require access to a shared resource and a level of abstraction between the application and the hardware. The applications themselves could be all from the same vendor or a team of developers.

The OS itself I don't see requiring that much in terms of memory and CPU time, etc. Granted the OS I am referring to won't be a Linux clone or by no means a fully fledged OS, but rather a barebones OS capable of scheduling and basic resource management, which is what this community is best at doing (when they aren't arguing with you anyway). I only have 256KB of ram (which if needed I can add up to 256MB) to work with.

Which is why my question was is there any new advancements in the last 10 years in terms of scheduling or resource sharing?

Thanks

Brendan

stdcall wrote:So you need/want to implement to OS yourself ?
Why don't you use freeRTOS ?

This is the osdev forum. We don't need an actual reason to implement an OS.

Hi,

ziggyfish2 wrote:

Brendan wrote:For "single-CPU" the CPU can only do one thing at a time. To me it sounds like the best approach may be for the bootable application to have IRQ handlers that interrupt the main work and do whatever is necessary (where the main work includes "do nothing in low power state until an IRQ handler adds more work to a queue")

Isn't that what an operating system does? Add tasks to a queue, then runs a PIT to share the CPU amongst tasks, until those tasks need to wait for an interrupt again (in most cases either a mouse event or disk even, in my case its sensor data or a timing event)?

You've misread. What I'm suggesting is a single task (with interrupt handlers that interrupt that single task, sort of like signals in C do); and has nothing to do with anything used for multi-tasking whatsoever.

On its own; this would be a little bit like MS-DOS, where an application can hook whatever IRQs it feels like and can do whatever it wants when the IRQ occurs (including simulating "many things happening at the same time" without multi-tasking).

For example; maybe when an IRQ occurs because some sensor data arrived, your IRQ handler gets the sensor data and builds a networking packet and sends it over the network; and you've got a "main()" that's calculating the first 12 million digits of PI that's being constantly interrupted but doesn't have a reason to care; where everything seems like it's all happening in parallel without any multi-tasking at all.

For a different example; maybe when an IRQ occurs because some sensor data arrived, your IRQ handler gets the sensor data and puts it on a FIFO queue; and you've got a "main()" that's checking that FIFO queue and processing any work that arrives; where everything seems like it's all happening in parallel without any multi-tasking at all.

ziggyfish2 wrote:The OS itself I don't see requiring that much in terms of memory and CPU time, etc. Granted the OS I am referring to won't be a Linux clone or by no means a fully fledged OS, but rather a barebones OS capable of scheduling and basic resource management, which is what this community is best at doing (when they aren't arguing with you anyway). I only have 256KB of ram (which if needed I can add up to 256MB) to work with.

You don't need multi-tasking and don't need a scheduler. You don't need file systems. You can't need virtual memory management (there is no MMU). You won't be running third-partly software, or multiple applications. You also won't be handling unknown or hot-plug devices (e.g. USB ports that user can plug anything into). What exactly do you think you need an OS for?

Let's look it from a different perceptive. Assume you've made a little "OS" (kernel) and you've got a nice small kernel ELF file. Why can't you statically link that directly into the application (and use link time optimisation to rip out all the dead weight)?

You have 512 KiB of flash and 128 KiB of RAM. If you use more than that (e.g. have to use the 1000 KiB of flash and 256 KiB of RAM version) the price of the SoC increases by about 35% (and profit and sales decrease). Maybe with efficient code you can use a cheaper SoC and improve profit/sales instead.


Cheers,

Brendan

Brendan wrote:You don't need multi-tasking and don't need a scheduler.

It depends a lot on the task and how you want to implement it.

Consider, for example, a smart battery charger that uses an algorithm to charge a battery. It reads from ADCs and writes to DACs to control current/voltage and temperature. You implement the algorithm in code and it works, your battery charges. Now you want the charger to charge multiple batteries simultaneously, each of the batteries handled individually, and you still have plenty of unused CPU time to do it. What do you do? You can redo the implementation in such a way that it is now a state machine parametrized by the battery number (and the associated state) and does not block for more than N milliseconds in any of its state, allowing you to cycle through all of your battery states continuously while still charging each battery independently. Another way is to put "yield" calls into all potentially long loops and use cooperative scheduling to cycle through the battery states. Yet another way is to leave the code mostly unchanged and run several instances of it in threads. In a simple example like this you can use any of the three approaches, unless you're extremely restricted in resources.

But what if you have something significantly more complex and heavy? I once worked on an implementation of an encrypted phone, which used a modem library, a crypto library and a voice compression library. If we disregard the encryption, we're left with two pieces of code that are driven by ADC/DAC interrupts and have to run in real time without interfering with each other in undesired ways. Each of the two work on a buffer of input bytes/samples and produce a buffer of output samples/bytes. Say, the modem handles 2 ms worth of data at a time (and takes 1 ms to process it) while the voice compressor handles 30 ms worth of data at a time (and takes 16 ms to process it). You can't run them back to back like calling the modem 15 times after every compressor call because you need not only continuity in modem operation but also a small processing delay. So, you have to somehow run both (semi-)simultaneously. You can't easily rewrite or adjust the code of the modem or the compressor to introduce cooperative scheduling. You may not even have the source code for either or both. So, you use two separate threads.

Right?

Brendan wrote:Hi,

You don't need multi-tasking and don't need a scheduler. You don't need file systems. You can't need virtual memory management (there is no MMU). You won't be running third-partly software, or multiple applications. You also won't be handling unknown or hot-plug devices (e.g. USB ports that user can plug anything into). What exactly do you think you need an OS for?

Let's look it from a different perceptive. Assume you've made a little "OS" (kernel) and you've got a nice small kernel ELF file. Why can't you statically link that directly into the application (and use link time optimisation to rip out all the dead weight)?

You have 512 KiB of flash and 128 KiB of RAM. If you use more than that (e.g. have to use the 1000 KiB of flash and 256 KiB of RAM version) the price of the SoC increases by about 35% (and profit and sales decrease). Maybe with efficient code you can use a cheaper SoC and improve profit/sales instead.


Cheers,

Brendan

There are 6 PWM (this is one of the reasons for using this MCU, the other is power consumption), 2 SPI, and 1 I2C(+1) and 1 JTAG chips/devices physically connected to the device. Plus the PIT. So yeah I may be able to get away with what you suggest.

Yay, old OSDev without me and some other children like NunoLava1998 ]

Sorry for offtopic.

catnikita255 wrote:Yay, old OSDev without me and some other children like NunoLava1998 ]

Sorry for offtopic.

Incidentally, you have incorrectly stated your age on your profile. You are 13, AREN'T YOU

MichaelFarthing wrote:

catnikita255 wrote:Yay, old OSDev without me and some other children like NunoLava1998 ]

Sorry for offtopic.

Incidentally, you have incorrectly stated your age on your profile. You are 13, AREN'T YOU

I am 13 as well