OSDev.org

I have created multiple POSIX threads (on readhat Linux) in a C program in my app. What I am doing is - I am creating threads equal to the number of CPUs in the system and and equal number of instances of a certain data structure, basically a queue implementation. I am assigning one ID to the thread and same ID is given to the queue. Threads process data from the queue. Thread #1 always process data from queue #1 (and from this queue only) and so on. However, data that is processed from queue #1 has some state dependency wrt the previously processed data from the queue. Think of a network traffic analyser - to establish a connection, we need to take previous state into consideration.

My question is -
(a) in such a scenario, will my app be (thread) safe ?
(b) Yes, data from a queue is picked by a corresponding thread but what if the threads switch CPUs ?
(c) Do posix threads switch CPUs ?

There are (in some sense) two concepts here. If a piece of data can be modified from two threads at the same time, then accessing that data is not threadsafe. That data needs at least to have an atomic lock on it (for a single-CPU architecture). If there is a period of time when the data is invalid (while it is being modified), you will still need to lock it atomically if it might be accessed by another thread.

If a piece of data can be modified from two CPUs at the same time then that data needs some form of multi-CPU locking on it. (Invalid periods still need to be handled atomically, too.)

Simply reading data, from as many threads as you want, on as many CPUs as you want, is never a problem.

So, the nice thing about your setup is: when you are accessing your queues, they are readonly, and only accessed by one thread, on one CPU at a time, for each queue. Also, each item within the queue is only accessed by one thread.

The case you asked about -- where a thread is migrated from one CPU to another -- is not a problem. The OS must verify that the thread is completely terminated on CPU#1 before it is migrated and restarted on CPU#2 ... but that is the scheduler's problem to solve, not yours.

So, as I said, "popping" data off your queue is not a problem. But adding items to your queue might very well need locking. You will need to look at that separately, and very carefully.

Simply reading data, from as many threads as you want, on as many CPUs as you want, is never a problem.

Unless its being written to and its not aligned, because then the operation isn't guaranteed to be atomic. Which shouldn't be a problem unless you pack all your structures or your compiler is really really bad.

If you use only atomic writes but don't have synchronisation, there still may be issues if what you write depends on what you have read before the write, or if you have an operation which performs two writes and doesn't ensure the structure's consistency between them.