What are primitives for the fastest user-space IPC?

[kotovalexarian]

Hello. Let's say I want to implement the fastest user-space IPC. The obvious solution is to use shared memory. However, we need more primitives because sometimes one process can wait for data from other one. What we want is to:

• Notify OS scheduler that current process have nothing to do.
• Notify OS scheduler that current process waits for data from specific other process.

The first requirement can be satisfied with sleep, pthread mutex or POSIX semaphore. The second requirement can't be satisfied with those primitives, I believe. For example, let's inspect how the POSIX semaphore works. It is implemented mostly in user space (in Linux using futex). When waiting for the locked semaphore, syscall is made to notify OS scheduler that current process have nothing to do. However, how can OS scheduler know which process it wait's data from? I suppose that this can magically happen because OS knows which processes share the region of memory where semaphore is located in, but documentation doesn't say anything about this. How can I get such guarantee? Native OS IPC primitives such as pipes provide such guarantee, and I want to have an advantage over them.

[bzt]

The Minix kernel solves the second requirement by simply removing the waiting process from the active tasks list, and puts it on the sender process' needs-to-be-awaken list. So when the sender process is finished with the task and wants to send a response message, the waiting process is automatically put back to the active tasks list, but not sooner. No additional API required.

Cheers,
bzt

[Korona]

A priority inheriting pthread mutex can fulfill both requirements. Note that the second requirement is only really important in real-time contexts.

At the kernel level, all current mainstream OSes use futexes to implement this functionality (but Windows has some other options, too).

Note that in general, there is no "fastest" IPC. It depends a lot on the use case. Also note that using shared memory becomes a lot harder if processes do not trust each other and that it might be useful to have another IPC mechanism to establish the shared memory in the first place.

[Schol-R-LEA]

Is there a specific goal for this, or are you just trying to optimize your IPC time as tightly as possible? As Korona said, IPC timings can depend a lot on the specific use case. 'As fast as possible' isn't going to have a one-size-fits-all answer.

Also, IPC speed is rarely a limiting factor for most purposes; other aspects unrelated to the speed of the IPC mechanism are likely to be bigger concerns in a given instance. I would serious ask myself whether this is premature optimization.

[kotovalexarian]

The Minix kernel solves the second requirement by simply removing the waiting process from the active tasks list, and puts it on the sender process' needs-to-be-awaken list. No additional API required.

I develop my own user-space IPC.

A priority inheriting pthread mutex can fulfill both requirements.

Thank you, I'll take a look.

Note that the second requirement is only really important in real-time contexts.

I think that it can be important, for example, for high-loaded web servers. However, I don't have benchmarks for now.

Note that in general, there is no "fastest" IPC. It depends a lot on the use case. Also note that using shared memory becomes a lot harder if processes do not trust each other and that it might be useful to have another IPC mechanism to establish the shared memory in the first place.

Is there a specific goal for this, or are you just trying to optimize your IPC time as tightly as possible? As Korona said, IPC timings can depend a lot on the specific use case. 'As fast as possible' isn't going to have a one-size-fits-all answer.
Also, IPC speed is rarely a limiting factor for most purposes; other aspects unrelated to the speed of the IPC mechanism are likely to be bigger concerns in a given instance. I would serious ask myself whether this is premature optimization.

I also want to minimize syscalls for better portability.