In a multi-threaded environment, several threads may access the same stream, the same module, or even the same queue at the same time. In order to protect the STREAMS resources (queues and other specific data), STREAMS provides per-thread resource synchronization. This synchronization is ensured by STREAMS and is completely transparent to the user.
Read the following to learn more about STREAMS synchronization:
STREAMS uses a synchronization-queueing mechanism that maximizes execution throughput. A synchronization queue is a linked list of structures. Each structure encapsulates a callback to a function attempting to access a resource. A thread which cannot block (a service procedure, for example) can access the resource using a transparent call.
In either case, the call returns immediately. Routines performing synchronous operations, like stream head routines, are blocked until they gain access to the resource. Although the mechanism is completely transparent, the user needs to set the adequate synchronization level.
On multiprocessor systems, the timeout and bufcall utilities present a particular problem to the synchronization mechanism. These utilities specify a callback function. Multiprocessor-safe modules or drivers require that the callback functions be interrupt-safe.
Multiprocessor-safe modules or drivers are designed to run on any processor. They are very similar to multiprocessor-safe device drivers. Interrupt-safe functions serialize their code with interrupt handlers. Functions such as the qenable utility or the wakeup kernel service are interrupt-safe.
To support callback functions that are not interrupt-safe, the STR_QSAFETY flag can be set when calling the str_install utility. When this flag is set, STREAMS ensures the data integrity of the module. Using this flag imposes an overhead to the module or driver, thus it should only be used when porting old code. When writing new code, callback functions must be interrupt-safe.
The STREAMS synchronization mechanism offers flexible selection of synchronization levels. It is possible to select the set of resources serialized by one synchronization queue.
The synchronization levels are set dynamically by calling the str_install utility when a module or a driver is loaded. The synchronization levels are implemented by linking synchronization queues together, so that one synchronization queue is used for several resources. The following synchronization levels are defined:
No synchronization level indicates that each queue can be accessed by more than one thread at the same time. The protection of internal data and of put and service routines against the timeout or bufcall utilities is done by the module or driver itself.
This synchronization level is typically used by multiprocessor-efficient modules.
Queue-level synchronization protects an individual message queue. The module must ensure that no data inconsistency may occur when two different threads access both upstream and downstream queues at the same time.
This is the lowest level of synchronization available. It is typically used by modules with no need for synchronization, either because they share no state or provide their own synchronization or locking.
In the STREAMS Queue-Level Synchronization figure, the queue Bd (downstream queue of module B) is protected by queue-level synchronization. The bolded box shows the protected area; only one thread can access this area.
Figure 10-4. STREAMS Queue-Level Synchronization. This diagram shows two streams, with two modules each, where the first module in each stream is an instance of the same module (Module B). The first stream (on the left) contains the protected Queue "Bd", which is downstream in the first instance of Module B.
Queue pair-level synchronization protects the pair of message queues (downstream and upstream) of one instance of a module. The module may share common data between both queues, but it cannot assume that two instances of the module are accessed by two different threads at the same time.
Queue pair-level synchronization is a common synchronization level for most modules that have only per-stream data, such as TTY line disciplines. All stream-head queues are synchronized at this level.
In the Queue Pair-Level Synchronization figure, the queue pair of module B's left instance is protected by queue pair-level synchronization. The boxes highlighted in bold show the protected area; only one thread can access this area.
Figure 10-5. Queue Pair-Level Synchronization. This diagram shows two streams, with two modules each, where the first module in each stream is an instance of the same module (Module B). The first stream (on the left) contains two protected queues: Queue "Bd "which is downstream in the first instance of Module B, and Queue "Bu" which is upstream in the first instance of Module B.
Module-level synchronization protects all instances of one module or driver. The module (or driver) can have global data, shared among all instances of the module. This data and all message queues are protected against concurrent access.
Module-level synchronization is the default synchronization level. Modules protected at this level are not required to be thread-safe, because multiple threads cannot access the module. Module-level synchronization is also used by modules that maintain shared state information.
In the Module-Level Synchronization figure, module B (both instances) is protected by module-level synchronization. The boxes highlighted in bold show the protected area; only one thread can access this area.
Figure 10-6. Module-Level Synchronization. This diagram shows two streams, with two modules each, where the first module in each stream is an instance of the same module (Module B). Each instance of Module B in each of the two streams is protected by module-level synchronization.
Arbitrary-level synchronization protects an arbitrary group of modules or drivers (including all instances of each module or driver). A name passed when setting this level (with the str_install utility) is used to associate modules together. The name is decided by convention among cooperating modules.
Arbitrary-level synchronization is used for synchronizing a group of modules that access each other's data. An example might be a networking stack such as a Transmission Control Protocol (TCP) module and an Internet Protocol (IP) module, both of which share data. Such modules might agree to pass the string "tcp/ip".
In the Arbitrary-Level Synchronization figure, modules A and B are protected by arbitrary-level synchronization. Module A and both instances of module B are in the same group. The boxes highlighted in bold show the protected area; only one thread can access this area.
Figure 10-7. Arbitrary-Level Synchronization. This diagram shows two streams, with two modules each, where the first module in each stream is an instance of the same module (Module B). Each instance of Module B in each of the two streams is protected. Module A, the other module in the first stream, is also protected by the arbitrary-level synchronization.
Global-level synchronization protects the entire STREAMS.
Note: This level can be useful for debugging purposes, but should not be used otherwise.
Synchronization levels take all their signification in multiprocessor systems. In a uniprocessor system, the benefit of synchronization is reduced; and sometimes it is better to provide serialization rather than concurrent execution. The per-stream synchronization provides this serialization on a whole stream and can be applied only if the whole stream accepts to run on this mode. Two conditions are required for a module or driver to run at per-stream-synchronization level:
If a module that does not support the per-stream synchronization is pushed in the stream, then all other modules and drivers will be reset to their original synchronization level (queue level or the queue-pair level).
In the same way, if a module that was not supporting the per-stream synchronization is popped out of the stream, a new check of the stream is done to see if it now deals with a per-stream synchronization.
The STREAMS synchronization-queueing mechanism allows only one queue to be accessed at any one time. In some cases, however, it is necessary for a thread to establish queue connections between modules that are not in the same stream.
These queue connections (welding mechanism) are especially useful for STREAMS multiplexing and for echo-like STREAMS drivers.
STREAMS uses a special synchronization queue for welding queues. As for individual queue synchronization, the welding and unwelding requests are queued. The actual operation is done safely by STREAMS, without any risk of deadlocks.
The weldq and unweldq utilities, respectively, establish and remove connections between one or two pairs of module or driver queues. Because the actual operation is done asynchronously, the utilities specify a callback function and its argument. The callback function is typically the qenable utility or the e_wakeup kernel service.
During the welding or unwelding operation, both pairs of queues are acquired, as shown in the STREAMS Queue-Welding Synchronization figure. However, it may be necessary to prevent another queue, queue pair, module, or group of modules from being accessed during the operation. Therefore, an additional queue can be specified when calling the weldq or unweldq utility; this queue will also be acquired during the operation. Depending on the synchronization level of the module to which this queue belongs, the queue, the queue pair, the module instance, all module instances, or an arbitrary group of modules will be acquired.
Figure 10-8. STREAMS Queue-Welding Synchronization. This diagram shows two streams side-by side, each acquiring a queue from the other. The first stream (on the left) contains two modules with two queues each, as follows (from the top): Module B with Queue "Bd" and Queue "Bu", as well as Module A with Queues "Ad" and "Au". The second stream contains two modules with two queues each as follows (from the top): Module D with Queue "Dd" and Queue "Du", as well as Module C with Queues "Cd" and "Cu". A dotted arrow leads from Queue "Ad" to Queue "Cu". Another dotted arrow leads from Queue "Cd" to Queue "Au". The four queues involved are highlighted; they are Queues "Ad", "Cu", "Cd", and "Au".
For example, in the Queue Welding Using an Extra Queue figure, the welding is done using the queue Bd as an extra synchronization queue. Module B is synchronized at module level. Therefore, the queues Ad, Au, Cd, and Cu and all instances of module B will all be acquired for performing the weld operation.
Figure 10-9. Queue Welding Using an Extra Queue. This diagram shows two streams side-by side. The first stream (on the left) contains two modules with two queues each, as follows (from the top): Module B with Queue "Bd" and Queue "Bu", as well as Module A with Queue "Ad "and "Au". The second stream contains two modules with two queues each as follows (from the top): Module D with Queue "Dd" and Queue "Du", as well as Module C with Queue "Cd" and "Cu". A dotted arrow leads from Queue "Ad "to Queue "Cu". Another dotted arrow leads from Queue "Cd" to Queue "Au". The module and four queues involved are highlighted; they are Module B as well as Queues "Ad", "Cu", "Cd", and "Au".
