TAO Real-Time Architecture
This page describes and compares the two main ORB designs we considered for
supporting Real-Time CORBA 1.0 in TAO. The first design, codenamed reactor-per-lane
and shown in Figure 1, was chosen for our initial implementation. The
second design, queue-per-lane, might also get implemented in the future,
as a part of a project evaluating alternative ORB architectures for Real-Time
CORBA.
Design I: Reactor-per-Lane
In this design, each Threadpool Lane has its own reactor, acceptor, connector
and connection cache. Both I/O and application-level processing for a
request happen in the same threadpool thread: there are no context switches, and
the ORB does not create any internal threads. Objects registered with any
Real-Time POA that does not have the ThreadpoolPolicy are serviced by default
threadpool, which is a set of of all the application threads that invoked
orb->run ()
.
Figure 1: Reactor-per-lane
When a Real-Time POA is creating an IOR, it includes one or more of its
threadpool's acceptor endpoints into that IOR according to the following
rules. If the POA's priority model is server declared, we use the
acceptor from the lane whose priority is equal to the priority of the target
object. If the
priority model is client propagated, all endpoints from the POA's
threadpool are included into the IOR. Finally, if the PriorityBandedConnectionPolicy
is set, then endpoints from the threadpool lanes with priorities falling into one of the specified
priority bands are selected. The endpoints, selected according to the
rules above, and their corresponding priorities are stored in IORs using a special
TaggedComponent TAO_TAG_ENDPOINTS
. During each
invocation, to obtain a connection to the server, the client-side ORB selects
one of these endpoints based on the effective policies. For example, if the object has
client propagated priority
model, the ORB selects the endpoint whose priority is equal to the priority of the client thread making the
invocation. If on the client-side, during invocation, or on the
server-side, during IOR creation, an endpoint of the right priority is not available, it
means the system has not been configured properly, and the ORB throws an exception.
The design and rules described above ensure that when a threadpool with lanes is
used, client requests are processed by the thread of desired priority from very
beginning, and priority inversions are minimized.
Some applications need less rigid, more dynamic environment. They do
not have the advanced knowledge or cannot afford the cost of configuring all of the
resources ahead of time, and have a greater tolerance for priority
inversions. For such applications, threadpools without lanes are a way to go.
In TAO, threadpools without lanes have different semantics that their with-lanes
counterparts. Pools without lanes have a single acceptor endpoint used in
all IORs, and their threads change priorities on the fly, as necessary, to
service all types of requests.
Design II: Queue-per-Lane
In this design, each threadpool lane has its own request queue. There is a separate
I/O layer, which is shared by all lanes in all threadpools. The I/O layer has one
set of resources - reactor, connector, connection cache, and I/O thread(s) - for each priority
level used in the threadpools. I/O layer threads perform I/O processing
and demultiplexing, and threadpool threads are used for application-level request
processing.
Figure 2: Queue-per-lane
Global acceptor is listening for connections from clients. Once a
connection is established, it is moved to the appropriate reactor during the
first request, once its priority is known. Threads in each lane are blocked on
condition variable, waiting for requests to show up in their queue. I/O
threads read incoming requests, determine their target POA and its threadpool,
and deposit the requests into the right queue for processing.
Design Comparison
Reactor-per-lane advantages:
- Better performance
Unlike in queue-per-lane, since each request is serviced in one
thread, there are no context switches, and there are opportunities for stack and TSS optimizations.
- No priority inversions during connection establishment
In reactor-per-lane, threads accepting connections are the same
threads that will be servicing the requests coming in on those connections, i.e.,
priority of the accepting thread is equal to the priority of requests on the
connection. In queue-per-lane,
however, because of a global acceptor, there is no differentiation between
high priority and low priority clients until the first request.
- Control over all threads with standard threadpool API
In reactor-per-lane, the ORB does not create any threads of its
own, so application programmer has full control over the number and
properties of all the threads with the Real-Time CORBA Threadpool
APIs. Queue-per-lane, on the other hand, has I/O layer threads,
so either a proprietary API has to be added or application programmer will
not have full control over all the thread resources..
Queue-per-lane advantages:
- Better feature support and adaptability
Queue-per-lane supports ORB-level request buffering, while reactor-per-lane
can only provide buffering in the transport. With its two-layer
structure, queue-per-lane is a more decoupled design than reactor-per-lane,
making it easier to add new features or introduce changes
- Better scalability
Reactor, connector and connection cache are per-priority resources in
queue-per-lane, and per-lane in reactor-per-lane.
If a server is configured with many threadpools that have similar lane
priorities, queue-per-lane might require significantly fewer of the
above-mentioned resources. It also uses fewer acceptors, and its IORs
are a bit smaller.
- Easier piece-by-piece integration into the ORB
Ease of implementation and integration are important practical
considerations in any project. Because of its two-layer structure, queue-per-lane
is an easier design to implement, integrate and test piece-by-piece.