Configuring TAO's Components

Overview

As described in the options documentation, various components in TAO can be customized by specifying options for those components. This document illustrates how to combine these options in order to affect ORB behavior and performance, particularly its concurrency model.

TAO configures itself using the ACE Service Configurator framework. Thus, options are specified in the familiar svc.conf file (if you want to use a different file name, use the -ORBsvcconf option).

Roadmap

Details for the following configurations are provided.

• Configurating key components:
  • Server Concurrency Strategy.
  • ORB and other resources.
  • POA.
  • Collocation Table.
  • Forwarding IIOP Profile
• Configuration examples
  • Single-threaded, reactive model.
  • Multiple threads, thread-per-connection model.
  • Multiple threads, ORB-per-Reactor-thread model.
  • Multiple threads, ORB-per-thread, thread-per-connection model.
  • Multiple threads, thread-pool model. (Not yet implemented.)
  • Multiple threads, ORB-per-thread, thread-pool model. (Not yet implemented.)
  • Each configuration has the following information:

    Typical Use	A brief description of the scenario and its typical use.
    Number of Threads	The number of threads used by ORB-related activities.
    Thread Creator	Identifies the creator of the threads discussed above.
    Resource Location	Where information on various resources is stored.
    Thread task	Describes what task is undertaken for each thread.
    Options	Specifies the options for each service in order to utilize this configuration.

• Configuration for homogenous systems
  • Compile time options
  • Runtime time

Configuring key components

• Server concurrency strategy specifies the concurrency strategy an ORB uses. It says nothing about how many ORBs (or, threads) are there in a process.
  • reactive: The ORB handles requests reactively, i.e., the ORB runs in one thread and service multiple requests/connections simultaneously using "select" call. You can have multiple ORBs accepting requests reactively and running in separate threads.

  • thread-per-connection: The ORB handles new connections by spawning a new thread whose job is to service requests coming from the connection. The new threads inherits all properties from the ORB threads (see below.)

  • thread-pool (not yet implemented): ... to be continued ...

• ORB and other resources.
  • global: There's only one ORB process-wide. ORB_init () must be called only once. Every thread accesses the same ORB.

  • tss: When using tss ORB, the programmer is responsible for spawning the ORB threads and setting up the ORB by calling ORB_init () for each ORB threads. Any ORB spawned thread (i.e., thru thread-per-connection) shares the same resource the spawning ORB uses.

• POA.
  • global: All ORBs share the same POA. The advantage of using a global POA is that once an object is registered to the POA under an ORB, it can be externalized from other ORB.

  • per ORB (tss): Each ORB has its own POA, which means, the programmer should also instantiate the POA for each ORB (otherwise, a default RootPOA gets created, which might not be what you what and thus, is discouraged.)

• Collocation Table: ^*Care must be taken when using CORBA objects to control the ORB directly. For you are actually executing the collocated object, not in the object's ORB context, but in the calling ORB's context.
  • global: Process keeps a global collocation table which contains tuples of listening endpoint and its corresponding RootPOA.

  • per ORB (tss): At this moment, since TAO only supports one listening endpoint per ORB, there is no per-ORB collocation Table. Checking of collocated objects is done by comparing object's IIOP profile and the calling ORB's listening endpoint.

• Forwarding IIOP Profile: In the case of multiple threads using the same CORBA::Object and using forwarding, it is necessary to protect the forwarding IIOP_Profile, which is part of the IIOP_Object, which is part of the CORBA::Object against multiple access. Therefore a mutex lock is used by default to ensure proper access. Using the switch -ORBiiopprofilelock this policy can be deactivated specifying -ORBiiopprofilelock null. A motivation to do this might be performance reasons in cases, where no forwarding is used or no multithreading with access to shared CORBA::Object's. Deactivating forces the ORB to use a null mutex, which does introduce only a very small overhead, compared with overhead introduced by a regular mutex lock.

Configuration Example

• Single-threaded, reactive model.

  Typical Use	This is the default configuration of TAO, where one thread handles requests from multiple clients via a single Reactor. It is appropriate when the requests (1) take a fixed, relatively uniform amount of time and (2) are largely compute bound.
  Number of Threads	1
  Thread Creator	OS or whomever creates the main ORB thread in a process.
  Resource Location	Resources are stored process-wide.
  Thread task	The single thread processes all connection requests and CORBA messages.
  Options	TAO_Resource_Manager: -ORBresources global TAO_Server_Strategy_Factory: -ORBconcurrency reactive

• Multiple threads, thread-per-connection model.

  Typical Use	This configuration spawns a new thread to serve requests from a new connection. This approach works well when there are multiple connections active simultaneously and each request-per-connection may take a fair amount of time to execute.
  Number of Threads	1 plus the number of connections.
  Thread Creator	Programmer must set up the main thread which is responsible to create new threads for new connections.
  Resource Location	Process-wise.
  Thread task	The main thread handles new connections and spawns new threads for them. Other threads handle requests for established connections.
  Options	TAO_Resource_Manager: -ORBresources global TAO_Server_Strategy_Factory: -ORBconcurrency thread-per-connection

• Multiple threads, ORB-per-thread model.

  Typical Use	In this configuration, there multiple ORBs per process each running in its own thread. Each thread handles requests reactively. It's good for hard real-time applications that require different thread priorities for the various ORBs.
  Number of Threads	The number of ORBs.
  Thread Creator	The main process (thread).
  Resource Location	Thread specific.
  Thread task	Service the requests from associating ORB.
  Options	TAO_Resource_Manager: -ORBresources tss TAO_Server_Strategy_Factory: -ORBconcurrency reactive

• Multiple threads, ORB-per-thread, thread-per-connection model.

  Typical Use	This approach provides a range of thread priorities plus connections that don't interfere with each others.
  Number of Threads	Number of ORBs plus number of connections.
  Thread Creator	Main threads creates threads running ORBs. They, in turns, create connection handling threads.
  Resource Location	Thread specific.
  Thread task	There are ORB threads which handle connection requests and handler threads which service requests form establiched connections.
  Options	TAO_Resource_Manager: -ORBresources tss TAO_Server_Strategy_Factory: -ORBconcurrency thread-per-connection

• Multiple threads, thread-pool model. (Not yet implemented.)

  Typical Use	This model implements a highly optimized thread pool that minimizes context switching, synchronization, dynamic memory allocations, and data movement between threads.
  Number of Threads	The number of threads used by ORB-related activities.
  Thread Creator	Identifies the creator of the threads discussed above.
  Resource Location	Where information on various resources is stored.
  Thread task	Describes what task is undertaken for each thread.

• Multiple threads, ORB-per-thread, thread-pool model. (Not yet implemented.)

  Typical Use	A brief description of the scenario and its typical use.
  Number of Threads	The number of threads used by ORB-related activities.
  Thread Creator	Identifies the creator of the threads discussed above.
  Resource Location	Where information on various resources is stored.
  Thread task	Describes what task is undertaken for each thread.

Configuration for homogenous systems

• Compile time options

  Many real-time applications run on homogenous environments, TAO can take advantage of this fact by simplifying the server side demarshaling; to enable this feature you have to edit the $TAO_ROOT/tao/orbconf.h file and enable the macro TAO_DISABLE_SWAP_ON_READ.

  In this systems it is also common that server and the client startup and shutdown simultaneously, in those circumstances there is no need to check the timestamps in the POA, another macro (POA_NO_TIMESTAMP) can be used for this purpose.

  Users running in embebbed systems may also need to modify the default options for TAO, the macros TAO_DEFAULT_RESOURCE_FACTORY_ARGS, TAO_DEFAULT_CLIENT_STRATEGY_FACTORY_ARGS and TAO_DEFAULT_SERVER_STRATEGY_FACTORY_ARGS can be used for those purposes. If the footprint size is an issue users may consider writing custom strategy factories that only create the right strategies, this eliminates the parsing code for the different options.

• Runtime options

  If the only ORB running is TAO and there is no need to be IIOP interoperable the option -ORBiioplite can be used to reduce the message size and the processing time.

  Some embedded systems run without the benefit of a DNS server, in that case they can use the -ORBdotteddecimaladdresses option; the ORB will avoid the use of hostnames in the profiles it generates, thus clients don't need to do any name resolution. The compile-time define TAO_USES_DOTTED_DECIMAL_ADDRESSES in $TAO_ROOT/tao/orbconf.h to make this the default behavior.

Hints

Choosing the right configuration is hard and, of course, depends on your application. In the following section we will attempt to describe some motivations for features in TAO, hopefully that can guide you through the choice of your configuration options.

• ORB-per-thread The main motivation behind this options is to minimize priority invertion, since threads share no ORB resources no locking is required and thus, priority is preserved in most cases (assuming proper support from the OS). If you are not too concerned about priority inversion try to use a global ORB, using ORB-per-thread has some tradeoffs (like calling ORB_init on each thread, activation of a servant is more complicated, etc.) Some of the problems, can be minimized, but they require even more careful analysis. For example, object activation can be simplified by using a global POA; the careful reader will wonder how could global POA be useful in anyway since it will require locks and thus introduce priority inversions again; some applications activate all their objects beforehand so locks in the POA are not always needed; other applications only activate a few objects after startup, so they can use a child POA with the right locking policy for the dynamic servants and the root poa (with no locking) for the majority of the servants.

  As the reader will note this is a delicate configuration option, the rule of thumb should be not to use ORB-per-thread unless it is really required.

• Collocation tables Why could the application what a non-global collocation table? If objects are to serve requests only at a well known priority the application can be configured with the ORB-per-thread option, and the object is activated only in the thread (ORB) corresponding to the desired priority. But using a global table would subert the priority assignment (because calls would run at the priority of the client).

• Single-threaded vs. Multi-threaded Connection Handlers The Client_Connection_Handler is the component in TAO that writes the requests to the underlying transport socket; this is also the component that reads the response back from the server.

  While waiting for this response new requests to the local ORB can arrive, this is the so-called nested upcall support. TAO supports two mechanisms for handling nested upcalls, the default uses the leader-follower model to allow multiple threads to wait on a single reactor for several concurrent requests; sometimes this configuration can be an overkill, if only one thread is using a reactor at the same time a lighter weight implementation can be used.

  This configuration is controled by the -ORBclientconnectionhandler option, good opportunities to use this option are:

  • Single threaded servers
  • Servers running in ORB-per-thread mode
  • Pure clients that will never receive a request

• Allocator for input CDR streams Normally the application has no access to this buffer, and it is only used on the demarshaling of arguments (or results). It is almost always better to use the "-ORBinputcdrallocator tss" option since it will allocate memory from a thread specific allocator and it will not need locks to manage that memory.

  In some cases the user may gain access to the CDR stream buffer: TAO makes no copies when demarshaling octet sequences, instead the octet sequence simply points to the CDR buffer, since the octet sequence does not own this buffer a copy must be made if the user wants to keep the buffer after the upcall.

  The user can, however, increase the reference count on the CDR stream buffer, thus allowing her to extend the lifetime of this buffer. Still passing this buffer to another thread and attempting to release it in that thread will result in some memory leak or corruption. Users willing to use this feature of TAO can still do so, if they use a global allocator for their input CDR stream, but that will introduce extra locking on the critical path.

  As the reader can see this is an option that has limited applicability and requires careful consideration of the tradeoffs involved.

Back to the TAO components documentation.