ChangeLogTag: Fri May 30 18:18:25 2003 Jeff Parsons <j.parsons@vanderbilt.edu>

3 files changed, 14 insertions, 483 deletions

diff --git a/TAO/CIAO/CIDLC/DesignNotes b/TAO/CIAO/CIDLC/DesignNotes
deleted file mode 100644
index b01fd452762..00000000000
--- a/TAO/CIAO/CIDLC/DesignNotes
+++ /dev/null
@@ -1,457 +0,0 @@
-
-Intention of this file is to capture and document CIDL complier design 
-ideas/decisions.
-
-Conceptual parts of CIDL compiler design
-----------------------------------------
-
-Option Parser                       Consists of option parser and option
-                                    database. 
-
-C Preprocessor Interfacing          Represents mechanism of preprocessing
-                                    cidl files.
-
-IDL Compiler Interfacing            Represents mechanism of invoking IDL
-                                    compiler.
-
-Scanner                             Scanner for preprocessed cidl file.
-
-Parser                              CIDL grammar parser. Consists of grammar
-                                    and semantic rules.
-
-Syntax Tree                         Intermediate representation of cidl file.
-                                    Consists of syntax tree nodes itself and
-                                    perhaps symbol tables.
-
-Semantic Analyzer                   Traverses Syntax Tree and performs
-                                    semantic analysis as well as some
-                                    semantic expansions.
-
-
-Code Generation Stream              Stream to output generated code to. Used 
-                                    by concrete Code Generators
-
-Code Generators 
-{
-
-  Executor Mapping Generator          Generator for local executor mapping.
-
-  Executor Implementation Generator   Generator for partial implementation
-                                      of local executor mapping.
-
-  Skeleton Thunk Generator            Generator for skeleton thunks i.e.
-                                      code that implements skeleton and
-                                      thunks user-defined functions to
-                                      executor mapping.
-}
-
-Compiler driver                     Establishes order of execution of
-                                    different components as part of 
-                                    compilation process.
-
-
-How everything works together
------------------------------
-
-(1) Compiler Driver executes Option Parser to populate Option Database
-
-(2) Compiler Driver executes C Preprocessor on a supplied cidl file
-
-(3) Compiler Driver executes Parser which uses Scanner to scan preprocessed
-    cidl file and generates Syntax Tree by means of semantic rules.
-
-(4) At this point we have Syntax Tree corresponding to the original cidl
-    file. Compiler Driver executes Executor Mapping Generator, 
-    Executor Implementation Generator and Skeleton Thunk Generator on 
-    Syntax Tree.
-
-
-
-General Design Ideas/Decision
--------------
-
-[IDEA]: There is an effort to use autoconf/automake in ACE/TAO. Maybe it's 
-        a good idea to start using it with CIDLC? There is one side advantage
-        of this approach: if we decide to embed GCC CPP then we will have to
-        use configure (or otherwise ACE-ify the code which doesn't sound like
-        a right solution).
-
-[IDEA]: CIDLC is a prototype for a new IDLC, PSDLC and IfR model. Here are
-        basic concepts:
-
-          - use common IDL grammar, semantic rules and syntax tree nodes 
-            for IDLC, CIDLC, PSDLC and IfR. Possibly have several libraries
-            for example ast_idl-2.so, ast_idl-3.so, scaner_idl-2.so
-            scaner_idl-3.so, parser_idl-2.so, parser_idl-3.so. Dependency
-            graph would look like this:
-
-
-                 ast_idl-2.so                     scanner_idl-2.so
-                      |                                 |
-                      |---------------------------------|
-                      |               |                 |
-                      |               |                 |
-                      |         parser_idl-2.so         |
-                      |               |                 |
-                 ast_idl-3.so         |           scanner_idl-3.so
-                      |               |                 |
-                      |               |                 |
-                      |               |                 |
-                       ---------parser_idl-3.so---------
-
-            Same idea applies for CIDL and PSDL.
-                                             
-
-          - use the same internal representation (syntax tree) in all 
-            compilers and IfR. This way at some stage if we will need
-            to make one of the compilers IfR-integrated (import keyword?)
-            then it will be a much easier task than it's now. This internal
-            representation may also be usable in typecodes 
-
-            @@ boris: not clear to me. 
-            
-            @@ jeff: A typecode is like a piece of the Syntax Tree with these
-               exceptions -
-
-                (1) There is no typecode for an IDL module.
-
-                (2) Typecodes for interfaces and valuetypes lack some of the
-                    information in the corresponding Syntax Tree nodes.
-
-               With these exceptions in mind, a typecode can be composed and
-               traversed in the same manner as a Syntax Tree, perhaps with
-               different classes than used to compose the ST itself.
-
-            @@ boris: Ok, let me see if I got it right. So when typecode
-               is kept in parsed state (as opposite to binary) (btw, when 
-               does it happen?) it makes sense to apply the same techniques
-               (if in fact not the same ST nodes and traversal mechs) as
-               for XIDL compilation.
-
-[IDEA]: We should be consistent with the way external compilers that we call
-        report errors. For now those are CPP and IDLC.
-                       
-Option Parser
--------------
-
-[IDEA]: Use Spirit parser framework to generate option parser.
-
-[IDEA]: Option Database is probably a singleton. 
-
-        @@ jeff: This is a good idea, especially when passing some of the 
-           options to a preprocessor or spawned IDL compier. But I think we 
-           will still need 'state' classes for the front and back ends (to 
-           hold values set by command line options and default values) so 
-           we can keep them decoupled).
-
-
-        @@ boris: I understand what you mean. Though I think we will be 
-           able to do with one 'runtime database'. Each 'compiler module'
-           will be able to populate its 'namespace' with (1) default
-           values, (2) with module-specific options and (3) arbitrary 
-           runtime information. I will present prototopy design shortly.
-           
-
-[IDEA]: It seems we will have to execute at least two external programs
-        as part of CIDLC execution: CPP and IDLC. Why wouldn't we follow
-        GCC specs model (gcc -dumpspecs). Here are candidates to be put into
-        specs:        
-
-          - default CPP name and options
-          - default  IDLC name and options
-          - default file extensions and formats for different mappings
-          - other ideas? 
-
-[IDEA]: Provide short and long option names (e.g. -o and --output-dir)
-        for every option (maybe except -I, -D, etc).
-
-
-C Preprocessor Interfacing
---------------------------
-
-[IDEA]: Embed/require GCC CPP
-
-[IDEA]: We need a new model of handling includes in CIDLC (as well as IDLC).
-        Right now I'm mentally testing a new model (thanks to Carlos for the
-        comments). Soon I will put the description here.
-
-[IDEA]: We cannot move cidl file being preprocessed to for example /tmp
-        as it's currently the case with IDLC.
-
-[IDEA]: Can we use pipes (ACE Pipes) portably to avoid temporary files?
-        (Kitty, you had some ideas about that?)
-
-
-
-IDL Compiler Interfacing
-------------------------
-
-[IDEA]: Same as for CPP: Can we use pipes? 
-
-        @@ jeff: check with Nanbor on this. I think there may be CCM/CIAO 
-           use cases where we need the intermediate IDL file.
-
-[IDEA]: Will need a mechanism to pass options to IDLC from CIDLC command
-        line (would be nice to have this ability for CPP as well).
-        Something like -x in xterm? Better ideas?
-
-
-
-Scanner
-------
-
-[IDEA]: Use Spirit framework to construct scanner. The resulting sequence
-        can be sequence of objects? BTW, Spirit parser expects a "forward
-        iterator"-based scanner. So this basically mean that we may have to
-        keep the whole sequence in memory. BTW, this is another good reason
-        to have scanner: if we manage to make scanner a predictable parser
-        (i.e. no backtracking) then we don't have to keep the whole 
-        preprocessed cidl file in memory.
-
-
-
-Parser
-------
-
-[IDEA]: Use Spirit framework to construct parser.
-
-[IDEA]: Define IDL grammar as a number of grammar capsules. This way it's
-        much easier to reuse/inherit even dynamically. Need to elaborate
-        this idea.
-
-[IDEA]: Use functors as semantic actions. This way we can specify (via 
-        functor's data member) on which Syntax Tree they are working.
-        Bad side: semantic rules are defined during grammar construction.
-        However we can use a modification of the factory method pattern.
-        Better ideas? 
-
-        @@ jeff: I think ST node creation with a factory
-           is a good idea - another ST implementation could be plugged in,
-           as long as it uses a factory with the same method names.
-
-        @@ boris: Right. In fact it's our 'improved' way of handling 'BE' 
-           usecases.
-
-
-
-Syntax Tree
------------
-
-[IDEA]: Use interface repository model as a base for Syntax Tree hierarchy.
-
-[IDEA]: Currently (in IDLC) symbol lookup is accomplished by AST navigation,
-        and is probably the biggest single bottleneck in performance. Perhaps 
-        a separate symbol table would be preferable. Also, lookups could be 
-        specialized, e.g., for declaration, for references, and perhaps a 
-        third type for argument-related lookups.
-
-[NOTE]: If we are to implement symbol tables then we need to think how we
-        are going to inherit (extend) this tables.
-
-[NOTE]: Inheritance/supports graphs: these graphs need to be traversed at
-        several points in the back end. Currently they are rebuilt for each 
-        use, using an n-squared algorithm. We could at least build them only 
-        once for each interface/valuetype, perhaps even with a better 
-        algorithm. It could be integrated into inheritance/supports error 
-        checking at node creation time, which also be streamlined. 
-
-        @@ boris: Well, I think we should design our Syntax Tree so that 
-           every interface/valuetype has a list (flat?) of interfaces it 
-           inherits from/supports.
-
-[IDEA]: We will probably want to use factories to instantiate Syntax Tree
-        Nodes (STN). This will allow a concrete code generators to alter (i.e.
-        inherit off and extend) vanilla STNs (i.e. alternative to BE nodes
-        in current IDLC design).
-
-
-Common Syntax Tree traversal Design Ideas/Decision
---------------------------------------------------
-
-[IDEA] If we specify Syntax Tree traversal facility then we will be able
-       to specify (or even plug dynamically) Syntax Tree traversal agents
-       that may not only generate something but also annotate or modify 
-       Syntax Tree. We are already using this technique for a number of
-       features (e.g. AMI, IDL3 extension, what else?) but all these agents
-       are hardwired inside TAO IDLC. If we have this facility then we will
-       be able to produce modular and highly extensible design. Notes:
-
-       - Some traversal agents can change Syntax Tree so that it will be
-         unusable by some later traversal agents. So maybe the more 
-         generic approach would be to produce new Syntax Tree? 
-
-         @@ jeff: Yes, say for example that we were using a common ST 
-            representation for the IDL compiler and the IFR. We would not 
-            want to send the extra AMI nodes to the IFR so in that case 
-            simple modification of the ST might not be best.
-
-[IDEA] Need a generic name for "Syntax Tree Traversal Agents". What about
-       "Syntax Tree Traverser"?
-
-
-Code Generation Stream
-----------------------
-
-[IDEA] Use language indentation engines for code generation (like a c-mode
-       in emacs). The idea is that code like this
-
-       out << "long foo (long arg0,  "  << endl
-           << "          long arg1)  "  << endl
-           << "{                     "  << endl
-           << "  return arg0 + arg1; "  << endl
-           << "}                     "  << endl;
-
-	will result in a generated code like this:
-
-       namespace N
-       {      
-         ...         
-
-         long foo (long arg0,
-                   long arg1)
-         {
-           return arg0 + arg1;
-         }
-             
-         ...      
-       }
-
-       Note that no special actions were taken to ensure proper indentation.
-       Instead the stream's indentation engine is responsible for that.
-       The same mech can be used for different languages (e.g. XML).
-
-
-Code Generators
----------------
-
-[IDEA] It makes sense to establish a general concept of code generators.
-       "Executor Mapping Generator", "Executor Implementation Generator"
-       and "Skeleton Thunk Generator" would be a concrete code generators.
-
-[IDEA] Expression evaluation: currently the result (not the expression)
-       is generated, which may not always be necessary.
-
-       @@ boris: I would say may not always be correct
-
-
-       However, for purposes of type coercion and other checking (such as 
-       for positive integer values in string, array and sequence bounds) 
-       evaluation must be done internally.
-       
-       @@ boris: note that evaluation is needed to only verify that things
-          are correct. You don't have to (shouldn't?) substitute original 
-          (const) expression with what's been evaluated.
-
-
-       @@ jeff: it may be necessary in some cases to append 'f' or 'U' to 
-          a generated number to avoid a C++ compiler warning.
-
-       @@ boris: shouldn't this 'f' and 'U' be in IDL as well?
-
-[IDEA] I wonder if it's a good idea to use a separate pass over syntax tree
-       for semantic checking (e.g. type coercion, positive values for 
-       sequence bounds). 
-
-       @@ jeff: This may hurt performance a little - more lookups - but it 
-          will improve error reporting.
-      
-       @@ boris:  As we dicussed earlier this pass could be used to do
-          'semantic expansions' (e.g. calculate a flat list of interface's
-          children, etc). Also I don't think we should worry about speed
-          very much here (of course I don't say we have to be stupid ;-)
-          In fact if we are trading better design vs faster compilation
-          at this stage we should always go for better design.
-      
-
-Executor Mapping Generator
---------------------------
-
-
-
-Executor Implementation Generator
---------------------------------
-
-[IDEA]: Translate CIDL composition to C++ namespace.
-
-
-
-Skeleton Thunk Generator
-------------------------
-
-
-
-
-Compiler driver
----------------
-
-
-
-Vault
------
-
-Some thoughts from Jeff that I are not directly related to CIDLC and are
-rather current IDLC design defects:
-
-* AMI/AMH implied IDL: more can be done in the BE preprocessing pass, 
-  hopefully eliminating a big chunk of the huge volume of AMI/AMH visitor 
-  code. The implied IDL generated for CCM types, for example, leaves almost 
-  nothing extra for the visitors to do.
-
-* Fwd decl redefinition: forward declaration nodes all initially contain a 
-  heap-allocated dummy full-definition member, later replaced by a copy 
-  of the full definition. This needs to be streamlined.
-
-* Memory leaks: inconsistent copying/passing policies make it almost 
-  impossible to eliminate the huge number of leaks. The front end will be 
-  more and more reused, and it may be desirable to make it executable as a 
-  function call, in which case it will important to eliminate the leaks. 
-  Perhaps copying of AST nodes can be eliminated with reference counting or 
-  just with careful management, similarly for string identifiers and literals.
-  Destroy() methods have been put in all the node classes, and are called 
-  recursively from the AST root at destruction time, but they are far from 
-  doing a complete job.
-
-* Visitor instantiation: the huge visitor factory has already been much 
-  reduced, and the huge enum of context state values is being reduced. 
-  However there will still be an abundance of switch statements at nearly 
-  every instance of visitor creation at scope nesting. We could make better 
-  use of polymorphism to get rid of them.
-
-* Node narrowing: instead of the impenetrable macros we use now, we
-  could either generate valuetype-like downcast methods for the (C)IDL 
-  types, or we could just use dynamic_cast.
-
-* Error reporting: making error messages more informative, and error recovery
-  could both be a lot better, as they are in most other IDL compilers. If a 
-  recursive descent parser is used (such as Spirit), there is a simple 
-  generic algorithm for error recovery.
-
-
-* FE/BE node classes: if BE node classes are implemented at all, there should 
-  be a complete separation of concerns - BE node classes should contain only 
-  info related to code generation, and FE node classes should contain only 
-  info related to the AST representation. As the front end becomes more 
-  modular and reusable, this will become more and more necessary. 
-
-  @@ boris: It doesn't seem we will need two separate and parallel hierarhies.
-
-* Undefined fwd decls: now that we have dropped support for platforms without
-  namespaces, the code generated for fwd declarations not defined in the same 
-  translation unit can be much improved, most likely by the elimination of 
-  generated flat-name global methods, and perhaps other improvements as well.
-
-* Strategized code generation: many places now have either lots of 
-  duplication, or an explosion of branching in a single visitor. Adding code 
-  generation for use cases incrementally may give us an opportunity to 
-  refactor and strategize it better.
-
-* Node generator: this class does nothing more than call 'new' and pass 
-  unchanged the arguments it gets to the appropriate constructor - it can be 
-  eliminated.
-
-* Virtual methods: there are many member functions in the IDL compiler that 
-  are needlessly virtual.
-
-* Misc. leveraging: redesign of mechanisms listed above can have an effect 
-  on other mechanisms, such as the handling of pragma prefix, typeprefix, and
-  reopened modules.
diff --git a/TAO/CIAO/CIDLC/README b/TAO/CIAO/CIDLC/README
index 5cc8c2628ec..8e211286ca0 100644
--- a/TAO/CIAO/CIDLC/README
+++ b/TAO/CIAO/CIDLC/README
@@ -1,35 +1,19 @@
-A place holder for CIAO's CIDL compiler. 
 
-Please see some infromation about rationale behind naming convention at
-the end of this file.
+The C++ source files in this directory are compiled into the 
+code generators and driver for the CIAO CIDL compiler.
 
-* Executor mapping can be found at:  Chapter 3.3 of formal/02-06-65
+The CIDL compiler's front end is built from files in the
+$CIAO_ROOT/CCF directory.
 
-  <filename>E.*
+The CIAO CIDL compiler uses the Boost library, which may be
+downloaded from:
 
-* Templates for generic servants (container glue code) will be
-  available from $CIAO_ROOT/docs/templates/
+http://www.boost.org
 
-  These classes should be put into CIAO_GLUE namespace.
-
-  <filename>G.*
-
-* Component descriptors: Chapter 6.4 of formal/02-06-65
-
-  <flatname>.ccd
-
-  How to import this information into GME 2000?
-
-* I think it may be a good thing to do to be able to represent IDL
-  definition in XML format and be able to exchange (import/export) IDL
-  definitions via XML documents.  What is the spec. for this?
-
-* Likewise, it would be cool to have component-aware IfR so we can
-  either import component definitions into GME tools through the XML
-  files, or allow GME tools to query component capabilities via IfR.
-
-* Need to fix the collocated valuetype passed by reference bug.
+See documentation files in CIAO_ROOT/CCF for more information
+on supported platforms, design, etc.
 
+-jeff
 
 -----------------------------------------------------------------
 Naming Convention.
diff --git a/TAO/CIAO/ChangeLog b/TAO/CIAO/ChangeLog
index 56b1c803582..019e1c747fe 100644
--- a/TAO/CIAO/ChangeLog
+++ b/TAO/CIAO/ChangeLog
@@ -8,6 +8,10 @@ Fri May 30 18:18:25 2003  Jeff Parsons <j.parsons@vanderbilt.edu>
           in this directory are no longer useful or up to date. Soon
           parts of the CIDL compiler will be checked into this directory.
 
+        * READEME:
+
+          Updated file.
+
 Fri May 30 17:41:05 2003  Jeff Parsons <j.parsons@vanderbilt.edu>
 
         * examples/OEP/BasicSP/BMClosedED/BMClosedED_svnt.dsp: