diff options
Diffstat (limited to 'sandbox/tibs/pysource/notes/thoughts.txt')
| -rw-r--r-- | sandbox/tibs/pysource/notes/thoughts.txt | 170 |
1 files changed, 0 insertions, 170 deletions
diff --git a/sandbox/tibs/pysource/notes/thoughts.txt b/sandbox/tibs/pysource/notes/thoughts.txt deleted file mode 100644 index 59691717b..000000000 --- a/sandbox/tibs/pysource/notes/thoughts.txt +++ /dev/null @@ -1,170 +0,0 @@ -Thoughts on pysource -==================== - -:Author: Tibs -:Date: 19 Sep 2002 - -.. - - [This is the contents of an email sent from Tibs to me, to tie up - -- or at least identify -- loose ends. When Tibs writes "you", - he's referring to me. It is published with Tibs' permission. I - mostly agree with what Tibs has written. I've added my own notes, - indented in square brackets, like this one. - - --David Goodger] - -Well, having looked back at the pysource sources, I'm confirmed -in my opinion that, if I were to start work on the project again, -I would probably start a new version of pysource from scratch, -using what I have learned (essentially, the content of visit.py), -and taking advantage of all the new technology that is in the -modern DOCUTILS. - -Thus I would (at least): - -1. produce a "frontend" using Optik -2. rewrite visit.py more neatly -3. construct DOCUTILS nodes to match pysource.dtd -4. *possibly* meld those into the classes in visit.py - (but possibly not - some of the stuff in visit.py - seems to me to be, of necessity, a bit messy as it - is constructing stuff, and thus it may be premature - to "DOCUTILS" it before it is necessary). -5. produce an example transform to amend the Python - specific DOCUTILS tree into a generic DOCUTILS - tree. - -That's a fair amount of work, and I'm not convinced that I can -find the sustained effort [1]_, especially if you might be -willing to take the task on (and if I would have been refactoring -anyway, a whole different head may be a significant benefit). - -.. [1] I believe that's called "British understatement". - -Some comments on pysource.txt and pysource.dtd ----------------------------------------------- - -I believe that a <docstring> node is a Good Thing. It's a natural -construct (if one believes that the other nodes are good things -as well!), and it allows the "transform" of the Python specific -tree to be done more sensibly. - - [I think I finally understand what Tibs is getting at here. I - thought he meant to have a <docstring> node in an otherwise - standard Docutils doctree, which just contained the raw docstring. - Instead, I believe he means that the Python-specific Docutils - doctree elements like <class_section> and - <module_section> (see pysource.dtd), should each have a <docstring> - element which contains ``%structure.model;``, instead of the - current ``%structure.model;`` directly inside the element. In - other words, a <docstring> element would be a simple container for - all the parsed elements of the docstring. On second thought, each - Python-specific section element would still have to have - ``%structure.model;``, in order to contain subsections. A - <package_section> would contain <module_section>s, which would - contain <class_section>s, and so on. - - So a <docstring> element may be useful as a convenience to - transforms. It would be a trivial change anyhow. - - The initial (internal) data structure resulting from the parsing - of Python modules & packages is a completely different beast. It - should contain <docstring> nodes, along with <package>, <module>, - <class>, etc., nodes. But these should *not* be subclasses of - docutils.nodes.Node, and this tree should not be a "Docutils - document tree". It should be a separate, parallel structure. - Document tree nodes (docutils.nodes.Node objects) are not suited - for this work. - - --DG] - -I recommend some initial "surgery" on the initial parse tree to -make finding the docstrings for nodes easier. - -I reckon that one produces a Python-specific doctree, and then -chooses one of several transforms to produce generic doctrees. - -By the way, I still vote for "static" introspection - that is, -without actually importing the module. It does limit what one can -do in some respects, but there are modules one might want to -document that one does not wish to import (and indeed, at work we -have some Python files that essentially cannot be introspected in -this manner with any ease - they are "designed" to be imported by -other modules after ``sys.path`` has been mangled suitably, and -just don't work stand-alone). - -I've tried to expand out the ideas you had on how the "pysource" -tool should work below. - - although it occurs to me that this is all terribly obvious, - really. Oh well... - -The pysource tool ------------------ - -The "input mode" should be the "Python Source Reader". - - You can see where I'm starting from in pysource.txt. - -This produces, as its output, a Python-specific doctree, -containing extra nodes as defined in pysource.dtd (etc.). - -One of the tasks undertaken by the Python Source Reader is to -decide what to do with docstrings. For each Python file, it -discovers (from ``__docformat__``) if the input parser is -"reStructuredText". If it is, then the contents of each docstring -in that file will be parsed as such, including sorting out -interpreted text (i.e., rendering it into links across the tree). -If it is not, then each docstring will be treated as a literal -block. - - This admits the posibility of adding other parsers *for - docstrings* at a later date - which I suspect is how HappyDoc - does it. It is just necessary to publicise the API for the - Docstring class, and then someone else can provide alternate - plugins. - -The output of the Python Source Reader is thus a Python-specific -doctree, with all docstrings fully processed (as appropriate), -and held inside <docstring> elements. - -The next stage is handled by the Layout Transformer. - - [What I call "stylist transforms". --DG] - -This determines the layout of the document to be produced - the -*kind* or *style* of output that the user wants (e.g., PyDoc -style, LibRefMan style, generic docutils style, etc.). - -Each layout is represented by a class that walks the -Python-specific doctree, replacing the Python-specific nodes with -appropriate generic nodes. The output of the Layout Transformer -is thus a generic doctree. - -The final stage is thus a normal DOCUTILS Writer - since it is -taking input that is a generic doctree, this makes perfect -sense. This also means that we get maximum leverage from existing -Writers, which is essential (not just a Good Thing). - -As you point out, some layouts will probably only be appropriate -for some output formats. Well, that's OK. On the other hand, if -the output of the Layout stage *is* a generic doctree, we're not -likely to get fatal errors by putting it through the wrong -Writer, so we needn't worry too much. - -Thus one might envisage a command line something like:: - - pysource --layout:book --format:html this_module - -Of course, other command line switches would be options for -particular phases (e.g., to use frames or not for HTML output). I -can see wanting a series of configuration files that one could -reference, to save specifying lots of switches. - -One specific thing to be decided, particularly for HTML, is -whether one is outputting a "cluster" of files (e.g., as javadoc -does). I reckon this can be left for later on, though (as can -such issues as saying "other interesting sources are *over -there*, so reference them if you can" - e.g., the Python -library). |