IOError handling improvements. More tests. Doc update

1 files changed, 109 insertions, 15 deletions

diff --git a/doc/ply.html b/doc/ply.html
index 631d76f..808ed1d 100644
--- a/doc/ply.html
+++ b/doc/ply.html
@@ -74,6 +74,7 @@ dave@dabeaz.com<br>
 <li><a href="#ply_nn45">Debugging the lex() and yacc() commands</a>
 <li><a href="#ply_nn46">Run-time Debugging</a>
 </ul>
+<li><a href="#ply_nn49">Packaging Advice</a>
 <li><a href="#ply_nn39">Where to go from here?</a>
 </ul>
 </div>
@@ -82,6 +83,7 @@ dave@dabeaz.com<br>
 
 
 
+
 <H2><a name="ply_nn1"></a>1. Preface and Requirements</H2>
 
 
@@ -778,7 +780,7 @@ approach substantially improves the startup time of the lexer and it
 works in Python's optimized mode.
 
 <p>
-To change the name of the lexer-generated file, use the <tt>lextab</tt> keyword argument.  For example:
+To change the name of the lexer-generated module, use the <tt>lextab</tt> keyword argument.  For example:
 </p>
 
 <blockquote>
@@ -787,15 +789,6 @@ lexer = lex.lex(optimize=1,lextab="footab")
 </pre>
 </blockquote>
 
-<p>To change the output directory of the file, use the <tt>outputdir</tt> keyword argument. For example:
-</p>
-
-<blockquote>
-<pre>
-lexer = lex.lex(optimize=1, outputdir="/some/directory")
-</pre>
-</blockquote>
-
 When running in optimized mode, it is important to note that lex disables most error checking.  Thus, this is really only recommended
 if you're sure everything is working correctly and you're ready to start releasing production code.
 
@@ -1748,12 +1741,17 @@ executions, <tt>yacc</tt> will reload the table from
 <tt>parsetab.py</tt> unless it has detected a change in the underlying
 grammar (in which case the tables and <tt>parsetab.py</tt> file are
 regenerated).  Both of these files are written to the same directory
-as the module in which the parser is specified.   The output directory
-can be changed by giving an <tt>outputdir</tt> keyword argument to <tt>yacc()</tt>. 
-The name of the <tt>parsetab</tt> module can also be changed using the
-<tt>tabmodule</tt> keyword argument to <tt>yacc()</tt>.
+as the module in which the parser is specified.  
+The name of the <tt>parsetab</tt> module can be changed using the
+<tt>tabmodule</tt> keyword argument to <tt>yacc()</tt>.  For example:
 </p>
 
+<blockquote>
+<pre>
+parser = yacc.yacc(tabmodule='fooparsetab')
+</pre>
+</blockquote>
+
 <p>
 If any errors are detected in your grammar specification, <tt>yacc.py</tt> will produce
 diagnostic messages and possibly raise an exception.  Some of the errors that can be detected include:
@@ -3104,6 +3102,13 @@ parser = yacc.yacc(tabmodule="foo")
 </pre>
 </blockquote>
 
+<P>
+Normally, the <tt>parsetab.py</tt> file is placed into the same directory as
+the module where the parser is defined. If you want it to go somewhere else, you can
+given an absolute package name for <tt>tabmodule</tt> instead.  In that case, the 
+tables will be written there.
+</p>
+
 <p>
 <li>To change the directory in which the <tt>parsetab.py</tt> file (and other output files) are written, use:
 <blockquote>
@@ -3113,6 +3118,12 @@ parser = yacc.yacc(tabmodule="foo",outputdir="somedirectory")
 </blockquote>
 
 <p>
+Note: Be aware that unless the directory specified is also on Python's path (<tt>sys.path</tt>), subsequent
+imports of the table file will fail.   As a general rule, it's better to specify a destination using the
+<tt>tabmodule</tt> argument instead of directly specifying a directory using the <tt>outputdir</tt> argument.
+</p>
+
+<p>
 <li>To prevent yacc from generating any kind of parser table file, use:
 <blockquote>
 <pre>
@@ -3348,7 +3359,90 @@ For very complicated problems, you should pass in a logging object that
 redirects to a file where you can more easily inspect the output after
 execution.
 
-<H2><a name="ply_nn39"></a>10. Where to go from here?</H2>
+<H2><a name="ply_nn49"></a>10. Packaging Advice</H2>
+
+
+<p>
+If you are distributing a package that makes use of PLY, you should
+spend a few moments thinking about how you want to handle the files
+that are automatically generated.  For example, the <tt>parsetab.py</tt>
+file generated by the <tt>yacc()</tt> function.</p>
+
+<p>
+Starting in PLY-3.6, the table files are created in the same directory
+as the file where a parser is defined.   This means that the
+<tt>parsetab.py</tt> file will live side-by-side with your parser
+specification.  In terms of packaging, this is probably the easiest and
+most sane approach to manage.  You don't need to give <tt>yacc()</tt>
+any extra arguments and it should just "work."</p>
+
+<p>
+One concern is the management of the <tt>parsetab.py</tt> file itself.
+For example, should you have this file checked into version control (e.g., GitHub),
+should it be included in a package distribution as a normal file, or should you
+just let PLY generate it automatically for the user when they install your package?
+</p>
+
+<p>
+As of PLY-3.6, the <tt>parsetab.py</tt> file should be compatible across all versions
+of Python including Python 2 and 3.  Thus, a table file generated in Python 2 should
+work fine if it's used on Python 3.  Because of this, it should be relatively harmless 
+to distribute the <tt>parsetab.py</tt> file yourself if you need to. However, be aware
+that older/newer versions of PLY may try to regenerate the file if there are future 
+enhancements or changes to its format.
+</p>
+
+<p>
+To make the generation of table files easier for the purposes of installation, you might
+way to make your parser files executable using the <tt>-m</tt> option or similar.  For
+example:
+</p>
+
+<blockquote>
+<pre>
+# calc.py
+...
+...
+def make_parser():
+    parser = yacc.yacc()
+    return parser
+
+if __name__ == '__main__':
+    make_parser()
+</pre>
+</blockquote>
+
+<p>
+You can then use a command such as <tt>python -m calc.py</tt> to generate the tables. Alternatively,
+a <tt>setup.py</tt> script, can import the module and use <tt>make_parser()</tt> to create the
+parsing tables.
+</p>
+
+<p>
+If you're willing to sacrifice a little startup time, you can also instruct PLY to never write the
+tables using <tt>yacc.yacc(write_tables=False, debug=False)</tt>.   In this mode, PLY will regenerate
+the parsing tables from scratch each time.  For a small grammar, you probably won't notice.  For a 
+large grammar, you should probably reconsider--the parsing tables are meant to dramatically speed up this process.
+</p>
+
+<p>
+During operation, is is normal for PLY to produce diagnostic error
+messages (usually printed to standard error).  These are generated
+entirely using the <tt>logging</tt> module.  If you want to redirect
+these messages or silence them, you can provide your own logging
+object to <tt>yacc()</tt>.  For example:
+</p>
+
+<blockquote>
+<pre>
+import logging
+log = logging.getLogger('ply')
+...
+parser = yacc.yacc(errorlog=log)
+</pre>
+</blockquote>
+
+<H2><a name="ply_nn39"></a>11. Where to go from here?</H2>
 
 
 The <tt>examples</tt> directory of the PLY distribution contains several simple examples.   Please consult a