Update documentation for the "csv-table" directive.

Document current behaviour and the roadmap to bring it in line
with the specification.

Add deprecation warnings for to-be-removed objects.

git-svn-id: https://svn.code.sf.net/p/docutils/code/trunk@9356 929543f6-e4f2-0310-98a6-ba3bd3dd1d04

5 files changed, 92 insertions, 35 deletions

diff --git a/docutils/RELEASE-NOTES.txt b/docutils/RELEASE-NOTES.txt
index a7e9144e7..858ce2086 100644
--- a/docutils/RELEASE-NOTES.txt
+++ b/docutils/RELEASE-NOTES.txt
@@ -102,6 +102,18 @@ Drop support for Python 3.7 and 3.8 in Docutils 0.21.
 
   - Remove the input encoding auto-detection code in Docutils 1.0 or later.
 
+* "csv-table_" directive:
+
+  - Use the same CSV format for the main CSV data and the :header: option
+    (as specified in the documentation since addition of "csv-table_")
+    in Docutils 0.21.
+
+  - Move `parsers.rst.directives.Table.process_header_option()` to
+    `parsers.rst.directives.CSVTable` in Docutils 0.21.
+
+  - Remove `parsers.rst.directives.CSVTable.HeaderDialect`
+    in Docutils 0.22.
+
 * "html5" writer:
 
   - Stop setting the "footnote-reference" class value for footnote
@@ -178,6 +190,7 @@ Drop support for Python 3.7 and 3.8 in Docutils 0.21.
 .. _literal_block_env: docs/user/config.html#literal-block-env
 .. _use_latex_citations: docs/user/config.html#use-latex-citations
 .. _buildhtml.py: docs/user/tools.html#buildhtml-py
+.. _csv-table: docs/ref/rst/directives.html#csv-table
 
 
 Release 0.20 (unpublished)
@@ -270,7 +283,7 @@ Release 0.18.1 (2021-12-23)
 
 * re-add module ``parsers.rst.directives.html``
   (stub, emits deprecation warning and loads
-  "Meta" directive from ist new place at ``parsers.rst.directives.misc``.)
+  "Meta" directive from its new place at ``parsers.rst.directives.misc``.)
 
 * Small bugfixes (see HISTORY_).
 
diff --git a/docutils/docs/ref/rst/directives.txt b/docutils/docs/ref/rst/directives.txt
index f3d0bd1cf..36e9f8cfe 100644
--- a/docutils/docs/ref/rst/directives.txt
+++ b/docutils/docs/ref/rst/directives.txt
@@ -838,12 +838,13 @@ CSV Table
 :Doctree Element: table_
 :Directive Arguments: One, optional (table title).
 :Directive Options: Possible (see below).
-:Directive Content: A CSV (comma-separated values) table.
+:Directive Content: A CSV (comma-separated values) table
+                    or (with the `:file:`_ or `:url:`_ options) None
 
 .. WARNING::
 
-   The "csv-table" directive's ":file:" and ":url:" options represent
-   a potential security holes.  They can be disabled with the
+   The "csv-table" directive's `:file:`_ and `:url:`_ options represent
+   potential security holes.  They can be disabled with the
    "file_insertion_enabled_" runtime setting.
 
 The "csv-table" directive is used to create a table from CSV
@@ -853,7 +854,7 @@ internal (an integral part of the document) or external (a separate
 file).
 
 * Block markup and inline markup within cells is supported.  Line ends
-  are recognized within cells.
+  are recognized within quoted cells.
 
 * There is no support for checking that the number of columns in each
   row is the same. The directive automatically adds empty entries at
@@ -868,34 +869,38 @@ Example::
        :widths: 15, 10, 30
 
        "Albatross", 2.99, "On a stick!"
-       "Crunchy Frog", 1.49, "If we took the bones out, it wouldn't be
-       crunchy, now would it?"
+       "Crunchy Frog", 1.49, "If we took the bones out,
+       it wouldn't be crunchy, now would it?"
        "Gannet Ripple", 1.99, "On a stick!"
 
 The following options are recognized:
 
 ``align`` : "left", "center", or "right"
     The horizontal alignment of the table. (New in Docutils 0.13)
+    
+.. _delimiter:    
 
-``delim`` : char | "tab" | "space" [#whitespace-delim]_
-    A one-character string used to separate fields.
-    Defaults to ``,`` (comma).  May be specified as a Unicode code
-    point; see the unicode_ directive for syntax details.
+``delim`` : char | "tab" | "space"
+    A one-character string [#char-or-uni]_ used to separate data fields.
+    The special values "tab" and "space" are converted to the respective
+    whitespace characters. [#tab-expansion]_
+    Defaults to ``,`` (comma).
 
 ``encoding`` : string
     The text encoding of the external CSV data (file or URL).
     Defaults to the document's `input_encoding`_.
 
 ``escape`` : char
-    A one-character string used to escape the
-    delimiter or quote characters.  May be specified as a Unicode
-    code point; see the unicode_ directive for syntax details.  Used
-    when the delimiter is used in an unquoted field, or when quote
-    characters are used within a field.  The default is to double-up
-    the character, e.g. "He said, ""Hi!"""
+    A one-character string [#char-or-uni]_ used to escape the delimiter_
+    or quote_ characters from the CSV parser. 
+    The default is no CSV-escape character -- 
+    quote data fields containing the delimiter and double-up the quote
+    character, e.g.  ``"He said, ""Hi!"""``.
+    The reStructuredText `escaping mechanism`_ is applied
+    after CSV parsing as part of parsing the field content for
+    reStructuredText markup.
 
-    .. Add another possible value, "double", to explicitly indicate
-       the default case?
+.. _`:file:`:
 
 ``file`` : string (newlines removed)
     The local filesystem path to a CSV data file.
@@ -905,6 +910,10 @@ The following options are recognized:
     before any ``header-rows`` from the main CSV data.  Must use the
     same CSV format as the main CSV data.
 
+    .. note:: Currently, the header option uses a hard-coded
+              CSV dialect variant with the backslash as escape character.
+              This will change to the documented behaviour in Docutils 0.21.
+
 ``header-rows`` : integer
     The number of rows of CSV data to use in the table header.
     Defaults to 0.
@@ -913,16 +922,19 @@ The following options are recognized:
     Treat whitespace immediately following the delimiter as
     significant.  The default is to ignore such whitespace.
 
+.. _quote:
+
 ``quote`` : char
-    A one-character string used to quote elements
-    containing the delimiter or which start with the quote
-    character.  Defaults to ``"`` (quote).  May be specified as a
-    Unicode code point; see the unicode_ directive for syntax
-    details.
+    A one-character string [#char-or-uni]_ used to quote fields
+    containing special characters, such as the delimiter, quote,
+    or new-line characters.
+    Defaults to ``"`` (quote).
 
 ``stub-columns`` : integer
-    The number of table columns to use as stubs (row titles, on the
-    left).  Defaults to 0.
+    The number of table columns to use as stubs (row titles, on the left).
+    Defaults to 0.
+
+.. _`:url:`:
 
 ``url`` : string (whitespace removed)
     An Internet URL reference to a CSV data file.
@@ -941,8 +953,16 @@ The following options are recognized:
 
 and the common options class_ and name_.
 
-.. [#whitespace-delim] Whitespace delimiters are supported only for external
-   CSV files.
+.. [#char-or-uni] May be specified as a *Unicode character code*;
+   see the unicode_ directive for syntax details.
+
+.. [#tab-expansion] Note, that tabs can be used as separator only in
+   external files because all hard `tabs in rST sources are converted to
+   spaces`__.
+
+   __ restructuredtext.html#whitespace
+
+.. _escaping mechanism: restructuredtext.html#escaping-mechanism
 
 
 List Table
diff --git a/docutils/docs/user/config.txt b/docutils/docs/user/config.txt
index e6e7716e4..a6d5ed398 100644
--- a/docutils/docs/user/config.txt
+++ b/docutils/docs/user/config.txt
@@ -828,10 +828,12 @@ New in Docutils 0.9.
 tab_width
 ~~~~~~~~~
 
-Number of spaces for hard tab expansion.
+Number of spaces for `hard tab expansion`_.
 
 Default: 8.  Option: ``--tab-width``.
 
+.. _hard tab expansion: ../ref/rst/restructuredtext.html#whitespace
+
 trim_footnote_reference_space
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
diff --git a/docutils/docutils/parsers/rst/directives/__init__.py b/docutils/docutils/parsers/rst/directives/__init__.py
index 397558041..b27886352 100644
--- a/docutils/docutils/parsers/rst/directives/__init__.py
+++ b/docutils/docutils/parsers/rst/directives/__init__.py
@@ -334,7 +334,7 @@ def unicode_code(code):
 
 def single_char_or_unicode(argument):
     """
-    A single character is returned as-is.  Unicode characters codes are
+    A single character is returned as-is.  Unicode character codes are
     converted as in `unicode_code`.  (Directive option conversion function.)
     """
     char = unicode_code(argument)
diff --git a/docutils/docutils/parsers/rst/directives/tables.py b/docutils/docutils/parsers/rst/directives/tables.py
index 01db47629..8ba858a9a 100644
--- a/docutils/docutils/parsers/rst/directives/tables.py
+++ b/docutils/docutils/parsers/rst/directives/tables.py
@@ -53,6 +53,11 @@ class Table(Directive):
         return title, messages
 
     def process_header_option(self):
+        # Provisional
+        # * Will move to CSVTable in Docutils 0.21
+        #   as it calls `self.HeaderDialect()` only defined in CSVTable.
+        # * Will change to use the same CSV dialect as the body to get in line
+        #   with the specification in ref/rst/directives.txt in Docutils 0.21.
         source = self.state_machine.get_source(self.lineno - 1)
         table_head = []
         max_header_cols = 0
@@ -132,6 +137,11 @@ class Table(Directive):
 
 
 class RSTTable(Table):
+    """
+    Class for the `"table" directive`__ for formal tables using rST syntax.
+
+    __ https://docutils.sourceforge.io/docs/ref/rst/directives.html
+    """
 
     def run(self):
         if not self.content:
@@ -219,11 +229,14 @@ class CSVTable(Table):
             if 'escape' in options:
                 self.doublequote = False
                 self.escapechar = options['escape']
-            csv.Dialect.__init__(self)
+            super().__init__()
 
     class HeaderDialect(csv.Dialect):
 
-        """CSV dialect to use for the "header" option data."""
+        """CSV dialect used for the "header" option data.
+
+        Deprecated. Will be removed in Docutils 0.22.
+        """
 
         delimiter = ','
         quotechar = '"'
@@ -234,8 +247,18 @@ class CSVTable(Table):
         lineterminator = '\n'
         quoting = csv.QUOTE_MINIMAL
 
-    def check_requirements(self):
-        pass
+        def __init__(self):
+            warnings.warn('CSVTable.HeaderDialect will be removed '
+                          'in Docutils 0.22.',
+                          PendingDeprecationWarning, stacklevel=2)
+            super().__init__()
+
+    @staticmethod
+    def check_requirements():
+        warnings.warn('CSVTable.check_requirements()'
+                      ' is not required with Python 3'
+                      ' and will be removed in Docutils 0.22.',
+                      DeprecationWarning, stacklevel=2)
 
     def run(self):
         try:
@@ -247,7 +270,6 @@ class CSVTable(Table):
                     nodes.literal_block(self.block_text, self.block_text),
                     line=self.lineno)
                 return [warning]
-            self.check_requirements()
             title, messages = self.make_title()
             csv_data, source = self.get_csv_data()
             table_head, max_header_cols = self.process_header_option()