users_guide: Convert mkUserGuidePart generation to a Sphinx extension

This removes all dependencies the users guide had on `mkUserGuidePart`.
The generation of the flag reference table and the various pieces of the
man page is now entirely contained within the Spinx extension
`flags.py`. You can see the man page generation on the orphan page
https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/ghc.html

The extension works by collecting all of the meta-data attached to the
`ghc-flag` directives and then formatting and displaying it at
`flag-print` directives. There is a single printing directive that can
be customized with two options, what format to display (table, list, or
block of flags) and an optional category to limit the output to
(verbosity, warnings, codegen, etc.).

New display formats can be added by creating a function
`generate_flag_xxx` (where `xxx` is a description of the format) which
takes a list of flags and a category and returns a new `xxx`. Then just
add a reference in the dispatch table `handlers`. That display can now
be run by passing `:type: xxx` to the `flag-print` directive.

`flags.py` contains two maps of settings that can be adjusted. The first
is a canonical list of flag categories, and the second sets default
categories for files.

The only functionality that Sphinx could not replace was the
`what_glasgow_exts_does.gen.rst` file. `mkUserGuidePart` actually just
reads the list of flags from `compiler/main/DynFlags.hs` which Sphinx
cannot do. As the flag is deprecated, I added the list as a static file
which can be updated manually.

Additionally, this patch updates every single documented flag with the
data from `mkUserGuidePart` to generate the reference table.

Fixes #11654 and, incidentally, #12155.

Reviewers: austin, bgamari

Subscribers: rwbarton, thomie

GHC Trac Issues: #11654, #12155

Differential Revision: https://phabricator.haskell.org/D3839

1 files changed, 233 insertions, 36 deletions

diff --git a/docs/users_guide/flags.rst b/docs/users_guide/flags.rst
index 4064f46bdf..a3ec0f6f9e 100644
--- a/docs/users_guide/flags.rst
+++ b/docs/users_guide/flags.rst
@@ -12,63 +12,135 @@ Verbosity options
 
 More details in :ref:`options-help`
 
-.. include:: flags-verbosity.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.33\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.26\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: verbosity
 
 Alternative modes of operation
 ------------------------------
 
 More details in :ref:`modes`
 
-.. include:: flags-modes.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: modes
 
 Which phases to run
 -------------------
 
 More details in :ref:`options-order`
 
-.. include:: flags-phases.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: phases
 
 Redirecting output
 ------------------
 
 More details in :ref:`options-output`
 
-.. include:: flags-redirecting-output.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: redirect-output
 
 Keeping intermediate files
 --------------------------
 
 More details in :ref:`keeping-intermediates`
 
-.. include:: flags-keeping-intermediates.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: keep-intermediates
 
 Temporary files
 ---------------
 
 More details in :ref:`temp-files`
 
-.. include:: flags-temporary-files.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: temp-files
 
 Finding imports
 ---------------
 
 More details in :ref:`search-path`
 
-.. include:: flags-finding-imports.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: search-path
 
 Interface file options
 ----------------------
 
 More details in :ref:`hi-options`
 
-.. include:: flags-interface-files.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: interface-files
 
 Recompilation checking
 ----------------------
 
 More details in :ref:`recomp`
 
-.. include:: flags-recompilation-checking.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: recompilation
 
 .. _interactive-mode-options:
 
@@ -77,14 +149,30 @@ Interactive-mode options
 
 More details in :ref:`ghci-dot-files`
 
-.. include:: flags-interactive.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: interactive
 
 Packages
 --------
 
 More details in :ref:`packages`
 
-.. include:: flags-packages.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: packages
 
 Language options
 ----------------
@@ -94,14 +182,30 @@ Language options can be enabled either by a command-line option
 See :ref:`options-language`. Some options are enabled using ``-f*``
 flags.
 
-.. include:: flags-language.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.36\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.25\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: language
 
 Warnings
 --------
 
 More details in :ref:`options-sanity`
 
-.. include:: flags-warnings.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: warnings
 
 Optimisation levels
 -------------------
@@ -111,7 +215,15 @@ These options are described in more detail in :ref:`options-optimise`.
 See :ref:`options-f-compact` for a list of optimisations enabled on
 level 1 and level 2.
 
-.. include:: flags-optimization-levels.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: optimization-levels
 
 .. _options-f-compact:
 
@@ -123,90 +235,175 @@ flag is implied by ``-O`` then it is also implied by ``-O2`` (unless
 flag description explicitly says otherwise). If a flag is implied by
 ``-O0`` only then the flag is not implied by ``-O`` and ``-O2``.
 
-.. include:: flags-optimization.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: optimization
 
 Profiling options
 -----------------
 
 More details in :ref:`profiling`
 
-.. include:: flags-profiling.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: profiling
 
 Program coverage options
 ------------------------
 
 More details in :ref:`hpc`
 
-.. include:: flags-program-coverage.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: coverage
 
 C pre-processor options
 -----------------------
 
 More details in :ref:`c-pre-processor`
 
-.. include:: flags-cpp.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: cpp
 
 Code generation options
 -----------------------
 
 More details in :ref:`options-codegen`
 
-.. include:: flags-codegen.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: codegen
 
 Linking options
 ---------------
 
 More details in :ref:`options-linker`
 
-.. include:: flags-linking.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.35\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.44\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.10\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: linking
 
 Plugin options
 --------------
 
 More details in :ref:`compiler-plugins`
 
-.. include:: flags-plugin.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: plugins
 
 Replacing phases
 ----------------
 
 More details in :ref:`replacing-phases`
 
-.. include:: flags-phase-programs.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
 
-.. index::
-   single: -pgmL
-   single: -pgmP
-   single: -pgmc
-   single: -pgmlo
-   single: -pgmlc
-   single: -pgma
-   single: -pgml
-   single: -pgmdll
-   single: -pgmF
+.. flag-print::
+    :type: table
+    :category: phase-programs
 
 Forcing options to particular phases
 ------------------------------------
 
 More details in :ref:`forcing-options-through`
 
-.. include:: flags-phase-specific.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: phase-options
 
 Platform-specific options
 -------------------------
 
 More details in :ref:`options-platform`
 
-.. include:: flags-platform-specific.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.30\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.31\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.29\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: platform-options
 
 Compiler debugging options
 --------------------------
 
 More details in :ref:`options-debugging`
 
-.. include:: flags-compiler-debugging.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.35\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.44\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.10\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: debugging
 
 Miscellaneous compiler options
 ------------------------------
 
-.. include:: flags-misc.gen.rst
+.. tabularcolumns::
+    | p{\dimexpr 0.35\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.44\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.11\textwidth-2\tabcolsep} |
+      p{\dimexpr 0.10\textwidth-2\tabcolsep} |
+
+.. flag-print::
+    :type: table
+    :category: misc