Update userguide on `miscellaneous` and `extension`

This is a continuation of the update effort to de-emphasize `distutils`
and make the documentation more consistent.

The main targets of the changes are the files
`docs/userguide/miscellaneous` and `docs/userguide/extension`.

Changes:

- Extracted text about automatic resource extraction and the zip-safe flag
  from `userguide/miscellaneous` to `deprecated/resource_extraction` and
  `deprecated/zip_safe`.
  - These configuration parameters are commonly associated with
    ``eggs``/``easy_install``/``pkg_resources``, and therefore are
    obsolete. Leaving them around in the main parts of the documentation
    just confuses users.
  - The text in the new files were updated.
- Extracted text about additional metadata from
  `userguide/miscellaneous` into the existing `userguide/extension`
  document.
- Updated `userguide/extension` to better reflect the status of the
  setuptools project.
  The text was also changed to explain a little bit more what is the
  relationship between ``setuptools`` and ``distutils``.
- Removed `userguide/functionalities_rewrite`.
  This file was virtually empty and not bringing any extra value to the
  docs.

3 files changed, 117 insertions, 177 deletions

diff --git a/docs/userguide/extension.rst b/docs/userguide/extension.rst
index 21fb05b6..4ecfc3ee 100644
--- a/docs/userguide/extension.rst
+++ b/docs/userguide/extension.rst
@@ -1,60 +1,73 @@
 .. _Creating ``distutils`` Extensions:
 
-Creating ``distutils`` Extensions
-=================================
+Extending or Customizing Setuptools
+===================================
 
-It can be hard to add new commands or setup arguments to the distutils.  But
-the ``setuptools`` package makes it a bit easier, by allowing you to distribute
-a distutils extension as a separate project, and then have projects that need
-the extension just refer to it in their ``setup_requires`` argument.
+Setuptools design is based on the distutils_ package originally distributed
+as part of Python's standard library, effectively serving as its successor
+(as established in :pep:`632`).
 
-With ``setuptools``, your distutils extension projects can hook in new
+This means that ``setuptools`` strives to honour the extension mechanisms
+provided by ``distutils``, and allows developers to create third party packages
+that modify or augment the build process behaviour.
+
+A simple way of doing that is to hook in new or existing
 commands and ``setup()`` arguments just by defining "entry points".  These
 are mappings from command or argument names to a specification of where to
 import a handler from.  (See the section on :ref:`Dynamic Discovery of
 Services and Plugins` above for some more background on entry points.)
 
+The following sections describe the most common procedures for extending
+the ``distutils`` functionality used by ``setuptools``.
 
-Adding Commands
----------------
+Customizing Commands
+--------------------
 
-You can add new ``setup`` commands by defining entry points in the
-``distutils.commands`` group.  For example, if you wanted to add a ``foo``
-command, you might add something like this to your distutils extension
-project's setup script::
+Both ``setuptools`` and ``distutils`` are structured around the *command design
+pattern*. This means that each main action executed when building a
+distribution package (such as creating a :term:`sdist <Source Distribution (or "sdist")>`
+or :term:`wheel`) correspond to the implementation of a Python class.
 
-    setup(
-        # ...
-        entry_points={
-            "distutils.commands": [
-                "foo = mypackage.some_module:foo",
-            ],
-        },
-    )
+Originally in ``distutils``, these commands would correspond to actual CLI
+arguments that could be passed to the ``setup.py`` script to trigger a
+different aspect of the build. In ``setuptools``, however, these command
+objects are just a design abstraction that encapsulate logic and help to
+organise the code.
+
+You can overwrite exiting commands (or add new ones) by defining entry
+points in the ``distutils.commands`` group.  For example, if you wanted to add
+a ``foo`` command, you might add something like this to your project:
+
+.. code-block:: ini
+
+    # setup.cfg
+    ...
+    [options.entry_points]
+    distutils.commands =
+         foo = mypackage.some_module:foo
 
 (Assuming, of course, that the ``foo`` class in ``mypackage.some_module`` is
 a ``setuptools.Command`` subclass.)
 
 Once a project containing such entry points has been activated on ``sys.path``,
-(e.g. by running "install" or "develop" with a site-packages installation
-directory) the command(s) will be available to any ``setuptools``-based setup
-scripts.  It is not necessary to use the ``--command-packages`` option or
-to monkeypatch the ``distutils.command`` package to install your commands;
-``setuptools`` automatically adds a wrapper to the distutils to search for
-entry points in the active distributions on ``sys.path``.  In fact, this is
+(e.g. by running ``pip install``) the command(s) will be available to any
+``setuptools``-based project. In fact, this is
 how setuptools' own commands are installed: the setuptools project's setup
 script defines entry points for them!
 
-.. note::
-   When creating commands, and specially when defining custom ways of building
-   compiled extensions (for example via ``build_ext``), consider
-   handling exceptions such as ``CompileError``, ``LinkError``, ``LibError``,
-   among others.  These exceptions are available in the ``setuptools.errors``
-   module.
+The commands ``sdist``, ``build_py`` and ``build_ext`` are specially useful
+to customize ``setuptools`` builds. Note however that when overwriting existing
+commands you should be very careful to maintain API compatibility.
+Custom commands should try to replicate the same overall behaviour as the
+original classes, and when possible, even inherit from them.
 
+You should also consider handling exceptions such as ``CompileError``,
+``LinkError``, ``LibError``, among others.  These exceptions are available in
+the ``setuptools.errors`` module.
 
-Adding ``setup()`` Arguments
-----------------------------
+
+Adding Arguments
+----------------
 
 .. warning:: Adding arguments to setup is discouraged as such arguments
    are only supported through imperative execution and not supported through
@@ -64,19 +77,17 @@ Sometimes, your commands may need additional arguments to the ``setup()``
 call.  You can enable this by defining entry points in the
 ``distutils.setup_keywords`` group.  For example, if you wanted a ``setup()``
 argument called ``bar_baz``, you might add something like this to your
-distutils extension project's setup script::
+extension project:
 
-    setup(
-        # ...
-        entry_points={
-            "distutils.commands": [
-                "foo = mypackage.some_module:foo",
-            ],
-            "distutils.setup_keywords": [
-                "bar_baz = mypackage.some_module:validate_bar_baz",
-            ],
-        },
-    )
+.. code-block:: ini
+
+    # setup.cfg
+    ...
+    [options.entry_points]
+    distutils.commands =
+         foo = mypackage.some_module:foo
+    distutils.setup_keywords =
+        bar_baz = mypackage.some_module:validate_bar_baz
 
 The idea here is that the entry point defines a function that will be called
 to validate the ``setup()`` argument, if it's supplied.  The ``Distribution``
@@ -93,7 +104,7 @@ a non-None value.  Here's an example validation function::
 
 Your function should accept three arguments: the ``Distribution`` object,
 the attribute name, and the attribute value.  It should raise a
-``DistutilsSetupError`` (from the ``distutils.errors`` module) if the argument
+``SetupError`` (from the ``setuptools.errors`` module) if the argument
 is invalid.  Remember, your function will only be called with non-None values,
 and the default value of arguments defined this way is always None.  So, your
 commands should always be prepared for the possibility that the attribute will
@@ -101,15 +112,9 @@ be ``None`` when they access it later.
 
 If more than one active distribution defines an entry point for the same
 ``setup()`` argument, *all* of them will be called.  This allows multiple
-distutils extensions to define a common argument, as long as they agree on
+extensions to define a common argument, as long as they agree on
 what values of that argument are valid.
 
-Also note that as with commands, it is not necessary to subclass or monkeypatch
-the distutils ``Distribution`` class in order to add your arguments; it is
-sufficient to define the entry points in your extension, as long as any setup
-script using your extension lists your project in its ``setup_requires``
-argument.
-
 
 Customizing Distribution Options
 --------------------------------
@@ -130,22 +135,36 @@ plugin is encouraged to load the configuration/settings for their behavior
 independently.
 
 
+Defining Additional Metadata
+----------------------------
+
+Some extensible applications and frameworks may need to define their own kinds
+of metadata, which they can then access using the :mod:`importlib.metadata` APIs.
+Ordinarily, this is done by having plugin
+developers include additional files in their ``ProjectName.egg-info``
+directory.  However, since it can be tedious to create such files by hand, you
+may want to create an extension that will create the necessary files
+from arguments to ``setup()``, in much the same way that ``setuptools`` does
+for many of the ``setup()`` arguments it adds.  See the section below for more
+details.
+
+
 .. _Adding new EGG-INFO Files:
 
 Adding new EGG-INFO Files
--------------------------
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Some extensible applications or frameworks may want to allow third parties to
 develop plugins with application or framework-specific metadata included in
 the plugins' EGG-INFO directory, for easy access via the ``pkg_resources``
-metadata API.  The easiest way to allow this is to create a distutils extension
+metadata API.  The easiest way to allow this is to create an extension
 to be used from the plugin projects' setup scripts (via ``setup_requires``)
 that defines a new setup keyword, and then uses that data to write an EGG-INFO
 file when the ``egg_info`` command is run.
 
 The ``egg_info`` command looks for extension points in an ``egg_info.writers``
-group, and calls them to write the files.  Here's a simple example of a
-distutils extension defining a setup argument ``foo_bar``, which is a list of
+group, and calls them to write the files.  Here's a simple example of an
+extension defining a setup argument ``foo_bar``, which is a list of
 lines that will be written to ``foo_bar.txt`` in the EGG-INFO directory of any
 project that uses the argument::
 
@@ -161,6 +180,16 @@ project that uses the argument::
         },
     )
 
+.. code-block:: ini
+
+    # setup.cfg
+    ...
+    [options.entry_points]
+    distutils.setup_keywords =
+        foo_bar = setuptools.dist:assert_string_list
+    egg_info.writers =
+        foo_bar.txt = setuptools.command.egg_info:write_arg
+
 This simple example makes use of two utility functions defined by setuptools
 for its own use: a routine to validate that a setup keyword is a sequence of
 strings, and another one that looks up a setup argument and writes it to
@@ -179,11 +208,11 @@ write (e.g. ``foo_bar.txt``), and the actual full filename that should be
 written to.
 
 In general, writer functions should honor the command object's ``dry_run``
-setting when writing files, and use the ``distutils.log`` object to do any
-console output.  The easiest way to conform to this requirement is to use
+setting when writing files, and use ``logging`` to do any console output.
+The easiest way to conform to this requirement is to use
 the ``cmd`` object's ``write_file()``, ``delete_file()``, and
-``write_or_delete_file()`` methods exclusively for your file operations.  See
-those methods' docstrings for more details.
+``write_or_delete_file()`` methods exclusively for your file operations.
+See those methods' docstrings for more details.
 
 
 .. _Adding Support for Revision Control Systems:
@@ -212,13 +241,16 @@ called "foobar", you would write a function something like this:
     def find_files_for_foobar(dirname):
         ...  # loop to yield paths that start with `dirname`
 
-And you would register it in a setup script using something like this::
+And you would register it in a setup script using something like this:
+
+.. code-block:: ini
+
+    # setup.cfg
+    ...
 
-    entry_points={
-        "setuptools.file_finders": [
-            "foobar = my_foobar_module:find_files_for_foobar",
-        ]
-    }
+    [options.entry_points]
+    setuptools.file_finders =
+        foobar = my_foobar_module:find_files_for_foobar
 
 Then, anyone who wants to use your plugin can simply install it, and their
 local setuptools installation will be able to find the necessary files.
@@ -248,3 +280,18 @@ A few important points for writing revision control file finders:
   with the absence of needed programs (i.e., ones belonging to the revision
   control system itself.  It *may*, however, use ``distutils.log.warn()`` to
   inform the user of the missing program(s).
+
+
+.. _distutils: https://docs.python.org/3.9/library/distutils.html
+
+
+Final Remarks
+-------------
+
+* To use a ``setuptools`` plugin, your users will need to add your package as a
+  build requirement to their build-system configuration. Please check out our
+  guides on :doc:`/userguide/dependency_management` for more information.
+
+* Directly calling ``python setup.py ...`` is considered a **deprecated** practice.
+  You should not add new commands to ``setuptools`` expecting them to be run
+  via this interface.
diff --git a/docs/userguide/functionalities_rewrite.rst b/docs/userguide/functionalities_rewrite.rst
deleted file mode 100644
index d0997ca6..00000000
--- a/docs/userguide/functionalities_rewrite.rst
+++ /dev/null
@@ -1,9 +0,0 @@
-========================================================
-Using setuptools to package and distribute your project
-========================================================
-
-``setuptools`` offers a variety of functionalities that make it easy to
-build and distribute your python package. Here we provide an overview on
-the commonly used ones.
-
-
diff --git a/docs/userguide/miscellaneous.rst b/docs/userguide/miscellaneous.rst
index 5fd2f0a8..776f12f6 100644
--- a/docs/userguide/miscellaneous.rst
+++ b/docs/userguide/miscellaneous.rst
@@ -1,105 +1,7 @@
-.. _Automatic Resource Extraction:
-
-Automatic Resource Extraction
------------------------------
-
-If you are using tools that expect your resources to be "real" files, or your
-project includes non-extension native libraries or other files that your C
-extensions expect to be able to access, you may need to list those files in
-the ``eager_resources`` argument to ``setup()``, so that the files will be
-extracted together, whenever a C extension in the project is imported.
-
-This is especially important if your project includes shared libraries *other*
-than distutils-built C extensions, and those shared libraries use file
-extensions other than ``.dll``, ``.so``, or ``.dylib``, which are the
-extensions that setuptools 0.6a8 and higher automatically detects as shared
-libraries and adds to the ``native_libs.txt`` file for you.  Any shared
-libraries whose names do not end with one of those extensions should be listed
-as ``eager_resources``, because they need to be present in the filesystem when
-he C extensions that link to them are used.
-
-The ``pkg_resources`` runtime for compressed packages will automatically
-extract *all* C extensions and ``eager_resources`` at the same time, whenever
-*any* C extension or eager resource is requested via the ``resource_filename()``
-API.  (C extensions are imported using ``resource_filename()`` internally.)
-This ensures that C extensions will see all of the "real" files that they
-expect to see.
-
-Note also that you can list directory resource names in ``eager_resources`` as
-well, in which case the directory's contents (including subdirectories) will be
-extracted whenever any C extension or eager resource is requested.
-
-Please note that if you're not sure whether you need to use this argument, you
-don't!  It's really intended to support projects with lots of non-Python
-dependencies and as a last resort for crufty projects that can't otherwise
-handle being compressed.  If your package is pure Python, Python plus data
-files, or Python plus C, you really don't need this.  You've got to be using
-either C or an external program that needs "real" files in your project before
-there's any possibility of ``eager_resources`` being relevant to your project.
-
-Defining Additional Metadata
-----------------------------
-
-Some extensible applications and frameworks may need to define their own kinds
-of metadata to include in eggs, which they can then access using the
-``pkg_resources`` metadata APIs.  Ordinarily, this is done by having plugin
-developers include additional files in their ``ProjectName.egg-info``
-directory.  However, since it can be tedious to create such files by hand, you
-may want to create a distutils extension that will create the necessary files
-from arguments to ``setup()``, in much the same way that ``setuptools`` does
-for many of the ``setup()`` arguments it adds.  See the section below on
-:ref:`Creating ``distutils\`\` Extensions` for more details, especially the
-subsection on :ref:`Adding new EGG-INFO Files`.
-
-Setting the ``zip_safe`` flag
------------------------------
-
-For some use cases (such as bundling as part of a larger application), Python
-packages may be run directly from a zip file.
-Not all packages, however, are capable of running in compressed form, because
-they may expect to be able to access either source code or data files as
-normal operating system files.  So, ``setuptools`` can install your project
-as a zipfile or a directory, and its default choice is determined by the
-project's ``zip_safe`` flag.
-
-You can pass a True or False value for the ``zip_safe`` argument to the
-``setup()`` function, or you can omit it.  If you omit it, the ``bdist_egg``
-command will analyze your project's contents to see if it can detect any
-conditions that would prevent it from working in a zipfile.  It will output
-notices to the console about any such conditions that it finds.
-
-Currently, this analysis is extremely conservative: it will consider the
-project unsafe if it contains any C extensions or datafiles whatsoever.  This
-does *not* mean that the project can't or won't work as a zipfile!  It just
-means that the ``bdist_egg`` authors aren't yet comfortable asserting that
-the project *will* work.  If the project contains no C or data files, and does
-no ``__file__`` or ``__path__`` introspection or source code manipulation, then
-there is an extremely solid chance the project will work when installed as a
-zipfile.  (And if the project uses ``pkg_resources`` for all its data file
-access, then C extensions and other data files shouldn't be a problem at all.
-See the :ref:`Accessing Data Files at Runtime` section above for more information.)
-
-However, if ``bdist_egg`` can't be *sure* that your package will work, but
-you've checked over all the warnings it issued, and you are either satisfied it
-*will* work (or if you want to try it for yourself), then you should set
-``zip_safe`` to ``True`` in your ``setup()`` call.  If it turns out that it
-doesn't work, you can always change it to ``False``, which will force
-``setuptools`` to install your project as a directory rather than as a zipfile.
-
-In the future, as we gain more experience with different packages and become
-more satisfied with the robustness of the ``pkg_resources`` runtime, the
-"zip safety" analysis may become less conservative.  However, we strongly
-recommend that you determine for yourself whether your project functions
-correctly when installed as a zipfile, correct any problems if you can, and
-then make an explicit declaration of ``True`` or ``False`` for the ``zip_safe``
-flag, so that it will not be necessary for ``bdist_egg`` to try to guess
-whether your project can work as a zipfile.
-
-
 .. _Controlling files in the distribution:
 
 Controlling files in the distribution
--------------------------------------
+=====================================
 
 For the most common use cases, ``setuptools`` will automatically find out which
 files are necessary for distributing the package.