[backport-2.13] 📦 Integrate manpage builds into PEP 517 build backend (#80130)

This change includes backports of the PRs #79606, #80098, #80253, #80255 and #80262, with a number of backward compatibility changes that make it work and be testable in older CI environments.

This patch creates a thin wrapper around the `setuptools`' PEP 517 build backend in-tree. It features an ability to request generating the manpage files in-memory, in the process of building a source distribution. The build is happening in an isolated temporary directory. This toggle is implemented using the `config_settings` mechanism of PEP 517.
One must explicitly pass it a CLI option to the build front-end to trigger said behavior. The packagers are expected to use the following call:

    python -m build --config-setting=--build-manpages

This option has no effect on building wheels.

🧪 The change includes integration tests

This test runs building and re-building sdists and wheels with and without the `--build-manpages` config setting under the oldest-supported and new `setuptools` pinned.

It is intended to preserve the interoperability of the packaging setup across Python runtimes.

An extra smoke test also verifies that non PEP 517 interfaces remain functional.

📦 The sdist build-system is switched to pure setuptools upon the first build from Git checkout

This patch modifies the in-tree build backend to build sdists that swap
out pointers to it in the `pyproject.toml`'s `[build-system]` section.

The effect of this is that the first build from source (for example,
from a Git checkout) uses our PEP 517 in-tree build backend. But the
produced tarball has `build-backend` set to `setuptools.build_meta`
which is the native build backend of `setuptools`. So any following
builds from that sdist will skip using the in-tree build backend,
calling the setuptools' one.
The good news is that if the first build generated the manpages, they
will be included and won't go anywhere even though, a different build
system is in place.

Combined with #80253, this makes sure not to modify the current
source checkout on that first build.

Co-Authored-By: Matt Clay <matt@mystile.com>
Co-Authored-By: Matt Davis <6775756+nitzmahone@users.noreply.github.com>

(cherry picked from commit 56036013cd09db0f1b20402a69be44345ddafb22)
(cherry picked from commit 67bafafbc0d61751d2ec10b0aeccb6b01482a074)
(cherry picked from commit 888abf5d6e13880ba82d02c5f8a57ca1f90a992f)
(cherry picked from commit 7097df3eed979446830fc579613ffb9b7e7c19bf)
(cherry picked from commit eebfd71a6da396b011e664604c9991543205d780)

1 files changed, 2 insertions, 1 deletions

diff --git a/pyproject.toml b/pyproject.toml
index 38c5a47d62..4a583ee5a3 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,3 +1,4 @@
 [build-system]
 requires = ["setuptools >= 39.2.0", "wheel"]
-build-backend = "setuptools.build_meta"
+backend-path = ["packaging"]  # requires 'Pip>=20' or 'pep517>=0.6.0'
+build-backend = "pep517_backend.hooks"  # wraps `setuptools.build_meta`