[stable-2.12] ansible-test - Managed venv fixes. (#77100)

* ansible-test - Remove cap on cryptography version.

(cherry picked from commit 00a2b7788e0ef7067dc58c9a13659376e38cd352)

* ansible-test - Fix consistency of managed venvs. (#77028)

(cherry picked from commit 68fb3bf90efa3a722ba5ab7d66b1b22adc73198c)

* Avoid system-site-packages in AZP coverage venvs.

The use of `--venv-system-site-packages` was an optimization to use the `coverage` package pre-installed in the AZP test container.
However, now that the venv is bootstrapped by ansible-test that optimization no longer makes sense, since other downloads are already taking place.

(cherry picked from commit 177336a9d33a395cf77098a65f59e0fc449ecaad)

* ansible-test - Clean up venv code.

(cherry picked from commit addb9baec20fcede2a2494069c01c73b3d2e4fc9)

* Adjust virtualenv version for Python 2.6.

* Adjust bootstrap URL to work with Python 2.6.

* Enable PyPI proxy for generic tests.

This is needed to support ansible-test integration tests on Python 2.6.

* Freeze plugin import sanity test requirements.

Based on https://github.com/ansible/ansible/pull/76308

16 files changed, 253 insertions, 64 deletions

diff --git a/.azure-pipelines/scripts/aggregate-coverage.sh b/.azure-pipelines/scripts/aggregate-coverage.sh
index 1ccfcf2073..cb2f017794 100755
--- a/.azure-pipelines/scripts/aggregate-coverage.sh
+++ b/.azure-pipelines/scripts/aggregate-coverage.sh
@@ -9,7 +9,7 @@ PATH="${PWD}/bin:${PATH}"
 
 mkdir "${agent_temp_directory}/coverage/"
 
-options=(--venv --venv-system-site-packages --color -v)
+options=(--venv --color -v)
 
 ansible-test coverage combine --group-by command --export "${agent_temp_directory}/coverage/" "${options[@]}"
 
diff --git a/.azure-pipelines/scripts/report-coverage.sh b/.azure-pipelines/scripts/report-coverage.sh
index 297169d9f9..4db905eae2 100755
--- a/.azure-pipelines/scripts/report-coverage.sh
+++ b/.azure-pipelines/scripts/report-coverage.sh
@@ -14,4 +14,4 @@ fi
 
 # Generate stubs using docker (if supported) otherwise fall back to using a virtual environment instead.
 # The use of docker is required when Powershell code is present, but Ansible 2.12 was the first version to support --docker with coverage.
-ansible-test coverage xml --group-by command --stub --docker --color -v || ansible-test coverage xml --group-by command --stub --venv --venv-system-site-packages --color -v
+ansible-test coverage xml --group-by command --stub --docker --color -v || ansible-test coverage xml --group-by command --stub --venv --color -v
diff --git a/changelogs/fragments/ansible-test-managed-venv.yml b/changelogs/fragments/ansible-test-managed-venv.yml
new file mode 100644
index 0000000000..ce7f51b283
--- /dev/null
+++ b/changelogs/fragments/ansible-test-managed-venv.yml
@@ -0,0 +1,15 @@
+bugfixes:
+  - ansible-test - Virtual environments managed by ansible-test now use consistent versions of ``pip``, ``setuptools`` and ``wheel``.
+                   This avoids issues with virtual environments containing outdated or dysfunctional versions of these tools.
+                   The initial bootstrapping of ``pip`` is done by ansible-test from an HTTPS endpoint instead of creating the virtual environment with it already present.
+  - ansible-test - Sanity tests run with the ``--requirements` option for Python 2.x now install ``virtualenv`` when it is missing or too old.
+                   Previously it was only installed if missing.
+                   Version 16.7.12 is now installed instead of the latest version on Python 2.7.
+  - ansible-test - All virtual environments managed by ansible-test are marked as usable after being bootstrapped, to avoid errors caused by use of incomplete environments.
+                   Previously this was only done for sanity tests.
+                   Existing environments from previous versions of ansible-test will be recreated on demand due to lacking the new marker.
+minor_changes:
+  - ansible-test - The ``pip`` and ``wheel`` packages are removed from all sanity test virtual environments after installation completes to reduce their size.
+                   Previously they were only removed from the environments used for the ``import`` sanity test.
+  - ansible-test - The hash for all managed sanity test virtual environments has changed.
+                   Containers that include ``ansible-test sanity --prime-venvs`` will need to be rebuilt to continue using primed virtual environments.
diff --git a/changelogs/fragments/ansible-test-pyopenssl.yaml b/changelogs/fragments/ansible-test-pyopenssl.yaml
index f372679118..c6d93f3be9 100644
--- a/changelogs/fragments/ansible-test-pyopenssl.yaml
+++ b/changelogs/fragments/ansible-test-pyopenssl.yaml
@@ -1,2 +1,2 @@
-bugfixes:
-  - ansible-test - When installing ``cryptography < 3.4`` also install ``pyopenssl < 22`` to avoid conflicts (except for sanity tests).
+minor_changes:
+  - ansible-test - Installation of ``cryptography`` is no longer version constrained when ``openssl`` 1.1.0 or later is installed.
diff --git a/changelogs/fragments/ansible-test-sanity-requirements-update.yml b/changelogs/fragments/ansible-test-sanity-requirements-update.yml
new file mode 100644
index 0000000000..5dff4a9c4f
--- /dev/null
+++ b/changelogs/fragments/ansible-test-sanity-requirements-update.yml
@@ -0,0 +1,2 @@
+minor_changes:
+  - ansible-test - Requirements for the plugin import test are now frozen.
diff --git a/test/lib/ansible_test/_data/requirements/sanity.import.plugin.txt b/test/lib/ansible_test/_data/requirements/sanity.import.plugin.txt
new file mode 100644
index 0000000000..76d16725b2
--- /dev/null
+++ b/test/lib/ansible_test/_data/requirements/sanity.import.plugin.txt
@@ -0,0 +1,12 @@
+jinja2 == 3.0.1
+PyYAML == 5.4.1
+cryptography == 3.3.2
+packaging == 21.0
+resolvelib == 0.5.4
+
+# dependencies
+MarkupSafe == 2.0.1
+cffi == 1.15.0
+pycparser == 2.20
+pyparsing == 2.4.7
+six == 1.16.0
diff --git a/test/lib/ansible_test/_internal/commands/sanity/__init__.py b/test/lib/ansible_test/_internal/commands/sanity/__init__.py
index aa7ffff197..8c1340f2fc 100644
--- a/test/lib/ansible_test/_internal/commands/sanity/__init__.py
+++ b/test/lib/ansible_test/_internal/commands/sanity/__init__.py
@@ -242,7 +242,7 @@ def command_sanity(args):  # type: (SanityConfig) -> None
                     elif isinstance(test, SanitySingleVersion):
                         # single version sanity tests use the controller python
                         test_profile = host_state.controller_profile
-                        virtualenv_python = create_sanity_virtualenv(args, test_profile.python, test.name, context=test.name)
+                        virtualenv_python = create_sanity_virtualenv(args, test_profile.python, test.name)
 
                         if virtualenv_python:
                             virtualenv_yaml = check_sanity_virtualenv_yaml(virtualenv_python)
@@ -1077,10 +1077,8 @@ def create_sanity_virtualenv(
         args,  # type: SanityConfig
         python,  # type: PythonConfig
         name,  # type: str
-        ansible=False,  # type: bool
         coverage=False,  # type: bool
         minimize=False,  # type: bool
-        context=None,  # type: t.Optional[str]
 ):  # type: (...) -> t.Optional[VirtualPythonConfig]
     """Return an existing sanity virtual environment matching the requested parameters or create a new one."""
     commands = collect_requirements(  # create_sanity_virtualenv()
@@ -1088,14 +1086,11 @@ def create_sanity_virtualenv(
         controller=True,
         virtualenv=False,
         command=None,
-        # used by import tests
-        ansible=ansible,
-        cryptography=ansible,
+        ansible=False,
+        cryptography=False,
         coverage=coverage,
         minimize=minimize,
-        # used by non-import tests
-        sanity=context,
-        sanity_cryptography=ansible,
+        sanity=name,
     )
 
     if commands:
@@ -1130,6 +1125,7 @@ def create_sanity_virtualenv(
 
         write_text_file(meta_install, virtualenv_install)
 
+        # false positive: pylint: disable=no-member
         if any(isinstance(command, PipInstall) and command.has_package('pyyaml') for command in commands):
             virtualenv_yaml = yamlcheck(virtualenv_python)
         else:
diff --git a/test/lib/ansible_test/_internal/commands/sanity/import.py b/test/lib/ansible_test/_internal/commands/sanity/import.py
index 19a67b4b29..aa0239d522 100644
--- a/test/lib/ansible_test/_internal/commands/sanity/import.py
+++ b/test/lib/ansible_test/_internal/commands/sanity/import.py
@@ -73,6 +73,10 @@ from ...host_configs import (
     PythonConfig,
 )
 
+from ...venv import (
+    get_virtualenv_version,
+)
+
 
 def _get_module_test(module_restrictions):  # type: (bool) -> t.Callable[[str], bool]
     """Create a predicate which tests whether a path can be used by modules or not."""
@@ -100,9 +104,10 @@ class ImportTest(SanityMultipleVersion):
 
         paths = [target.path for target in targets.include]
 
-        if python.version.startswith('2.'):
+        if python.version.startswith('2.') and (get_virtualenv_version(args, python.path) or (0,)) < (13,):
             # hack to make sure that virtualenv is available under Python 2.x
             # on Python 3.x we can use the built-in venv
+            # version 13+ is required to use the `--no-wheel` option
             try:
                 install_requirements(args, python, virtualenv=True, controller=False)  # sanity (import)
             except PipUnavailableError as ex:
@@ -112,9 +117,9 @@ class ImportTest(SanityMultipleVersion):
 
         messages = []
 
-        for import_type, test, controller in (
-                ('module', _get_module_test(True), False),
-                ('plugin', _get_module_test(False), True),
+        for import_type, test in (
+                ('module', _get_module_test(True)),
+                ('plugin', _get_module_test(False)),
         ):
             if import_type == 'plugin' and python.version in REMOTE_ONLY_PYTHON_VERSIONS:
                 continue
@@ -124,7 +129,7 @@ class ImportTest(SanityMultipleVersion):
             if not data and not args.prime_venvs:
                 continue
 
-            virtualenv_python = create_sanity_virtualenv(args, python, f'{self.name}.{import_type}', ansible=controller, coverage=args.coverage, minimize=True)
+            virtualenv_python = create_sanity_virtualenv(args, python, f'{self.name}.{import_type}', coverage=args.coverage, minimize=True)
 
             if not virtualenv_python:
                 display.warning(f'Skipping sanity test "{self.name}" on Python {python.version} due to missing virtual environment support.')
@@ -143,7 +148,7 @@ class ImportTest(SanityMultipleVersion):
             )
 
             if data_context().content.collection:
-                external_python = create_sanity_virtualenv(args, args.controller_python, self.name, context=self.name)
+                external_python = create_sanity_virtualenv(args, args.controller_python, self.name)
 
                 env.update(
                     SANITY_COLLECTION_FULL_NAME=data_context().content.collection.full_name,
diff --git a/test/lib/ansible_test/_internal/host_profiles.py b/test/lib/ansible_test/_internal/host_profiles.py
index 0a08d68f18..e3aeeeebbc 100644
--- a/test/lib/ansible_test/_internal/host_profiles.py
+++ b/test/lib/ansible_test/_internal/host_profiles.py
@@ -209,11 +209,7 @@ class PosixProfile(HostProfile[TPosixConfig], metaclass=abc.ABCMeta):
             python = self.config.python
 
             if isinstance(python, VirtualPythonConfig):
-                python = VirtualPythonConfig(
-                    version=python.version,
-                    system_site_packages=python.system_site_packages,
-                    path=os.path.join(get_virtual_python(self.args, python), 'bin', 'python'),
-                )
+                python = get_virtual_python(self.args, python)
 
             self.state['python'] = python
 
diff --git a/test/lib/ansible_test/_internal/python_requirements.py b/test/lib/ansible_test/_internal/python_requirements.py
index 3a95d23222..aaaf44b8b3 100644
--- a/test/lib/ansible_test/_internal/python_requirements.py
+++ b/test/lib/ansible_test/_internal/python_requirements.py
@@ -109,6 +109,13 @@ class PipVersion(PipCommand):
     """Details required to get the pip version."""
 
 
+@dataclasses.dataclass(frozen=True)
+class PipBootstrap(PipCommand):
+    """Details required to bootstrap pip."""
+    pip_version: str
+    packages: t.List[str]
+
+
 # Entry Points
 
 
@@ -168,10 +175,25 @@ def install_requirements(
 
     run_pip(args, python, commands, connection)
 
+    # false positive: pylint: disable=no-member
     if any(isinstance(command, PipInstall) and command.has_package('pyyaml') for command in commands):
         check_pyyaml(python)
 
 
+def collect_bootstrap(python):  # type: (PythonConfig) -> t.List[PipCommand]
+    """Return the details necessary to bootstrap pip into an empty virtual environment."""
+    infrastructure_packages = get_venv_packages(python)
+    pip_version = infrastructure_packages['pip']
+    packages = [f'{name}=={version}' for name, version in infrastructure_packages.items()]
+
+    bootstrap = PipBootstrap(
+        pip_version=pip_version,
+        packages=packages,
+    )
+
+    return [bootstrap]
+
+
 def collect_requirements(
         python,  # type: PythonConfig
         controller,  # type: bool
@@ -182,19 +204,21 @@ def collect_requirements(
         minimize,  # type: bool
         command,  # type: t.Optional[str]
         sanity,  # type: t.Optional[str]
-        sanity_cryptography=False,  # type: bool
 ):  # type: (...) -> t.List[PipCommand]
     """Collect requirements for the given Python using the specified arguments."""
     commands = []  # type: t.List[PipCommand]
 
     if virtualenv:
-        commands.extend(collect_package_install(packages=['virtualenv']))
+        # sanity tests on Python 2.x install virtualenv when it is too old or is not already installed and the `--requirements` option is given
+        # the last version of virtualenv with no dependencies is used to minimize the changes made outside a virtual environment
+        virtualenv_version = '15.2.0' if python.version == '2.6' else '16.7.12'
+        commands.extend(collect_package_install(packages=[f'virtualenv=={virtualenv_version}'], constraints=False))
 
     if coverage:
         commands.extend(collect_package_install(packages=[f'coverage=={COVERAGE_REQUIRED_VERSION}'], constraints=False))
 
     if cryptography:
-        commands.extend(collect_package_install(packages=get_cryptography_requirements(python, sanity_cryptography)))
+        commands.extend(collect_package_install(packages=get_cryptography_requirements(python)))
 
     if ansible or command:
         commands.extend(collect_general_install(command, ansible))
@@ -208,15 +232,20 @@ def collect_requirements(
     if command in ('integration', 'windows-integration', 'network-integration'):
         commands.extend(collect_integration_install(command, controller))
 
-    if minimize:
-        # In some environments pkg_resources is installed as a separate pip package which needs to be removed.
-        # For example, using Python 3.8 on Ubuntu 18.04 a virtualenv is created with only pip and setuptools.
-        # However, a venv is created with an additional pkg-resources package which is independent of setuptools.
-        # Making sure pkg-resources is removed preserves the import test consistency between venv and virtualenv.
-        # Additionally, in the above example, the pyparsing package vendored with pkg-resources is out-of-date and generates deprecation warnings.
-        # Thus it is important to remove pkg-resources to prevent system installed packages from generating deprecation warnings.
-        commands.extend(collect_uninstall(packages=['pkg-resources'], ignore_errors=True))
-        commands.extend(collect_uninstall(packages=['setuptools', 'pip']))
+    if (sanity or minimize) and any(isinstance(command, PipInstall) for command in commands):
+        # bootstrap the managed virtual environment, which will have been created without any installed packages
+        # sanity tests which install no packages skip this step
+        commands = collect_bootstrap(python) + commands
+
+        # most infrastructure packages can be removed from sanity test virtual environments after they've been created
+        # removing them reduces the size of environments cached in containers
+        uninstall_packages = list(get_venv_packages(python))
+
+        if not minimize:
+            # installed packages may have run-time dependencies on setuptools
+            uninstall_packages.remove('setuptools')
+
+        commands.extend(collect_uninstall(packages=uninstall_packages))
 
     return commands
 
@@ -378,6 +407,46 @@ def collect_uninstall(packages, ignore_errors=False):  # type: (t.List[str], boo
 # Support
 
 
+def get_venv_packages(python):  # type: (PythonConfig) -> t.Dict[str, str]
+    """Return a dictionary of Python packages needed for a consistent virtual environment specific to the given Python version."""
+
+    # NOTE: This same information is needed for building the base-test-container image.
+    #       See: https://github.com/ansible/base-test-container/blob/main/files/installer.py
+
+    default_packages = dict(
+        pip='21.3.1',
+        setuptools='60.8.2',
+        wheel='0.37.1',
+    )
+
+    override_packages = {
+        '2.6': dict(
+            pip='9.0.3',  # 10.0 requires Python 2.7+
+            setuptools='36.8.0',  # 37.0.0 requires Python 2.7+
+            wheel='0.29.0',  # 0.30.0 requires Python 2.7+
+        ),
+        '2.7': dict(
+            pip='20.3.4',  # 21.0 requires Python 3.6+
+            setuptools='44.1.1',  # 45.0.0 requires Python 3.5+
+            wheel=None,
+        ),
+        '3.5': dict(
+            pip='20.3.4',  # 21.0 requires Python 3.6+
+            setuptools='50.3.2',  # 51.0.0 requires Python 3.6+
+            wheel=None,
+        ),
+        '3.6': dict(
+            pip='21.3.1',  # 22.0 requires Python 3.7+
+            setuptools='59.6.0',  # 59.7.0 requires Python 3.7+
+            wheel=None,
+        ),
+    }
+
+    packages = {name: version or default_packages[name] for name, version in override_packages.get(python.version, default_packages).items()}
+
+    return packages
+
+
 def requirements_allowed(args, controller):  # type: (EnvironmentConfig, bool) -> bool
     """
     Return True if requirements can be installed, otherwise return False.
@@ -439,7 +508,7 @@ def is_cryptography_available(python):  # type: (str) -> bool
     return True
 
 
-def get_cryptography_requirements(python, sanity):  # type: (PythonConfig, bool) -> t.List[str]
+def get_cryptography_requirements(python):  # type: (PythonConfig) -> t.List[str]
     """
     Return the correct cryptography and pyopenssl requirements for the given python version.
     The version of cryptography installed depends on the python version and openssl version.
@@ -453,17 +522,11 @@ def get_cryptography_requirements(python, sanity):  # type: (PythonConfig, bool)
         # pyopenssl 20.0.0 requires cryptography 3.2 or later
         pyopenssl = 'pyopenssl < 20.0.0'
     else:
-        if sanity:
-            # cryptography 3.4+ builds require a working rust toolchain
-            cryptography = 'cryptography < 3.4'
-            # pyopenssl not required, don't install it
-            pyopenssl = ''
-        else:
-            # cryptography 3.4+ fails to install on many systems
-            # this is a temporary work-around until a more permanent solution is available
-            cryptography = 'cryptography < 3.4'
-            # pyopenssl 20.0.0 requires cryptography 35 or later
-            pyopenssl = 'pyopenssl < 22.0.0'
+        # cryptography 3.4+ builds require a working rust toolchain
+        # systems bootstrapped using ansible-core-ci can access additional wheels through the spare-tire package index
+        cryptography = 'cryptography'
+        # any future installation of pyopenssl is free to use any compatible version of cryptography
+        pyopenssl = ''
 
     requirements = [
         cryptography,
diff --git a/test/lib/ansible_test/_internal/venv.py b/test/lib/ansible_test/_internal/venv.py
index 2cfd978dd4..cf436775bd 100644
--- a/test/lib/ansible_test/_internal/venv.py
+++ b/test/lib/ansible_test/_internal/venv.py
@@ -3,6 +3,7 @@ from __future__ import annotations
 
 import json
 import os
+import pathlib
 import sys
 import typing as t
 
@@ -31,11 +32,16 @@ from .host_configs import (
     PythonConfig,
 )
 
+from .python_requirements import (
+    collect_bootstrap,
+    run_pip,
+)
+
 
 def get_virtual_python(
         args,  # type: EnvironmentConfig
         python,  # type: VirtualPythonConfig
-):
+):  # type: (...) -> VirtualPythonConfig
     """Create a virtual environment for the given Python and return the path to its root."""
     if python.system_site_packages:
         suffix = '-ssp'
@@ -43,24 +49,40 @@ def get_virtual_python(
         suffix = ''
 
     virtual_environment_path = os.path.join(ResultType.TMP.path, 'delegation', f'python{python.version}{suffix}')
+    virtual_environment_marker = os.path.join(virtual_environment_path, 'marker.txt')
+
+    virtual_environment_python = VirtualPythonConfig(
+        version=python.version,
+        path=os.path.join(virtual_environment_path, 'bin', 'python'),
+        system_site_packages=python.system_site_packages,
+    )
+
+    if os.path.exists(virtual_environment_marker):
+        display.info('Using existing Python %s virtual environment: %s' % (python.version, virtual_environment_path), verbosity=1)
+    else:
+        # a virtualenv without a marker is assumed to have been partially created
+        remove_tree(virtual_environment_path)
 
-    if not create_virtual_environment(args, python, virtual_environment_path, python.system_site_packages):
-        raise ApplicationError(f'Python {python.version} does not provide virtual environment support.')
+        if not create_virtual_environment(args, python, virtual_environment_path, python.system_site_packages):
+            raise ApplicationError(f'Python {python.version} does not provide virtual environment support.')
 
-    return virtual_environment_path
+        commands = collect_bootstrap(virtual_environment_python)
+
+        run_pip(args, virtual_environment_python, commands, None)  # get_virtual_python()
+
+    # touch the marker to keep track of when the virtualenv was last used
+    pathlib.Path(virtual_environment_marker).touch()
+
+    return virtual_environment_python
 
 
 def create_virtual_environment(args,  # type: EnvironmentConfig
                                python,  # type: PythonConfig
                                path,  # type: str
                                system_site_packages=False,  # type: bool
-                               pip=True,  # type: bool
+                               pip=False,  # type: bool
                                ):  # type: (...) -> bool
     """Create a virtual environment using venv or virtualenv for the requested Python version."""
-    if os.path.isdir(path):
-        display.info('Using existing Python %s virtual environment: %s' % (python.version, path), verbosity=1)
-        return True
-
     if not os.path.exists(python.path):
         # the requested python version could not be found
         return False
@@ -207,6 +229,9 @@ def run_virtualenv(args,  # type: EnvironmentConfig
 
     if not pip:
         cmd.append('--no-pip')
+        # these options provide consistency with venv, which does not install them without pip
+        cmd.append('--no-setuptools')
+        cmd.append('--no-wheel')
 
     cmd.append(path)
 
diff --git a/test/lib/ansible_test/_util/target/setup/bootstrap.sh b/test/lib/ansible_test/_util/target/setup/bootstrap.sh
index b79d28fb50..53e2ca7177 100644
--- a/test/lib/ansible_test/_util/target/setup/bootstrap.sh
+++ b/test/lib/ansible_test/_util/target/setup/bootstrap.sh
@@ -183,6 +183,13 @@ bootstrap_remote_freebsd()
         sed -i '' 's/^# *PermitRootLogin.*$/PermitRootLogin yes/;' /etc/ssh/sshd_config
         service sshd restart
     fi
+
+    # make additional wheels available for packages which lack them for this platform
+    echo "# generated by ansible-test
+[global]
+extra-index-url = https://spare-tire.testing.ansible.com/simple/
+prefer-binary = yes
+" > /etc/pip.conf
 }
 
 bootstrap_remote_macos()
diff --git a/test/lib/ansible_test/_util/target/setup/quiet_pip.py b/test/lib/ansible_test/_util/target/setup/quiet_pip.py
index 99f09649bd..fc65c88b4c 100644
--- a/test/lib/ansible_test/_util/target/setup/quiet_pip.py
+++ b/test/lib/ansible_test/_util/target/setup/quiet_pip.py
@@ -3,6 +3,7 @@ from __future__ import (absolute_import, division, print_function)
 __metaclass__ = type
 
 import logging
+import os
 import re
 import runpy
 import sys
@@ -70,8 +71,16 @@ def main():
         #   Python 2.7 cannot use the -W option to match warning text after a colon. This makes it impossible to match specific warning messages.
         warnings.filterwarnings('ignore', message_filter)
 
+    get_pip = os.environ.get('GET_PIP')
+
     try:
-        runpy.run_module('pip.__main__', run_name='__main__', alter_sys=True)
+        if get_pip:
+            directory, filename = os.path.split(get_pip)
+            module = os.path.splitext(filename)[0]
+            sys.path.insert(0, directory)
+            runpy.run_module(module, run_name='__main__', alter_sys=True)
+        else:
+            runpy.run_module('pip.__main__', run_name='__main__', alter_sys=True)
     except ImportError as ex:
         print('pip is unavailable: %s' % ex)
         sys.exit(1)
diff --git a/test/lib/ansible_test/_util/target/setup/requirements.py b/test/lib/ansible_test/_util/target/setup/requirements.py
index 50ca7c6427..f460c5c541 100644
--- a/test/lib/ansible_test/_util/target/setup/requirements.py
+++ b/test/lib/ansible_test/_util/target/setup/requirements.py
@@ -38,6 +38,11 @@ except ImportError:
     # noinspection PyProtectedMember
     from pipes import quote as cmd_quote
 
+try:
+    from urllib.request import urlopen
+except ImportError:
+    from urllib import urlopen
+
 ENCODING = 'utf-8'
 PAYLOAD = b'{payload}'  # base-64 encoded JSON payload which will be populated before this script is executed
 
@@ -70,6 +75,38 @@ def main():  # type: () -> None
                 sys.exit(1)
 
 
+# noinspection PyUnusedLocal
+def bootstrap(pip, options):  # type: (str, t.Dict[str, t.Any]) -> None
+    """Bootstrap pip and related packages in an empty virtual environment."""
+    pip_version = options['pip_version']
+    packages = options['packages']
+
+    url = 'https://ansible-ci-files.s3.amazonaws.com/ansible-test/get-pip-%s.py' % pip_version
+    cache_path = os.path.expanduser('~/.ansible/test/cache/get_pip_%s.py' % pip_version.replace(".", "_"))
+    temp_path = cache_path + '.download'
+
+    if os.path.exists(cache_path):
+        log('Using cached pip %s bootstrap script: %s' % (pip_version, cache_path))
+    else:
+        log('Downloading pip %s bootstrap script: %s' % (pip_version, url))
+
+        make_dirs(os.path.dirname(cache_path))
+        download_file(url, temp_path)
+        shutil.move(temp_path, cache_path)
+
+        log('Cached pip %s bootstrap script: %s' % (pip_version, cache_path))
+
+    env = common_pip_environment()
+    env.update(GET_PIP=cache_path)
+
+    options = common_pip_options()
+    options.extend(packages)
+
+    command = [sys.executable, pip] + options
+
+    execute_command(command, env=env)
+
+
 def install(pip, options):  # type: (str, t.Dict[str, t.Any]) -> None
     """Perform a pip install."""
     requirements = options['requirements']
@@ -92,7 +129,9 @@ def install(pip, options):  # type: (str, t.Dict[str, t.Any]) -> None
 
         command = [sys.executable, pip, 'install'] + options
 
-        execute_command(command, tempdir)
+        env = common_pip_environment()
+
+        execute_command(command, env=env, cwd=tempdir)
     finally:
         remove_tree(tempdir)
 
@@ -107,8 +146,10 @@ def uninstall(pip, options):  # type: (str, t.Dict[str, t.Any]) -> None
 
     command = [sys.executable, pip, 'uninstall', '-y'] + options
 
+    env = common_pip_environment()
+
     try:
-        execute_command(command, capture=True)
+        execute_command(command, env=env, capture=True)
     except SubprocessError:
         if not ignore_errors:
             raise
@@ -123,7 +164,16 @@ def version(pip, options):  # type: (str, t.Dict[str, t.Any]) -> None
 
     command = [sys.executable, pip, '-V'] + options
 
-    execute_command(command, capture=True)
+    env = common_pip_environment()
+
+    execute_command(command, env=env, capture=True)
+
+
+def common_pip_environment():  # type: () -> t.Dict[str, str]
+    """Return common environment variables used to run pip."""
+    env = os.environ.copy()
+
+    return env
 
 
 def common_pip_options():  # type: () -> t.List[str]
@@ -143,6 +193,13 @@ def devnull():  # type: () -> t.IO[bytes]
     return devnull.file
 
 
+def download_file(url, path):  # type: (str, str) -> None
+    """Download the given URL to the specified file path."""
+    with open(to_bytes(path), 'wb') as saved_file:
+        download = urlopen(url)
+        shutil.copyfileobj(download, saved_file)
+
+
 class ApplicationError(Exception):
     """Base class for application exceptions."""
 
@@ -170,7 +227,7 @@ def log(message, verbosity=0):  # type: (str, int) -> None
     CONSOLE.flush()
 
 
-def execute_command(cmd, cwd=None, capture=False):  # type: (t.List[str], t.Optional[str], bool) -> None
+def execute_command(cmd, cwd=None, capture=False, env=None):  # type: (t.List[str], t.Optional[str], bool, t.Optional[t.Dict[str, str]]) -> None
     """Execute the specified command."""
     log('Execute command: %s' % ' '.join(cmd_quote(c) for c in cmd), verbosity=1)
 
@@ -183,7 +240,8 @@ def execute_command(cmd, cwd=None, capture=False):  # type: (t.List[str], t.Opti
         stdout = None
         stderr = None
 
-    process = subprocess.Popen(cmd_bytes, cwd=to_optional_bytes(cwd), stdin=devnull(), stdout=stdout, stderr=stderr)  # pylint: disable=consider-using-with
+    cwd_bytes = to_optional_bytes(cwd)
+    process = subprocess.Popen(cmd_bytes, cwd=cwd_bytes, stdin=devnull(), stdout=stdout, stderr=stderr, env=env)  # pylint: disable=consider-using-with
     stdout_bytes, stderr_bytes = process.communicate()
     stdout_text = to_optional_text(stdout_bytes) or u''
     stderr_text = to_optional_text(stderr_bytes) or u''
diff --git a/test/sanity/ignore.txt b/test/sanity/ignore.txt
index 5dde8ba17b..058331e1d3 100644
--- a/test/sanity/ignore.txt
+++ b/test/sanity/ignore.txt
@@ -184,6 +184,7 @@ test/integration/targets/win_script/files/test_script_with_splatting.ps1 pslint:
 test/integration/targets/windows-minimal/library/win_ping_syntax_error.ps1 pslint!skip
 test/lib/ansible_test/_data/requirements/sanity.pslint.ps1 pslint:PSCustomUseLiteralPath # Uses wildcards on purpose
 test/lib/ansible_test/_util/target/setup/ConfigureRemotingForAnsible.ps1 pslint:PSCustomUseLiteralPath
+test/lib/ansible_test/_util/target/setup/requirements.py replace-urlopen
 test/support/integration/plugins/inventory/aws_ec2.py pylint:use-a-generator
 test/support/integration/plugins/module_utils/network/common/utils.py pylint:use-a-generator
 test/support/integration/plugins/modules/ec2_group.py pylint:use-a-generator
diff --git a/test/utils/shippable/generic.sh b/test/utils/shippable/generic.sh
index 28eb12688e..367f98191f 100755
--- a/test/utils/shippable/generic.sh
+++ b/test/utils/shippable/generic.sh
@@ -15,4 +15,4 @@ fi
 
 # shellcheck disable=SC2086
 ansible-test integration --color -v --retry-on-error "${target}" ${COVERAGE:+"$COVERAGE"} ${CHANGED:+"$CHANGED"} ${UNSTABLE:+"$UNSTABLE"} \
-    --docker default --python "${python}"
+    --docker default --python "${python}" --pypi-proxy