diff options
Diffstat (limited to 'Doc/library/venv.rst')
| -rw-r--r-- | Doc/library/venv.rst | 426 |
1 files changed, 426 insertions, 0 deletions
diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst new file mode 100644 index 0000000000..be0cad4231 --- /dev/null +++ b/Doc/library/venv.rst @@ -0,0 +1,426 @@ +:mod:`venv` --- Creation of virtual environments +================================================ + +.. module:: venv + :synopsis: Creation of virtual environments. +.. moduleauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk> +.. sectionauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk> + + +.. index:: pair: Environments; virtual + +.. versionadded:: 3.3 + +**Source code:** :source:`Lib/venv` + +-------------- + +The :mod:`venv` module provides support for creating lightweight "virtual +environments" with their own site directories, optionally isolated from system +site directories. Each virtual environment has its own Python binary (allowing +creation of environments with various Python versions) and can have its own +independent set of installed Python packages in its site directories. + +See :pep:`405` for more information about Python virtual environments. + +Creating virtual environments +----------------------------- + +.. include:: /using/venv-create.inc + + +.. _venv-def: + +.. note:: A virtual environment (also called a ``venv``) is a Python + environment such that the Python interpreter, libraries and scripts + installed into it are isolated from those installed in other virtual + environments, and (by default) any libraries installed in a "system" Python, + i.e. one which is installed as part of your operating system. + + A venv is a directory tree which contains Python executable files and + other files which indicate that it is a venv. + + Common installation tools such as ``Setuptools`` and ``pip`` work as + expected with venvs - i.e. when a venv is active, they install Python + packages into the venv without needing to be told to do so explicitly. + Of course, you need to install them into the venv first: this could be + done by running ``ez_setup.py`` with the venv activated, + followed by running ``easy_install pip``. Alternatively, you could download + the source tarballs and run ``python setup.py install`` after unpacking, + with the venv activated. + + When a venv is active (i.e. the venv's Python interpreter is running), the + attributes :attr:`sys.prefix` and :attr:`sys.exec_prefix` point to the base + directory of the venv, whereas :attr:`sys.base_prefix` and + :attr:`sys.base_exec_prefix` point to the non-venv Python installation + which was used to create the venv. If a venv is not active, then + :attr:`sys.prefix` is the same as :attr:`sys.base_prefix` and + :attr:`sys.exec_prefix` is the same as :attr:`sys.base_exec_prefix` (they + all point to a non-venv Python installation). + + When a venv is active, any options that change the installation path will be + ignored from all distutils configuration files to prevent projects being + inadvertently installed outside of the virtual environment. + + When working in a command shell, users can make a venv active by running an + ``activate`` script in the venv's executables directory (the precise filename + is shell-dependent), which prepends the venv's directory for executables to + the ``PATH`` environment variable for the running shell. There should be no + need in other circumstances to activate a venv -- scripts installed into + venvs have a shebang line which points to the venv's Python interpreter. This + means that the script will run with that interpreter regardless of the value + of ``PATH``. On Windows, shebang line processing is supported if you have the + Python Launcher for Windows installed (this was added to Python in 3.3 - see + :pep:`397` for more details). Thus, double-clicking an installed script in + a Windows Explorer window should run the script with the correct interpreter + without there needing to be any reference to its venv in ``PATH``. + + +API +--- + +.. highlight:: python + +The high-level method described above makes use of a simple API which provides +mechanisms for third-party virtual environment creators to customize environment +creation according to their needs, the :class:`EnvBuilder` class. + +.. class:: EnvBuilder(system_site_packages=False, clear=False, symlinks=False, upgrade=False) + + The :class:`EnvBuilder` class accepts the following keyword arguments on + instantiation: + + * ``system_site_packages`` -- a Boolean value indicating that the system Python + site-packages should be available to the environment (defaults to ``False``). + + * ``clear`` -- a Boolean value which, if true, will delete any existing target + directory instead of raising an exception (defaults to ``False``). + + * ``symlinks`` -- a Boolean value indicating whether to attempt to symlink the + Python binary (and any necessary DLLs or other binaries, + e.g. ``pythonw.exe``), rather than copying. Defaults to ``True`` on Linux and + Unix systems, but ``False`` on Windows. + + * ``upgrade`` -- a Boolean value which, if true, will upgrade an existing + environment with the running Python - for use when that Python has been + upgraded in-place (defaults to ``False``). + + + + Creators of third-party virtual environment tools will be free to use the + provided ``EnvBuilder`` class as a base class. + + The returned env-builder is an object which has a method, ``create``: + + .. method:: create(env_dir) + + This method takes as required argument the path (absolute or relative to + the current directory) of the target directory which is to contain the + virtual environment. The ``create`` method will either create the + environment in the specified directory, or raise an appropriate + exception. + + The ``create`` method of the ``EnvBuilder`` class illustrates the hooks + available for subclass customization:: + + def create(self, env_dir): + """ + Create a virtualized Python environment in a directory. + env_dir is the target directory to create an environment in. + """ + env_dir = os.path.abspath(env_dir) + context = self.ensure_directories(env_dir) + self.create_configuration(context) + self.setup_python(context) + self.setup_scripts(context) + self.post_setup(context) + + Each of the methods :meth:`ensure_directories`, + :meth:`create_configuration`, :meth:`setup_python`, + :meth:`setup_scripts` and :meth:`post_setup` can be overridden. + + .. method:: ensure_directories(env_dir) + + Creates the environment directory and all necessary directories, and + returns a context object. This is just a holder for attributes (such as + paths), for use by the other methods. The directories are allowed to + exist already, as long as either ``clear`` or ``upgrade`` were + specified to allow operating on an existing environment directory. + + .. method:: create_configuration(context) + + Creates the ``pyvenv.cfg`` configuration file in the environment. + + .. method:: setup_python(context) + + Creates a copy of the Python executable (and, under Windows, DLLs) in + the environment. On a POSIX system, if a specific executable + ``python3.x`` was used, symlinks to ``python`` and ``python3`` will be + created pointing to that executable, unless files with those names + already exist. + + .. method:: setup_scripts(context) + + Installs activation scripts appropriate to the platform into the virtual + environment. + + .. method:: post_setup(context) + + A placeholder method which can be overridden in third party + implementations to pre-install packages in the virtual environment or + perform other post-creation steps. + + In addition, :class:`EnvBuilder` provides this utility method that can be + called from :meth:`setup_scripts` or :meth:`post_setup` in subclasses to + assist in installing custom scripts into the virtual environment. + + .. method:: install_scripts(context, path) + + *path* is the path to a directory that should contain subdirectories + "common", "posix", "nt", each containing scripts destined for the bin + directory in the environment. The contents of "common" and the + directory corresponding to :data:`os.name` are copied after some text + replacement of placeholders: + + * ``__VENV_DIR__`` is replaced with the absolute path of the environment + directory. + + * ``__VENV_NAME__`` is replaced with the environment name (final path + segment of environment directory). + + * ``__VENV_BIN_NAME__`` is replaced with the name of the bin directory + (either ``bin`` or ``Scripts``). + + * ``__VENV_PYTHON__`` is replaced with the absolute path of the + environment's executable. + + The directories are allowed to exist (for when an existing environment + is being upgraded). + +There is also a module-level convenience function: + +.. function:: create(env_dir, system_site_packages=False, clear=False, symlinks=False) + + Create an :class:`EnvBuilder` with the given keyword arguments, and call its + :meth:`~EnvBuilder.create` method with the *env_dir* argument. + +An example of extending ``EnvBuilder`` +-------------------------------------- + +The following script shows how to extend :class:`EnvBuilder` by implementing a +subclass which installs setuptools and pip into a created venv:: + + import os + import os.path + from subprocess import Popen, PIPE + import sys + from threading import Thread + from urllib.parse import urlparse + from urllib.request import urlretrieve + import venv + + class ExtendedEnvBuilder(venv.EnvBuilder): + """ + This builder installs setuptools and pip so that you can pip or + easy_install other packages into the created environment. + + :param nodist: If True, setuptools and pip are not installed into the + created environment. + :param nopip: If True, pip is not installed into the created + environment. + :param progress: If setuptools or pip are installed, the progress of the + installation can be monitored by passing a progress + callable. If specified, it is called with two + arguments: a string indicating some progress, and a + context indicating where the string is coming from. + The context argument can have one of three values: + 'main', indicating that it is called from virtualize() + itself, and 'stdout' and 'stderr', which are obtained + by reading lines from the output streams of a subprocess + which is used to install the app. + + If a callable is not specified, default progress + information is output to sys.stderr. + """ + + def __init__(self, *args, **kwargs): + self.nodist = kwargs.pop('nodist', False) + self.nopip = kwargs.pop('nopip', False) + self.progress = kwargs.pop('progress', None) + self.verbose = kwargs.pop('verbose', False) + super().__init__(*args, **kwargs) + + def post_setup(self, context): + """ + Set up any packages which need to be pre-installed into the + environment being created. + + :param context: The information for the environment creation request + being processed. + """ + os.environ['VIRTUAL_ENV'] = context.env_dir + if not self.nodist: + self.install_setuptools(context) + # Can't install pip without setuptools + if not self.nopip and not self.nodist: + self.install_pip(context) + + def reader(self, stream, context): + """ + Read lines from a subprocess' output stream and either pass to a progress + callable (if specified) or write progress information to sys.stderr. + """ + progress = self.progress + while True: + s = stream.readline() + if not s: + break + if progress is not None: + progress(s, context) + else: + if not self.verbose: + sys.stderr.write('.') + else: + sys.stderr.write(s.decode('utf-8')) + sys.stderr.flush() + stream.close() + + def install_script(self, context, name, url): + _, _, path, _, _, _ = urlparse(url) + fn = os.path.split(path)[-1] + binpath = context.bin_path + distpath = os.path.join(binpath, fn) + # Download script into the env's binaries folder + urlretrieve(url, distpath) + progress = self.progress + if self.verbose: + term = '\n' + else: + term = '' + if progress is not None: + progress('Installing %s ...%s' % (name, term), 'main') + else: + sys.stderr.write('Installing %s ...%s' % (name, term)) + sys.stderr.flush() + # Install in the env + args = [context.env_exe, fn] + p = Popen(args, stdout=PIPE, stderr=PIPE, cwd=binpath) + t1 = Thread(target=self.reader, args=(p.stdout, 'stdout')) + t1.start() + t2 = Thread(target=self.reader, args=(p.stderr, 'stderr')) + t2.start() + p.wait() + t1.join() + t2.join() + if progress is not None: + progress('done.', 'main') + else: + sys.stderr.write('done.\n') + # Clean up - no longer needed + os.unlink(distpath) + + def install_setuptools(self, context): + """ + Install setuptools in the environment. + + :param context: The information for the environment creation request + being processed. + """ + url = 'https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py' + self.install_script(context, 'setuptools', url) + # clear up the setuptools archive which gets downloaded + pred = lambda o: o.startswith('setuptools-') and o.endswith('.tar.gz') + files = filter(pred, os.listdir(context.bin_path)) + for f in files: + f = os.path.join(context.bin_path, f) + os.unlink(f) + + def install_pip(self, context): + """ + Install pip in the environment. + + :param context: The information for the environment creation request + being processed. + """ + url = 'https://raw.github.com/pypa/pip/master/contrib/get-pip.py' + self.install_script(context, 'pip', url) + + def main(args=None): + compatible = True + if sys.version_info < (3, 3): + compatible = False + elif not hasattr(sys, 'base_prefix'): + compatible = False + if not compatible: + raise ValueError('This script is only for use with ' + 'Python 3.3 or later') + else: + import argparse + + parser = argparse.ArgumentParser(prog=__name__, + description='Creates virtual Python ' + 'environments in one or ' + 'more target ' + 'directories.') + parser.add_argument('dirs', metavar='ENV_DIR', nargs='+', + help='A directory to create the environment in.') + parser.add_argument('--no-setuptools', default=False, + action='store_true', dest='nodist', + help="Don't install setuptools or pip in the " + "virtual environment.") + parser.add_argument('--no-pip', default=False, + action='store_true', dest='nopip', + help="Don't install pip in the virtual " + "environment.") + parser.add_argument('--system-site-packages', default=False, + action='store_true', dest='system_site', + help='Give the virtual environment access to the ' + 'system site-packages dir.') + if os.name == 'nt': + use_symlinks = False + else: + use_symlinks = True + parser.add_argument('--symlinks', default=use_symlinks, + action='store_true', dest='symlinks', + help='Try to use symlinks rather than copies, ' + 'when symlinks are not the default for ' + 'the platform.') + parser.add_argument('--clear', default=False, action='store_true', + dest='clear', help='Delete the contents of the ' + 'environment directory if it ' + 'already exists, before ' + 'environment creation.') + parser.add_argument('--upgrade', default=False, action='store_true', + dest='upgrade', help='Upgrade the environment ' + 'directory to use this version ' + 'of Python, assuming Python ' + 'has been upgraded in-place.') + parser.add_argument('--verbose', default=False, action='store_true', + dest='verbose', help='Display the output ' + 'from the scripts which ' + 'install setuptools and pip.') + options = parser.parse_args(args) + if options.upgrade and options.clear: + raise ValueError('you cannot supply --upgrade and --clear together.') + builder = ExtendedEnvBuilder(system_site_packages=options.system_site, + clear=options.clear, + symlinks=options.symlinks, + upgrade=options.upgrade, + nodist=options.nodist, + nopip=options.nopip, + verbose=options.verbose) + for d in options.dirs: + builder.create(d) + + if __name__ == '__main__': + rc = 1 + try: + main() + rc = 0 + except Exception as e: + print('Error: %s' % e, file=sys.stderr) + sys.exit(rc) + + +This script is also available for download `online +<https://gist.github.com/4673395>`_. |