using python venv to create virtual environment

python env 创建虚拟环境

前言

又断更了,长此以往,我可能无法完成100天打卡任务了。

红包是刺激自己前进的动力。

python 标准库03之虚拟环境创建

参考文档

python目前官方推荐的工具是venv

顺便介绍下目前常用的python虚拟环境工具

• virsualenv
• pyvenv (3.3,3.4中推荐的虚拟环境创建工具,3.6中被弃用)
• venv(3.6以后推荐的虚拟环境创建工具)

创建虚拟环境

使用下面的命令创建一个虚拟环境

python3 -m venv /path/to/new/virtual/environment

windows下使用venv来创建虚拟环境

c:\>c:\Python35\python -m venv c:\path\to\myenv

使用-h参数可以查看venv命令行的帮助文档

usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip] [--prompt PROMPT]
            ENV_DIR [ENV_DIR ...]

Creates virtual Python environments in one or more target directories.

positional arguments:
  ENV_DIR               A directory to create the environment in.

optional arguments:
  -h, --help            show this help message and exit
  --system-site-packages
                        Give the virtual environment access to the system
                        site-packages dir.
  --symlinks            Try to use symlinks rather than copies, when symlinks
                        are not the default for the platform.
  --copies              Try to use copies rather than symlinks, even when
                        symlinks are the default for the platform.
  --clear               Delete the contents of the environment directory if it
                        already exists, before environment creation.
  --upgrade             Upgrade the environment directory to use this version
                        of Python, assuming Python has been upgraded in-place.
  --without-pip         Skips installing or upgrading pip in the virtual
                        environment (pip is bootstrapped by default)
  --prompt PROMPT       Provides an alternative prompt prefix for this
                        environment.

除非你使用了--without-pip 参数,否则 ensurepip 模块会默认安装pip指令到创建好的虚拟环境中。

一旦一个虚拟环境被创建，可以通过下面的命令激活这个虚拟环境。不同平台下的激活方式略有不同,下面总结了各个平台下虚拟环境的激活命令脚本,请参考。

激活虚拟环境

平台	Shell	用于激活虚拟环境的命令
POSIX	bash/zsh	$ source /bin/activate
	fish	$ . /bin/activate.fish
	csh/tcsh	$ source /bin/activate.csh
	PowerShell Core	$ /bin/Activate.ps1
Windows	cmd.exe	C:> \Scripts\activate.bat
	PowerShell	PS C:> \Scripts\Activate.ps1

You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.

You can deactivate a virtual environment by typing “deactivate” in your shell. The exact mechanism is platform-specific and is an internal implementation detail (typically a script or shell function will be used).

你无需关心如何激活一个虚拟环境；使用相对于项目当前路径下的可执行方法,python解析器就可以激活虚拟环境。已经激活的虚拟环境, 就可以使用python的命令直接执行脚本或者安装其他软件包到当前的虚拟环境，而不需要关心是否会污染本地python环境的问题。同样，使用deactivate 就可以退出当前的虚拟环境。

API

下面展示的是一个使用EnvBuilder 创建一个自定义虚拟环境的脚本。

class venv.EnvBuilder(system_site_packages=False, clear=False, symlinks=False, upgrade=False, with_pip=False, prompt=None)

• create(env_dir)

  • system_site_packages – 一个Boolean值,是否将site-packages添加到虚拟环境中 默认是False.
  • clear – 一个Boolean值,如果为真,将会在创建环境之前清除已经存在的目标路径.
  • symlinks – 一个Boolean值,如果为真,将会在创建环境前删除包含内容的目录
  • upgrade – 一个Boolean值,如果为真, 则将用正在运行的Python升级现有环境 - 用于在Python就地升级（默认为False）时使用。.
  • with_pip – 一个Boolean值,如果为真,pip命令将会安装到虚拟环境中 ,使用 ensurepip 的默认 --default-pip 参数选项.
  • prompt – 在激活虚拟环境后使用的字符串（默认为None表示将使用环境的目录名称）。.

def create(self, env_dir):
    """
    Create a virtualized Python environment in a directory.
    env_dir is the target directory to create an environment in.
    """
    env_dir = os.path.abspath(env_dir)
    context = self.ensure_directories(env_dir)
    self.create_configuration(context)
    self.setup_python(context)
    self.setup_scripts(context)
    self.post_setup(context)

• ensure_directories(env_dir)

  创建环境目录和所有必需的目录，并返回一个上下文对象。这只是属性（如路径）的
  持有者，供其他方法使用。这些目录已被允许存在，只要其中一个clear或被upgrade
  指定为允许在现有环境目录上进行操作即可。

• create_configuration(context)

  pyvenv.cfg在环境中创建配置文件。

• setup_python(context)

  在环境中创建Python可执行文件（以及Windows下的DLL）的副本。在POSIX系统
  中，如果一个特定的可执行文件 python3.x使用，符号链接python和python3将创建指
  向该可执行文件，除非已存在具有这些名称的文件。

• setup_scripts(context)

  将适合该平台的激活脚本安装到虚拟环境中。

• post_setup(context)

  一种占位符方法，可以在第三方实现中重写，以在虚拟环境中预安装包或执行其他后
  创建步骤。

  此外，EnvBuilder提供这种工具方法，可以从被称为setup_scripts()或post_setup()在子
  类中，以协助安装自定义脚本到虚拟环境中。
  install_scripts（上下文，路径）
  路径是应包含子目录“common”，“posix”，“nt”的目录的路径，每个目录都包含指向环
  境中bin目录的脚本。os.name经过一些文本替换占位符后，“common”的内容和相应的
  目录被复制：

  • __VENV_DIR__ 被替换为环境目录的绝对路径。

  • __VENV_NAME__ 被替换为环境名称（环境目录的最终路径段）。

  • __VENV_PROMPT__ 被提示符替换（环境名称由括号括起来，并带有下面的空格）

  • __VENV_BIN_NAME__被替换为bin目录的名称（bin或者Scripts）。

  • __VENV_PYTHON__被替换为环境可执行文件的绝对路径。
    允许目录存在（用于在现有环境正在升级时）。

    还有一个模块级的便利功能：

    venv.create（env_dir，system_site_packages = False，clear = False，symlinks =
    False，with_pip = False ）

    EnvBuilder用给定的关键字参数创建一个参数，并调用create()并使用env_dir参数。

一个扩展EnvBuilder例子

以下脚本展示了如何EnvBuilder通过实现将setuptools和pip安装到创建的虚拟环境中的子类来进行扩展：

import os
import os.path
from subprocess import Popen, PIPE
import sys
from threading import Thread
from urllib.parse import urlparse
from urllib.request import urlretrieve
import venv

class ExtendedEnvBuilder(venv.EnvBuilder):
    """
    This builder installs setuptools and pip so that you can pip or
    easy_install other packages into the created virtual environment.

    :param nodist: If true, setuptools and pip are not installed into the
                   created virtual environment.
    :param nopip: If true, pip is not installed into the created
                  virtual environment.
    :param progress: If setuptools or pip are installed, the progress of the
                     installation can be monitored by passing a progress
                     callable. If specified, it is called with two
                     arguments: a string indicating some progress, and a
                     context indicating where the string is coming from.
                     The context argument can have one of three values:
                     'main', indicating that it is called from virtualize()
                     itself, and 'stdout' and 'stderr', which are obtained
                     by reading lines from the output streams of a subprocess
                     which is used to install the app.

                     If a callable is not specified, default progress
                     information is output to sys.stderr.
    """

    def __init__(self, *args, **kwargs):
        self.nodist = kwargs.pop('nodist', False)
        self.nopip = kwargs.pop('nopip', False)
        self.progress = kwargs.pop('progress', None)
        self.verbose = kwargs.pop('verbose', False)
        super().__init__(*args, **kwargs)

    def post_setup(self, context):
        """
        Set up any packages which need to be pre-installed into the
        virtual environment being created.

        :param context: The information for the virtual environment
                        creation request being processed.
        """
        os.environ['VIRTUAL_ENV'] = context.env_dir
        if not self.nodist:
            self.install_setuptools(context)
        # Can't install pip without setuptools
        if not self.nopip and not self.nodist:
            self.install_pip(context)

    def reader(self, stream, context):
        """
        Read lines from a subprocess' output stream and either pass to a progress
        callable (if specified) or write progress information to sys.stderr.
        """
        progress = self.progress
        while True:
            s = stream.readline()
            if not s:
                break
            if progress is not None:
                progress(s, context)
            else:
                if not self.verbose:
                    sys.stderr.write('.')
                else:
                    sys.stderr.write(s.decode('utf-8'))
                sys.stderr.flush()
        stream.close()

    def install_script(self, context, name, url):
        _, _, path, _, _, _ = urlparse(url)
        fn = os.path.split(path)[-1]
        binpath = context.bin_path
        distpath = os.path.join(binpath, fn)
        # Download script into the virtual environment's binaries folder
        urlretrieve(url, distpath)
        progress = self.progress
        if self.verbose:
            term = '\n'
        else:
            term = ''
        if progress is not None:
            progress('Installing %s ...%s' % (name, term), 'main')
        else:
            sys.stderr.write('Installing %s ...%s' % (name, term))
            sys.stderr.flush()
        # Install in the virtual environment
        args = [context.env_exe, fn]
        p = Popen(args, stdout=PIPE, stderr=PIPE, cwd=binpath)
        t1 = Thread(target=self.reader, args=(p.stdout, 'stdout'))
        t1.start()
        t2 = Thread(target=self.reader, args=(p.stderr, 'stderr'))
        t2.start()
        p.wait()
        t1.join()
        t2.join()
        if progress is not None:
            progress('done.', 'main')
        else:
            sys.stderr.write('done.\n')
        # Clean up - no longer needed
        os.unlink(distpath)

    def install_setuptools(self, context):
        """
        Install setuptools in the virtual environment.

        :param context: The information for the virtual environment
                        creation request being processed.
        """
        url = 'https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py'
        self.install_script(context, 'setuptools', url)
        # clear up the setuptools archive which gets downloaded
        pred = lambda o: o.startswith('setuptools-') and o.endswith('.tar.gz')
        files = filter(pred, os.listdir(context.bin_path))
        for f in files:
            f = os.path.join(context.bin_path, f)
            os.unlink(f)

    def install_pip(self, context):
        """
        Install pip in the virtual environment.

        :param context: The information for the virtual environment
                        creation request being processed.
        """
        url = 'https://raw.github.com/pypa/pip/master/contrib/get-pip.py'
        self.install_script(context, 'pip', url)

def main(args=None):
    compatible = True
    if sys.version_info < (3, 3):
        compatible = False
    elif not hasattr(sys, 'base_prefix'):
        compatible = False
    if not compatible:
        raise ValueError('This script is only for use with '
                         'Python 3.3 or later')
    else:
        import argparse

        parser = argparse.ArgumentParser(prog=__name__,
                                         description='Creates virtual Python '
                                                     'environments in one or '
                                                     'more target '
                                                     'directories.')
        parser.add_argument('dirs', metavar='ENV_DIR', nargs='+',
                            help='A directory in which to create the
                                 'virtual environment.')
        parser.add_argument('--no-setuptools', default=False,
                            action='store_true', dest='nodist',
                            help="Don't install setuptools or pip in the "
                                 "virtual environment.")
        parser.add_argument('--no-pip', default=False,
                            action='store_true', dest='nopip',
                            help="Don't install pip in the virtual "
                                 "environment.")
        parser.add_argument('--system-site-packages', default=False,
                            action='store_true', dest='system_site',
                            help='Give the virtual environment access to the '
                                 'system site-packages dir.')
        if os.name == 'nt':
            use_symlinks = False
        else:
            use_symlinks = True
        parser.add_argument('--symlinks', default=use_symlinks,
                            action='store_true', dest='symlinks',
                            help='Try to use symlinks rather than copies, '
                                 'when symlinks are not the default for '
                                 'the platform.')
        parser.add_argument('--clear', default=False, action='store_true',
                            dest='clear', help='Delete the contents of the '
                                               'virtual environment '
                                               'directory if it already '
                                               'exists, before virtual '
                                               'environment creation.')
        parser.add_argument('--upgrade', default=False, action='store_true',
                            dest='upgrade', help='Upgrade the virtual '
                                                 'environment directory to '
                                                 'use this version of '
                                                 'Python, assuming Python '
                                                 'has been upgraded '
                                                 'in-place.')
        parser.add_argument('--verbose', default=False, action='store_true',
                            dest='verbose', help='Display the output '
                                               'from the scripts which '
                                               'install setuptools and pip.')
        options = parser.parse_args(args)
        if options.upgrade and options.clear:
            raise ValueError('you cannot supply --upgrade and --clear together.')
        builder = ExtendedEnvBuilder(system_site_packages=options.system_site,
                                       clear=options.clear,
                                       symlinks=options.symlinks,
                                       upgrade=options.upgrade,
                                       nodist=options.nodist,
                                       nopip=options.nopip,
                                       verbose=options.verbose)
        for d in options.dirs:
            builder.create(d)

if __name__ == '__main__':
    rc = 1
    try:
        main()
        rc = 0
    except Exception as e:
        print('Error: %s' % e, file=sys.stderr)
    sys.exit(rc)

实践

在windows下vscode中通过venv 创建虚拟环境步骤:

1. powershell下,执行命令 python -m venv .venv

2. 执行激活命令 .\Scripts\activate.ps1

3. 退出虚拟环境 在项目目录下执行deactivate 即可。