using python venv to create virtual environment

python env 创建虚拟环境

前言

又断更了,长此以往,我可能无法完成100天打卡任务了。

红包是刺激自己前进的动力。

python 标准库03之虚拟环境创建

参考文档

python目前官方推荐的工具是venv

顺便介绍下目前常用的python虚拟环境工具

  • virsualenv
  • pyvenv (3.3,3.4中推荐的虚拟环境创建工具,3.6中被弃用)
  • venv(3.6以后推荐的虚拟环境创建工具)

创建虚拟环境

使用下面的命令创建一个虚拟环境

1
python3 -m venv /path/to/new/virtual/environment

windows下使用venv来创建虚拟环境

1
c:\>c:\Python35\python -m venv c:\path\to\myenv

使用-h参数可以查看venv命令行的帮助文档

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
[--upgrade] [--without-pip] [--prompt PROMPT]
ENV_DIR [ENV_DIR ...]

Creates virtual Python environments in one or more target directories.

positional arguments:
ENV_DIR A directory to create the environment in.

optional arguments:
-h, --help show this help message and exit
--system-site-packages
Give the virtual environment access to the system
site-packages dir.
--symlinks Try to use symlinks rather than copies, when symlinks
are not the default for the platform.
--copies Try to use copies rather than symlinks, even when
symlinks are the default for the platform.
--clear Delete the contents of the environment directory if it
already exists, before environment creation.
--upgrade Upgrade the environment directory to use this version
of Python, assuming Python has been upgraded in-place.
--without-pip Skips installing or upgrading pip in the virtual
environment (pip is bootstrapped by default)
--prompt PROMPT Provides an alternative prompt prefix for this
environment.

除非你使用了--without-pip 参数,否则 ensurepip 模块会默认安装pip指令到创建好的虚拟环境中。

一旦一个虚拟环境被创建,可以通过下面的命令激活这个虚拟环境。不同平台下的激活方式略有不同,下面总结了各个平台下虚拟环境的激活命令脚本,请参考。

激活虚拟环境

平台 Shell 用于激活虚拟环境的命令
POSIX bash/zsh $ source /bin/activate
fish $ . /bin/activate.fish
csh/tcsh $ source /bin/activate.csh
PowerShell Core $ /bin/Activate.ps1
Windows cmd.exe C:> \Scripts\activate.bat
PowerShell PS C:> \Scripts\Activate.ps1

You don’t specifically need to activate an environment; activation just prepends the virtual environment’s binary directory to your path, so that “python” invokes the virtual environment’s Python interpreter and you can run installed scripts without having to use their full path. However, all scripts installed in a virtual environment should be runnable without activating it, and run with the virtual environment’s Python automatically.

You can deactivate a virtual environment by typing “deactivate” in your shell. The exact mechanism is platform-specific and is an internal implementation detail (typically a script or shell function will be used).

你无需关心如何激活一个虚拟环境;使用相对于项目当前路径下的可执行方法,python解析器就可以激活虚拟环境。已经激活的虚拟环境, 就可以使用python的命令直接执行脚本或者安装其他软件包到当前的虚拟环境,而不需要关心是否会污染本地python环境的问题。同样,使用deactivate 就可以退出当前的虚拟环境。

API

下面展示的是一个使用EnvBuilder 创建一个自定义虚拟环境的脚本。

1
class venv.EnvBuilder(system_site_packages=False, clear=False, symlinks=False, upgrade=False, with_pip=False, prompt=None)
  • create(env_dir)

    • system_site_packages – 一个Boolean值,是否将site-packages添加到虚拟环境中 默认是False.
    • clear – 一个Boolean值,如果为真,将会在创建环境之前清除已经存在的目标路径.
    • symlinks – 一个Boolean值,如果为真,将会在创建环境前删除包含内容的目录
    • upgrade – 一个Boolean值,如果为真, 则将用正在运行的Python升级现有环境 - 用于在Python就地升级(默认为False)时使用。.
    • with_pip – 一个Boolean值,如果为真,pip命令将会安装到虚拟环境中 ,使用 ensurepip 的默认 --default-pip 参数选项.
    • prompt – 在激活虚拟环境后使用的字符串(默认为None表示将使用环境的目录名称)。.
1
2
3
4
5
6
7
8
9
10
11
def create(self, env_dir):
"""
Create a virtualized Python environment in a directory.
env_dir is the target directory to create an environment in.
"""
env_dir = os.path.abspath(env_dir)
context = self.ensure_directories(env_dir)
self.create_configuration(context)
self.setup_python(context)
self.setup_scripts(context)
self.post_setup(context)
  • ensure_directories(env_dir)

    创建环境目录和所有必需的目录,并返回一个上下文对象。这只是属性(如路径)的
    持有者,供其他方法使用。这些目录已被允许存在,只要其中一个clear或被upgrade
    指定为允许在现有环境目录上进行操作即可。

  • create_configuration(context)

    pyvenv.cfg在环境中创建配置文件。

  • setup_python(context)

    在环境中创建Python可执行文件(以及Windows下的DLL)的副本。在POSIX系统
    中,如果一个特定的可执行文件 python3.x使用,符号链接python和python3将创建指
    向该可执行文件,除非已存在具有这些名称的文件。

  • setup_scripts(context)

    将适合该平台的激活脚本安装到虚拟环境中。

  • post_setup(context)

    一种占位符方法,可以在第三方实现中重写,以在虚拟环境中预安装包或执行其他后
    创建步骤。

    此外,EnvBuilder提供这种工具方法,可以从被称为setup_scripts()或post_setup()在子
    类中,以协助安装自定义脚本到虚拟环境中。
    install_scripts(上下文,路径)
    路径是应包含子目录“common”,“posix”,“nt”的目录的路径,每个目录都包含指向环
    境中bin目录的脚本。os.name经过一些文本替换占位符后,“common”的内容和相应的
    目录被复制:

    • __VENV_DIR__ 被替换为环境目录的绝对路径。

    • __VENV_NAME__ 被替换为环境名称(环境目录的最终路径段)。

    • __VENV_PROMPT__ 被提示符替换(环境名称由括号括起来,并带有下面的空格)

    • __VENV_BIN_NAME__被替换为bin目录的名称(bin或者Scripts)。

    • __VENV_PYTHON__被替换为环境可执行文件的绝对路径。
      允许目录存在(用于在现有环境正在升级时)。

      还有一个模块级的便利功能:

      1
      2
      venv.create(env_dir,system_site_packages = False,clear = False,symlinks =
      False,with_pip = False )

      EnvBuilder用给定的关键字参数创建一个参数,并调用create()并使用env_dir参数。

一个扩展EnvBuilder例子

以下脚本展示了如何EnvBuilder通过实现将setuptoolspip安装到创建的虚拟环境中的子类来进行扩展:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
import os
import os.path
from subprocess import Popen, PIPE
import sys
from threading import Thread
from urllib.parse import urlparse
from urllib.request import urlretrieve
import venv

class ExtendedEnvBuilder(venv.EnvBuilder):
"""
This builder installs setuptools and pip so that you can pip or
easy_install other packages into the created virtual environment.

:param nodist: If true, setuptools and pip are not installed into the
created virtual environment.
:param nopip: If true, pip is not installed into the created
virtual environment.
:param progress: If setuptools or pip are installed, the progress of the
installation can be monitored by passing a progress
callable. If specified, it is called with two
arguments: a string indicating some progress, and a
context indicating where the string is coming from.
The context argument can have one of three values:
'main', indicating that it is called from virtualize()
itself, and 'stdout' and 'stderr', which are obtained
by reading lines from the output streams of a subprocess
which is used to install the app.

If a callable is not specified, default progress
information is output to sys.stderr.
"""

def __init__(self, *args, **kwargs):
self.nodist = kwargs.pop('nodist', False)
self.nopip = kwargs.pop('nopip', False)
self.progress = kwargs.pop('progress', None)
self.verbose = kwargs.pop('verbose', False)
super().__init__(*args, **kwargs)

def post_setup(self, context):
"""
Set up any packages which need to be pre-installed into the
virtual environment being created.

:param context: The information for the virtual environment
creation request being processed.
"""
os.environ['VIRTUAL_ENV'] = context.env_dir
if not self.nodist:
self.install_setuptools(context)
# Can't install pip without setuptools
if not self.nopip and not self.nodist:
self.install_pip(context)

def reader(self, stream, context):
"""
Read lines from a subprocess' output stream and either pass to a progress
callable (if specified) or write progress information to sys.stderr.
"""
progress = self.progress
while True:
s = stream.readline()
if not s:
break
if progress is not None:
progress(s, context)
else:
if not self.verbose:
sys.stderr.write('.')
else:
sys.stderr.write(s.decode('utf-8'))
sys.stderr.flush()
stream.close()

def install_script(self, context, name, url):
_, _, path, _, _, _ = urlparse(url)
fn = os.path.split(path)[-1]
binpath = context.bin_path
distpath = os.path.join(binpath, fn)
# Download script into the virtual environment's binaries folder
urlretrieve(url, distpath)
progress = self.progress
if self.verbose:
term = '\n'
else:
term = ''
if progress is not None:
progress('Installing %s ...%s' % (name, term), 'main')
else:
sys.stderr.write('Installing %s ...%s' % (name, term))
sys.stderr.flush()
# Install in the virtual environment
args = [context.env_exe, fn]
p = Popen(args, stdout=PIPE, stderr=PIPE, cwd=binpath)
t1 = Thread(target=self.reader, args=(p.stdout, 'stdout'))
t1.start()
t2 = Thread(target=self.reader, args=(p.stderr, 'stderr'))
t2.start()
p.wait()
t1.join()
t2.join()
if progress is not None:
progress('done.', 'main')
else:
sys.stderr.write('done.\n')
# Clean up - no longer needed
os.unlink(distpath)

def install_setuptools(self, context):
"""
Install setuptools in the virtual environment.

:param context: The information for the virtual environment
creation request being processed.
"""
url = 'https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py'
self.install_script(context, 'setuptools', url)
# clear up the setuptools archive which gets downloaded
pred = lambda o: o.startswith('setuptools-') and o.endswith('.tar.gz')
files = filter(pred, os.listdir(context.bin_path))
for f in files:
f = os.path.join(context.bin_path, f)
os.unlink(f)

def install_pip(self, context):
"""
Install pip in the virtual environment.

:param context: The information for the virtual environment
creation request being processed.
"""
url = 'https://raw.github.com/pypa/pip/master/contrib/get-pip.py'
self.install_script(context, 'pip', url)

def main(args=None):
compatible = True
if sys.version_info < (3, 3):
compatible = False
elif not hasattr(sys, 'base_prefix'):
compatible = False
if not compatible:
raise ValueError('This script is only for use with '
'Python 3.3 or later')
else:
import argparse

parser = argparse.ArgumentParser(prog=__name__,
description='Creates virtual Python '
'environments in one or '
'more target '
'directories.')
parser.add_argument('dirs', metavar='ENV_DIR', nargs='+',
help='A directory in which to create the
'virtual environment.')
parser.add_argument('--no-setuptools', default=False,
action='store_true', dest='nodist',
help="Don't install setuptools or pip in the "
"virtual environment.")
parser.add_argument('--no-pip', default=False,
action='store_true', dest='nopip',
help="Don't install pip in the virtual "
"environment.")
parser.add_argument('--system-site-packages', default=False,
action='store_true', dest='system_site',
help='Give the virtual environment access to the '
'system site-packages dir.')
if os.name == 'nt':
use_symlinks = False
else:
use_symlinks = True
parser.add_argument('--symlinks', default=use_symlinks,
action='store_true', dest='symlinks',
help='Try to use symlinks rather than copies, '
'when symlinks are not the default for '
'the platform.')
parser.add_argument('--clear', default=False, action='store_true',
dest='clear', help='Delete the contents of the '
'virtual environment '
'directory if it already '
'exists, before virtual '
'environment creation.')
parser.add_argument('--upgrade', default=False, action='store_true',
dest='upgrade', help='Upgrade the virtual '
'environment directory to '
'use this version of '
'Python, assuming Python '
'has been upgraded '
'in-place.')
parser.add_argument('--verbose', default=False, action='store_true',
dest='verbose', help='Display the output '
'from the scripts which '
'install setuptools and pip.')
options = parser.parse_args(args)
if options.upgrade and options.clear:
raise ValueError('you cannot supply --upgrade and --clear together.')
builder = ExtendedEnvBuilder(system_site_packages=options.system_site,
clear=options.clear,
symlinks=options.symlinks,
upgrade=options.upgrade,
nodist=options.nodist,
nopip=options.nopip,
verbose=options.verbose)
for d in options.dirs:
builder.create(d)

if __name__ == '__main__':
rc = 1
try:
main()
rc = 0
except Exception as e:
print('Error: %s' % e, file=sys.stderr)
sys.exit(rc)

实践

在windows下vscode中通过venv 创建虚拟环境步骤:

  1. powershell下,执行命令 python -m venv .venv

  2. 执行激活命令 .\Scripts\activate.ps1

  3. 退出虚拟环境 在项目目录下执行deactivate 即可。

坚持原创技术分享,您的支持将鼓励我继续创作!