How to create a Python virtual environment on Lewis

Locales: Data Centers and Campus RC University of Missouri - Columbia
1

Python Virtual Environments

(Python >= 3.3) The venv module provides support for creating lightweight
“virtual environments” with their own site directories, optionally isolated
from system site directories.

In the following examples, we will interactively install a Python Virtual

Environment in our home folder using the preferred method for the version of

Python used. We will install a Python Module, in this case NumPy.

Please note that all work is done in an Interactive Slurm Job

Usage (Python 3.5)

For Python 3.3 to Python 3.5, the preferred tool for creating virtual
environments is pyvenv. In this example, we will create a new virtual
environment called my_virtenv3 in our home folder:

Command:

module load python/python-3.5.2
pyvenv ~/my_virtenv3

Output:

[user@lewis4-c8k-hpc2-node279 ~]$ module load python/python-3.5.2
[user@lewis4-c8k-hpc2-node279 ~]$ pyvenv ~/my_virtenv3
[user@lewis4-c8k-hpc2-node279 ~]$

Now that our new virtual environment is installed, we can activate using the

activate script within our virtual environment. Once activated, your prompt

will change to indicate what virtual environment you are using. You can also

confirm what python and pip executables you are using via the which

command:

Command:

source ~/my_virtenv3/bin/activate
which pip
which python

Output:

[user@lewis4-c8k-hpc2-node279 ~]$ source ~/my_virtenv3/bin/activate
(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$ which pip
~/my_virtenv3/bin/pip
(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$ which python
~/my_virtenv3/bin/python
(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$

Optionally, you can now upgrade pip:

Command:

srun pip install --upgrade pip

Output:

(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$ srun pip install --upgrade pip
Collecting pip
Using cached pip-9.0.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 8.1.1
Uninstalling pip-8.1.1:
Successfully uninstalled pip-8.1.1
Successfully installed pip-9.0.1
(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$

You can now install any needed python modules within you virtual environment:

Command:

srun pip install numpy

Output:

(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$ srun pip install numpy
srun: job 645607 queued and waiting for resources
srun: job 645607 has been allocated resources
Collecting numpy
Downloading numpy-1.12.0-cp35-cp35m-manylinux1_x86_64.whl (16.8MB)
Installing collected packages: numpy
Successfully installed numpy-1.12.0
(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$

To clear the virtual environment, use deactivate.:

Command:

deactivate

Output:

(my_virtenv3) [user@lewis4-c8k-hpc2-node279 ~]$ deactivate
[user@lewis4-c8k-hpc2-node279 ~]$

Usage (Python 3.6)

For Python 3.6, the pyvenv script has been deprecated in favor of using

python3 -m venv to help prevent any potential confusion as to which Python

interpreter a virtual environment will be based on. In this example, we will

create a new virtual environment called my_virtenv3.6.6 in our home folder:

Command:

module load python/python-3.6.6-tk
python3 -m venv ~/my_virtenv3.6.6

Output:

[user@lewis4-c8k-hpc2-node279 ~]$ module load python/python-3.6.6-tk
[user@lewis4-c8k-hpc2-node279 ~]$ python3 -m venv ~/my_virtenv3.6.6
[user@lewis4-c8k-hpc2-node279 ~]$

Now that our new virtual environment is installed, we can activate using the

activate script within our virtual environment. Once activated, your prompt

will change to indicate what virtual environment you are using. You can also

confirm what python and pip executables you are using via the which

command:

Command:

source ~/my_virtenv3.6.6/bin/activate
which pip
which python

Output:

[user@lewis4-c8k-hpc2-node279 ~]$ source ~/my_virtenv3.6.6/bin/activate
(my_virtenv3.6.6) [user@lewis4-c8k-hpc2-node279 ~]$ which pip
~/my_virtenv3.6.6/bin/pip
(my_virtenv3.6.6) [user@lewis4-c8k-hpc2-node279 ~]$ which python
~/my_virtenv3.6.6/bin/python
(my_virtenv3.6.6) [user@lewis4-c8k-hpc2-node279 ~]$

Optionally, you can now upgrade pip:

Command:

srun pip install --upgrade pip

Output:

(my_virtenv3.6.6) [user@lewis4-c8k-hpc2-node279 ~]$ srun pip install --upgrade pip
Requirement already up-to-date: pip in ./my_virtenv3.6.6/lib/python3.6/site-packages

You can now install any needed python modules within you virtual environment:

Command:

srun pip install numpy

Output:

(my_virtenv3.6.6) [user@lewis4-c8k-hpc2-node279 ~]$ srun pip install numpy
srun: job 645610 queued and waiting for resources
srun: job 645610 has been allocated resources
Collecting numpy
Downloading numpy-1.12.0-cp36-cp36m-manylinux1_x86_64.whl (16.8MB)
Installing collected packages: numpy
Successfully installed numpy-1.12.0
(my_virtenv3.6.6) [user@lewis4-c8k-hpc2-node279 ~]$

To clear the virtual environment, use deactivate.:

Command:

deactivate

Output:

(my_virtenv3.6.6) [user@lewis4-c8k-hpc2-node279 ~]$ deactivate
[user@lewis4-c8k-hpc2-node279 ~]$