What is a Virtual Environment?
A virtual environment is a way to isolate a specific working environment from the rest of your system. It is highly suggested to use a Python virtual environment when working on computing projects, as it allows the user to tightly control the packages available to the project, and will allow multiple projects that may have conflicting software requirements to be worked on from the same machine.
Virtual environments are also fairly easy to use and create, especially when working with Python 3, where the venv package is now included in all standard Python 3.x installations.
This tutorial will cover the creation of a virtual environment on UNIX-based systems, in particular Linux derivatives.
Step 1: Getting Python
It is highly suggested to use an alternate version of Python from the system default to base your virtual environment off of, as this version can be easily overwritten. To do this, it depends on your use case.
On a cluster
The simplest way of getting an alternative version of Python is to simply find an acceptable version of Python supplied via the module command. To see the list of modules available on the cluster you are working with, use the command
module avail
Then, load the desired version of Python with
module load Python/version
where Python/version would be replaced by the specific module you were attempting to load.
A note with this method: If using sbatch to submit jobs, you will need to load the Python version before activating your virtual environment. This is accomplished by repeating the above process of loading a module after specifying the SLURM options at the head of your sbatch script.
On a local machine:
When the machine you are working on is under your control, the easiest way is to simply install another version of Python alongside the system version of Python, and specify to use this version when creating the virtual environment.
To do this, first create the location of your alternative Python install. For this tutorial, we will create it as a hidden subdirectory of the default $HOME directory.
mkdir $HOME/.localpython
Next, use cd to enter this directory:
cd ~/.localpython
and then use wget to download the tarball of your desired Python version:
wget https://www.python.org/ftp/python/3.x/Python-3.x.tar.xz
where 3.x would be replaced by the desired version of Python.
Next, extract and install to the created local directory:
tar -xvf Python-3.x.tar.xz
cd Python-3.x
./configure --prefix=$HOME/.localpython
make && make altinstall
Step 2: Creating the virtual environment
If you do not already have a folder where your project is/will be stored, create one as a subdirectory of your HOME directory, or in another convenient place
mkdir ~/project-folder
Now, create the virtual environment:
Cluster:
python3 -m venv path/to/project/folder
Local machine:
$HOME/.localpython/bin/python3 -m venv path/to/project/folder
The virtual environment may then be activated by running
source path/to/project/folder/bin/activate
You will know if the virtual environment successfully launched if your terminal now looks like this:
(project-folder) [user@hostname ~]$