We provide Python including a variety of packages (including numpy, scipy, pandas, scikit-learn, and other computational packages) through the Anaconda distribution.

Python versions

On our Linux servers Python 3.8 is the default. We also have Python 3.7 available. Since switching to Ubuntu 18.04 we do not provide Python 2, but if you need it, please contact us and we will find a solution for you.

You can use Linux environment modules to switch between different Python versions. This can be done on a one-time basis in a given terminal session or cluster submission script, or can be done in your `.bashrc` (after the stanza involving `~skel/std.bashrc`) to set a default different than the system default.

To switch from Python 3.8 to Python 3.7:

module switch python/3.8 python/3.7

To see what Python is being used (if nothing is listed here then the default machine Python with very few packages will be used):

module list

If no Python is listed you can use

module load python

in this case to load Python 3.8.

Packages

To see what Python packages are available, invoke

conda list

To install packages locally in your home directory use the `--user` flag to `pip`:

pip install --user package_to_install

It is possible to use `conda install` to install packages outside of a Conda environment, but we don't recommend it as it can cause confusing interference between dependencies.

virtualenv and Conda environments

Environments provide a way to manage Python packages (and with Conda environments even the version(s) of Python and other software) in a context that can be isolated and controlled. This allows one to more easily manage dependencies and provide for reproducibility.

virtualenvs

If you would like to override system-installed libraries, for example if you want to use a newer or older version, try virtualenv, "a tool to create isolated Python environments".

virtualenv --system-site-packages ~/path/for/your/env
source ~/path/for/your/env/bin/activate

At this point you can `pip install` your library or do something more involved:

git clone https://github.com/somerepo/somelibrary.git
cd somelibrary
python setup.py install
# optionally, to delete source files
cd .. && rm -rf somelibrary

When you want to escape out of this environment, run `deactivate`. To re-enter, run the `source` line as above.

Conda environments

Alternatively you can use Conda to create environments:

conda create --name myenv
source activate myenv

To escape out of this environment, run `source deactivate`.

Note that use of `source` in `source activate` and `source deactivate` is deprecated, but you should be able to use these without problems.  

You can also use `conda activate` and `conda deactivate`, but there are some issues to be aware of. When you first try to use `conda activate`, Conda will prompt you to run `conda init` to initialize conda for the shell you are using. You can do this, but note that it will modify your `.bashrc` so that Conda commands are available whenever you log in. As part of this, Conda will put you in the base Conda environment automatically when a new shell starts, which will prevent you from accessing the Python version and related packages that the SCF provides. To avoid this, we recommend running `conda config --set auto_activate_base False` after running `conda init`.