Using Python on the SCF

We provide Python including a variety of packages (including numpy, scipy, pandas, scikit-learn, and other computational packages) through the Anaconda distribution.

On our Linux servers Python 3.7 is the default as of January 2019. We also have Python 3.6 available. Python 3.5 may be usable but not all packages will work. Since switching to Ubuntu 18.04 we do not provide Python 2, but if you need it, please contact us and we will find a solution for you.

Packages

To see what Python packages are available, invoke

conda list

To install packages locally in your home directory use the `--user` flag to `pip`:

pip install --user package_to_install

It is possible to install packages using `conda`, but we don't recommend it as `conda` can cause confusing interference between dependencies.

virtualenv and Conda environments

If you would like to override system-installed libraries, for example if you want to use a newer or older version, try virtualenv, "a tool to create isolated Python environments".

virtualenv --system-site-packages ~/path/for/your/env
source ~/path/for/your/env/bin/activate

At this point you can `pip install` your library or do something more involved:

git clone https://github.com/somerepo/somelibrary.git
cd somelibrary
python setup.py install
# optionally, to delete source files
cd .. && rm -rf somelibrary

When you want to escape out of this environment, run `deactivate`. To re-enter, run the `source` line as above.

Alternatively you can use Conda to create environments:

conda create --name myenv
source activate myenv

To escape out of this environment, run `source deactivate`.

Switch Versions

You can use Linux environment modules to switch between different Python versions. This can be done on a one-time basis in a given terminal session or cluster submission script, or can be done in your .bashrc (after the stanza involving ~skel/std.bashrc) to set a default different than the system default. Note that the python/3 and python/3.5 modules are the same.

To switch from Python 3.7 to Python 3.6:

module switch python/3.7 python/3.6

To see what Python is being used (if nothing is listed here then the default machine Python with very few packages will be used):

module list

If no Python is listed you can use

module load python

in this case to load Python 3.7.