Linear algebra and parallelized linear algebra (using BLAS)
This documentation covers using fast parallelized linear algebra in R, Python, and Julia. All of these rely on the BLAS library for basic linear algebra calculations and can make use of fast parallel versions of the BLAS, usually OpenBLAS, MKL, or (for MacOS) Apple's Accelerate (vecLib) BLAS.
The OpenBLAS threaded BLAS is installed on all the compute servers, including the cluster. This allows parallelization of linear algebra routines, in particular any linear algebra done in R, via a mechanism known as threading. For Python, numpy on the SCF uses OpenBLAS as well
In some cases using multiple threads can actually slow down a job, or more commonly, can give negligible speed-up. It's worth testing your job using a single thread vs. multiple threads to see what the speed-up is.
Modifying the number of threads used
For any software using OpenBLAS, MKL, or Apple's Accelerate framework, you can restrict your job to a single thread by setting the appropriate environment variable before starting your job. To restrict to a single thread, do the following at the command line (in bash):
export OMP_NUM_THREADS=1 # OpenBLAS export MKL_NUM_THREADS=1 # MKL export VECLIB_MAXIMUM_THREADS=1 # Apple Accelerate for older Intel-based Macs; not relevant for M1/M2 Macs
Or you could set that to some other number.
Note that newer Macs (Apple Silicon-based M1 and M2 Macs) use the Accelerate (vecLib) BLAS, but apparently they use the Mac's AMX co-processor. This gives fast computation, but the calculations are not using the regular CPU cores, so one doesn't use `VECLIB_MAXIMUM_THREADS`.
In R, you can also use the `RhpcBLASctl` package, which provides `blas_get_num_threads()` and `blas_set_num_threads()`.
In Julia, you can also use `LinearAlgebra.BLAS.get_num_threads()` and `LinearAlgebra.BLAS.set_num_threads()`
Checking use of a fast BLAS
> sessionInfo() R version 4.3.0 (2023-04-21) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 22.04.1 LTS Matrix products: default BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so; LAPACK version 3.10.0
On MacOS, it may not be clear:
> sessionInfo() R version 4.2.2 (2022-10-31) Platform: aarch64-apple-darwin20 (64-bit) Running under: macOS Ventura 13.6 Matrix products: default LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib
However, if you look at the files in the directory containing the BLAS and LAPACK dylib files, you should be able to see if `libRblas.dylib` is symbolically linked to the Accelerate BLAS dylib file:
$ ls -l /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib total 16504 -rwxrwxr-x 1 root admin 3988192 Oct 31 2022 libR.dylib drwxrwxr-x 3 root admin 96 Oct 31 2022 libR.dylib.dSYM -rwxrwxr-x 1 root admin 193440 Oct 31 2022 libRblas.0.dylib drwxrwxr-x 3 root admin 96 Oct 31 2022 libRblas.0.dylib.dSYM lrwxr-xr-x 1 scflocal admin 21 Nov 10 2022 libRblas.dylib -> libRblas.vecLib.dylib drwxrwxr-x 3 root admin 96 Oct 31 2022 libRblas.dylib.dSYM -rwxrwxr-x 1 root admin 154464 Oct 31 2022 libRblas.vecLib.dylib drwxrwxr-x 3 root admin 96 Oct 31 2022 libRblas.vecLib.dylib.dSYM -rwxrwxr-x 1 root admin 1711824 Oct 31 2022 libRlapack.dylib drwxrwxr-x 3 root admin 96 Oct 31 2022 libRlapack.dylib.dSYM -rw-rw-r-- 1 root admin 157792 Oct 31 2022 libgcc_s.1.1.dylib -rwxrwxr-x 1 root admin 1865904 Oct 31 2022 libgfortran.5.dylib -rwxrwxr-x 1 root admin 367040 Oct 31 2022 libquadmath.0.dylib