Does Python SciPy need BLAS?
Question:
numpy.distutils.system_info.BlasNotFoundError:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
Which tar do I need to download off this site?
I’ve tried the fortrans, but I keep getting this error (after setting the environment variable obviously).
Answers:
I guess you are talking about installation in Ubuntu. Just use:
apt-get install python-numpy python-scipy
That should take care of the BLAS libraries compiling as well. Else, compiling the BLAS libraries is very difficult.
The SciPy webpage used to provide build and installation instructions, but the instructions there now rely on OS binary distributions. To build SciPy (and NumPy) on operating systems without precompiled packages of the required libraries, you must build and then statically link to the Fortran libraries BLAS and LAPACK:
mkdir -p ~/src/
cd ~/src/
wget http://www.netlib.org/blas/blas.tgz
tar xzf blas.tgz
cd BLAS-*
## NOTE: The selected Fortran compiler must be consistent for BLAS, LAPACK, NumPy, and SciPy.
## For GNU compiler on 32-bit systems:
#g77 -O2 -fno-second-underscore -c *.f # with g77
#gfortran -O2 -std=legacy -fno-second-underscore -c *.f # with gfortran
## OR for GNU compiler on 64-bit systems:
#g77 -O3 -m64 -fno-second-underscore -fPIC -c *.f # with g77
gfortran -O3 -std=legacy -m64 -fno-second-underscore -fPIC -c *.f # with gfortran
## OR for Intel compiler:
#ifort -FI -w90 -w95 -cm -O3 -unroll -c *.f
# Continue below irrespective of compiler:
ar r libfblas.a *.o
ranlib libfblas.a
rm -rf *.o
export BLAS=~/src/BLAS-*/libfblas.a
Execute only one of the five g77/gfortran/ifort commands. I have commented out all, but the gfortran which I use. The subsequent LAPACK installation requires a Fortran 90 compiler, and since both installs should use the same Fortran compiler, g77 should not be used for BLAS.
Next, you’ll need to install the LAPACK stuff. The SciPy webpage’s instructions helped me here as well, but I had to modify them to suit my environment:
mkdir -p ~/src
cd ~/src/
wget http://www.netlib.org/lapack/lapack.tgz
tar xzf lapack.tgz
cd lapack-*/
cp INSTALL/make.inc.gfortran make.inc # On Linux with lapack-3.2.1 or newer
make lapacklib
make clean
export LAPACK=~/src/lapack-*/liblapack.a
Update on 3-Sep-2015:
Verified some comments today (thanks to all): Before running make lapacklib edit the make.inc file and add -fPIC option to OPTS and NOOPT settings. If you are on a 64bit architecture or want to compile for one, also add -m64. It is important that BLAS and LAPACK are compiled with these options set to the same values. If you forget the -fPIC SciPy will actually give you an error about missing symbols and will recommend this switch. The specific section of make.inc looks like this in my setup:
FORTRAN = gfortran
OPTS = -O2 -frecursive -fPIC -m64
DRVOPTS = $(OPTS)
NOOPT = -O0 -frecursive -fPIC -m64
LOADER = gfortran
On old machines (e.g. RedHat 5), gfortran might be installed in an older version (e.g. 4.1.2) and does not understand option -frecursive. Simply remove it from the make.inc file in such cases.
The lapack test target of the Makefile fails in my setup because it cannot find the blas libraries. If you are thorough you can temporarily move the blas library to the specified location to test the lapack. I’m a lazy person, so I trust the devs to have it working and verify only in SciPy.
Following the instructions given by ‘cfi’ works for me, although there are a few pieces they left out that you might need:
1) Your lapack directory, after unzipping, may be called lapack-X-Y (some version number), so you can just rename that to LAPACK.
cd ~/src
mv lapack-[tab] LAPACK
2) In that directory, you may need to do:
cd ~/src/LAPACK
cp lapack_LINUX.a libflapack.a
If you need to use the latest versions of SciPy rather than the packaged version, without going through the hassle of building BLAS and LAPACK, you can follow the below procedure.
Install linear algebra libraries from repository (for Ubuntu),
sudo apt-get install gfortran libopenblas-dev liblapack-dev
Then install SciPy, (after downloading the SciPy source): python setup.py install or
pip install scipy
As the case may be.
On Fedora, this works:
yum install lapack lapack-devel blas blas-devel
pip install numpy
pip install scipy
Remember to install ‘lapack-devel‘ and ‘blas-devel‘ in addition to ‘blas’ and ‘lapack’ otherwise you’ll get the error you mentioned or the “numpy.distutils.system_info.LapackNotFoundError” error.
For Windows users there is a nice binary package by Chris (warning: it’s a pretty large download, 191 MB):
Try using
sudo apt-get install python3-scipy
numpy.distutils.system_info.BlasNotFoundError:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [blas]) or by setting
the BLAS environment variable.
Which tar do I need to download off this site?
I’ve tried the fortrans, but I keep getting this error (after setting the environment variable obviously).
I guess you are talking about installation in Ubuntu. Just use:
apt-get install python-numpy python-scipy
That should take care of the BLAS libraries compiling as well. Else, compiling the BLAS libraries is very difficult.
The SciPy webpage used to provide build and installation instructions, but the instructions there now rely on OS binary distributions. To build SciPy (and NumPy) on operating systems without precompiled packages of the required libraries, you must build and then statically link to the Fortran libraries BLAS and LAPACK:
mkdir -p ~/src/
cd ~/src/
wget http://www.netlib.org/blas/blas.tgz
tar xzf blas.tgz
cd BLAS-*
## NOTE: The selected Fortran compiler must be consistent for BLAS, LAPACK, NumPy, and SciPy.
## For GNU compiler on 32-bit systems:
#g77 -O2 -fno-second-underscore -c *.f # with g77
#gfortran -O2 -std=legacy -fno-second-underscore -c *.f # with gfortran
## OR for GNU compiler on 64-bit systems:
#g77 -O3 -m64 -fno-second-underscore -fPIC -c *.f # with g77
gfortran -O3 -std=legacy -m64 -fno-second-underscore -fPIC -c *.f # with gfortran
## OR for Intel compiler:
#ifort -FI -w90 -w95 -cm -O3 -unroll -c *.f
# Continue below irrespective of compiler:
ar r libfblas.a *.o
ranlib libfblas.a
rm -rf *.o
export BLAS=~/src/BLAS-*/libfblas.a
Execute only one of the five g77/gfortran/ifort commands. I have commented out all, but the gfortran which I use. The subsequent LAPACK installation requires a Fortran 90 compiler, and since both installs should use the same Fortran compiler, g77 should not be used for BLAS.
Next, you’ll need to install the LAPACK stuff. The SciPy webpage’s instructions helped me here as well, but I had to modify them to suit my environment:
mkdir -p ~/src
cd ~/src/
wget http://www.netlib.org/lapack/lapack.tgz
tar xzf lapack.tgz
cd lapack-*/
cp INSTALL/make.inc.gfortran make.inc # On Linux with lapack-3.2.1 or newer
make lapacklib
make clean
export LAPACK=~/src/lapack-*/liblapack.a
Update on 3-Sep-2015:
Verified some comments today (thanks to all): Before running make lapacklib edit the make.inc file and add -fPIC option to OPTS and NOOPT settings. If you are on a 64bit architecture or want to compile for one, also add -m64. It is important that BLAS and LAPACK are compiled with these options set to the same values. If you forget the -fPIC SciPy will actually give you an error about missing symbols and will recommend this switch. The specific section of make.inc looks like this in my setup:
FORTRAN = gfortran
OPTS = -O2 -frecursive -fPIC -m64
DRVOPTS = $(OPTS)
NOOPT = -O0 -frecursive -fPIC -m64
LOADER = gfortran
On old machines (e.g. RedHat 5), gfortran might be installed in an older version (e.g. 4.1.2) and does not understand option -frecursive. Simply remove it from the make.inc file in such cases.
The lapack test target of the Makefile fails in my setup because it cannot find the blas libraries. If you are thorough you can temporarily move the blas library to the specified location to test the lapack. I’m a lazy person, so I trust the devs to have it working and verify only in SciPy.
Following the instructions given by ‘cfi’ works for me, although there are a few pieces they left out that you might need:
1) Your lapack directory, after unzipping, may be called lapack-X-Y (some version number), so you can just rename that to LAPACK.
cd ~/src
mv lapack-[tab] LAPACK
2) In that directory, you may need to do:
cd ~/src/LAPACK
cp lapack_LINUX.a libflapack.a
If you need to use the latest versions of SciPy rather than the packaged version, without going through the hassle of building BLAS and LAPACK, you can follow the below procedure.
Install linear algebra libraries from repository (for Ubuntu),
sudo apt-get install gfortran libopenblas-dev liblapack-dev
Then install SciPy, (after downloading the SciPy source): python setup.py install or
pip install scipy
As the case may be.
On Fedora, this works:
yum install lapack lapack-devel blas blas-devel
pip install numpy
pip install scipy
Remember to install ‘lapack-devel‘ and ‘blas-devel‘ in addition to ‘blas’ and ‘lapack’ otherwise you’ll get the error you mentioned or the “numpy.distutils.system_info.LapackNotFoundError” error.
For Windows users there is a nice binary package by Chris (warning: it’s a pretty large download, 191 MB):
Try using
sudo apt-get install python3-scipy