How to run sklearn library with native tensorflow on m1 mac
Question:
I have installed TensorFlow using a virtual environment running python 3.8 as described by Apple. This should theoretically run natively and utilise the GPU. I tried installing TensorFlow using miniforge last time and it was not able to use the GPU as miniforge uses python 3.9 and Tensorflow for m1 macs currently require python 3.8.
On sklearns website, the only way to install sklearn libraries currently is by using conda install sklearn which is through miniforge.
Is there a way to install sklearn on a tensorflow environment created using
python3 -m venv TFGPU
I have already tried pip. I was able to install most other libraries other than sklearn which I use for pre-processing.
Answers:
Hi and welcome to SO 🙂
I’m a pip/virtualvenv user, so I had to fix my venv to work with my M1 mac using the conda/miniforge, without switching to conda’s venv. So, I believe this should work for you as well:
# if not yet installed
xcode-select --install
git clone git://github.com/scikit-learn/scikit-learn.git
cd scikit-learn
# mac / mac m1 specific
brew install libomp
brew install miniforge
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh
conda init bash
conda create -n conda-sklearn-dev -c conda-forge python numpy scipy cython joblib threadpoolctl pytest compilers llvm-openmp
conda activate conda-sklearn-dev
pip install cython
pip install --verbose --no-build-isolation --editable .
Now:
- Test it on the conda venv. You should get version 0.24.2 (at the
time of writing)
- Deactivate all conda venv-s
- Activate your regular venv and test again. you should get version .dev0
- In case sklearn is missing – do a simple pip install for it – it should grab the compiled one
I have installed TensorFlow using a virtual environment running python 3.8 as described by Apple. This should theoretically run natively and utilise the GPU. I tried installing TensorFlow using miniforge last time and it was not able to use the GPU as miniforge uses python 3.9 and Tensorflow for m1 macs currently require python 3.8.
On sklearns website, the only way to install sklearn libraries currently is by using conda install sklearn which is through miniforge.
Is there a way to install sklearn on a tensorflow environment created using
python3 -m venv TFGPU
I have already tried pip. I was able to install most other libraries other than sklearn which I use for pre-processing.
Hi and welcome to SO 🙂
I’m a pip/virtualvenv user, so I had to fix my venv to work with my M1 mac using the conda/miniforge, without switching to conda’s venv. So, I believe this should work for you as well:
# if not yet installed
xcode-select --install
git clone git://github.com/scikit-learn/scikit-learn.git
cd scikit-learn
# mac / mac m1 specific
brew install libomp
brew install miniforge
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh
conda init bash
conda create -n conda-sklearn-dev -c conda-forge python numpy scipy cython joblib threadpoolctl pytest compilers llvm-openmp
conda activate conda-sklearn-dev
pip install cython
pip install --verbose --no-build-isolation --editable .
Now:
- Test it on the conda venv. You should get version 0.24.2 (at the
time of writing) - Deactivate all conda venv-s
- Activate your regular venv and test again. you should get version .dev0
- In case sklearn is missing – do a simple pip install for it – it should grab the compiled one