Cannot use keras models on Mac M1 with BigSur
Question:
I am trying to use Sequential model from keras of tensorflow. When I am executing following statement:
model.fit(x_train, y_train, epochs=20, verbose=True, validation_data=(x_dev, y_dev), batch_size=10)
I am getting following errors:
I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
W tensorflow/core/platform/profile_utils/cpu_utils.cc:126] Failed to get CPU frequency: 0 Hz
F tensorflow/core/grappler/costs/op_level_cost_estimator.cc:710] Check failed: 0 < gflops (0 vs. 0)type: "CPU"
I am not able to understand how to fix it. Can anyone please help me.
From this issue on github, I understood that device.frequency()
returned 0 probably because NominalCPUFrequency()
returned 1.
However, this information seems too abstract for me and I cannot understand.
Answers:
First two ones are nothing to worry about.
The third one is a problem. You have installed an improper version of TensorFlow. Use one that supports the Mac M1 chip.
Run the following bash script to download and install TensorFlow.
#!/bin/bash
set -e
VERSION=0.1alpha3
INSTALLER_PACKAGE=tensorflow_macos-$VERSION.tar.gz
INSTALLER_PATH=https://github.com/apple/tensorflow_macos/releases/download/v$VERSION/$INSTALLER_PACKAGE
INSTALLER_SCRIPT=install_venv.sh
echo
# Check to make sure we're good to go.
if [[ $(uname) != Darwin ]] || [[ $(sw_vers -productName) != macOS ]] || [[ $(sw_vers -productVersion) != "11."* ]] ; then
echo "ERROR: TensorFlow with ML Compute acceleration is only available on macOS 11.0 and later."
exit 1
fi
# This
echo "Installation script for pre-release tensorflow_macos $VERSION. Please visit https://github.com/apple/tensorflow_macos "
echo "for instructions and license information."
echo
echo "This script will download tensorflow_macos $VERSION and needed binary dependencies, then install them into a new "
echo "or existing Python 3.8 virtual environment."
# Make sure the user knows what's going on.
read -p 'Continue [y/N]? '
if [[ ! $REPLY =~ ^[Yy]$ ]]
then
exit 1
fi
echo
echo "Downloading installer."
tmp_dir=$(mktemp -d)
pushd $tmp_dir
curl -LO $INSTALLER_PATH
echo "Extracting installer."
tar xf $INSTALLER_PACKAGE
cd tensorflow_macos
function graceful_error () {
echo
echo "Error running installation script with default options. Please fix the above errors and proceed by running "
echo
echo " $PWD/$INSTALLER_SCRIPT --prompt"
echo
echo
exit 1
}
bash ./$INSTALLER_SCRIPT --prompt || graceful_error
popd
rm -rf $tmp_dir
I’ve done as follows on macOS 11.4 (Even though the ref says "OS Requirements macOS 12.0+"), python==3.8.2 and worked [ref: https://developer.apple.com/metal/tensorflow-plugin/]:
-
Create a venv on x86 terminal, i.e. Rosetta Terminal (see: https://dev.to/courier/tips-and-tricks-to-setup-your-apple-m1-for-development-547g)
i.e. Environment Setup:
x86 : AMD
Create venv: python3 -m venv ~/PATH/tensorflow-metal
(Substitute PATH with your real PATH)
Activate the venv: source ~/PATH/tensorflow-metal/bin/activate
Update pip: python -m pip install -U pip
-
Install any library/package you need. For instance:
For instance: pip install matplotlib jupyterlab
-
Install base tensorflow:
python -m pip install tensorflow-macos
-
Install metal plugin:
python -m pip install tensorflow-metal
Good Luck & Cheers!
This might not help at all, but since I was running into the same problem I managed to get the model to train without the solutions provided here (that I will soon try), simply by changing my Y_test (0s and 1s) like this when making the train_test_split: (to_categorical(label
). So:
X_train, X_test, Y_train, Y_test = train_test_split(dataset,
to_categorical(label),
test_size=.2,
random_state=42)
Then, when training the model, I get the following message – that I do not understand fully:
2022-04-03 23:10:08.941296: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
So, this is not really a solution, but more a temporary workaround – or it might give insight in where it goes wrong.
I am trying to use Sequential model from keras of tensorflow. When I am executing following statement:
model.fit(x_train, y_train, epochs=20, verbose=True, validation_data=(x_dev, y_dev), batch_size=10)
I am getting following errors:
I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
W tensorflow/core/platform/profile_utils/cpu_utils.cc:126] Failed to get CPU frequency: 0 Hz
F tensorflow/core/grappler/costs/op_level_cost_estimator.cc:710] Check failed: 0 < gflops (0 vs. 0)type: "CPU"
I am not able to understand how to fix it. Can anyone please help me.
From this issue on github, I understood that device.frequency()
returned 0 probably because NominalCPUFrequency()
returned 1.
However, this information seems too abstract for me and I cannot understand.
First two ones are nothing to worry about.
The third one is a problem. You have installed an improper version of TensorFlow. Use one that supports the Mac M1 chip.
Run the following bash script to download and install TensorFlow.
#!/bin/bash
set -e
VERSION=0.1alpha3
INSTALLER_PACKAGE=tensorflow_macos-$VERSION.tar.gz
INSTALLER_PATH=https://github.com/apple/tensorflow_macos/releases/download/v$VERSION/$INSTALLER_PACKAGE
INSTALLER_SCRIPT=install_venv.sh
echo
# Check to make sure we're good to go.
if [[ $(uname) != Darwin ]] || [[ $(sw_vers -productName) != macOS ]] || [[ $(sw_vers -productVersion) != "11."* ]] ; then
echo "ERROR: TensorFlow with ML Compute acceleration is only available on macOS 11.0 and later."
exit 1
fi
# This
echo "Installation script for pre-release tensorflow_macos $VERSION. Please visit https://github.com/apple/tensorflow_macos "
echo "for instructions and license information."
echo
echo "This script will download tensorflow_macos $VERSION and needed binary dependencies, then install them into a new "
echo "or existing Python 3.8 virtual environment."
# Make sure the user knows what's going on.
read -p 'Continue [y/N]? '
if [[ ! $REPLY =~ ^[Yy]$ ]]
then
exit 1
fi
echo
echo "Downloading installer."
tmp_dir=$(mktemp -d)
pushd $tmp_dir
curl -LO $INSTALLER_PATH
echo "Extracting installer."
tar xf $INSTALLER_PACKAGE
cd tensorflow_macos
function graceful_error () {
echo
echo "Error running installation script with default options. Please fix the above errors and proceed by running "
echo
echo " $PWD/$INSTALLER_SCRIPT --prompt"
echo
echo
exit 1
}
bash ./$INSTALLER_SCRIPT --prompt || graceful_error
popd
rm -rf $tmp_dir
I’ve done as follows on macOS 11.4 (Even though the ref says "OS Requirements macOS 12.0+"), python==3.8.2 and worked [ref: https://developer.apple.com/metal/tensorflow-plugin/]:
-
Create a venv on x86 terminal, i.e. Rosetta Terminal (see: https://dev.to/courier/tips-and-tricks-to-setup-your-apple-m1-for-development-547g)
i.e. Environment Setup:
x86 : AMD
Create venv:python3 -m venv ~/PATH/tensorflow-metal
(Substitute PATH with your real PATH)
Activate the venv:source ~/PATH/tensorflow-metal/bin/activate
Update pip:python -m pip install -U pip
-
Install any library/package you need. For instance:
For instance:pip install matplotlib jupyterlab
-
Install base tensorflow:
python -m pip install tensorflow-macos
-
Install metal plugin:
python -m pip install tensorflow-metal
Good Luck & Cheers!
This might not help at all, but since I was running into the same problem I managed to get the model to train without the solutions provided here (that I will soon try), simply by changing my Y_test (0s and 1s) like this when making the train_test_split: (to_categorical(label
). So:
X_train, X_test, Y_train, Y_test = train_test_split(dataset,
to_categorical(label),
test_size=.2,
random_state=42)
Then, when training the model, I get the following message – that I do not understand fully:
2022-04-03 23:10:08.941296: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:112] Plugin optimizer for device_type GPU is enabled.
So, this is not really a solution, but more a temporary workaround – or it might give insight in where it goes wrong.