arch x86_64 and arm64e is available but python3 is saying incompatible architecture on Mac M1
Question:
I am trying to run this reading-text-in-the-wild
on Mac M1.
When I attempt to run this code
python3 make_keras_charnet_model.py
I get the error
Using Theano backend.
Traceback (most recent call last):
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 305, in <module>
from cutils_ext.cutils_ext import * # noqa
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 316, in <module>
from cutils_ext.cutils_ext import * # noqa
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "make_keras_charnet_model.py", line 10, in <module>
from keras.models import Sequential, model_from_json
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/models.py", line 15, in <module>
from . import backend as K
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/backend/__init__.py", line 47, in <module>
from .theano_backend import *
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/backend/theano_backend.py", line 1, in <module>
import theano
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/__init__.py", line 76, in <module>
from theano.scan_module import scan, map, reduce, foldl, foldr, clone
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/scan_module/__init__.py", line 40, in <module>
from theano.scan_module import scan_opt
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/scan_module/scan_opt.py", line 59, in <module>
from theano import tensor, scalar
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/tensor/__init__.py", line 7, in <module>
from theano.tensor.subtensor import *
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/tensor/subtensor.py", line 27, in <module>
import theano.gof.cutils # needed to import cutils_ext
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 319, in <module>
compile_cutils()
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 283, in compile_cutils
cmodule.GCC_compiler.compile_str('cutils_ext', code, location=loc,
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cmodule.py", line 2212, in compile_str
return dlimport(lib_filename)
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cmodule.py", line 299, in dlimport
rval = __import__(module_name, {}, {}, [module_name])
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)
I have duplicated my terminal to have the duplicate open with Rosetta and still I get the error.
when I run the command below to check the architecture available on my M1 Mac
file /bin/bash
I get this output
/bin/bash: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/bin/bash (for architecture x86_64): Mach-O 64-bit executable x86_64
/bin/bash (for architecture arm64e): Mach-O 64-bit executable arm64e
I looks like I have both x86_64 and arm64e available but the error is saying
(mach-o file, but is an incompatible architecture (have ‘x86_64’, need ‘arm64e’)), ‘/usr/local/lib/cutils_ext.so’ (no such file), ‘/usr/lib/cutils_ext.so’ (no such file)
what is causing this error and how can I fix it?
Answers:
the package you are using is not compatible with the new mac hardware by the looks of it. You will need to run this using Rosetta 2, which is an apple system component.
I found that i need to specify the architecture
so instead of
python3 make_keras_charnet_model.py
i now use this
arch -arm64 python3 make_keras_charnet_model.py
While the virtual env is active, I usually reinstall the package that complains with
pip install -U --force <package name>
if you install Python 3.8 or 3.9 with the macOS 64-bit universal2 installer package. you may try using python3-intel64 instead of python3.
in my MacBook M1
/usr/local/bin/python3-intel64
using Rosetta
arch -arm64 python3 -m pip install theano
I’ve read A LOT of information about this problem and since I’ve installed python3-intel64 from official Python website I have no more troubles)
Solution:
- Go to https://www.python.org/downloads/macos/
- Find your version of Python for project
- Press "Download macOS 64-bit universal2 installer"
- Install Python from archive (since that moment you can use "python3-intel64" command to run Python instead of just "python")
- Create new venv in project from base interpreter which you are installed
- Install all requirements to venv
- PROFIT!!!
Use the next command to install your libraries or tu run your script:
arch -x86_64 python3 -pip install [your lib]
I recommend to remove the previously installed packages to avoid the error
I am trying to run this reading-text-in-the-wild
on Mac M1.
When I attempt to run this code
python3 make_keras_charnet_model.py
I get the error
Using Theano backend.
Traceback (most recent call last):
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 305, in <module>
from cutils_ext.cutils_ext import * # noqa
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 316, in <module>
from cutils_ext.cutils_ext import * # noqa
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "make_keras_charnet_model.py", line 10, in <module>
from keras.models import Sequential, model_from_json
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/models.py", line 15, in <module>
from . import backend as K
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/backend/__init__.py", line 47, in <module>
from .theano_backend import *
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/keras/backend/theano_backend.py", line 1, in <module>
import theano
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/__init__.py", line 76, in <module>
from theano.scan_module import scan, map, reduce, foldl, foldr, clone
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/scan_module/__init__.py", line 40, in <module>
from theano.scan_module import scan_opt
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/scan_module/scan_opt.py", line 59, in <module>
from theano import tensor, scalar
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/tensor/__init__.py", line 7, in <module>
from theano.tensor.subtensor import *
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/tensor/subtensor.py", line 27, in <module>
import theano.gof.cutils # needed to import cutils_ext
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 319, in <module>
compile_cutils()
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cutils.py", line 283, in compile_cutils
cmodule.GCC_compiler.compile_str('cutils_ext', code, location=loc,
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cmodule.py", line 2212, in compile_str
return dlimport(lib_filename)
File "/Users/name/miniforge3/envs/ocr_env/lib/python3.8/site-packages/theano/gof/cmodule.py", line 299, in dlimport
rval = __import__(module_name, {}, {}, [module_name])
ImportError: dlopen(/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so, 0x0002): tried: '/Users/name/.theano/compiledir_macOS-12.0-arm64-i386-64bit-i386-3.8.6-64/cutils_ext/cutils_ext.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e')), '/usr/local/lib/cutils_ext.so' (no such file), '/usr/lib/cutils_ext.so' (no such file)
I have duplicated my terminal to have the duplicate open with Rosetta and still I get the error.
when I run the command below to check the architecture available on my M1 Mac
file /bin/bash
I get this output
/bin/bash: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
/bin/bash (for architecture x86_64): Mach-O 64-bit executable x86_64
/bin/bash (for architecture arm64e): Mach-O 64-bit executable arm64e
I looks like I have both x86_64 and arm64e available but the error is saying
(mach-o file, but is an incompatible architecture (have ‘x86_64’, need ‘arm64e’)), ‘/usr/local/lib/cutils_ext.so’ (no such file), ‘/usr/lib/cutils_ext.so’ (no such file)
what is causing this error and how can I fix it?
the package you are using is not compatible with the new mac hardware by the looks of it. You will need to run this using Rosetta 2, which is an apple system component.
I found that i need to specify the architecture
so instead of
python3 make_keras_charnet_model.py
i now use this
arch -arm64 python3 make_keras_charnet_model.py
While the virtual env is active, I usually reinstall the package that complains with
pip install -U --force <package name>
if you install Python 3.8 or 3.9 with the macOS 64-bit universal2 installer package. you may try using python3-intel64 instead of python3.
in my MacBook M1
/usr/local/bin/python3-intel64
using Rosetta
arch -arm64 python3 -m pip install theano
I’ve read A LOT of information about this problem and since I’ve installed python3-intel64 from official Python website I have no more troubles)
Solution:
- Go to https://www.python.org/downloads/macos/
- Find your version of Python for project
- Press "Download macOS 64-bit universal2 installer"
- Install Python from archive (since that moment you can use "python3-intel64" command to run Python instead of just "python")
- Create new venv in project from base interpreter which you are installed
- Install all requirements to venv
- PROFIT!!!
Use the next command to install your libraries or tu run your script:
arch -x86_64 python3 -pip install [your lib]
I recommend to remove the previously installed packages to avoid the error