Web3 python crypto cypher issue on M1 Mac

Question:

When I try to build a blockchain transaction using Web3 on python, I’m getting an error that is apparently because I’m using an Apple Silicon computer (2020 M1 MacBook Pro). I’m following a popular Solidity, Blockchain, and Smart Contract course on YouTube and I’m unable to get it working. Could someone please help me out?

My code fails when I try to build a transaction.

The error message I get when I run my code is below:


(myvenv) leonkc@Leons-MacBook-Pro web3_py_simple_storage % python deploy.py
Traceback (most recent call last):
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/deploy.py", line 3, in <module>
from web3 import Web3
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/web3/__init__.py", line 6, in <module>
from eth_account import (
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/eth_account/__init__.py", line 1, in <module>
from eth_account.account import (
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/eth_account/account.py", line 11, in <module>
from eth_keyfile import (
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/eth_keyfile/__init__.py", line 7, in <module>
from eth_keyfile.keyfile import (  # noqa: F401
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/eth_keyfile/keyfile.py", line 6, in <module>
from Crypto.Cipher import AES
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/Crypto/Cipher/__init__.py", line 27, in <module>
from Crypto.Cipher._mode_ecb import _create_ecb_cipher
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/Crypto/Cipher/_mode_ecb.py", line 35, in <module>
raw_ecb_lib = load_pycryptodome_raw_lib("Crypto.Cipher._raw_ecb", """
File "/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/Crypto/Util/_raw_api.py", line 309, in load_pycryptodome_raw_lib
raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Not found '_raw_ecb.cpython-39-darwin.so', Cannot load '_raw_ecb.abi3.so': dlopen(/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/Crypto/Util/../Cipher/_raw_ecb.abi3.so, 0x0006): tried: '/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/Crypto/Util/../Cipher/_raw_ecb.abi3.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e))), '/Users/leonkc/solidityDemo/web3_py_simple_storage/myvenv/lib/python3.9/site-packages/Crypto/Cipher/_raw_ecb.abi3.so' (mach-o file, but is an incompatible architecture (have (x86_64), need (arm64e))), Not found '_raw_ecb.so'

I’ve tried reinstalling pycryptodome as per another Stack Overflow answer but I still get the same error (with pip install pycryptodome --no-cache-dir --verbose --force-reinstall )

I’m using a Ganache Ethereum blockchain with an unimportant test address.

Asked By: Leon KC

||
Source

Answers:

I have almost the same issue.
It is very strange but, I manage to use web3 "Crypto.Cipher._raw_ecb" lib by reinstall some libs.
I explain.
To manage virtual environment, I use miniforge (https://github.com/conda-forge/miniforge).
From a new fresh env : conda create --name envTest python=3.10
I install web3 with pip (because no osx-arm64 build in conda):
pip install web3 --no-cache-dir --verbose --force-reinstall
After I reinstalled conda version of 2 libs :
pip uninstall pycryptodome (v3.16.0)
conda install pycryptodome (v3.15.0)
pip uninstall lru-dict (v1.1.8)
conda install lru-dict (v1.1.8)

For the moment, it works for me.
I think it is not a final solution.
You can try.

If something gone wrong with conda, because it just use cache and don’t take the new fresh builds, use this command : conda clean -p : purge all packages caches on your local storage.

Answered By: jeugregg

Update: I finally ended up fixing this. It was a combination of jeugregg‘s suggestion here and luchanos‘s solution in this thread.

That is to say, the things that finally made it work for me were:

  • Downloading Python 3.9.5 from https://www.python.org/downloads/macos/
    with "Download macOS 64-bit universal2 installer"
  • Use python3-intel64 command to run Python instead of just python
  • Create new venv in project from the base interpreter that you installed
  • Install all requirements to venv
  • Uninstall pycryptodome with pip (I think I had to do pip uninstall pycryptodome twice) and reinstalling with conda install pycryptodome (crucial step)
  • If I recall correctly I had to do similar reinstallations with a few other libraries (e.g. cytoolz, bitarray), which came up as errors when trying to install eth-brownie and which I addressed ad-hoc, but this may not be a problem for you
  • pipx install eth-brownie=1.14.6

Python 3.9.5 and brownie version 1.14.6 seem to be the best combination.

Answered By: Leon KC