Pyenv's python is missing bzip2 module
Question:
I used pyenv to install python 3.8.2 and to create a virtualenv.
In the virtualenv, I used pipenv to install pandas.
However, when importing pandas, I’m getting the following:
[...]
File "/home/luislhl/.pyenv/versions/poc-prefect/lib/python3.8/site-packages/pandas/io/common.py", line 3, in <module>
import bz2
File "/home/luislhl/.pyenv/versions/3.8.2/lib/python3.8/bz2.py", line 19, in <module>
from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'
After some googling, I found out some people suggesting I rebuild Python from source after installing bzip2 library in my system.
However, after trying installing it with sudo dnf install bzip2-devel I see that I already had it installed.
As far as I know, pyenv builds python from source when installing some version.
So, why wasn’t it capable of including the bzip2 module when building?
How can I manage to rebuild Python using pyenv in order to make bzip2 available?
I’m in Fedora 30
Thanks in advance
UPDATE
I tried installing another version of python with pyenv in verbose mode, to see the compilation output.
There is this message in the end of the compilation:
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
But as I stated before, I checked I already have bzip2 installed in my system. So I don’t know what to do.
Answers:
Ok, I have found the solution after some time. It was simple, but I took some time to realize it.
It turns out the problem was the bzip2-devel I had installed was a 32-bit version.
The compilation process was looking for the 64-bit one, and didn’t find it.
So I had to specifically install the 64-bit version:
sudo dnf install bzip2-devel-1.0.6-29.fc30.x86_64
On macOS Big Sur, to get pyenv ( via homebrew ) to work I had to install zlib and bzip2 via homebrew and then add the exports in my ~/.zshrc ( or ~/.bashrc for bash I guess). The answer above by luislhl leads the way to my solution.
brew install zlib bzip2
#Add the following to your ~/.zshrc
# For pyenv to build
export LDFLAGS="-L/usr/local/opt/zlib/lib -L/usr/local/opt/bzip2/lib"
export CPPFLAGS="-I/usr/local/opt/zlib/include -I/usr/local/opt/bzip2/include"
# Then the install worked
pyenv install 3.7.9
Thanks, that helped, just with small modification in ~/.zshrc:
export LDFLAGS="-L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/bzip2/lib"
export CPPFLAGS="-I/opt/homebrew/opt/zlib/include -I/opt/homebrew/opt/bzip2/include"
and then pyenv install 3.7.9
Apple M1, macOS 11.1 20C69 arm64;
➜ brew --version
Homebrew 2.7.1
Homebrew/homebrew-core (git revision ad6fd8; last commit 2021-01-05)
Homebrew/homebrew-cask (git revision 5c3de; last commit 2021-01-04)
but this didn’t help with No module named '_ctypes' on M1 🙁
On Ubuntu 22 LTS
Missing Library Problem in Python Installation with Pyenv
Before the fix:
$> pyenv install 3.11.0
command result:
pyenv: /home/user/.pyenv/versions/3.11.0 already exists
continue with installation? (y/N) y
Downloading Python-3.11.0.tar.xz...
-> https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz
Installing Python-3.11.0...
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
WARNING: The Python lzma extension was not compiled. Missing the lzma lib?
TLDR;
Recipe to fix:
sudo apt-get install build-essential zlib1g-dev libffi-dev libssl-dev libbz2-dev libreadline-dev libsqlite3-dev liblzma-dev
Result
After the fix:
$> pyenv install 3.11.0
Command result:
pyenv: /home/user/.pyenv/versions/3.11.0 already exists
continue with installation? (y/N) y
Downloading Python-3.11.0.tar.xz...
-> https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz
Installing Python-3.11.0...
Installed Python-3.11.0 to /home/user/.pyenv/versions/3.11.0
Deve ter ocorrido algum erro no momento da instalação das dependencias do pyenv, recomendo você abrir seu terminal e digitar o comandos abaixo referente a sua distro
Ubuntu/Debian/Mint:
sudo apt update; sudo apt install build-essential libssl-dev zlib1g-dev
libbz2-dev libreadline-dev libsqlite3-dev curl
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
Fonte (Veja sua o comando pra sua distro aqui):
https://github.com/pyenv/pyenv/wiki#suggested-build-environment
I used pyenv to install python 3.8.2 and to create a virtualenv.
In the virtualenv, I used pipenv to install pandas.
However, when importing pandas, I’m getting the following:
[...]
File "/home/luislhl/.pyenv/versions/poc-prefect/lib/python3.8/site-packages/pandas/io/common.py", line 3, in <module>
import bz2
File "/home/luislhl/.pyenv/versions/3.8.2/lib/python3.8/bz2.py", line 19, in <module>
from _bz2 import BZ2Compressor, BZ2Decompressor
ModuleNotFoundError: No module named '_bz2'
After some googling, I found out some people suggesting I rebuild Python from source after installing bzip2 library in my system.
However, after trying installing it with sudo dnf install bzip2-devel I see that I already had it installed.
As far as I know, pyenv builds python from source when installing some version.
So, why wasn’t it capable of including the bzip2 module when building?
How can I manage to rebuild Python using pyenv in order to make bzip2 available?
I’m in Fedora 30
Thanks in advance
UPDATE
I tried installing another version of python with pyenv in verbose mode, to see the compilation output.
There is this message in the end of the compilation:
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
But as I stated before, I checked I already have bzip2 installed in my system. So I don’t know what to do.
Ok, I have found the solution after some time. It was simple, but I took some time to realize it.
It turns out the problem was the bzip2-devel I had installed was a 32-bit version.
The compilation process was looking for the 64-bit one, and didn’t find it.
So I had to specifically install the 64-bit version:
sudo dnf install bzip2-devel-1.0.6-29.fc30.x86_64
On macOS Big Sur, to get pyenv ( via homebrew ) to work I had to install zlib and bzip2 via homebrew and then add the exports in my ~/.zshrc ( or ~/.bashrc for bash I guess). The answer above by luislhl leads the way to my solution.
brew install zlib bzip2
#Add the following to your ~/.zshrc
# For pyenv to build
export LDFLAGS="-L/usr/local/opt/zlib/lib -L/usr/local/opt/bzip2/lib"
export CPPFLAGS="-I/usr/local/opt/zlib/include -I/usr/local/opt/bzip2/include"
# Then the install worked
pyenv install 3.7.9
Thanks, that helped, just with small modification in ~/.zshrc:
export LDFLAGS="-L/opt/homebrew/opt/zlib/lib -L/opt/homebrew/opt/bzip2/lib"
export CPPFLAGS="-I/opt/homebrew/opt/zlib/include -I/opt/homebrew/opt/bzip2/include"
and then pyenv install 3.7.9
Apple M1, macOS 11.1 20C69 arm64;
➜ brew --version
Homebrew 2.7.1
Homebrew/homebrew-core (git revision ad6fd8; last commit 2021-01-05)
Homebrew/homebrew-cask (git revision 5c3de; last commit 2021-01-04)
but this didn’t help with No module named '_ctypes' on M1 🙁
On Ubuntu 22 LTS
Missing Library Problem in Python Installation with Pyenv
Before the fix:
$> pyenv install 3.11.0
command result:
pyenv: /home/user/.pyenv/versions/3.11.0 already exists
continue with installation? (y/N) y
Downloading Python-3.11.0.tar.xz...
-> https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz
Installing Python-3.11.0...
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
WARNING: The Python lzma extension was not compiled. Missing the lzma lib?
TLDR;
Recipe to fix:
sudo apt-get install build-essential zlib1g-dev libffi-dev libssl-dev libbz2-dev libreadline-dev libsqlite3-dev liblzma-dev
Result
After the fix:
$> pyenv install 3.11.0
Command result:
pyenv: /home/user/.pyenv/versions/3.11.0 already exists
continue with installation? (y/N) y
Downloading Python-3.11.0.tar.xz...
-> https://www.python.org/ftp/python/3.11.0/Python-3.11.0.tar.xz
Installing Python-3.11.0...
Installed Python-3.11.0 to /home/user/.pyenv/versions/3.11.0
Deve ter ocorrido algum erro no momento da instalação das dependencias do pyenv, recomendo você abrir seu terminal e digitar o comandos abaixo referente a sua distro
Ubuntu/Debian/Mint:
sudo apt update; sudo apt install build-essential libssl-dev zlib1g-dev
libbz2-dev libreadline-dev libsqlite3-dev curl
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
Fonte (Veja sua o comando pra sua distro aqui):
https://github.com/pyenv/pyenv/wiki#suggested-build-environment
