ModuleNotFoundError: No module named 'pandas._libs.interval'
Question:
Suddenly, I can’t import pandas in python. I am using anaconda as package manager, but it seems that no matter how many times I uninstall and install pandas, I still get the same error:
(base) C:>conda install pandas
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:Usersau207178Anaconda3
added / updated specs:
- pandas
The following NEW packages will be INSTALLED:
blas pkgs/main/win-64::blas-1.0-mkl
bottleneck pkgs/main/win-64::bottleneck-1.3.2-py38h2a96729_1
intel-openmp pkgs/main/win-64::intel-openmp-2021.4.0-haa95532_3556
mkl pkgs/main/win-64::mkl-2021.4.0-haa95532_640
mkl-service pkgs/main/win-64::mkl-service-2.4.0-py38h2bbff1b_0
mkl_fft pkgs/main/win-64::mkl_fft-1.3.1-py38h277e83a_0
mkl_random pkgs/main/win-64::mkl_random-1.2.2-py38hf11a4ad_0
numexpr pkgs/main/win-64::numexpr-2.8.1-py38hb80d3ca_0
numpy pkgs/main/win-64::numpy-1.21.2-py38hfca59bb_0
numpy-base pkgs/main/win-64::numpy-base-1.21.2-py38h0829f74_0
pandas pkgs/main/win-64::pandas-1.3.5-py38h6214cd6_0
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(base) C:>python
Python 3.8.12 (default, Oct 12 2021, 03:01:40) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandas__init__.py", line 22, in <module>
from pandas.compat import (
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandascompat__init__.py", line 15, in <module>
from pandas.compat.numpy import (
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandascompatnumpy__init__.py", line 7, in <module>
from pandas.util.version import Version
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandasutil__init__.py", line 1, in <module>
from pandas.util._decorators import ( # noqa
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandasutil_decorators.py", line 14, in <module>
from pandas._libs.properties import cache_readonly # noqa
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandas_libs__init__.py", line 13, in <module>
from pandas._libs.interval import Interval
ModuleNotFoundError: No module named 'pandas._libs.interval'
I have also tried running ‘conda update conda’.
What’s the next thing to do?
Update:
I checked, and the file is definitely there, in "C:Usersau207178AppDataRoamingPythonPython310site-packagespandas_libsinterval.pyx".
However, I notice that part of the path to this library says ‘python310’. However, my python version is 3.8.8. Is it possible that python is somehow looking in the wrong place when trying to import pandas._libs.interval? I have tried both python 3.10 (couldn’t even install pandas) and python 3.9.7 (no effect).
I have also tried conda install –revision 0, which also had no effect…
Answers:
Yes, it appears to be loading pandas from a user-level installation. User-level installs can leak into Conda environments and lead to unpredictable behavior, such as what you are seeing.
There are two routes of action of which I know. You may want to try the second one first, which would confirm the cause. However, the first option is likely more manageable going forward, since once it’s done, the issue should be resolved.
Option 1: Remove External Python(s)
If you would like Conda to simply work as expected, then uninstall the user-level Python. Note that the one detected (Python 3.10) may not be the only one, so you may have to track down multiple copies. I’m not on Windows, so I can’t suggest concrete steps for uninstallation.
However, you may be already using this user-level Python for other projects, so this option may not be practicable.
Option 2: Launch Python with Isolation Flags
There are some pertinent flags that Python provides that has it ignore the various sources that can lead to leaking in other site-packages. Here are the three important ones:
$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
...
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
...
Try them individually, or all together, and see if this fixes the issue. Individually, you may be able to narrow down whether it is due to PYTHONPATH (-E alone would fix) or generic user-level searching (-s alone would fix).
Additional Thoughts
I find it odd that a Python 3.10 installation is leaking into a Python 3.8 interpreter run. Usually, the site module only loads within matching major+minor versions (e.g., 3.8 should only pick up another 3.8). This might be indicative that something else is going on. Perhaps the PYTHONPATH environment is set? (It really never should be.)
Generally, I don’t recommend using the base environment for work. This is never documented in Anaconda documentation, but it is the painful truth that many long-term Conda users come to, usually after their base either breaks or get’s so overloaded with packages that it takes unreasonably long for them to upgrade the conda package. Instead, create a new environment and have that activate by default instead of base.
Only run
pip3 install -U pandas
And should works!!!!
Suddenly, I can’t import pandas in python. I am using anaconda as package manager, but it seems that no matter how many times I uninstall and install pandas, I still get the same error:
(base) C:>conda install pandas
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:Usersau207178Anaconda3
added / updated specs:
- pandas
The following NEW packages will be INSTALLED:
blas pkgs/main/win-64::blas-1.0-mkl
bottleneck pkgs/main/win-64::bottleneck-1.3.2-py38h2a96729_1
intel-openmp pkgs/main/win-64::intel-openmp-2021.4.0-haa95532_3556
mkl pkgs/main/win-64::mkl-2021.4.0-haa95532_640
mkl-service pkgs/main/win-64::mkl-service-2.4.0-py38h2bbff1b_0
mkl_fft pkgs/main/win-64::mkl_fft-1.3.1-py38h277e83a_0
mkl_random pkgs/main/win-64::mkl_random-1.2.2-py38hf11a4ad_0
numexpr pkgs/main/win-64::numexpr-2.8.1-py38hb80d3ca_0
numpy pkgs/main/win-64::numpy-1.21.2-py38hfca59bb_0
numpy-base pkgs/main/win-64::numpy-base-1.21.2-py38h0829f74_0
pandas pkgs/main/win-64::pandas-1.3.5-py38h6214cd6_0
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(base) C:>python
Python 3.8.12 (default, Oct 12 2021, 03:01:40) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandas__init__.py", line 22, in <module>
from pandas.compat import (
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandascompat__init__.py", line 15, in <module>
from pandas.compat.numpy import (
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandascompatnumpy__init__.py", line 7, in <module>
from pandas.util.version import Version
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandasutil__init__.py", line 1, in <module>
from pandas.util._decorators import ( # noqa
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandasutil_decorators.py", line 14, in <module>
from pandas._libs.properties import cache_readonly # noqa
File "C:Usersau207178AppDataRoamingPythonPython310site-packagespandas_libs__init__.py", line 13, in <module>
from pandas._libs.interval import Interval
ModuleNotFoundError: No module named 'pandas._libs.interval'
I have also tried running ‘conda update conda’.
What’s the next thing to do?
Update:
I checked, and the file is definitely there, in "C:Usersau207178AppDataRoamingPythonPython310site-packagespandas_libsinterval.pyx".
However, I notice that part of the path to this library says ‘python310’. However, my python version is 3.8.8. Is it possible that python is somehow looking in the wrong place when trying to import pandas._libs.interval? I have tried both python 3.10 (couldn’t even install pandas) and python 3.9.7 (no effect).
I have also tried conda install –revision 0, which also had no effect…
Yes, it appears to be loading pandas from a user-level installation. User-level installs can leak into Conda environments and lead to unpredictable behavior, such as what you are seeing.
There are two routes of action of which I know. You may want to try the second one first, which would confirm the cause. However, the first option is likely more manageable going forward, since once it’s done, the issue should be resolved.
Option 1: Remove External Python(s)
If you would like Conda to simply work as expected, then uninstall the user-level Python. Note that the one detected (Python 3.10) may not be the only one, so you may have to track down multiple copies. I’m not on Windows, so I can’t suggest concrete steps for uninstallation.
However, you may be already using this user-level Python for other projects, so this option may not be practicable.
Option 2: Launch Python with Isolation Flags
There are some pertinent flags that Python provides that has it ignore the various sources that can lead to leaking in other site-packages. Here are the three important ones:
$ python --help
usage: python [option] ... [-c cmd | -m mod | file | -] [arg] ...
Options and arguments (and corresponding environment variables):
...
-E : ignore PYTHON* environment variables (such as PYTHONPATH)
-s : don't add user site directory to sys.path; also PYTHONNOUSERSITE
-S : don't imply 'import site' on initialization
...
Try them individually, or all together, and see if this fixes the issue. Individually, you may be able to narrow down whether it is due to PYTHONPATH (-E alone would fix) or generic user-level searching (-s alone would fix).
Additional Thoughts
I find it odd that a Python 3.10 installation is leaking into a Python 3.8 interpreter run. Usually, the site module only loads within matching major+minor versions (e.g., 3.8 should only pick up another 3.8). This might be indicative that something else is going on. Perhaps the PYTHONPATH environment is set? (It really never should be.)
Generally, I don’t recommend using the base environment for work. This is never documented in Anaconda documentation, but it is the painful truth that many long-term Conda users come to, usually after their base either breaks or get’s so overloaded with packages that it takes unreasonably long for them to upgrade the conda package. Instead, create a new environment and have that activate by default instead of base.
Only run
pip3 install -U pandas
And should works!!!!