Modules are installed using pip on OSX but not found when importing
Question:
I successfully install different modules using pip and they are shown in the
pip list
such as:
beautifulsoup4 (4.4.1)
requests (2.10.0)
Scrapy (1.1.0)
From Terminal
However, whenever I try to import it
import beautifulsoup4 / import bs4 or import Scrapy or import requests
the following error is shown:
$ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
Update: if I launch python when I am at the correct site-packages directory
$ pwd
/usr/local/lib/python2.7/site-packages
$ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
>>> import requests
>>> import bs4
>>> import scrapy
Then it works. This would solve the issue if writing directly on the Terminal. However, I have no clue about how to make it work inside a file.py, which will be the normal use.
As far as I know, I only have Python2.7 installed.
From file.py
If I have a file.py saved in some local folder. This contains, for instance
import requests
from bs4 import BeautifulSoup
when I try
python file.py
I get the same error.
Approach
Same happens with any other module from the list.
I would think pip is installing them in a directory that Python is not reading, but as per what I read, it is the correct one.
They are all installed here:
/usr/local/lib/python2.7/site-packages
Output requested by Padraic Cunningham:
$ which -a pip
/usr/local/bin/pip
$ which -a python
/usr/bin/python
/usr/local/bin/python
Output requested by leovp:
$ pip -V
pip 8.1.2 from /usr/local/lib/python2.7/site-packages (python 2.7)
Threads already checked
I have checked the following threads, but unfortunately they did not help me to solve the issue:
- installing pyside using PIP – nmake not found
- PIp installs but module is not found ==> might have provided the right answer, but the links given do not work anymore
- google.protobuf installed, but module not found
- Python pip install module is not found. How to link python to pip location?
Any ideas of what the problem is?
Answers:
Since your problem maybe caused due to various reason, I have listed down a few of them here :
- This is possibly because of what ever is stated here : Pip installs but module is not found. Have updated the answer with newer link.
The link you were looking for : https://pythonhosted.org/setuptools/setuptools.html#development-mode
- It may also happen if you have two versions of python installed. If the
pip that you are accessing is of one version & the python interpreter used is another.
So just see to that you are using the same version of python to install and use the package.
You may fix this using alias,
First, set up a shell alias:
alias python=/usr/local/bin/python3
Then, type that at a prompt, or put it in your ~/.bashrc so that whenever you open python from the terminal the correct version opens.
- If both of the above methods don’t work for you then check this :
Here the answer that worked, which is basically what has been explained in the comments of the question. However, I thought it would be useful to have it explained as a clear and well structured answer.
As highlighted, the problem was that I was not using the interpreter that pip was installing for.
The command which shows where pip was installing the modules:
$ which -a pip
/usr/local/bin/pip
and where the different python versions were located:
$ which -a python
/usr/bin/python
/usr/local/bin/python
That is, my system/default python was
/usr/bin/python
while pip was installing for
/usr/local/bin/python
Therefore, I could not import anything I installed when I just typed python, because the /usr/bin/python interpreter was the one started.
Solution
Install pip again specifying the destination of the modules that will be installed. This must be the destination for the system/default python.
This has been done in two steps:
-
Downloding get-pip.py from bootstrap.pypa.io/get-pip.py. (You may need to use the deprecated one for Python 2: bootstrap.pypa.io/2.7/get-pip.py)
-
Installing it with the following command
sudo /usr/bin/python get-pip.py
Note that without the sudo I got an error and was not able to install pip.
I have just fixed a similar issue.
To give some background, I install pip with homebrew by executing brew install python. One drawback by executing this command, it will install both python2 and python3(maybe not a disadvantage in some case), then
pip install scrapy
but when I try to import scrapy, it complained ImportError: No module named scrapy.
My Solution:
run brew doctor, it should report you a link is broken, it asks you to run brew link python, you might encounter some errors, but follow the prompt suggestion to move forward, after successfully executing brew link python, everything should work now.
Run brew doctorin the terminal
it should give you a warning that says:
Warning: The following directories do not exist:
/usr/local/sbin
You should create these directories and change their ownership to your account.
sudo mkdir -p /usr/local/sbin
sudo chown -R $(whoami) /usr/local/sbin
type sudo mkdir -p /usr/local/sbin in the terminal and hit enter
then type sudo chown -R $(whoami) /usr/local/sbin
then run python and try importing your module again.
I’m adding this in case it helps anyone else out. For me the issue was I was running Anaconda and pip3 was installing to a different directory than Anaconda was linked with. To fix this run conda deactivate. You can reactivate later with conda activate
I was able to fix this problem by running:
$ brew doctor
and got the following:
Consider setting your PATH so that /usr/local/bin occurs before /usr/bin.
Here is a one-liner:
echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bash_profile
Once I ran the one-liner, I was able to access the installed package from /usr/local/bin
In my case I changed the VSCode interpreter to ~/opt/anaconda3/bin/python. This fixed the issue.
I successfully install different modules using pip and they are shown in the
pip list
such as:
beautifulsoup4 (4.4.1)
requests (2.10.0)
Scrapy (1.1.0)
From Terminal
However, whenever I try to import it
import beautifulsoup4 / import bs4 or import Scrapy or import requests
the following error is shown:
$ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
Update: if I launch python when I am at the correct site-packages directory
$ pwd
/usr/local/lib/python2.7/site-packages
$ python
Python 2.7.5 (default, Mar 9 2014, 22:15:05)
>>> import requests
>>> import bs4
>>> import scrapy
Then it works. This would solve the issue if writing directly on the Terminal. However, I have no clue about how to make it work inside a file.py, which will be the normal use.
As far as I know, I only have Python2.7 installed.
From file.py
If I have a file.py saved in some local folder. This contains, for instance
import requests
from bs4 import BeautifulSoup
when I try
python file.py
I get the same error.
Approach
Same happens with any other module from the list.
I would think pip is installing them in a directory that Python is not reading, but as per what I read, it is the correct one.
They are all installed here:
/usr/local/lib/python2.7/site-packages
Output requested by Padraic Cunningham:
$ which -a pip
/usr/local/bin/pip
$ which -a python
/usr/bin/python
/usr/local/bin/python
Output requested by leovp:
$ pip -V
pip 8.1.2 from /usr/local/lib/python2.7/site-packages (python 2.7)
Threads already checked
I have checked the following threads, but unfortunately they did not help me to solve the issue:
- installing pyside using PIP – nmake not found
- PIp installs but module is not found ==> might have provided the right answer, but the links given do not work anymore
- google.protobuf installed, but module not found
- Python pip install module is not found. How to link python to pip location?
Any ideas of what the problem is?
Since your problem maybe caused due to various reason, I have listed down a few of them here :
- This is possibly because of what ever is stated here : Pip installs but module is not found. Have updated the answer with newer link.
The link you were looking for : https://pythonhosted.org/setuptools/setuptools.html#development-mode
- It may also happen if you have two versions of python installed. If the
pipthat you are accessing is of one version & the python interpreter used is another.
So just see to that you are using the same version of python to install and use the package.
You may fix this using alias,
First, set up a shell alias:
alias python=/usr/local/bin/python3
Then, type that at a prompt, or put it in your ~/.bashrc so that whenever you open python from the terminal the correct version opens.
- If both of the above methods don’t work for you then check this :
Here the answer that worked, which is basically what has been explained in the comments of the question. However, I thought it would be useful to have it explained as a clear and well structured answer.
As highlighted, the problem was that I was not using the interpreter that pip was installing for.
The command which shows where pip was installing the modules:
$ which -a pip
/usr/local/bin/pip
and where the different python versions were located:
$ which -a python
/usr/bin/python
/usr/local/bin/python
That is, my system/default python was
/usr/bin/python
while pip was installing for
/usr/local/bin/python
Therefore, I could not import anything I installed when I just typed python, because the /usr/bin/python interpreter was the one started.
Solution
Install pip again specifying the destination of the modules that will be installed. This must be the destination for the system/default python.
This has been done in two steps:
-
Downloding
get-pip.pyfrom bootstrap.pypa.io/get-pip.py. (You may need to use the deprecated one for Python 2: bootstrap.pypa.io/2.7/get-pip.py) -
Installing it with the following command
sudo /usr/bin/python get-pip.py
Note that without the sudo I got an error and was not able to install pip.
I have just fixed a similar issue.
To give some background, I install pip with homebrew by executing brew install python. One drawback by executing this command, it will install both python2 and python3(maybe not a disadvantage in some case), then
pip install scrapy
but when I try to import scrapy, it complained ImportError: No module named scrapy.
My Solution:
run brew doctor, it should report you a link is broken, it asks you to run brew link python, you might encounter some errors, but follow the prompt suggestion to move forward, after successfully executing brew link python, everything should work now.
Run brew doctorin the terminal
it should give you a warning that says:
Warning: The following directories do not exist:
/usr/local/sbin
You should create these directories and change their ownership to your account.
sudo mkdir -p /usr/local/sbin
sudo chown -R $(whoami) /usr/local/sbin
type sudo mkdir -p /usr/local/sbin in the terminal and hit enter
then type sudo chown -R $(whoami) /usr/local/sbin
then run python and try importing your module again.
I’m adding this in case it helps anyone else out. For me the issue was I was running Anaconda and pip3 was installing to a different directory than Anaconda was linked with. To fix this run conda deactivate. You can reactivate later with conda activate
I was able to fix this problem by running:
$ brew doctor
and got the following:
Consider setting your PATH so that
/usr/local/binoccurs before/usr/bin.Here is a one-liner:
echo 'export PATH="/usr/local/bin:$PATH"' >> ~/.bash_profile
Once I ran the one-liner, I was able to access the installed package from /usr/local/bin
In my case I changed the VSCode interpreter to ~/opt/anaconda3/bin/python. This fixed the issue.