Jupyter notebook and No module named 'bs4'

Question

I’m trying to use beautiful soup however the python kernel can’t find it in jypter. When running:

!pip3 list
!pip3 install BeautifulSoup4
from bs4 import BeautifulSoup

I receive this error:

Package         Version
--------------- -------
beautifulsoup4  4.9.3
bs4             0.0.1
numpy           1.20.1
pandas          1.2.3
pip             21.0.1
python-dateutil 2.8.1
pytz            2021.1
setuptools      53.0.0
six             1.15.0
soupsieve       2.2
wheel           0.36.2
Requirement already satisfied: BeautifulSoup4 in /usr/local/lib/python3.9/site-packages (4.9.3)
Requirement already satisfied: soupsieve>1.2 in /usr/local/lib/python3.9/site-packages (from BeautifulSoup4) (2.2)
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-11-c89f195c7568> in <module>
      1 get_ipython().system('pip3 list')
      2 get_ipython().system('pip3 install BeautifulSoup4')
----> 3 from bs4 import BeautifulSoup
      4 print("scapy time")
      5 for url in stillAlive:

beautiful soup is also installed outside of jupyter through pip

Asked By: Osian

||

Source

Answer 1

I think that the problem is with bs4 package: uninstall bs4 and try again. It overlaps with beautifulsoup4, and because of this you are getting an error.

Answered By: alv2017

Answer 2

Jupiter notebook was not pointing at the correct pip packages. To resolve this I placed this at the top of my code:

import sys
sys.path.append("/usr/local/lib/python3.9/site-packages")

Answered By: Osian

Answer 3

I think this is the problem of the virtual environment. I had same problem and want to share one of the solutions.

Solution: Make sure when you run jupyter notebook in a virtual environment. There should be installed both jupyter notebook and BeautifulSoup. If they are missing, one of them in your existing environment won’t be worked. Confusion raised because the default jupyter notebook launched in any virtual environment even though it hasn’t been installed in the current working environment.

For instance:

Suppose default package pip list:

Package              Version
-------------------- -----------
argon2-cffi          21.3.0
argon2-cffi-bindings 21.2.0
asttokens            2.0.5
attrs                22.1.0
backcall             0.2.0
notebook             6.4.12

This is venv (Virtual Environment) (Missing notebook):

Package              Version
-------------------- -----------
argon2-cffi          21.3.0
argon2-cffi-bindings 21.2.0
asttokens            2.0.5
attrs                22.1.0
backcall             0.2.0
beautifulsoup4       4.11.1

When I run jupyter notebook in my venv, it run from the default package installed. Therefore, it give me following error:

from bs4 import BeautifulSoup

Traceback (most recent call last):
  File "/Users/lyons/Documents/scrape/scrape.py", line 2, in <module>
    from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'

After installation, of the jupyter notebook in venv. It fixed it

Answered By: code_conundrum

Jupyter notebook and No module named 'bs4'

Question:

Answers: