Parser returns either 'None' or '[]'

Question

Here’s the code:

import requests
from bs4 import BeautifulSoup as b
URL = 'https://ege.sdamgia.ru/'
r = requests.get(URL)
soup = b(r.text, 'html.parser')
topics = soup.find_all(class_="Link-U Link_wrap-U Link_pseudo-U Link_pseudoBlack-U")
print(topics)

Please help guys

I want my parser to return topic titles from that site

Asked By: UPTOWN

||

Source

Answer 1

First, install selenium, because this site is dynamic.

pip install selenium
Download chrome or firefox driver (see google)

Examle using chromedriver:

from selenium import webdriver
from bs4 import BeautifulSoup as bs
from time import sleep

URL = 'https://ege.sdamgia.ru/'

driver = webdriver.Chrome("chromedriver.exe")
driver.get(URL)

#wait some time

sleep(5)

soup = bs(driver.page_source, 'html.parser')
topics = soup.find_all(class_="Link-U Link_wrap-U Link_pseudo-U Link_pseudoBlack-U")

print(topics)

Reading more about dynamic sites would be helpful.

Answered By: Ilya Gorunov

Parser returns either 'None' or '[]'

Question:

Answers: