Date search and date output from the same class name when parsing

Question

I’m trying to parse a site. It has the same named classes, while the number of such classes varies from page to page. I’m interested in the class that contains the date, which is written in the following pattern: 24 January 2020.

Code:

import pandas as pd
import numpy as np
import requests
from bs4 import BeautifulSoup
import re
from user_agent import generate_user_agent
import time

my_user_agent = generate_user_agent(os=('mac', 'linux'))

headers = {
    "Accept": "*/*",
    "user-agent": my_user_agent}

source = 'https://coronavirus-graph.ru/kanada'
req = requests.get(source, headers=headers)
html = req.content
soup = BeautifulSoup(html, 'lxml')
items_list = soup.find_all('div', class_='rr')

print(items_list)
[<div class="rr"><span>In Canada, ≈</span><b>11</b><span> people die out of </span><b>1000</b><span> infected ( 1.1%)</span></div>, <div class="rr"><b>99</b><span> a person in serious condition</span></div>, <div class="rr"><span>First person infected</span><b>24 January 2020</b></div>]

In this case, I’m interested in the last class and its value in <b>.
That is the date. How can I get it?

Asked By: kostya ivanov

||

Source

Answer 1

Here is one way of getting that data:

[...]
headers = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36'
}

source = 'https://coronavirus-graph.ru/kanada'
r = requests.get(source, headers=headers)
soup = BeautifulSoup(r.text, 'lxml')
items_list = soup.select('div[class="rr"]')[-1].select_one('b')

print(items_list.text)

Result in terminal:

24 января 2020 г.

Answered By: Barry the Platipus

Answer 2

Use css selectors and pseudo class last-child:

soup.select_one('.rr b:last-child').text

Example

from bs4 import BeautifulSoup

html = '''<div class="rr"><span>In Canada, ≈</span><b>11</b><span> people die out of </span><b>1000</b><span> infected ( 1.1%)</span></div>, <div class="rr"><b>99</b><span> a person in serious condition</span></div>, <div class="rr"><span>First person infected</span><b>24 January 2020</b></div>'''
soup = BeautifulSoup(html)

soup.select_one('.rr b:last-child').text

Answered By: HedgeHog

Date search and date output from the same class name when parsing

Question:

Answers:

Example