Python/Selenium – Not able to select element by class
Question:
I’m trying to retrieve all the usernames of the people who replied to a specific tweet.
The specific tweet is the following but from what I’ve seen it should work on any tweet.
HTML:
The code I used to try retrieving the usernames:
As you can see I’m trying to retrieve the href where the username of the user who replied is stored via the class name.
TwitterClass = 'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l'
driver.implicitly_wait(10)
users = driver.find_elements(By.CLASS_NAME,TwitterClass)
UserList= [elem.get_attribute('href') for elem in users]
print(UserList)
#Out: []
Although I’ve selected the class name when i run this code I end up getting an empty list.
I haven’t used selenium in a while and I’m still new to it but I’ve been searching posts on StackOverflow that might help and none seems to work I’ve also used a similar code for another project and it worked.
Can somebody help me figure out what am I doing wrong?
Thanks in advance for any answer.
Answers:
The whitespace in between each words in the class attribute css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l is the reason your code is not working and is returning empty list. Whitespace refers to more than one class. So, By.CLASS_NAME locator will not work. You can either use By.CSS_SELECTOR or By.XPATH as below:
Change the below line:
users = driver.find_elements(By.CLASS_NAME,TwitterClass)
To:
users = driver.find_elements(By.XPATH, "//a[contains(@class,'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l')]")
Working code:
driver.get("https://twitter.com/JustYGami/status/1637949323651723264")
# TwitterClass = 'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l'
driver.implicitly_wait(10)
users = driver.find_elements(By.XPATH, "//a[contains(@class,'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l')]")
UserList = [elem.get_attribute('href') for elem in users]
print(UserList)
Console output:
['https://twitter.com/JustYGami', 'https://twitter.com/JustYGami']
Process finished with exit code 0
Note: This code will not guarantee to return all the usernames who replied which you are after. This will only return the list of users which are visible when the page loads.
I’m trying to retrieve all the usernames of the people who replied to a specific tweet.
The specific tweet is the following but from what I’ve seen it should work on any tweet.
HTML:
The code I used to try retrieving the usernames:
As you can see I’m trying to retrieve the href where the username of the user who replied is stored via the class name.
TwitterClass = 'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l'
driver.implicitly_wait(10)
users = driver.find_elements(By.CLASS_NAME,TwitterClass)
UserList= [elem.get_attribute('href') for elem in users]
print(UserList)
#Out: []
Although I’ve selected the class name when i run this code I end up getting an empty list.
I haven’t used selenium in a while and I’m still new to it but I’ve been searching posts on StackOverflow that might help and none seems to work I’ve also used a similar code for another project and it worked.
Can somebody help me figure out what am I doing wrong?
Thanks in advance for any answer.
The whitespace in between each words in the class attribute css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l is the reason your code is not working and is returning empty list. Whitespace refers to more than one class. So, By.CLASS_NAME locator will not work. You can either use By.CSS_SELECTOR or By.XPATH as below:
Change the below line:
users = driver.find_elements(By.CLASS_NAME,TwitterClass)
To:
users = driver.find_elements(By.XPATH, "//a[contains(@class,'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l')]")
Working code:
driver.get("https://twitter.com/JustYGami/status/1637949323651723264")
# TwitterClass = 'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l'
driver.implicitly_wait(10)
users = driver.find_elements(By.XPATH, "//a[contains(@class,'css-4rbku5 css-18t94o4 css-1dbjc4n r-1loqt21 r-1wbh5a2 r-dnmrzs r-1ny4l3l')]")
UserList = [elem.get_attribute('href') for elem in users]
print(UserList)
Console output:
['https://twitter.com/JustYGami', 'https://twitter.com/JustYGami']
Process finished with exit code 0
Note: This code will not guarantee to return all the usernames who replied which you are after. This will only return the list of users which are visible when the page loads.