How can I get text from an A tag with Selenium?

Question

I have been trying to scrape some products online, but when I try and print the title from an A tag it gives me this output:

<selenium.webdriver.remote.webelement.WebElement (session="48e7924c296324a7a5a843d9ccab36fb", element="b8871651-23af-42c6-a49a-5b93fe932653")>

Now this is my code:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd

PATH = "C:Program Files (x86)chromedriver.exe"

driver = webdriver.Chrome(PATH)
driver.get("https://egypt.souq.com")

dotd = "/html/body/div[2]/div/main/div[1]/div[1]/div/div[1]/a/img"

driver.find_element_by_xpath(dotd).click()

def get_deals():
    title_xpath = "/html/body/div[1]/div/main/div/div[4]/div[3]/div[2]/div[1]/div[1]/div/div[2]/ul/li[1]/h6/span/a"
    titles = driver.find_elements_by_xpath(title_xpath)
    for title in titles:
        print(title)

get_deals()
print("successful")

Asked By: eyadali05

||

Source

Answer 1

The problem is that you are printing the object, which contains all the properties, instead of the text property.

So, the only thing that you need to change is, instead of using print(title), use print(title.text)

Answered By: Yoshi

Answer 2

This output from print()…

<selenium.webdriver.remote.webelement.WebElement (session="48e7924c296324a7a5a843d9ccab36fb", element="b8871651-23af-42c6-a49a-5b93fe932653")>

…isn’t any error, but the WebElement itself.

It seems you were close. As you were able to extract the element, to extract the text within the element, you can use either of the following Locator Strategies:

Using text attribute:

for title in titles:
    print(title.text)

Using get_attribute() attribute:

for title in titles:
    print(title.get_attribute("innerHTML"))

Reference

You can find a couple of relevant discussions in:

Difference between text and innerHTML using Selenium

Answered By: undetected Selenium

How can I get text from an A tag with Selenium?

Question:

Answers:

Reference