How to locate the LOAD MORE button using Selenium Python

Question:

I’m having trouble trying to figure out how to locate the path for a "Load More Button" on this site I am trying to scrape.

https://www.ufc.com/athletes/all

The button html:

<a class="button" href="?gender=All&amp;search=&amp;page=1" title="Load more items" 
 rel="next">Load More</a>  
 "Load More" == $0
 ::after == 0$
Asked By: Nick_Lopez43

||
Source

Answers:

You didn’t provide any attempts and I can see 3 right off the top of my head that should work…

  1. CSS selector, a[title='Load more items']
  2. XPath, //a[text()='Load More']
  3. By link text, "Load More"

In general you should prefer CSS selectors because they are more widely supported, faster, and have simpler syntax.

Answered By: JeffC

To click on the element LOAD MORE you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR:

    driver.get('https://www.ufc.com/athletes/all')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[title='Load more items']"))).click()

  • Using XPATH:

    driver.get('https://www.ufc.com/athletes/all')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@id='onetrust-accept-btn-handler']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@title='Load more items']"))).click()

  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Answered By: undetected Selenium

this is the XPATH for that button that you want
you can right-click on an element to inspect and right-click on it to copy XPath

driver.find_element(By.XPATH, ' //*[@id="block- 
 mainpagecontent"]/div/div/div[2]/div/div/ul/li/a').click()
Answered By: Zachary Rizzo