Find all elements (even outside viewport) using Selenium


Its my understanding that when the find_elements-method is called on a selenium.webdriver-instance, it returns a reference of all the elements in the DOM that matches with the provided locator.

When I run the following code I only get <30 elements whereas there are many more on the page in question.

from selenium import webdriver
from import WebDriverWait
from import By
import as EC

DRIVER_PATH = '/chromedriver'

driver = webdriver.Chrome(executable_path=DRIVER_PATH)
wait = WebDriverWait(driver, 30)

URL = ''
PGA_button = wait.until(EC.presence_of_element_located((By.XPATH, ".//button[@class='btn btn-light' and text()='PGA']")))
driver.execute_script("arguments[0].click();", PGA_button)

logos = driver.find_elements(By.XPATH, ".//*[@class='odds-logo ml-1']")

The desired elements:

enter image description here

How do I do get all the logos? When I debug and scroll the page down I get a list of different elements. So, I assume it has something to do with the viewport.

Is there a way to get all the elements on the page (even if they are outside the viewport)?

Asked By: HJA24



A good strategy is to find all rows visible in the page and then scroll down to the last one so that new rows are loaded in the HTML. In the meanwhile you add the images urls to a set. I choose set instead of list so that duplicate urls are not added, but if you want all the urls then replace the first command with img = [] and img.add with img.append.

img = set()
number_of_loops = 4

for i in range(number_of_loops):
    rows = WebDriverWait(driver, 9).until(EC.visibility_of_all_elements_located((By.XPATH, '//div[@class="ag-center-cols-container"]/div[@role="row"]')))
    row_idx = []

    for row in rows:
        img.add(row.find_element(By.XPATH, './/descendant::div[@class="odds-logo ml-1"]/img').get_attribute('src'))
    idx_of_max = row_idx.index(max(row_idx))
    driver.execute_script('arguments[0].scrollIntoView({block: "center", behavior: "smooth"});', rows[idx_of_max])



{'[email protected]',
 '[email protected]'}
Answered By: sound wave
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.