How can I download a csv file from a website whose element is a button or rather the csv is accessible via a button?

Question

I’m trying to download or rather read a csv from a website. The csv is hidden under a button. Here is the website link.

The csv I’m trying to save is on the furthest right and is denoted as a blue button labelled ‘Download all data in .csv’

Here is a sample code I tried to use to download it, but the download fails:

# import the required libraries
from selenium import webdriver
from selenium.webdriver.common.by import By

options = webdriver.ChromeOptions()
prefs = {"download.default_directory" : "D:/Profession/Data Extraction and Web Scraping/Stocks Data Extraction - Core Scientific/Output"}

#example: prefs = {"download.default_directory" : "C:Tutorialdown"};
options.add_experimental_option("prefs",prefs)

driver = webdriver.Chrome(executable_path="D:/Software/Selenium WebDrivers/chromedriver_win32/chromedriver", options=options)

try:
    driver.implicitly_wait(5)
    driver.get("https://defillama.com/chains")
    downloadcsv = driver.find_element(By.CLASS_NAME, 'sc-8f0f10aa-1')
    download_button = downloadcsv.find_element(By.TAG_NAME, 'button')
    download_button.click() # this should save it to the folder I specified in prefs
    time.sleep(3)
    driver.close()
except:
    print('There is an Error!')

This code tries to download the csv, however the download fails. Is there a better way I could download the CSV, especially one that is under a button? Thanks!

Asked By: Steel8

||

Source

Answer 1

Looks like you need to use a proper wait method.
implicitly_wait waits for element presence while you need to wait for element to be clickable, this is more mature element state. WebDriverWait is the best practice to be used in such (and almost all the others) cases.
The following code works:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:webdriverschromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 10)

url = "https://defillama.com/chains"

driver.get(url)
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".sc-8f0f10aa-1 button"))).click()

Answered By: Prophet

How can I download a csv file from a website whose element is a button or rather the csv is accessible via a button?

Question:

Answers: