How can I use WebDriverWait from Selenium properly through Python?

Question

I just thought I’d add an edit now that this has been resolved. Replacing those two time.sleep() took my program from 180 seconds down to 30. WebDriverWait creates a substantial improvement in runtime.

I am just trying to determine if I’m setting up WebDriverWait correctly. This is my working script, I use time.sleep()

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        time.sleep(5)
        element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
        time.sleep(5)

In short, it goes through a table, clicks a button which opens a pop up, extracts text from pop up, and closes said pop up. I have to wait for the pop up to open, and to fully close. I’m currently using time.sleep(), but I’m trying to switch to WebDriverWait. This is how I’ve implemented it.

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        element = WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
        time.sleep(5)

However, the above does not work. And I receive this error:

driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button').click()
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable

This is gone if I return to time.sleep(), thus making me think I must have setup my WebDriverWait incorrectly. The waiting is for the opening and closing of the browser, so we have to wait till the button appears, thus I put the xpath of the button itself in the WebDriverWait. Is this the proper setup?

Edit:
Thank you @DebanjanB for the answer. However, I have come across another issue in trying to remove my time.sleep(), this is what I currently have.

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
        driver.find_element_by_xpath(f'//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr[{x}]/td[9]/span[2]').click()
        time.sleep(5)
        #WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')))
        seq_info=driver.find_element_by_xpath('//*[@id="gwzViewResultsModalDialog"]/div/div/div[2]/div')
        seq_list.append([seq_info.text])
        WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button'))).click()

The time.sleep() above is waiting for the pop up to open (from the previous click), before it extracts the data. If I remove the time.sleep with the below #WebDriver, it still moves forward, but for some reason seq_info.text is now blank (it no longer finds the text). I don’t quite understand why this is happening. This is not a click or button, I just wanted to check that the pop up is open before I extract the info from it.

Asked By: samman

||

Source

Answer 1

Short answer, no, though syntactically correct, but you aren’t using WebDriverWait optimally.

Along with WebDriverWait you are also using time.sleep().

time.sleep(secs)

time.sleep(secs) suspends the execution of the current thread for the given number of seconds. The argument may be a floating point number to indicate a more precise sleep time. The actual suspension time may be less than that requested because any caught signal will terminate the sleep() following execution of that signal’s catching routine. Also, the suspension time may be longer than requested by an arbitrary amount because of the scheduling of other activity in the system.

You can find a detailed discussion in How to sleep webdriver in python for milliseconds

Moreover,

In the for loop as you intend to iterate instead of /tr[{x}] you need //tr[{x}]
To collect the desired text you you need to use visibility_of_element_located().
<button> are interactive in nature, so instead of presence_of_element_located() you need to use element_to_be_clickable() just when you need to interact with them.

A probable solution:

for x,sequence in enumerate(table.find_elements_by_xpath('//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody/tr/td[9]'),1):
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzSngrOrderResultPanelRoot"]/table/tbody//tr[{x}]/td[9]/span[2]'))).click()
      seq_list.append(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[@id='gwzViewResultsModalDialog']/div/div/div[2]/div"))).text)
      WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="gwzViewResultsModalDialog"]/div/div/div[3]/button'))).click()

Answered By: undetected Selenium

How can I use WebDriverWait from Selenium properly through Python?

Question:

Answers:

time.sleep(secs)