Checking if an element exists with Python Selenium
Question:
I have a problem; I am using the Selenium (Firefox) web driver to open a webpage, click a few links, etc., and then capture a screenshot.
My script runs fine from the CLI, but when run via a cron job it is not getting past the first find_element() test. I need to add some debug, or something to help me figure out why it is failing.
Basically, I have to click a ‘log in’ anchor before going to the login page. The construct of the element is:
<a class="lnk" rel="nofollow" href="/login.jsp?destination=/secure/Dash.jspa">log in</a>
I am using the find_element By LINK_TEXT method:
login = driver.find_element(By.LINK_TEXT, "log in").click()
A) How do I check that the link is actually being picked up by Python? Should I use try/catch block?
B) Is there a better/more reliable way to locate the DOM element than by LINK_TEXT? E.g., in jQuery, you can use a more specific selector, $(‘a.lnk:contains(log in)’).do_something();
I have solved the main problem and it was just finger trouble. I was calling the script with incorrect parameters – a simple mistake.
I’d still like some pointers on how to check whether an element exists. Also, an example/explanation of implicit / explicit Waits instead of using a crappy time.sleep() call.
Answers:
A) Yes. The easiest way to check if an element exists is to simply call find_element inside a try/catch.
B) Yes, I always try to identify elements without using their text for two reasons:
- the text is more likely to change and;
- if it is important to you, you won’t be able to run your tests against localized builds.
The solution is either:
- You can use XPath to find a parent or ancestor element that has an ID or some other unique identifier and then find its child/descendant that matches or;
- you could request an ID or name or some other unique identifier for the link itself.
For the follow-up questions, using try/catch is how you can tell if an element exists or not and good examples of waits can be found here: http://seleniumhq.org/docs/04_webdriver_advanced.html
For a):
from selenium.common.exceptions import NoSuchElementException
def check_exists_by_xpath(xpath):
try:
webdriver.find_element_by_xpath(xpath)
except NoSuchElementException:
return False
return True
For b): Moreover, you can take the XPath expression as a standard throughout all your scripts and create functions as above mentions for universal use.
I recommend to use CSS selectors. I recommend not to mix/use "by id", "by name", etc. and use one single approach instead.
You can grab a list of elements instead of a single element. An empty list in python is falsey. Example:
if driver.find_elements(By.CSS_SELECTOR, '#element'):
print "Element exists!"
You can also use By.ID and By.NAME, but that just turns your id or name into a css selector anyway. Source
The same as Brian, but add to this answer from tstempko:
I tried and it works quickly:
driver.implicitly_wait(0)
if driver.find_element_by_id("show_reflist"):
driver.find_element_by_id("show_reflist").find_element_by_tag_name("img").click()
After this, I restore my default value:
driver.implicitly_wait(30)
You could also do it more concisely using
driver.find_element_by_id("some_id").size != 0
driver.find_element_by_id("some_id").size() is a class method.
We need:
driver.find_element_by_id("some_id").size which is a dictionary, so:
if driver.find_element_by_id("some_id").size['width'] != 0:
print 'button exist'
A solution without try&catch and without new imports:
if len(driver.find_elements_by_id('blah')) > 0: # Pay attention: find_element*s*
driver.find_element_by_id('blah').click # Pay attention: find_element
You could use is_displayed() like below:
res = driver.find_element_by_id("some_id").is_displayed()
assert res, 'element not displayed!'
You can find elements by available methods and check response array length if the length of an array equal the 0 element not exist.
element_exist = False if len(driver.find_elements_by_css_selector('div.eiCW-')) > 0 else True
When you know the element could not be there, the implicit wait could be a problem. I’ve created a simple context manager to avoid those waiting times
class PauseExplicitWait(object):
"""
Example usage:
with PauseExplicitWait(driver, 0):
driver.find_element(By.ID, 'element-that-might-not-be-there')
"""
def __init__(self, driver, new_wait=0):
self.driver = driver
self.original_wait = driver.timeouts.implicit_wait
self.new_wait = new_wait
def __enter__(self):
self.driver.implicitly_wait(self.new_wait)
def __exit__(self, exc_type, exc_value, exc_tb):
self.driver.implicitly_wait(self.original_wait)
You can check by find_elements. If the result is null, that element does not exist.
if driver.find_elements(By.SOMETHING, "#someselector") == []:
continue # That element does not exist
With the latest Selenium, you can now use you can now use .is_displayed():
https://www.selenium.dev/documentation/webdriver/elements/information/
A) Use .is_displayed() as explained at Information about web elements:
# After navigating to the URL,
# get the Boolean value for if this element is displayed
is_button_visible = driver.find_element(By.CSS_SELECTOR, "[name='login']").is_displayed()
Continue your Visual Studio Code logic using "is_button_visible" for branching.
B) Refer to Sam Woods’s answer
el = WebDriverWait(driver, timeout=3).until(lambda d: d.find_element(By.TAG_NAME,"p"))
I have a problem; I am using the Selenium (Firefox) web driver to open a webpage, click a few links, etc., and then capture a screenshot.
My script runs fine from the CLI, but when run via a cron job it is not getting past the first find_element() test. I need to add some debug, or something to help me figure out why it is failing.
Basically, I have to click a ‘log in’ anchor before going to the login page. The construct of the element is:
<a class="lnk" rel="nofollow" href="/login.jsp?destination=/secure/Dash.jspa">log in</a>
I am using the find_element By LINK_TEXT method:
login = driver.find_element(By.LINK_TEXT, "log in").click()
A) How do I check that the link is actually being picked up by Python? Should I use try/catch block?
B) Is there a better/more reliable way to locate the DOM element than by LINK_TEXT? E.g., in jQuery, you can use a more specific selector, $(‘a.lnk:contains(log in)’).do_something();
I have solved the main problem and it was just finger trouble. I was calling the script with incorrect parameters – a simple mistake.
I’d still like some pointers on how to check whether an element exists. Also, an example/explanation of implicit / explicit Waits instead of using a crappy time.sleep() call.
A) Yes. The easiest way to check if an element exists is to simply call find_element inside a try/catch.
B) Yes, I always try to identify elements without using their text for two reasons:
- the text is more likely to change and;
- if it is important to you, you won’t be able to run your tests against localized builds.
The solution is either:
- You can use XPath to find a parent or ancestor element that has an ID or some other unique identifier and then find its child/descendant that matches or;
- you could request an ID or name or some other unique identifier for the link itself.
For the follow-up questions, using try/catch is how you can tell if an element exists or not and good examples of waits can be found here: http://seleniumhq.org/docs/04_webdriver_advanced.html
For a):
from selenium.common.exceptions import NoSuchElementException
def check_exists_by_xpath(xpath):
try:
webdriver.find_element_by_xpath(xpath)
except NoSuchElementException:
return False
return True
For b): Moreover, you can take the XPath expression as a standard throughout all your scripts and create functions as above mentions for universal use.
I recommend to use CSS selectors. I recommend not to mix/use "by id", "by name", etc. and use one single approach instead.
You can grab a list of elements instead of a single element. An empty list in python is falsey. Example:
if driver.find_elements(By.CSS_SELECTOR, '#element'):
print "Element exists!"
You can also use By.ID and By.NAME, but that just turns your id or name into a css selector anyway. Source
The same as Brian, but add to this answer from tstempko:
I tried and it works quickly:
driver.implicitly_wait(0)
if driver.find_element_by_id("show_reflist"):
driver.find_element_by_id("show_reflist").find_element_by_tag_name("img").click()
After this, I restore my default value:
driver.implicitly_wait(30)
You could also do it more concisely using
driver.find_element_by_id("some_id").size != 0
driver.find_element_by_id("some_id").size() is a class method.
We need:
driver.find_element_by_id("some_id").size which is a dictionary, so:
if driver.find_element_by_id("some_id").size['width'] != 0:
print 'button exist'
A solution without try&catch and without new imports:
if len(driver.find_elements_by_id('blah')) > 0: # Pay attention: find_element*s*
driver.find_element_by_id('blah').click # Pay attention: find_element
You could use is_displayed() like below:
res = driver.find_element_by_id("some_id").is_displayed()
assert res, 'element not displayed!'
You can find elements by available methods and check response array length if the length of an array equal the 0 element not exist.
element_exist = False if len(driver.find_elements_by_css_selector('div.eiCW-')) > 0 else True
When you know the element could not be there, the implicit wait could be a problem. I’ve created a simple context manager to avoid those waiting times
class PauseExplicitWait(object):
"""
Example usage:
with PauseExplicitWait(driver, 0):
driver.find_element(By.ID, 'element-that-might-not-be-there')
"""
def __init__(self, driver, new_wait=0):
self.driver = driver
self.original_wait = driver.timeouts.implicit_wait
self.new_wait = new_wait
def __enter__(self):
self.driver.implicitly_wait(self.new_wait)
def __exit__(self, exc_type, exc_value, exc_tb):
self.driver.implicitly_wait(self.original_wait)
You can check by find_elements. If the result is null, that element does not exist.
if driver.find_elements(By.SOMETHING, "#someselector") == []:
continue # That element does not exist
With the latest Selenium, you can now use you can now use .is_displayed():
https://www.selenium.dev/documentation/webdriver/elements/information/
A) Use .is_displayed() as explained at Information about web elements:
# After navigating to the URL,
# get the Boolean value for if this element is displayed
is_button_visible = driver.find_element(By.CSS_SELECTOR, "[name='login']").is_displayed()
Continue your Visual Studio Code logic using "is_button_visible" for branching.
B) Refer to Sam Woods’s answer
el = WebDriverWait(driver, timeout=3).until(lambda d: d.find_element(By.TAG_NAME,"p"))