Unable to locate elements in Selenium (Python)
Question:
I tried to use Selenium with Chrome, but I was unable to find elements on the page. I tried it with link text, XPath, and full XPath, but there was just one error and it wasn’t clicking on the element.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver= webdriver.Chrome()
driver.get("https://www.youtube.com/")
print(driver.title)
driver.find_element_by_xpath("//*[@id='search']").click()
Output:
UserWarning: find_element_by_* commands are deprecated. Please use find_element() instead
warnings.warn("find_element_by_* commands are deprecated. Please use find_element() instead")
I’m using ChromeDriver 81.0.4044.69 and version 81.0.4044.113 of the browser.
Answers:
As per your comment, you are using Selenium WebDriver version 4.0.0a5 which is not stable though. There is the potential that features may be added/removed between these releases. You may switch back to 3.141.59 and give it a try:
driver = webdriver.Chrome(executable_path="C:New folderchromedriver.exe",chrome_options=chrome_options)
url = 'http://www.youtube.com'
driver.get(url)
driver.maximize_window()
wait = WebDriverWait(driver, 20)
element = wait.until(EC.presence_of_element_located((By.XPATH, "//form[@id='search-form']//div[@id='container']//div[@id='search-input']//input")))
actionChains = ActionChains(driver)
actionChains.move_to_element(element).click().perform()
actionChains.move_to_element(element).send_keys("Test",Keys.RETURN).perform()
Note: please add the below imports to your solution
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
I changed the xpath of the search box to (there are multiple elements with id ‘search’):
driver.find_element_by_xpath("//input[@id='search']").click()
I then ran your code and it worked with no errors or warnings. Do this fix it for you?
Also, I don’t know what you’re trying to accomplish overall, but if you want to search something then you don’t need to click the search bar first, you could just do.
driver.find_element_by_xpath("//input[@id='search']").send_keys("Your search query")
driver.find_element_by_xpath("//button[@id='search-icon-legacy']").click()
As for the warning:
"find_element_by_* commands are deprecated. Please use find_element() instead"
I believe that is because selenium is wanting you to use:
from selenium.webdriver.common.by import By
driver.find_element(By.XPATH, '//input[@id='search']')
I just found something in the Selenium documentation. Try to use this:
driver.findElement(By.id("search"))
It’s just because of the version.
pip install selenium==3.14.0
You would have to first import the By from Selenium:
from selenium.webdriver.common.by import By
driver.find_element(By.XPATH, '//input[@id='search']')
This would work.
This error message…
UserWarning: find_element_by_* commands are deprecated. Please use find_element() instead
warnings.warn("find_element_by_* commands are deprecated. Please use find_element() instead")
…implies that the find_element_by_* commands are now associated with a deprecation warning in the latest Selenium Python libraries.
This change is in line with the Selenium 4 Release Candidate 1 changelog which mentions:
Specify that the "find_element_by_* …" warning is a deprecation warning (#9700)
Solution
Instead of find_element_by_* you have to use find_element(). As an example:
You need to add the following import:
from selenium.webdriver.common.by import By
-
Using xpath:
driver.find_element_by_xpath("//*[@id='search']").click()
Needs be replaced with:
driver.find_element(By.XPATH, "//*[@id='search']").click()
Likewise you also have to consider the following changes:
-
Using class_name:
element = find_element_by_class_name("element_class_name")
Needs be replaced with:
element = driver.find_element(By.CLASS_NAME, "element_class_name")
-
Using id:
element = find_element_by_id("element_id")
Needs be replaced with:
element = driver.find_element(By.ID, "element_id")
-
Using name:
element = find_element_by_name("element_name")
Needs be replaced with:
element = driver.find_element(By.NAME, "element_name")
-
Using link_text:
element = find_element_by_link_text("element_link_text")
Needs be replaced with:
element = driver.find_element(By.LINK_TEXT, "element_link_text")
-
Using partial_link_text:
element = find_element_by_partial_link_text("element_partial_link_text")
Needs be replaced with:
element = driver.find_element(By.PARTIAL_LINK_TEXT, "element_partial_link_text")
-
Using tag_name:
element = find_element_by_tag_name("element_tag_name")
Needs be replaced with:
element = driver.find_element(By.TAG_NAME, "element_tag_name")
-
Using css_selector:
element = find_element_by_css_selector("element_css_selector")
Needs be replaced with:
element = driver.find_element(By.CSS_SELECTOR, "element_css_selector")
tl; dr
You can find a couple of relevant detailed discussions in:
I tried to use Selenium with Chrome, but I was unable to find elements on the page. I tried it with link text, XPath, and full XPath, but there was just one error and it wasn’t clicking on the element.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver= webdriver.Chrome()
driver.get("https://www.youtube.com/")
print(driver.title)
driver.find_element_by_xpath("//*[@id='search']").click()
Output:
UserWarning: find_element_by_* commands are deprecated. Please use find_element() instead
warnings.warn("find_element_by_* commands are deprecated. Please use find_element() instead")
I’m using ChromeDriver 81.0.4044.69 and version 81.0.4044.113 of the browser.
As per your comment, you are using Selenium WebDriver version 4.0.0a5 which is not stable though. There is the potential that features may be added/removed between these releases. You may switch back to 3.141.59 and give it a try:
driver = webdriver.Chrome(executable_path="C:New folderchromedriver.exe",chrome_options=chrome_options)
url = 'http://www.youtube.com'
driver.get(url)
driver.maximize_window()
wait = WebDriverWait(driver, 20)
element = wait.until(EC.presence_of_element_located((By.XPATH, "//form[@id='search-form']//div[@id='container']//div[@id='search-input']//input")))
actionChains = ActionChains(driver)
actionChains.move_to_element(element).click().perform()
actionChains.move_to_element(element).send_keys("Test",Keys.RETURN).perform()
Note: please add the below imports to your solution
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
I changed the xpath of the search box to (there are multiple elements with id ‘search’):
driver.find_element_by_xpath("//input[@id='search']").click()
I then ran your code and it worked with no errors or warnings. Do this fix it for you?
Also, I don’t know what you’re trying to accomplish overall, but if you want to search something then you don’t need to click the search bar first, you could just do.
driver.find_element_by_xpath("//input[@id='search']").send_keys("Your search query")
driver.find_element_by_xpath("//button[@id='search-icon-legacy']").click()
As for the warning:
"find_element_by_* commands are deprecated. Please use find_element() instead"
I believe that is because selenium is wanting you to use:
from selenium.webdriver.common.by import By
driver.find_element(By.XPATH, '//input[@id='search']')
I just found something in the Selenium documentation. Try to use this:
driver.findElement(By.id("search"))
It’s just because of the version.
pip install selenium==3.14.0
You would have to first import the By from Selenium:
from selenium.webdriver.common.by import By
driver.find_element(By.XPATH, '//input[@id='search']')
This would work.
This error message…
UserWarning: find_element_by_* commands are deprecated. Please use find_element() instead
warnings.warn("find_element_by_* commands are deprecated. Please use find_element() instead")
…implies that the find_element_by_* commands are now associated with a deprecation warning in the latest Selenium Python libraries.
This change is in line with the Selenium 4 Release Candidate 1 changelog which mentions:
Specify that the "find_element_by_* …" warning is a deprecation warning (#9700)
Solution
Instead of find_element_by_* you have to use find_element(). As an example:
You need to add the following import:
from selenium.webdriver.common.by import By
-
Using
xpath:driver.find_element_by_xpath("//*[@id='search']").click()Needs be replaced with:
driver.find_element(By.XPATH, "//*[@id='search']").click()
Likewise you also have to consider the following changes:
-
Using
class_name:element = find_element_by_class_name("element_class_name")Needs be replaced with:
element = driver.find_element(By.CLASS_NAME, "element_class_name") -
Using
id:element = find_element_by_id("element_id")Needs be replaced with:
element = driver.find_element(By.ID, "element_id") -
Using
name:element = find_element_by_name("element_name")Needs be replaced with:
element = driver.find_element(By.NAME, "element_name") -
Using
link_text:element = find_element_by_link_text("element_link_text")Needs be replaced with:
element = driver.find_element(By.LINK_TEXT, "element_link_text") -
Using
partial_link_text:element = find_element_by_partial_link_text("element_partial_link_text")Needs be replaced with:
element = driver.find_element(By.PARTIAL_LINK_TEXT, "element_partial_link_text") -
Using
tag_name:element = find_element_by_tag_name("element_tag_name")Needs be replaced with:
element = driver.find_element(By.TAG_NAME, "element_tag_name") -
Using
css_selector:element = find_element_by_css_selector("element_css_selector")Needs be replaced with:
element = driver.find_element(By.CSS_SELECTOR, "element_css_selector")
tl; dr
You can find a couple of relevant detailed discussions in: