Selenium python driver doesn't click or press the key for the button all the times

Question:

I’m using selenium to get to YouTube and write something on the search bar and then press the button or press the enter key.

Both clicking or pressing a key does sometimes work, but sometimes it does not.
I tried to wait with WebDriverWait, and I even changed the waiting time from 10 to 20 seconds, but it didn’t make any difference.

And if I add anything (like printing the new page title), it only shows me the first page title and not the title after the search.

Here is my code and what I tried:

from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


def get_driver():
    firefox_options = Options()
    # firefox_options.add_argument("--headless")
    driver = webdriver.Firefox(executable_path=r"C:Program FilesMozilla Firefoxgeckodriver.exe", options=firefox_options)
    driver.implicitly_wait(9)
    return driver

driver = get_driver()

driver.get('https://www.youtube.com/')
search = driver.find_element(By.XPATH, '//input[@id="search"]')
search.send_keys("python")
# search.send_keys(Keys.ENTER) #using the enter key # If I add nothing after this line it work
# searchbutton = driver.find_element(By.XPATH,'//*[@id="search-icon-legacy"]') # This also dose doesn't work
# searchbutton.click() # using the click method() #also dose not work
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//*[@id="search-icon-legacy"]'))).click() # Sometimes work
# driver.implicitly_wait(10)
# print(driver.title) # This show me only the title of the first page not the one after the search

Is it because I use the Firefox webdriver (should I change to Chrome)?
Or is it because of my internet connection?

Asked By: zaki_zardo

||
Source

Answers:

To make this working you need to click the search field input first, then add a short delay and then send the Keys.ENTER or click search-icon-legacy element.
So, this is not your fault, this is how YouTube webpage works. You may even call it a kind of bug. But since this webpage it built for human users it works good since human will never click on the input field and insert the search value there within zero time.
Anyway, the 2 following codes are working:
First.

import time

from selenium import webdriver
from selenium.webdriver import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")
options.add_argument('--disable-notifications')

webdriver_service = Service('C:webdriverschromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 10)

url = "https://www.youtube.com/"
driver.get(url)

search = wait.until(EC.element_to_be_clickable((By.XPATH, '//input[@id="search"]')))
search.click()
time.sleep(0.2)
search.send_keys("python")
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[@id="search-icon-legacy"]'))).click()

Second.

import time

from selenium import webdriver
from selenium.webdriver import Keys
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")
options.add_argument('--disable-notifications')

webdriver_service = Service('C:webdriverschromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 10)

url = "https://www.youtube.com/"
driver.get(url)

search = wait.until(EC.element_to_be_clickable((By.XPATH, '//input[@id="search"]')))
search.click()
time.sleep(0.2)
search.send_keys("python" + Keys.ENTER)
Answered By: Prophet