NoSuchElementException – Cannot find elements

Question

I am doing automation on this url: link

I am using Selenium Webdriver

Unable to get the Time fields and tags.

As you can see there are two fields date and time. Date has calendar picker, I am successful in sending the required date, but unable to do the same with Time field. The time option is basically a dropdown. But I can’t even click the dropdown field. It’s scraping nothing related to it. I have tried many options like with tags, ids and xpath but it returns nosuchelementexception or empty list

url = 'https://disneyland.disney.go.com/en-ca/dining/'
driver.get(url)
driver.find_element(By.ID, "custom-dropdown-options-list")

it returns this

---------------------------------------------------------------------------
NoSuchElementException                    Traceback (most recent call last)
~AppDataLocalTempipykernel_154843697586215.py in <module>
----> 1 driver.find_element(By.ID, "custom-dropdown-options-list")

~anaconda3libsite-packagesseleniumwebdriverremotewebdriver.py in find_element(self, by, value)
    828             value = f'[name="{value}"]'
    829 
--> 830         return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"]
    831 
    832     def find_elements(self, by=By.ID, value: Optional[str] = None) -> List[WebElement]:

~anaconda3libsite-packagesseleniumwebdriverremotewebdriver.py in execute(self, driver_command, params)
    438         response = self.command_executor.execute(driver_command, params)
    439         if response:
--> 440             self.error_handler.check_response(response)
    441             response["value"] = self._unwrap_value(response.get("value", None))
    442             return response

~anaconda3libsite-packagesseleniumwebdriverremoteerrorhandler.py in check_response(self, response)
    243                 alert_text = value["alert"].get("text")
    244             raise exception_class(message, screen, stacktrace, alert_text)  # type: ignore[call-arg]  # mypy is not smart enough here
--> 245         raise exception_class(message, screen, stacktrace)

NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="custom-dropdown-options-list"]"}
  (Session info: chrome=110.0.5481.97)
Stacktrace:
Backtrace:
    (No symbol) [0x004F37D3]
    (No symbol) [0x00488B81]
    (No symbol) [0x0038B36D]
    (No symbol) [0x003BD382]
    (No symbol) [0x003BD4BB]
    (No symbol) [0x003F3302]
    (No symbol) [0x003DB464]
    (No symbol) [0x003F1215]
    (No symbol) [0x003DB216]
    (No symbol) [0x003B0D97]
    (No symbol) [0x003B253D]
    GetHandleVerifier [0x0076ABF2+2510930]
    GetHandleVerifier [0x00798EC1+2700065]
    GetHandleVerifier [0x0079C86C+2714828]
    GetHandleVerifier [0x005A3480+645344]
    (No symbol) [0x00490FD2]
    (No symbol) [0x00496C68]
    (No symbol) [0x00496D4B]
    (No symbol) [0x004A0D6B]
    BaseThreadInitThunk [0x76FF00F9+25]
    RtlGetAppContainerNamedObjectPath [0x77B57BBE+286]
    RtlGetAppContainerNamedObjectPath [0x77B57B8E+238]

when i try with xpath of the dropdown button:

driver.find_element(By.XPATH, '//*[@id="custom-dropdown-button"]/div[2]/i')

again returns the same error

I cannot scrape anything under #shadow-root as in the image IMAGE

Asked By: Muhammad Daniyal

||

Source

Answer 1

You are currently trying to pick an element by its ID.
The id you gave is custom-dropdown-options-list. Which sounds more like a class than an ID.

You want to select two dropdown menues. They have the class "time" and "date" in the HTML file. You want to select these.

Also what is driver? Are you using selenium? You gave no information to that. I’ll however continue using Selenium for your solution.

Selecting by ID

The ID is give in the attribute id="X". You can use F12 and select the item you want or find it in the HTML file. Both of your selectors don’t have an ID. So you want to find them by class.

Selecting by Class

When you hover both objects you’ll see that they have the class wdpr.single-select.time and wdpr.single-select.date. So you want to select them not by ID, rather By.CLASS_NAME. You can find the class name under the attribute class="x"

An example listed on the page would be.

<html>
 <body>
  <p class="content">Site content goes here.</p>
</body>
</html>

content = driver.find_element(By.CLASS_NAME, 'content')

The example speaks for itself. So you either want to do

content = driver.find_element(By.CLASS_NAME, 'time')

or to be more specific (didn’t test, please provide feedback)

content = driver.find_element(By.CLASS_NAME, 'wdpr.single-select.time')

Further read with many examples: https://selenium-python.readthedocs.io/locating-elements.html#locating-elements

Answered By: SickerDude43

Answer 2

You can’t locate elements inside a shadow root without doing some work first. It’s kinda like an IFRAME, if that’s familiar.

Using Selenium 4.1, you can do

shadow_host = driver.find_element(By.CSS_SELECTOR, 'wdpr-single-select.time')
shadow_root = shadow_host.shadow_root
shadow_root.find_element(By.ID, 'custom-dropdown-button').click()
shadow_root.find_element(By.CSS_SELECTOR, 'li[aria-label="Breakfast"]').click()

Answered By: JeffC

NoSuchElementException – Cannot find elements

Question:

Answers:

Selecting by ID

Selecting by Class