Clicking dropdown option, getting base WebDriverException with empty Message

Question:

Using Python & Selenium in Safari browser; trying to select from a dropdown box.

The dropdown looks like this in the HTML:

<select name="ctl00$cph1$d1$cboExchange" onchange="javascript:setTimeout('__doPostBack('ctl00$cph1$d1$cboExchange','')', 0)" id="ctl00_cph1_d1_cboExchange" style="width:205px;margin-left:30px;">
                <option value="AMEX">American Stock Exchange</option>
                <option value="ASX">Australian Securities Exchange</option>
                <option value="CFE">Chicago Futures Exchange</option>
                <option value="EUREX">EUREX Futures Exchange</option>
                <option value="FOREX">Foreign Exchange</option>
                <option selected="selected" value="INDEX">Global Indices</option>
                <option value="HKEX">Hong Kong Stock Exchange</option>
                <option value="KCBT">Kansas City Board of Trade</option>
                <option value="LIFFE">LIFFE Futures and Options</option>
                <option value="LSE">London Stock Exchange</option>
                <option value="MGEX">Minneapolis Grain Exchange</option>
                <option value="USMF">Mutual Funds</option>
                <option value="NASDAQ">NASDAQ Stock Exchange</option>
                <option value="NYBOT">New York Board of Trade</option>
                <option value="NYSE">New York Stock Exchange</option>
                <option value="OTCBB">OTC Bulletin Board</option>
                <option value="SGX">Singapore Stock Exchange</option>
                <option value="TSX">Toronto Stock Exchange</option>
                <option value="TSXV">Toronto Venture Exchange</option>
                <option value="WCE">Winnipeg Commodity Exchange</option>

            </select>

My select-related code looks like this:

    try:
        exchange_dropdown_id = 'ctl00_cph1_d1_cboExchange'

        exchange_dropdown = driver.find_element(by=By.ID, value=exchange_dropdown_id)
        exchange_dropdown_select = Select(exchange_dropdown)

        print('-- clicking dropdown element')
        exchange_dropdown_select.select_by_value('AMEX')

        print('IT WORKED!')

    except (selenium.common.exceptions.ElementClickInterceptedException,
            selenium.common.exceptions.ElementNotInteractableException,
            selenium.common.exceptions.ElementNotSelectableException,
            selenium.common.exceptions.ElementNotVisibleException,
            selenium.common.exceptions.ImeActivationFailedException,
            selenium.common.exceptions.ImeNotAvailableException,
            selenium.common.exceptions.InsecureCertificateException,
            selenium.common.exceptions.InvalidCookieDomainException,
            selenium.common.exceptions.InvalidCoordinatesException,
            selenium.common.exceptions.InvalidElementStateException,
            selenium.common.exceptions.InvalidSelectorException,
            selenium.common.exceptions.InvalidSessionIdException,
            selenium.common.exceptions.InvalidSwitchToTargetException,
            selenium.common.exceptions.JavascriptException,
            selenium.common.exceptions.MoveTargetOutOfBoundsException,
            selenium.common.exceptions.NoAlertPresentException,
            selenium.common.exceptions.NoSuchAttributeException,
            selenium.common.exceptions.NoSuchCookieException,
            selenium.common.exceptions.NoSuchElementException,
            selenium.common.exceptions.NoSuchFrameException,
            selenium.common.exceptions.NoSuchShadowRootException,
            selenium.common.exceptions.NoSuchWindowException,
            selenium.common.exceptions.ScreenshotException,
            selenium.common.exceptions.SeleniumManagerException,
            selenium.common.exceptions.SessionNotCreatedException,
            selenium.common.exceptions.StaleElementReferenceException,
            selenium.common.exceptions.TimeoutException,
            selenium.common.exceptions.UnableToSetCookieException,
            selenium.common.exceptions.UnexpectedAlertPresentException,
            selenium.common.exceptions.UnexpectedTagNameException,
            selenium.common.exceptions.UnknownMethodException,
            ) as ex:
        print('*** POSSIBLE EXCEPTION FOUND:n' + repr(ex) + 'n*** END REPR')

    except selenium.common.exceptions.WebDriverException as ex:
        print('*** WEBDRIVER BASE EXCEPTION:n' + repr(ex) + 'n*** END BASE EXCEPTION DETAIL')

    except Exception as ex:
        print('*** Failure selecting item. Exception:n' + str(ex) + '*** END EXCEPTION MESSAGE ***')
        print('*** Exception Detail?:n' + repr(ex) + 'n*** END EXCEPTION MESSAGE ***')

The result is:

— clicking dropdown element
*** WEBDRIVER BASE EXCEPTION:
WebDriverException()
*** END BASE EXCEPTION DETAIL

If I use str(ex) instead of repr(ex), I get the word MESSAGE: with no other information.

I entered all the exceptions in the WebDriver API in hopes of catching "the one" that would tell me what’s happening.

I have tried using select_by_visible_text('American Stock Exchange') as well, with the same result.

I can iterate through all the options with the Select object:

        for option in exchange_dropdown_select.options:
            print(option.get_attribute('value'))
            if option.get_attribute('value') == 'AMEX':
                print('AMEX found')

and it finds the option. However, if I add a click on the option:

        for option in exchange_dropdown_select.options:
            print(option.get_attribute('value'))
            if option.get_attribute('value') == 'AMEX':
                print('AMEX found')
                option.click()
                print('clicked AMEX')

it still ends with the empty WebDriver exception:

AMEX found
*** WEBDRIVER BASE EXCEPTION:
WebDriverException()
*** END BASE EXCEPTION DETAIL.

The HTML is enclosed in a table, if that makes a difference. And, I’m using Python 3.9.

Any ideas what might be happening and how I might fix it?

Also, I’d prefer not to use the XPATH or index values as the developers might change the order of the options, and the XPATH doesn’t contain any ID information relating to the item(s) I want to select – again, a path that could change if the option order changes.

Oh, and my imports are:

import sys

import selenium # for all of those exceptions down there; will be removed when solved
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait

Update: to get to the issue –

  • start with the URL eoddata.com
  • log in (free account is fine)
  • click the Download button at the upper left (next to Home Page button)
  • click the ‘X’ to close the popup

Then, I’m looking at the Download Data portion in the upper left.

In this particular example, I want to select the Exchange – value = "AMEX" and visible text = "American Stock Exchange". It’s currently the first index, but again, this could change at any time, so I don’t want to count on an exact position, if I can help it.

Finally, in respectful netizenship, the current robots.txt is:

User-agent: *
Crawl-delay:10
Disallow: /images/
Disallow: /styles/
Disallow: /scripts/

User-agent: turnitinbot 
Disallow: /

Sitemap: http://www.eoddata.com/sitemapindex.xml

Update 2:
Apple Docs related to safaridriver (accessed 28 Feb 2023):

I have found a number of posts, without selected answers, asking about this issue in Safari. I’ve not found any instructional info with a SELECT specifically – and all of the rest of the steps I listed above are working perfectly.

It looks like I need a REST way of accessing the SELECT/OPTION item — or, I need to switch to Firefox or Chrome, lol.

Asked By: leanne

||

Answers:

To me this works without problems using chromedriver

from selenium.webdriver.support.ui import Select
exchange_dropdown = driver.find_element(By.ID, 'ctl00_cph1_d1_cboExchange')
Select(exchange_dropdown).select_by_value('AMEX')

Alternative (1)

exchange_dropdown = driver.find_element(By.ID, 'ctl00_cph1_d1_cboExchange')
option = exchange_dropdown.find_element(By.CSS_SELECTOR, 'option[value=AMEX]')
exchange_dropdown.send_keys(option.text)

Alternative (2)

exchange_dropdown = driver.find_element(By.ID, 'ctl00_cph1_d1_cboExchange')
option = exchange_dropdown.find_element(By.CSS_SELECTOR, 'option[value=AMEX]')
option.click()
Answered By: sound wave