selenium.common.exceptions.InvalidSelectorException

Question

Trying to scrape airbnb with selenium, i’ve written this class to get each listing

def getListings():
listings = browser.find_elements(By.CLASS_NAME, 'c4mnd7m dir dir-ltr')
for listing in listings:
    link = listing.find_element(By.CLASS_NAME, '.ln2bl2p dir dir-ltr').get_attribute('href')
    price = listing.find_element(By.CLASS_NAME, '.a8jt5op dir dir-ltr').text
    print(link,price)

When calling the fuction i get

>>> getListings()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/cat/Desktop/code/airbnb/webscraper.py", line 65, in getListings
    listings = browser.find_elements(By.CLASS_NAME, 'c4mnd7m dir dir-ltr')
  File "/Users/cat/Desktop/code/airbnb/env/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 888, in find_elements
    return self.execute(Command.FIND_ELEMENTS, {
  File "/Users/cat/Desktop/code/airbnb/env/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 428, in execute
    self.error_handler.check_response(response)
  File "/Users/cat/Desktop/code/airbnb/env/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: An invalid or illegal selector was specified
  (Session info: chrome=105.0.5195.125)
Stacktrace:
0   chromedriver                        0x00000001050b9a90 chromedriver + 3889808
1   chromedriver                        0x0000000105048b54 chromedriver + 3427156
2   chromedriver                        0x0000000104d3a238 chromedriver + 221752
3   chromedriver                        0x0000000104d3cdf8 chromedriver + 232952
4   chromedriver                        0x0000000104d3cc60 chromedriver + 232544
5   chromedriver                        0x0000000104d3ce94 chromedriver + 233108
6   chromedriver                        0x0000000104d6b254 chromedriver + 422484
7   chromedriver                        0x0000000104d94944 chromedriver + 592196
8   chromedriver                        0x0000000104d6054c chromedriver + 378188
9   chromedriver                        0x000000010508d2f8 chromedriver + 3707640
10  chromedriver                        0x0000000105090ea8 chromedriver + 3722920
11  chromedriver                        0x00000001050973d8 chromedriver + 3748824
12  chromedriver                        0x00000001050919a8 chromedriver + 3725736
13  chromedriver                        0x000000010506cde8 chromedriver + 3575272
14  chromedriver                        0x00000001050abd1c chromedriver + 3833116
15  chromedriver                        0x00000001050abe84 chromedriver + 3833476
16  chromedriver                        0x00000001050c0210 chromedriver + 3916304
17  libsystem_pthread.dylib             0x00000001afa5426c _pthread_start + 148
18  libsystem_pthread.dylib             0x00000001afa4f08c thread_start + 8

Unsure of why the class i’m selecting is not working. It seems to be the right class. Anything i’m missing?

Asked By: Cat Hat

||

Source

Answer 1

CLASS_NAME like c4mnd7m dir dir-ltr (with spaces) does not work, it should be a CSS_SELECTOR instead.

Removing the spaces and putting the . would do the job.

Effective code changes:

listings = browser.find_elements(By.CSS_SELECTOR, '.c4mnd7m.dir.dir-ltr')

similar changes you would have to make wherever you’ve class name with spaces

Answered By: cruisepandey

selenium.common.exceptions.InvalidSelectorException

Question:

Answers: