How to access the iframe using Selenium and Python on Glassdoor
Question:
I am trying to automate application process on Glassdoor using the EasyApply button. Now after identifying the EasyApply Button and successfully clicking it, I need to switch to the Frame so as to access the HTML content of the form to be able to send in my form details.
I used:
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "#indeedapply-modal-preload-1658752913396-iframe")))
to perform the switching but still could not access the frame’s html content.
Here is the block that performs this operation:
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
#if it has easy-apply, then perform application
if len(driver.find_elements_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/button')) > 0:
driver.find_element_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/button').click()
wait = WebDriverWait(driver, 50)
time.sleep(5)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "#indeedapply-modal-preload-1658752913396-iframe")))
name = driver.find_element(By.CSS_SELECTOR, '#input-applicant.name')
name.send_keys('Oluyele Anthony')
elif len(driver.find_elements_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/a')) > 0:
driver.find_element_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/a').click()
Here is the HTML content for the frame
<iframe name="indeedapply-modal-preload-1659146630884-iframe" id="indeedapply-modal-preload-1659146630884-iframe" scrolling="no" frameborder="0" title="Job application form container" src="https://apply.indeed.com/indeedapply/xpc?v=5#%7B%22cn%22:%224AaHlXdnW4%22,%22ppu%22:%22https://www.glassdoor.com/robots.txt%22,%22lpu%22:%22https://apply.indeed.com/robots.txt%22,%22setupms%22:1659146630959,%22preload%22:true,%22iaUid%22:%221g96dgr7uii3h800%22,%22parentURL%22:%22https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox%22%7D" style="border: 0px; vertical-align: bottom; width: 100%; height: 100%;"></iframe>
Apparently, after running the cell, the:
TimeoutException: Message:
Error occurs which shows that the frame is not being switched to.
Answers:
The middle part of the value of id attribute i.e. 1658752913396 is dynamically generated and is bound to change sooner/later. They may change next time you access the application afresh or even while next application startup. So can’t be used in locators.
Solution
To switch to the <iframe> you can use either of the following Locator Strategies:
-
Using id attribute:
-
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[title='Job application form container'][id^='indeedapply-modal-preload']")))
-
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[title='Job application form container' and starts-with(@id, 'indeedapply-modal-preload')]")))
-
Using name attribute:
-
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[title='Job application form container'][name^='indeedapply-modal-preload']")))
-
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[title='Job application form container' and starts-with(@name, 'indeedapply-modal-preload')]")))
-
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
There are 2 iframes in that page and the second iframe contains another (nested) iframe. The following code will sort out your problem (setup is for linux, but you can figure it out – pay attention to imports, and to the code after getting the url):
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time as t
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)
url = 'https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox'
browser.get(url)
button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//button[@data-test="applyButton"]')))
button.click()
iframes = WebDriverWait(browser, 20).until(EC.presence_of_all_elements_located((By.TAG_NAME, "iframe")))
print(len(iframes))
for iframe in iframes:
print(iframe.get_attribute('id'))
browser.switch_to.frame(iframes[1])
t.sleep(2)
WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form']")))
t.sleep(2)
applicant_name = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, 'input-applicant.name')))
applicant_name.click()
applicant_name.send_keys('hello dolly')
Let me know if it works for you.
UPDATE: It appears that website changed its structure since yesterday, and now there are 11/13 iframes per page. Also, if the job selected from the main column does not offer Easy Apply, you get an error. The updated code (below) is selecting the Easy Apply jobs from page, goes through each of them, clicks easyapply button, select the correct iframe/nested iframe, and does stuff to that form. It currently works, until that website will change its format again:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time as t
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)
url = 'https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox'
browser.get(url)
jobs = WebDriverWait(browser, 20).until(EC.presence_of_all_elements_located((By.XPATH, "//li[@data-is-easy-apply='true']")))
while True:
for job in jobs:
try:
print(job.text)
job.click()
try:
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[@alt='Close']"))).click()
except Exception as e:
print('no request to login')
t.sleep(2)
button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//button[@data-test="applyButton"]')))
button.click()
WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form container']")))
t.sleep(2)
WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form']")))
t.sleep(2)
applicant_name = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, 'input-applicant.name')))
applicant_name.click()
applicant_name.send_keys('hello dolly')
t.sleep(2)
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@id='form-action-cancel']"))).click()
t.sleep(2)
browser.switch_to.default_content()
t.sleep(2)
except Exception as e:
print(e)
break
I am trying to automate application process on Glassdoor using the EasyApply button. Now after identifying the EasyApply Button and successfully clicking it, I need to switch to the Frame so as to access the HTML content of the form to be able to send in my form details.
I used:
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "#indeedapply-modal-preload-1658752913396-iframe")))
to perform the switching but still could not access the frame’s html content.
Here is the block that performs this operation:
from selenium.webdriver.support.ui import WebDriverWait, Select
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
#if it has easy-apply, then perform application
if len(driver.find_elements_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/button')) > 0:
driver.find_element_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/button').click()
wait = WebDriverWait(driver, 50)
time.sleep(5)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "#indeedapply-modal-preload-1658752913396-iframe")))
name = driver.find_element(By.CSS_SELECTOR, '#input-applicant.name')
name.send_keys('Oluyele Anthony')
elif len(driver.find_elements_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/a')) > 0:
driver.find_element_by_xpath('//*[@id="JDCol"]/div/article/div/div[1]/div/div/div[1]/div[3]/div[2]/div/div[1]/div[1]/a').click()
Here is the HTML content for the frame
<iframe name="indeedapply-modal-preload-1659146630884-iframe" id="indeedapply-modal-preload-1659146630884-iframe" scrolling="no" frameborder="0" title="Job application form container" src="https://apply.indeed.com/indeedapply/xpc?v=5#%7B%22cn%22:%224AaHlXdnW4%22,%22ppu%22:%22https://www.glassdoor.com/robots.txt%22,%22lpu%22:%22https://apply.indeed.com/robots.txt%22,%22setupms%22:1659146630959,%22preload%22:true,%22iaUid%22:%221g96dgr7uii3h800%22,%22parentURL%22:%22https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox%22%7D" style="border: 0px; vertical-align: bottom; width: 100%; height: 100%;"></iframe>
Apparently, after running the cell, the:
TimeoutException: Message:
Error occurs which shows that the frame is not being switched to.
The middle part of the value of id attribute i.e. 1658752913396 is dynamically generated and is bound to change sooner/later. They may change next time you access the application afresh or even while next application startup. So can’t be used in locators.
Solution
To switch to the <iframe> you can use either of the following Locator Strategies:
-
Using id attribute:
-
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[title='Job application form container'][id^='indeedapply-modal-preload']"))) -
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[title='Job application form container' and starts-with(@id, 'indeedapply-modal-preload')]")))
-
-
Using name attribute:
-
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe[title='Job application form container'][name^='indeedapply-modal-preload']"))) -
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[title='Job application form container' and starts-with(@name, 'indeedapply-modal-preload')]")))
-
-
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.common.by import By from selenium.webdriver.support import expected_conditions as EC
There are 2 iframes in that page and the second iframe contains another (nested) iframe. The following code will sort out your problem (setup is for linux, but you can figure it out – pay attention to imports, and to the code after getting the url):
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time as t
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)
url = 'https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox'
browser.get(url)
button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//button[@data-test="applyButton"]')))
button.click()
iframes = WebDriverWait(browser, 20).until(EC.presence_of_all_elements_located((By.TAG_NAME, "iframe")))
print(len(iframes))
for iframe in iframes:
print(iframe.get_attribute('id'))
browser.switch_to.frame(iframes[1])
t.sleep(2)
WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form']")))
t.sleep(2)
applicant_name = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, 'input-applicant.name')))
applicant_name.click()
applicant_name.send_keys('hello dolly')
Let me know if it works for you.
UPDATE: It appears that website changed its structure since yesterday, and now there are 11/13 iframes per page. Also, if the job selected from the main column does not offer Easy Apply, you get an error. The updated code (below) is selecting the Easy Apply jobs from page, goes through each of them, clicks easyapply button, select the correct iframe/nested iframe, and does stuff to that form. It currently works, until that website will change its format again:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time as t
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)
url = 'https://www.glassdoor.com/Job/nigeria-data-science-jobs-SRCH_IL.0,7_IN177_KO8,20.htm?clickSource=searchBox'
browser.get(url)
jobs = WebDriverWait(browser, 20).until(EC.presence_of_all_elements_located((By.XPATH, "//li[@data-is-easy-apply='true']")))
while True:
for job in jobs:
try:
print(job.text)
job.click()
try:
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[@alt='Close']"))).click()
except Exception as e:
print('no request to login')
t.sleep(2)
button = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, '//button[@data-test="applyButton"]')))
button.click()
WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form container']")))
t.sleep(2)
WebDriverWait(browser, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//*[@title='Job application form']")))
t.sleep(2)
applicant_name = WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.ID, 'input-applicant.name')))
applicant_name.click()
applicant_name.send_keys('hello dolly')
t.sleep(2)
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@id='form-action-cancel']"))).click()
t.sleep(2)
browser.switch_to.default_content()
t.sleep(2)
except Exception as e:
print(e)
break