Is there a way to get the url of popup js onclick dialogue using selenium?

Question

This is the website link which I am trying to scrape for data https://tis.nhai.gov.in/tollplazasataglance.aspx?language=en#

There are links in 4th column in above site if clicked a popup window comes which has certain info along with href for the next link when we click More Information tab.We get to such links https://tis.nhai.gov.in/TollInformation.aspx?TollPlazaID=236

From selenium import webdriver

    driver = webdriver.Firefox()
    driver.maximize_window()

driver.get("https://tis.nhai.gov.in/tollplazasataglance.aspx?language=en#")


b = driver.find_element("xpath", '//*[@id="tollList"]/table/tbody/tr[2]/td[4]/a')


c = driver.execute_script("arguments[0].click();", b)

At this point I am stuck up as am unable to capture the href or url of the popup window… Kindly help me to get past to the other page from the pop up window

Asked By: Abhi

||

Source

Answer 1

Those pop-ups are the result of POST requests, where the payload is each location ID. Here is a way to get the locations IDs:

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
import time as t

chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('disable-notifications')
chrome_options.add_argument("window-size=1280,720")

webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
driver = webdriver.Chrome(service=webdriver_service, options=chrome_options)
wait = WebDriverWait(driver, 25)
url = 'https://tis.nhai.gov.in/tollplazasataglance.aspx?language=en#'
driver.get(url)
places = wait.until(EC.presence_of_all_elements_located((By.XPATH, '//div[@id="tollList"]//tbody/tr/td[4]' )))
for p in places:
    p_id = p.find_element(By.XPATH, './/a').get_attribute('onclick').split('(')[1].split(')')[0]
    print(p.text, p_id)

Result in terminal:

Aganampudi 236
Amakathadu 258
Badava 4486
Bandapalli 5697
Bandlapalli 5952
Basapuram 4542
Bathalapalli 5753
Bolapalli 252
[...]

Once you have the IDs, you can go to each place’ page with https://tis.nhai.gov.in/TollInformation.aspx?TollPlazaID={place_id}.

Answered By: Barry the Platipus

Is there a way to get the url of popup js onclick dialogue using selenium?

Question:

Answers: