How do I stop Selenium from closing the driver during execution?

Question

I am trying to learn Selenium to scrape some Javascript heavy websites. I can locate and extract information just fine. However, I find that for some sites I need to switch my user agent. I did it the following way to test it:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent 


PATH ="C:/my/path/to/chromedriver.exe"

ua = UserAgent()
userAgent = ua.random
print(userAgent)

options = Options()
options.add_argument(f'user-agent={userAgent}')
driver = webdriver.Chrome(chrome_options=options, executable_path=PATH)

driver.get("https://www.whatismybrowser.com/detect/what-is-my-user-agent")

The code works and my user agent is switched, however there is one bug that occurs now which did not occur before. The webdriver/browser (Chrome driver) automatically closes after displaying the website for a second without me specifying the driver.quit() argument. When I do not switch my user agent it does not close unless I do and I want to study the page a bit before closing it. I have tried to wait using time.sleep() but this doesn’t work.

How can I make the webdriver not close until specified?

Answers are greatly appreciated, preferably with a code example of how to implement the solution.

Asked By: Aite97

||

Source

Answer 1

I am not sure if it is possible that the problem is related to the webdriver version you are using or not but when I tried to add time.sleep(n) to your code while using webdriver_manager library to download most recent version of ChromeWebDriver I had the chance to look at the website and the browser didn’t close until the timer finished.

My code:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent 
from webdriver_manager.chrome import ChromeDriverManager
import time

# PATH ="C:/my/path/to/chromedriver.exe"

ua = UserAgent()
userAgent = ua.random
print(userAgent)

options = Options()
options.add_argument(f'user-agent={userAgent}')
driver = webdriver.Chrome(ChromeDriverManager().install(), chrome_options=options)
driver.get("https://www.whatismybrowser.com/detect/what-is-my-user-agent")
time.sleep(100)

Answered By: Mahmoud Embaby

Answer 2

This should do you nicely:

options.add_experimental_option("detach", True)

in your code:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from fake_useragent import UserAgent 


PATH ="C:/my/path/to/chromedriver.exe"

ua = UserAgent()
userAgent = ua.random
print(userAgent)

options = Options()
options.add_argument(f'user-agent={userAgent}')
options.add_experimental_option("detach", True)

driver = webdriver.Chrome(chrome_options=options, executable_path=PATH)

driver.get("https://www.whatismybrowser.com/detect/what-is-my-user-agent")

Answered By: Vlone

How do I stop Selenium from closing the driver during execution?

Question:

Answers: