Selenium Chromedriver Hangs?

Question:

I have a long running python app that will periodically (every 30-60 seconds) open a webpage with selenium and chrome driver, run some javascript and take a screenshot. Its running on an EC2 ubuntu instance with chrome in Xvfb and for the most part everything is working, except intermittently the program will hang. It is happening on one of these lines:

    options = Options()
    options.add_argument("--disable-web-security")
    options.add_argument("--webdriver-logfile=webdrive.log")
    dc = DesiredCapabilities.CHROME
    dc['loggingPrefs'] = {'browser': 'ALL'}
    driver = webdriver.Chrome(chrome_options=options, desired_capabilities=dc)
    driver.get(url);

(I don’t have an exact line but I know from debug statements I have put in that it is somewhere in between here)

Unfortunately, the program doesn’t crash so it doesn’t have any error messages, its just waiting endlessly since 7pm last night. Running strace -p 'python program pid' returns: wait4(-1, and running strace -p 'chromedriver pid' returns recvfrom(20,

I can see in ps axjf that the process is still running, its just not doing anything. I’m sort of at a loss of what to do now, any suggestions?

chromedriver version: 2.10.267518

Google Chrome 40.0.2214.111

Selenium (installed with pip): 2.42.1

#https://github.com/cgoldberg/xvfbwrapper
xvfb = Xvfb(width=1920, height=1920)
xvfb.start()

—- EDIT —-

I have just updated to ChromeDriver 2.14.313457 and Selenium 2.44.0, hopefully this will fix the issue. I’m going to leave this open for now. Thanks for the advice so far guys!

—- EDIT —-

So the service still ended up hanging. I’m wondering if this is because for every screenshot I close and restart google-chrome? Is this possibly causing a memory leak somehow? How could I diagnose this?

Asked By: Trevor

||
Source

Answers:

I never found out the specific piece of code that was causing this problem, but creating a fresh instance of Xvfb with each driver load seems to have fixed it. Perhaps there is a memory leak somewhere in the interaction between selenium and Xvfb? Either way, marking this as closed.

Answered By: Trevor

I ran into a similar issue and found the answer here and blogged about it here. Setting the environment variable DBUS_SESSION_BUS_ADDRESS=/dev/null worked for me without having to restart Xvfb all the time.

Answered By: Kevin Schroeder

In my case, the problem disappeared enclosing the call within a with block, like this:

options = Options()
path = f"{os.environ['LOCALAPPDATA']}{os.path.sep}Programs{os.path.sep}Chromium{os.path.sep}Win_x64{os.path.sep}1012736{os.path.sep}"
options.binary_location = f"{path}chrome-win{os.path.sep}chrome.exe"
executable_path = f"{path}chromedriver_win32{os.path.sep}chromedriver.exe"
with webdriver.Chrome(chrome_options=options, executable_path=executable_path) as browser:
    browser.get("https://live.euronext.com/en/products/equities")
    sleep(3)

Selenium 4.4.0, Python 3.10, Windows 10, ChromeDriver and Chromium 104.0.5112.0

Answered By: Antonio Miras