Gracefully exiting a child thread on uncaught main thread exception

Question

I have a setup with a worker thread that looks as follows:

from time import sleep
from threading import Event, Thread


class MyThread(Thread):
    
    def __init__(self, *args, **kwargs):
        # Following Doug Fort: "Terminating a Thread"
        # (https://www.oreilly.com/library/view/python-cookbook/0596001673/ch06s03.html)
        self._stop_request = Event()
        super().__init__(*args, **kwargs)
    
    def run(self):
        while not self._stop_request.is_set():
            print("My thread is running")
            sleep(.1)
        print("My thread is about to stop")  # Finish my thread's job
        
    def join(self, *args, **kwargs):
        self._stop_request.set()
        super().join(*args, **kwargs)
            
            
if __name__ == "__main__":
    my_thread = MyThread()
    my_thread.start()
    sleep(2)
    raise RuntimeError("Something went wrong!")

With this, I would like to achieve the following: once there occurs any uncaught exception in the main thread (like the deliberate RuntimeError on the last line), the worker thread should "finish its job" (i.e. run the line printing "My thread is about to stop") and then exit, as well.

In practice, the following happens:

On a Linux terminal (Python 3.5 on Debian WSL) this works as expected.
On a Windows PowerShell or command prompt (Python 3.7 on Windows 10), however, the
worker thread keeps on running, never exiting its while loop. What’s
worse, the prompt does not react to keyboard interrupts, so I have to
forcefully close the prompt window.

Using MyThread(daemon=True) does not seem to provide a solution, as it forcefully closes the worker thread immediately, without letting it finish its job. The only working version on Windows thus seems to be: once the worker thread has been started, wrap everything else into a try–except block, thus:

if __name__ == "__main__":
    my_thread = MyThread()
    my_thread.start()
    try:
        sleep(2)
        raise RuntimeError("Something went wrong!")
    except:
        my_thread.join()

This, however, looks somewhat clumsy. Also, I do not see why it should be necessary on Windows only. Am I missing something? Is there a better solution?

Edit: On a non-WSL Linux (Python 3.9 on Ubuntu 20.04), I experienced similar behavior as under Windows; that is, the worker thread continues after the RuntimeError – but at least I can use a keyboard interrupt here. So, it does not seem to be Windows-only behavior, but maybe hints at my expectations just being wrong (after all, no one ever explicitly calls my_thread.join() in the original setup, so why should its _stop_request ever be set?). My underlying question remains the same though: how do I get the worker thread to gracefully exit, as described above?

Asked By: simon

||

Source

Answer 1

I seem to have found a solution that works system-independent. It still feels a bit clumsy, though:

import sys
from time import sleep
from threading import Event, Thread


# Monkey-patch ``sys.excepthook`` to set a flag for notifying child threads
# about exceptions in the main thread
exception_raised_in_main_thread = Event()

def my_excepthook(type_, value, traceback):
    exception_raised_in_main_thread.set()
    sys.__excepthook__(type_, value, traceback)

sys.excepthook = my_excepthook


class MyThread(Thread):
    
    def run(self):
        while not exception_raised_in_main_thread.is_set():
            print("My thread is running")
            sleep(.1)
        print("My thread is about to stop")
        
            
if __name__ == "__main__":
    my_thread = MyThread()
    my_thread.start()
    sleep(2)
    raise RuntimeError("Something went wrong!")

By patching sys.excepthook (which, according to the
documentation, seems to be no misuse – quote "The handling of such top-level exceptions can be customized by assigning another three-argument function to sys.excepthook"), I will now set an event,
exception_raised_in_main_thread, to notify all child threads
about any uncaught exception that happens in the main thread.

Note that, in the code snippet above, I removed the other event
(self._stop_request) for brevity, but it might still come in
handy for terminating the worker thread under normal circumstances.

Answered By: simon

Gracefully exiting a child thread on uncaught main thread exception

Question:

Answers: