multiprocessing.Event.wait hangs when interrupted by a signal

Signals are only handled on the main thread. If the main thread is blocked in a system call, that system call will raise an InterruptedError.

From the Python docs:

they [signals] can only occur between the “atomic” instructions of the Python
interpreter

A time.sleep, for example, would raise an InterruptedError. It seems like the event.wait method does not deal with this scenario correctly. It does not raise an InterruptedError but simply starts to hang. This looks like a bug in Python to me?

UPDATE:

I narrowed this down to a deadlock in multiprocessing.Event. If the main thread is waiting for an event and at the same time, a signal sets that event on the interrupted main thread, then the multiprocessing.event.set() and multiprocessing.event.wait() methods deadlock each other.

Also, the behavior is heavily platform dependent. E.g. a time.sleep() would raise an InterruptedError on Windows but simply return on Linux.

A really clunky workaround is to keep the main thread free for processing signals.

import multiprocessing
import signal
import threading
import time


class Class1(object):
    _stop_event = multiprocessing.Event()

    @staticmethod
    def interrupt():
        Class1._stop_event.set()

    def wait_timeout(self):
        print("Waiting for _stop_event")
        if Class1._stop_event.wait(30):
            print("_stop_event set.")
        else:
            print("Timeout")


def stop(signum, frame):
    print("Received SIG")
    Class1.interrupt()
    exit_event.set()


def run():
    c = Class1()
    c.wait_timeout()

t = threading.Thread(target=run)
t.daemon = False
t.start()

exit_event = multiprocessing.Event()
signal.signal(signal.SIGINT, stop)

while not exit_event.is_set():
    # Keep a main thread around to handle the signal and
    # that thread must not be in a event.wait()!
    try:
        time.sleep(500)
    except InterruptedError:
        # We were interrupted by the incoming signal.
        # Let the signal handler stop the process gracefully.
        pass

This is ugly as fuck. Someone please provide a more elegant solution….

You could alternatively use the pause() function of the signal module instead of Event().wait(). signal.pause() sleeps until a signal is received by the process. In this case, when SIGINT is received, the signal.pause() exits, returning nothing. Note that the function does not work on Windows according to the documentation. I’ve tried it on Linux and it worked for me.

I came across this solution in this SO thread.

Building on Alex’s excellent investigation, it means that setting the flag from another thread would not cause the deadlock. The original code will work if the interrupt method is changed as such:

from threading import Thread

(...)
    @staticmethod
    def interrupt():
        Thread(target=Class1._stop_event.set).start()

As I wrote in Issue #85772:

I just spent a lot of time with this problem. I have a
multiprocessing.Event that sleeps the MainThread of a child process
and an interrupt handler that sets that event. As described here, when
the interrupt triggers the event, the process hangs. Doing some
research on the web, including on the cited stackoverflow topic, I got
the following solution:

Solutions for Linux with fork start method:

1. In the interrupt handler, call threading.Thread(target=exit_event.set).start()
2. Change multiprocessing.Event to threading.Event

Solution for Linux with spawn start method

1. In the interrupt handler, call threading.Thread(target=exit_event.set).start()

Windows:

1. Instead of using signal.signal() to change the interrupt handler, the solution was to use:

   import win32api

   win32api.SetConsoleCtrlHandler(handler, add)

Reference:
https://docs.microsoft.com/en-us/windows/console/setconsolectrlhandler

This works like a charm. The key difference is that it runs the
interrupt funcition on a thread (named Dummy-1).

2. (Not a great solution) In the interrupt handler, call threading.Thread(target=exit_event.set).start()
   • This solves the hang issue, but raises an InterruptedError exception