Why does sys.exit() not exit when called inside a thread in Python?
Question:
I am confused as to why the following code snippet would not exit when called in the thread, but would exit when called in the main thread.
import sys, time
from threading import Thread
def testexit():
time.sleep(5)
sys.exit()
print "post thread exit"
t = Thread(target = testexit)
t.start()
t.join()
print "pre main exit, post thread exit"
sys.exit()
print "post main exit"
The docs for sys.exit() state that the call should exit from Python. I can see from the output of this program that "post thread exit" is never printed, but the main thread just keeps on going even after the thread calls exit.
Is a separate instance of the interpreter being created for each thread, and the call to exit() is just exiting that separate instance? If so, how does the threading implementation manage access to shared resources? What if I did want to exit the program from the thread (not that I actually want to, but just so I understand)?
Answers:
sys.exit() raises the SystemExit exception, as does thread.exit(). So, when sys.exit() raises that exception inside that thread, it has the same effect as calling thread.exit(), which is why only the thread exits.
Is the fact that “pre main exit, post thread exit” is printed what’s bothering you?
Unlike some other languages (like Java) where the analog to sys.exit (System.exit, in Java’s case) causes the VM/process/interpreter to immediately stop, Python’s sys.exit just throws an exception: a SystemExit exception in particular.
Here are the docs for sys.exit (just print sys.exit.__doc__):
Exit the interpreter by raising SystemExit(status).
If the status is omitted or None, it defaults to zero (i.e., success).
If the status is numeric, it will be used as the system exit status.
If it is another kind of object, it will be printed and the system
exit status will be one (i.e., failure).
This has a few consequences:
- in a thread it just kills the current thread, not the entire process (assuming it gets all the way to the top of the stack…)
- object destructors (
__del__) are potentially invoked as the stack frames that reference those objects are unwound
- finally blocks are executed as the stack unwinds
- you can catch a
SystemExit exception
The last is possibly the most surprising, and is yet another reason why you should almost never have an unqualified except statement in your Python code.
What if I did want to exit the program
from the thread (not that I actually
want to, but just so I understand)?
My preferred method is Erlang-ish message passing. Slightly simlified, I do it like this:
import sys, time
import threading
import Queue # thread-safe
class CleanExit:
pass
ipq = Queue.Queue()
def testexit(ipq):
time.sleep(5)
ipq.put(CleanExit)
return
threading.Thread(target=testexit, args=(ipq,)).start()
while True:
print "Working..."
time.sleep(1)
try:
if ipq.get_nowait() == CleanExit:
sys.exit()
except Queue.Empty:
pass
What if I did want to exit the program
from the thread?
Apart from the method Deestan described you can call os._exit (notice the underscore). Before using it make sure that you understand that it does no cleanups (like calling __del__ or similar).
What if I did want to exit the program from the thread?
For Linux:
os.kill(os.getpid(), signal.SIGINT)
This sends a SIGINT to the main thread which raises a KeyboardInterrupt. With that you have a proper cleanup. Also you can register a handler, if you want to react differently.
For Windows:
The above does not work on Windows, as you can only send a SIGTERM signal, which is not handled by Python and has the same effect as os._exit().
The only option is to use:
os._exit()
This will exit the entire process without any cleanup. If you need cleanup, you need to communicate with the main thread in another way.
_thread.interrupt_main() is available since Python 3.7 (optional before that)
I am confused as to why the following code snippet would not exit when called in the thread, but would exit when called in the main thread.
import sys, time
from threading import Thread
def testexit():
time.sleep(5)
sys.exit()
print "post thread exit"
t = Thread(target = testexit)
t.start()
t.join()
print "pre main exit, post thread exit"
sys.exit()
print "post main exit"
The docs for sys.exit() state that the call should exit from Python. I can see from the output of this program that "post thread exit" is never printed, but the main thread just keeps on going even after the thread calls exit.
Is a separate instance of the interpreter being created for each thread, and the call to exit() is just exiting that separate instance? If so, how does the threading implementation manage access to shared resources? What if I did want to exit the program from the thread (not that I actually want to, but just so I understand)?
sys.exit() raises the SystemExit exception, as does thread.exit(). So, when sys.exit() raises that exception inside that thread, it has the same effect as calling thread.exit(), which is why only the thread exits.
Is the fact that “pre main exit, post thread exit” is printed what’s bothering you?
Unlike some other languages (like Java) where the analog to sys.exit (System.exit, in Java’s case) causes the VM/process/interpreter to immediately stop, Python’s sys.exit just throws an exception: a SystemExit exception in particular.
Here are the docs for sys.exit (just print sys.exit.__doc__):
Exit the interpreter by raising SystemExit(status).
If the status is omitted or None, it defaults to zero (i.e., success).
If the status is numeric, it will be used as the system exit status.
If it is another kind of object, it will be printed and the system
exit status will be one (i.e., failure).
This has a few consequences:
- in a thread it just kills the current thread, not the entire process (assuming it gets all the way to the top of the stack…)
- object destructors (
__del__) are potentially invoked as the stack frames that reference those objects are unwound - finally blocks are executed as the stack unwinds
- you can catch a
SystemExitexception
The last is possibly the most surprising, and is yet another reason why you should almost never have an unqualified except statement in your Python code.
What if I did want to exit the program
from the thread (not that I actually
want to, but just so I understand)?
My preferred method is Erlang-ish message passing. Slightly simlified, I do it like this:
import sys, time
import threading
import Queue # thread-safe
class CleanExit:
pass
ipq = Queue.Queue()
def testexit(ipq):
time.sleep(5)
ipq.put(CleanExit)
return
threading.Thread(target=testexit, args=(ipq,)).start()
while True:
print "Working..."
time.sleep(1)
try:
if ipq.get_nowait() == CleanExit:
sys.exit()
except Queue.Empty:
pass
What if I did want to exit the program
from the thread?
Apart from the method Deestan described you can call os._exit (notice the underscore). Before using it make sure that you understand that it does no cleanups (like calling __del__ or similar).
What if I did want to exit the program from the thread?
For Linux:
os.kill(os.getpid(), signal.SIGINT)
This sends a SIGINT to the main thread which raises a KeyboardInterrupt. With that you have a proper cleanup. Also you can register a handler, if you want to react differently.
For Windows:
The above does not work on Windows, as you can only send a SIGTERM signal, which is not handled by Python and has the same effect as os._exit().
The only option is to use:
os._exit()
This will exit the entire process without any cleanup. If you need cleanup, you need to communicate with the main thread in another way.
_thread.interrupt_main() is available since Python 3.7 (optional before that)