Extract traceback info from an exception object
Question:
Given an Exception object (of unknown origin) is there way to obtain its traceback? I have code like this:
def stuff():
try:
.....
return useful
except Exception as e:
return e
result = stuff()
if isinstance(result, Exception):
result.traceback <-- How?
How can I extract the traceback from the Exception object once I have it?
Answers:
The answer to this question depends on the version of Python you’re using.
In Python 3
It’s simple: exceptions come equipped with a __traceback__ attribute that contains the traceback. This attribute is also writable, and can be conveniently set using the with_traceback method of exceptions:
raise Exception("foo occurred").with_traceback(tracebackobj)
These features are minimally described as part of the raise documentation.
All credit for this part of the answer should go to Vyctor, who first posted this information. I’m including it here only because this answer is stuck at the top, and Python 3 is becoming more common.
In Python 2
It’s annoyingly complex. The trouble with tracebacks is that they have references to stack frames, and stack frames have references to the tracebacks that have references to stack frames that have references to… you get the idea. This causes problems for the garbage collector. (Thanks to ecatmur for first pointing this out.)
The nice way of solving this would be to surgically break the cycle after leaving the except clause, which is what Python 3 does. The Python 2 solution is much uglier: you are provided with an ad-hoc function,sys.exc_info(), which only works inside the except clause. It returns a tuple containing the exception, the exception type, and the traceback for whatever exception is currently being handled.
So if you are inside the except clause, you can use the output of sys.exc_info() along with the traceback module to do various useful things:
>>> import sys, traceback
>>> def raise_exception():
... try:
... raise Exception
... except Exception:
... ex_type, ex, tb = sys.exc_info()
... traceback.print_tb(tb)
... finally:
... del tb
...
>>> raise_exception()
File "<stdin>", line 3, in raise_exception
But as your edit indicates, you’re trying to get the traceback that would have been printed if your exception had not been handled, after it has already been handled. That’s a much harder question. Unfortunately, sys.exc_info returns (None, None, None) when no exception is being handled. Other related sys attributes don’t help either. sys.exc_traceback is deprecated and undefined when no exception is being handled; sys.last_traceback seems perfect, but it appears only to be defined during interactive sessions.
If you can control how the exception is raised, you might be able to use inspect and a custom exception to store some of the information. But I’m not entirely sure how that would work.
To tell the truth, catching and returning an exception is kind of an unusual thing to do. This might be a sign that you need to refactor anyway.
There’s a very good reason the traceback is not stored in the exception; because the traceback holds references to its stack’s locals, this would result in a circular reference and (temporary) memory leak until the circular GC kicks in. (This is why you should never store the traceback in a local variable.)
About the only thing I can think of would be for you to monkeypatch stuff‘s globals so that when it thinks it’s catching Exception it’s actually catching a specialised type and the exception propagates to you as the caller:
module_containing_stuff.Exception = type("BogusException", (Exception,), {})
try:
stuff()
except Exception:
import sys
print sys.exc_info()
Since Python 3.0[PEP 3109] the built in class Exception has a __traceback__ attribute which contains a traceback object (with Python 3.2.3):
>>> try:
... raise Exception()
... except Exception as e:
... tb = e.__traceback__
...
>>> tb
<traceback object at 0x00000000022A9208>
The problem is that after Googling __traceback__ for a while I found only few articles but none of them describes whether or why you should (not) use __traceback__.
However, the Python 3 documentation for raise says that:
A traceback object is normally created automatically when an exception is raised and attached to it as the __traceback__ attribute, which is writable.
So I assume it’s meant to be used.
A way to get traceback as a string from an exception object in Python 3:
import traceback
# `e` is an exception object that you get from somewhere
traceback_str = ''.join(traceback.format_tb(e.__traceback__))
traceback.format_tb(...) returns a list of strings. ''.join(...) joins them together. For more reference, please visit: https://docs.python.org/3/library/traceback.html#traceback.format_tb
As an aside, if you want to actually get the full traceback as you would see it printed to your terminal, you want this:
>>> try:
... print(1/0)
... except Exception as e:
... exc = e
...
>>> exc
ZeroDivisionError('division by zero')
>>> tb_str = traceback.format_exception(etype=type(exc), value=exc, tb=exc.__traceback__)
>>> tb_str
['Traceback (most recent call last):n', ' File "<stdin>", line 2, in <module>n', 'ZeroDivisionError: division by zeron']
>>> print("".join(tb_str))
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero
If you use format_tb as above answers suggest you’ll get less information:
>>> tb_str = "".join(traceback.format_tb(exc.__traceback__))
>>> print("".join(tb_str))
File "<stdin>", line 2, in <module>
You could use traceback.format_exc which returns a str
or traceback.print_exc prints to stdout
import traceback
try:
b"x81".decode()
except UnicodeError:
traceback.print_exc() # prints to stdout
my_traceback = traceback.format_exc() # returns a str
print(my_traceback)
If you need to get it from the actual exception (although I don’t see why)
traceback.format_exception returns a str
traceback.print_exception prints to stdout
import traceback
try:
b"x81".decode()
except UnicodeError as exc:
# etype is inferred from `value` since python3.5 so no need to pass a value...
# format_exception returns a list
my_traceback = "".join(traceback.format_exception(etype=None, value=exc, tb=exc.__traceback__))
traceback.print_exception(etype=None, value=exc, tb=exc.__traceback__)
Caution
Do not store a reference to __traceback__ (or exc) for later use because the traceback object contains references to all the stack frame objects, which comprise the call stack, and each stack frame contains references to all of its local variables. As such, the size of the transitive closure of objects reachable from the traceback object can be very large. And if you maintain that reference, these objects will not be garbage‑collected. Prefer to render tracebacks into another form for even short‑term storage in memory."
Robert Smallshire – Python Beyond the Basics – 11 – Exceptions and Errors – Traceback objects
Given an Exception object (of unknown origin) is there way to obtain its traceback? I have code like this:
def stuff():
try:
.....
return useful
except Exception as e:
return e
result = stuff()
if isinstance(result, Exception):
result.traceback <-- How?
How can I extract the traceback from the Exception object once I have it?
The answer to this question depends on the version of Python you’re using.
In Python 3
It’s simple: exceptions come equipped with a __traceback__ attribute that contains the traceback. This attribute is also writable, and can be conveniently set using the with_traceback method of exceptions:
raise Exception("foo occurred").with_traceback(tracebackobj)
These features are minimally described as part of the raise documentation.
All credit for this part of the answer should go to Vyctor, who first posted this information. I’m including it here only because this answer is stuck at the top, and Python 3 is becoming more common.
In Python 2
It’s annoyingly complex. The trouble with tracebacks is that they have references to stack frames, and stack frames have references to the tracebacks that have references to stack frames that have references to… you get the idea. This causes problems for the garbage collector. (Thanks to ecatmur for first pointing this out.)
The nice way of solving this would be to surgically break the cycle after leaving the except clause, which is what Python 3 does. The Python 2 solution is much uglier: you are provided with an ad-hoc function,sys.exc_info(), which only works inside the except clause. It returns a tuple containing the exception, the exception type, and the traceback for whatever exception is currently being handled.
So if you are inside the except clause, you can use the output of sys.exc_info() along with the traceback module to do various useful things:
>>> import sys, traceback
>>> def raise_exception():
... try:
... raise Exception
... except Exception:
... ex_type, ex, tb = sys.exc_info()
... traceback.print_tb(tb)
... finally:
... del tb
...
>>> raise_exception()
File "<stdin>", line 3, in raise_exception
But as your edit indicates, you’re trying to get the traceback that would have been printed if your exception had not been handled, after it has already been handled. That’s a much harder question. Unfortunately, sys.exc_info returns (None, None, None) when no exception is being handled. Other related sys attributes don’t help either. sys.exc_traceback is deprecated and undefined when no exception is being handled; sys.last_traceback seems perfect, but it appears only to be defined during interactive sessions.
If you can control how the exception is raised, you might be able to use inspect and a custom exception to store some of the information. But I’m not entirely sure how that would work.
To tell the truth, catching and returning an exception is kind of an unusual thing to do. This might be a sign that you need to refactor anyway.
There’s a very good reason the traceback is not stored in the exception; because the traceback holds references to its stack’s locals, this would result in a circular reference and (temporary) memory leak until the circular GC kicks in. (This is why you should never store the traceback in a local variable.)
About the only thing I can think of would be for you to monkeypatch stuff‘s globals so that when it thinks it’s catching Exception it’s actually catching a specialised type and the exception propagates to you as the caller:
module_containing_stuff.Exception = type("BogusException", (Exception,), {})
try:
stuff()
except Exception:
import sys
print sys.exc_info()
Since Python 3.0[PEP 3109] the built in class Exception has a __traceback__ attribute which contains a traceback object (with Python 3.2.3):
>>> try:
... raise Exception()
... except Exception as e:
... tb = e.__traceback__
...
>>> tb
<traceback object at 0x00000000022A9208>
The problem is that after Googling __traceback__ for a while I found only few articles but none of them describes whether or why you should (not) use __traceback__.
However, the Python 3 documentation for raise says that:
A traceback object is normally created automatically when an exception is raised and attached to it as the
__traceback__attribute, which is writable.
So I assume it’s meant to be used.
A way to get traceback as a string from an exception object in Python 3:
import traceback
# `e` is an exception object that you get from somewhere
traceback_str = ''.join(traceback.format_tb(e.__traceback__))
traceback.format_tb(...) returns a list of strings. ''.join(...) joins them together. For more reference, please visit: https://docs.python.org/3/library/traceback.html#traceback.format_tb
As an aside, if you want to actually get the full traceback as you would see it printed to your terminal, you want this:
>>> try:
... print(1/0)
... except Exception as e:
... exc = e
...
>>> exc
ZeroDivisionError('division by zero')
>>> tb_str = traceback.format_exception(etype=type(exc), value=exc, tb=exc.__traceback__)
>>> tb_str
['Traceback (most recent call last):n', ' File "<stdin>", line 2, in <module>n', 'ZeroDivisionError: division by zeron']
>>> print("".join(tb_str))
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ZeroDivisionError: division by zero
If you use format_tb as above answers suggest you’ll get less information:
>>> tb_str = "".join(traceback.format_tb(exc.__traceback__))
>>> print("".join(tb_str))
File "<stdin>", line 2, in <module>
You could use traceback.format_exc which returns a str
or traceback.print_exc prints to stdout
import traceback
try:
b"x81".decode()
except UnicodeError:
traceback.print_exc() # prints to stdout
my_traceback = traceback.format_exc() # returns a str
print(my_traceback)
If you need to get it from the actual exception (although I don’t see why)
traceback.format_exception returns a str
traceback.print_exception prints to stdout
import traceback
try:
b"x81".decode()
except UnicodeError as exc:
# etype is inferred from `value` since python3.5 so no need to pass a value...
# format_exception returns a list
my_traceback = "".join(traceback.format_exception(etype=None, value=exc, tb=exc.__traceback__))
traceback.print_exception(etype=None, value=exc, tb=exc.__traceback__)
Caution
Do not store a reference to
__traceback__(orexc) for later use because the traceback object contains references to all the stack frame objects, which comprise the call stack, and each stack frame contains references to all of its local variables. As such, the size of the transitive closure of objects reachable from the traceback object can be very large. And if you maintain that reference, these objects will not be garbage‑collected. Prefer to render tracebacks into another form for even short‑term storage in memory."
Robert Smallshire – Python Beyond the Basics – 11 – Exceptions and Errors – Traceback objects