Get full traceback
Question:
How can i get full traceback in the following case, including the calls of func2 and func functions?
import traceback
def func():
try:
raise Exception('Dummy')
except:
traceback.print_exc()
def func2():
func()
func2()
When i run this, i get:
Traceback (most recent call last):
File "test.py", line 5, in func
raise Exception('Dummy')
Exception: Dummy
traceback.format_stack() is not what i want, as need traceback object to be passed to a third party module.
I am particularly interested in this case:
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.exception("Something awful happened!")
def func2():
func()
func2()
In this case i am getting:
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "test.py", line 9, in func
raise Exception('Dummy')
Exception: Dummy
Answers:
As mechmind answered, the stack trace consists only of frames between the site where the exception was raised and the site of the try block. If you need the full stack trace, apparently you’re out of luck.
Except that it’s obviously possible to extract the stack entries from top-level to the current frame—traceback.extract_stack manages it just fine. The problem is that the information obtained by traceback.extract_stack comes from direct inspection of stack frames without creating a traceback object at any point, and the logging API requires a traceback object to affect traceback output.
Fortunately, logging doesn’t require an actual traceback object, it requires an object that it can pass to the formatting routines of the traceback module. traceback doesn’t care either—it only uses two attributes of the traceback, the frame and the line number. So, it should be possible to create a linked list of duck-typed faux-traceback objects and pass it off as the traceback.
import sys
class FauxTb(object):
def __init__(self, tb_frame, tb_lineno, tb_next):
self.tb_frame = tb_frame
self.tb_lineno = tb_lineno
self.tb_next = tb_next
def current_stack(skip=0):
try: 1/0
except ZeroDivisionError:
f = sys.exc_info()[2].tb_frame
for i in xrange(skip + 2):
f = f.f_back
lst = []
while f is not None:
lst.append((f, f.f_lineno))
f = f.f_back
return lst
def extend_traceback(tb, stack):
"""Extend traceback with stack info."""
head = tb
for tb_frame, tb_lineno in stack:
head = FauxTb(tb_frame, tb_lineno, head)
return head
def full_exc_info():
"""Like sys.exc_info, but includes the full traceback."""
t, v, tb = sys.exc_info()
full_tb = extend_traceback(tb, current_stack(1))
return t, v, full_tb
With these functions in place, your code only requires a trivial modification:
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.error("Something awful happened!", exc_info=full_exc_info())
def func2():
func()
func2()
…to give the expected output:
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "a.py", line 52, in <module>
func2()
File "a.py", line 49, in func2
func()
File "a.py", line 43, in func
raise Exception('Dummy')
Exception: Dummy
Note that the faux-traceback objects are fully usable for introspection—displaying local variables or as argument to pdb.post_mortem()—because they contain references to real stack frames.
Stack trace is collected when exception bubbles up. So you should print traceback on top of desired stack:
import traceback
def func():
raise Exception('Dummy')
def func2():
func()
try:
func2()
except:
traceback.print_exc()
There’s some more information that could be extracted from the traceback, and I sometimes prefer a neater, more ‘logical’ information instead of multi-line blob with files, line numbers and code snippetsgiven by traceback. Preferably one line should say all the essentials.
To achieve this I use following function:
def raising_code_info():
code_info = ''
try:
frames = inspect.trace()
if(len(frames)):
full_method_name = frames[0][4][0].rstrip('nr').strip()
line_number = frames[1][2]
module_name = frames[0][0].f_globals['__name__']
if(module_name == '__main__'):
module_name = os.path.basename(sys.argv[0]).replace('.py','')
class_name = ''
obj_name_dot_method = full_method_name.split('.', 1)
if len(obj_name_dot_method) > 1:
obj_name, full_method_name = obj_name_dot_method
try:
class_name = frames[0][0].f_locals[obj_name].__class__.__name__
except:
pass
method_name = module_name + '.'
if len(class_name) > 0:
method_name += class_name + '.'
method_name += full_method_name
code_info = '%s, line %d' % (method_name, line_number)
finally:
del frames
sys.exc_clear()
return code_info
It gives . and line number, e.g.:
(example module name: test.py):
(line 73:)
def function1():
print 1/0
class AClass(object):
def method2(self):
a = []
a[3] = 1
def try_it_out():
# try it with a function
try:
function1()
except Exception, what:
print '%s: "%s"' % (raising_code_info(), what)
# try it with a method
try:
my_obj_name = AClass()
my_obj_name.method2()
except Exception, what:
print '%s: "%s"' % (raising_code_info(), what)
if __name__ == '__main__':
try_it_out()
test.function1(), line 75: "integer division or modulo by zero"
test.AClass.method2(), line 80: "list assignment index out of range"
Which might be slightly neater in some use-cases.
i have written a module that writes a more complete traceback
The module is here documentation is docs
(also you can get the module from pypi
sudo pip install pd
)
To catch and pring exceptions do the following:
import pd
try:
<python code>
except BaseException:
pd.print_exception_ex( follow_objects = 1 )
The stack trace looks like this one here:
Exception: got it
#1 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 1) at t test_pd.py:29
Calls next frame at:
raise Exception('got it') at: test_pd.py:29
#2 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 2) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#3 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 3) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#4 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 4) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#5 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 5) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#6 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 6) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#7 def main() at test_pd.py:44
Local variables:
n = {'a': 42, 'b': [1, 2, 3, 4]}
Calls next frame at:
pd.print_exception_ex( follow_objects = 1 ) at: test_pd.py:44
follow_objects = 0 will not print out object content (with complex data structures follow_objects can take a lot of time).
This is based on user4815162342’s answer, but a bit more minimalistic:
import sys
import collections
FauxTb = collections.namedtuple("FauxTb", ["tb_frame", "tb_lineno", "tb_next"])
def full_exc_info():
"""Like sys.exc_info, but includes the full traceback."""
t, v, tb = sys.exc_info()
f = sys._getframe(2)
while f is not None:
tb = FauxTb(f, f.f_lineno, tb)
f = f.f_back
return t, v, tb
It avoids throwing the dummy exception, at the cost of requiring the usage of sys._getframe(). It assumes being used in the except clause where the exception was caught, as it goes up two stack frames (full_exc_info and the function that calls full_exc_info – that would be the function that calls the raising code, and as such is already included in the original traceback).
This gives the same output as the code in user4815162342’s answer.
If you don’t mind the slight differences in formatting, you can also use
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.exception("Something awful happened!", stack_info=True)
def func2():
func()
func2()
which results in
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "test.py", line 5, in func
raise Exception('Dummy')
Exception: Dummy
Stack (most recent call last):
File "test.py", line 12, in <module>
func2()
File "test.py", line 10, in func2
func()
File "test.py", line 7, in func
logging.exception("Something awful happened!", stack_info=True)
In this case, you’ll get a trace from the try to the exception, and a second from the root call to the location of the logging call.
How can i get full traceback in the following case, including the calls of func2 and func functions?
import traceback
def func():
try:
raise Exception('Dummy')
except:
traceback.print_exc()
def func2():
func()
func2()
When i run this, i get:
Traceback (most recent call last):
File "test.py", line 5, in func
raise Exception('Dummy')
Exception: Dummy
traceback.format_stack() is not what i want, as need traceback object to be passed to a third party module.
I am particularly interested in this case:
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.exception("Something awful happened!")
def func2():
func()
func2()
In this case i am getting:
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "test.py", line 9, in func
raise Exception('Dummy')
Exception: Dummy
As mechmind answered, the stack trace consists only of frames between the site where the exception was raised and the site of the try block. If you need the full stack trace, apparently you’re out of luck.
Except that it’s obviously possible to extract the stack entries from top-level to the current frame—traceback.extract_stack manages it just fine. The problem is that the information obtained by traceback.extract_stack comes from direct inspection of stack frames without creating a traceback object at any point, and the logging API requires a traceback object to affect traceback output.
Fortunately, logging doesn’t require an actual traceback object, it requires an object that it can pass to the formatting routines of the traceback module. traceback doesn’t care either—it only uses two attributes of the traceback, the frame and the line number. So, it should be possible to create a linked list of duck-typed faux-traceback objects and pass it off as the traceback.
import sys
class FauxTb(object):
def __init__(self, tb_frame, tb_lineno, tb_next):
self.tb_frame = tb_frame
self.tb_lineno = tb_lineno
self.tb_next = tb_next
def current_stack(skip=0):
try: 1/0
except ZeroDivisionError:
f = sys.exc_info()[2].tb_frame
for i in xrange(skip + 2):
f = f.f_back
lst = []
while f is not None:
lst.append((f, f.f_lineno))
f = f.f_back
return lst
def extend_traceback(tb, stack):
"""Extend traceback with stack info."""
head = tb
for tb_frame, tb_lineno in stack:
head = FauxTb(tb_frame, tb_lineno, head)
return head
def full_exc_info():
"""Like sys.exc_info, but includes the full traceback."""
t, v, tb = sys.exc_info()
full_tb = extend_traceback(tb, current_stack(1))
return t, v, full_tb
With these functions in place, your code only requires a trivial modification:
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.error("Something awful happened!", exc_info=full_exc_info())
def func2():
func()
func2()
…to give the expected output:
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "a.py", line 52, in <module>
func2()
File "a.py", line 49, in func2
func()
File "a.py", line 43, in func
raise Exception('Dummy')
Exception: Dummy
Note that the faux-traceback objects are fully usable for introspection—displaying local variables or as argument to pdb.post_mortem()—because they contain references to real stack frames.
Stack trace is collected when exception bubbles up. So you should print traceback on top of desired stack:
import traceback
def func():
raise Exception('Dummy')
def func2():
func()
try:
func2()
except:
traceback.print_exc()
There’s some more information that could be extracted from the traceback, and I sometimes prefer a neater, more ‘logical’ information instead of multi-line blob with files, line numbers and code snippetsgiven by traceback. Preferably one line should say all the essentials.
To achieve this I use following function:
def raising_code_info():
code_info = ''
try:
frames = inspect.trace()
if(len(frames)):
full_method_name = frames[0][4][0].rstrip('nr').strip()
line_number = frames[1][2]
module_name = frames[0][0].f_globals['__name__']
if(module_name == '__main__'):
module_name = os.path.basename(sys.argv[0]).replace('.py','')
class_name = ''
obj_name_dot_method = full_method_name.split('.', 1)
if len(obj_name_dot_method) > 1:
obj_name, full_method_name = obj_name_dot_method
try:
class_name = frames[0][0].f_locals[obj_name].__class__.__name__
except:
pass
method_name = module_name + '.'
if len(class_name) > 0:
method_name += class_name + '.'
method_name += full_method_name
code_info = '%s, line %d' % (method_name, line_number)
finally:
del frames
sys.exc_clear()
return code_info
It gives . and line number, e.g.:
(example module name: test.py):
(line 73:)
def function1():
print 1/0
class AClass(object):
def method2(self):
a = []
a[3] = 1
def try_it_out():
# try it with a function
try:
function1()
except Exception, what:
print '%s: "%s"' % (raising_code_info(), what)
# try it with a method
try:
my_obj_name = AClass()
my_obj_name.method2()
except Exception, what:
print '%s: "%s"' % (raising_code_info(), what)
if __name__ == '__main__':
try_it_out()
test.function1(), line 75: "integer division or modulo by zero"
test.AClass.method2(), line 80: "list assignment index out of range"
Which might be slightly neater in some use-cases.
i have written a module that writes a more complete traceback
The module is here documentation is docs
(also you can get the module from pypi
sudo pip install pd
)
To catch and pring exceptions do the following:
import pd
try:
<python code>
except BaseException:
pd.print_exception_ex( follow_objects = 1 )
The stack trace looks like this one here:
Exception: got it
#1 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 1) at t test_pd.py:29
Calls next frame at:
raise Exception('got it') at: test_pd.py:29
#2 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 2) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#3 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 3) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#4 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 4) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#5 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 5) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#6 def kuku2(self = {'a': 42, 'b': [1, 2, 3, 4]}, depth = 6) at test_pd.py:28
Calls next frame at:
self.kuku2( depth - 1 ) at: test_pd.py:28
#7 def main() at test_pd.py:44
Local variables:
n = {'a': 42, 'b': [1, 2, 3, 4]}
Calls next frame at:
pd.print_exception_ex( follow_objects = 1 ) at: test_pd.py:44
follow_objects = 0 will not print out object content (with complex data structures follow_objects can take a lot of time).
This is based on user4815162342’s answer, but a bit more minimalistic:
import sys
import collections
FauxTb = collections.namedtuple("FauxTb", ["tb_frame", "tb_lineno", "tb_next"])
def full_exc_info():
"""Like sys.exc_info, but includes the full traceback."""
t, v, tb = sys.exc_info()
f = sys._getframe(2)
while f is not None:
tb = FauxTb(f, f.f_lineno, tb)
f = f.f_back
return t, v, tb
It avoids throwing the dummy exception, at the cost of requiring the usage of sys._getframe(). It assumes being used in the except clause where the exception was caught, as it goes up two stack frames (full_exc_info and the function that calls full_exc_info – that would be the function that calls the raising code, and as such is already included in the original traceback).
This gives the same output as the code in user4815162342’s answer.
If you don’t mind the slight differences in formatting, you can also use
import logging
def func():
try:
raise Exception('Dummy')
except:
logging.exception("Something awful happened!", stack_info=True)
def func2():
func()
func2()
which results in
ERROR:root:Something awful happened!
Traceback (most recent call last):
File "test.py", line 5, in func
raise Exception('Dummy')
Exception: Dummy
Stack (most recent call last):
File "test.py", line 12, in <module>
func2()
File "test.py", line 10, in func2
func()
File "test.py", line 7, in func
logging.exception("Something awful happened!", stack_info=True)
In this case, you’ll get a trace from the try to the exception, and a second from the root call to the location of the logging call.