How to save traceback / sys.exc_info() values in a variable?
Question:
I want to save the name of the error and the traceback details into a variable. Here’s is my attempt.
import sys
try:
try:
print x
except Exception, ex:
raise NameError
except Exception, er:
print "0", sys.exc_info()[0]
print "1", sys.exc_info()[1]
print "2", sys.exc_info()[2]
Output:
0 <type 'exceptions.NameError'>
1
2 <traceback object at 0xbd5fc8>
Desired Output:
0 NameError
1
2 Traceback (most recent call last):
File "exception.py", line 6, in <module>
raise NameError
P.S. I know this can be done easily using the traceback module, but I want to know usage of sys.exc_info()[2] object here.
Answers:
This is how I do it:
>>> import traceback
>>> try:
... int('k')
... except:
... var = traceback.format_exc()
...
>>> print var
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ValueError: invalid literal for int() with base 10: 'k'
You should however take a look at the traceback documentation, as you might find there more suitable methods, depending to how you want to process your variable afterwards…
Use traceback.extract_stack() if you want convenient access to module and function names and line numbers.
Use ''.join(traceback.format_stack()) if you just want a string that looks like the traceback.print_stack() output.
Notice that even with ''.join() you will get a multi-line string, since the elements of format_stack() contain n. See output below.
Remember to import traceback.
Here’s the output from traceback.extract_stack(). Formatting added for readability.
>>> traceback.extract_stack()
[
('<string>', 1, '<module>', None),
('C:\Python\lib\idlelib\run.py', 126, 'main', 'ret = method(*args, **kwargs)'),
('C:\Python\lib\idlelib\run.py', 353, 'runcode', 'exec(code, self.locals)'),
('<pyshell#1>', 1, '<module>', None)
]
Here’s the output from ''.join(traceback.format_stack()). Formatting added for readability.
>>> ''.join(traceback.format_stack())
' File "<string>", line 1, in <module>n
File "C:\Python\lib\idlelib\run.py", line 126, in mainn
ret = method(*args, **kwargs)n
File "C:\Python\lib\idlelib\run.py", line 353, in runcoden
exec(code, self.locals)n File "<pyshell#2>", line 1, in <module>n'
sys.exc_info() returns a tuple with three values (type, value, traceback).
- Here type gets the exception type of the Exception being handled
- value is the arguments that are being passed to constructor of exception class
- traceback contains the stack information like where the exception occurred etc.
For Example, In the following program
try:
a = 1/0
except Exception,e:
exc_tuple = sys.exc_info()
Now If we print the tuple the values will be this.
- exc_tuple[0] value will be “ZeroDivisionError“
- exc_tuple[1] value will be “integer division or modulo by zero” (String passed as parameter to the exception class)
- exc_tuple[2] value will be “trackback object at (some memory address)“
The above details can also be fetched by simply printing the exception in string format.
print str(e)
The object can be used as a parameter in Exception.with_traceback() function:
except Exception as e:
tb = sys.exc_info()
print(e.with_traceback(tb[2]))
Be careful when you take the exception object or the traceback object out of the exception handler, since this causes circular references and gc.collect() will fail to collect. This appears to be of a particular problem in the ipython/jupyter notebook environment where the traceback object doesn’t get cleared at the right time and even an explicit call to gc.collect() in finally section does nothing. And that’s a huge problem if you have some huge objects that don’t get their memory reclaimed because of that (e.g. CUDA out of memory exceptions that w/o this solution require a complete kernel restart to recover).
In general if you want to save the traceback object, you need to clear it from references to locals(), like so:
import sys, traceback, gc
type, val, tb = None, None, None
try:
myfunc()
except:
type, val, tb = sys.exc_info()
traceback.clear_frames(tb)
# some cleanup code
gc.collect()
# and then use the tb:
if tb:
raise type(val).with_traceback(tb)
In the case of jupyter notebook, you have to do that at the very least inside the exception handler:
try:
myfunc()
except:
type, val, tb = sys.exc_info()
traceback.clear_frames(tb)
raise type(val).with_traceback(tb)
finally:
# cleanup code in here
gc.collect()
Tested with python 3.7.
p.s. the problem with ipython or jupyter notebook env is that it has %tb magic which saves the traceback and makes it available at any point later. And as a result any locals() in all frames participating in the traceback will not be freed until the notebook exits or another exception will overwrite the previously stored backtrace. This is very problematic. It should not store the traceback w/o cleaning its frames. Fix submitted here.
I want to save the name of the error and the traceback details into a variable. Here’s is my attempt.
import sys
try:
try:
print x
except Exception, ex:
raise NameError
except Exception, er:
print "0", sys.exc_info()[0]
print "1", sys.exc_info()[1]
print "2", sys.exc_info()[2]
Output:
0 <type 'exceptions.NameError'>
1
2 <traceback object at 0xbd5fc8>
Desired Output:
0 NameError
1
2 Traceback (most recent call last):
File "exception.py", line 6, in <module>
raise NameError
P.S. I know this can be done easily using the traceback module, but I want to know usage of sys.exc_info()[2] object here.
This is how I do it:
>>> import traceback
>>> try:
... int('k')
... except:
... var = traceback.format_exc()
...
>>> print var
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
ValueError: invalid literal for int() with base 10: 'k'
You should however take a look at the traceback documentation, as you might find there more suitable methods, depending to how you want to process your variable afterwards…
Use traceback.extract_stack() if you want convenient access to module and function names and line numbers.
Use ''.join(traceback.format_stack()) if you just want a string that looks like the traceback.print_stack() output.
Notice that even with ''.join() you will get a multi-line string, since the elements of format_stack() contain n. See output below.
Remember to import traceback.
Here’s the output from traceback.extract_stack(). Formatting added for readability.
>>> traceback.extract_stack()
[
('<string>', 1, '<module>', None),
('C:\Python\lib\idlelib\run.py', 126, 'main', 'ret = method(*args, **kwargs)'),
('C:\Python\lib\idlelib\run.py', 353, 'runcode', 'exec(code, self.locals)'),
('<pyshell#1>', 1, '<module>', None)
]
Here’s the output from ''.join(traceback.format_stack()). Formatting added for readability.
>>> ''.join(traceback.format_stack())
' File "<string>", line 1, in <module>n
File "C:\Python\lib\idlelib\run.py", line 126, in mainn
ret = method(*args, **kwargs)n
File "C:\Python\lib\idlelib\run.py", line 353, in runcoden
exec(code, self.locals)n File "<pyshell#2>", line 1, in <module>n'
sys.exc_info() returns a tuple with three values (type, value, traceback).
- Here type gets the exception type of the Exception being handled
- value is the arguments that are being passed to constructor of exception class
- traceback contains the stack information like where the exception occurred etc.
For Example, In the following program
try:
a = 1/0
except Exception,e:
exc_tuple = sys.exc_info()
Now If we print the tuple the values will be this.
- exc_tuple[0] value will be “ZeroDivisionError“
- exc_tuple[1] value will be “integer division or modulo by zero” (String passed as parameter to the exception class)
- exc_tuple[2] value will be “trackback object at (some memory address)“
The above details can also be fetched by simply printing the exception in string format.
print str(e)
The object can be used as a parameter in Exception.with_traceback() function:
except Exception as e:
tb = sys.exc_info()
print(e.with_traceback(tb[2]))
Be careful when you take the exception object or the traceback object out of the exception handler, since this causes circular references and gc.collect() will fail to collect. This appears to be of a particular problem in the ipython/jupyter notebook environment where the traceback object doesn’t get cleared at the right time and even an explicit call to gc.collect() in finally section does nothing. And that’s a huge problem if you have some huge objects that don’t get their memory reclaimed because of that (e.g. CUDA out of memory exceptions that w/o this solution require a complete kernel restart to recover).
In general if you want to save the traceback object, you need to clear it from references to locals(), like so:
import sys, traceback, gc
type, val, tb = None, None, None
try:
myfunc()
except:
type, val, tb = sys.exc_info()
traceback.clear_frames(tb)
# some cleanup code
gc.collect()
# and then use the tb:
if tb:
raise type(val).with_traceback(tb)
In the case of jupyter notebook, you have to do that at the very least inside the exception handler:
try:
myfunc()
except:
type, val, tb = sys.exc_info()
traceback.clear_frames(tb)
raise type(val).with_traceback(tb)
finally:
# cleanup code in here
gc.collect()
Tested with python 3.7.
p.s. the problem with ipython or jupyter notebook env is that it has %tb magic which saves the traceback and makes it available at any point later. And as a result any locals() in all frames participating in the traceback will not be freed until the notebook exits or another exception will overwrite the previously stored backtrace. This is very problematic. It should not store the traceback w/o cleaning its frames. Fix submitted here.