How to log error to file, and not fail on exception
Question:
I am downloading a file from the net, and it fails even though I am doing:
for p in query:
try:
except IOError as e:
print e;
If there is an error, I want to log it, and then continue on with the next file.
In this loop, I am trying to download an image, if for some reason the filename was bad, or the website was down, etc., I want to continue with the next item in the for loop.
Is there a more generic error that won’t fail and continue processing?
Also, how can I log errors to a file?
Answers:
You could use the logging module:
import logging
logging.basicConfig(filename='/tmp/myapp.log', level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(name)s %(message)s')
logger=logging.getLogger(__name__)
try:
1/0
except ZeroDivisionError as err:
logger.error(err)
Running the script writes in /tmp/myapp.log:
% cat /tmp/myapp.log
2010-08-01 17:50:45,960 ERROR __main__ integer division or modulo by zero
As pointed out by Lott, if a download is failing, unless the problem is fixed upstream (or with your download address), the best you can do is to try again. However, if the situation is that you have a list of downloads, and simply want to skip over failed downloads instead of exiting, then:
logf = open("download.log", "w")
for download in download_list:
try:
# code to process download here
except Exception as e: # most generic exception you can catch
logf.write("Failed to download {0}: {1}n".format(str(download), str(e)))
# optional: delete local version of failed download
finally:
# optional clean up code
pass
Things to note:
(1) Use of the “logging” module, as suggested by ~unutbu, gives you much more flexibility and power with your logging output, including time stamp, simultaneously writing to different channels (e.g., stderr, file) depending on error levels, etc. etc.
(2) You might consider implementing the above logic using the “with” construct.
This catches everything.
But it is much, much better to catch the exact exception.
python <= 2.7
while True:
try:
doStuff()
except Exception, e:
f = open('log.txt', 'w')
f.write('An exceptional thing happed - %s' % e)
f.close()
This will write your error to a log file and continue to run the code.
import traceback
#This line opens a log file
with open("log.txt", "w") as log:
try:
# some code
# Below line will print any print to log file as well.
print("Creating DB Connection", file = log)
except Exception:
traceback.print_exc(file=log)
continue
Add a continue statement in the except block. The continue statement will continue the loop at the next item, bypassing any remaining code within the current loop iteration.
for p in query:
try:
# The thing that might fail
except IOError as error:
print(error)
continue
For more information, see: How To Use Break, Continue, and Pass Statements when Working with Loops in Python 3
I am downloading a file from the net, and it fails even though I am doing:
for p in query:
try:
except IOError as e:
print e;
If there is an error, I want to log it, and then continue on with the next file.
In this loop, I am trying to download an image, if for some reason the filename was bad, or the website was down, etc., I want to continue with the next item in the for loop.
Is there a more generic error that won’t fail and continue processing?
Also, how can I log errors to a file?
You could use the logging module:
import logging
logging.basicConfig(filename='/tmp/myapp.log', level=logging.DEBUG,
format='%(asctime)s %(levelname)s %(name)s %(message)s')
logger=logging.getLogger(__name__)
try:
1/0
except ZeroDivisionError as err:
logger.error(err)
Running the script writes in /tmp/myapp.log:
% cat /tmp/myapp.log
2010-08-01 17:50:45,960 ERROR __main__ integer division or modulo by zero
As pointed out by Lott, if a download is failing, unless the problem is fixed upstream (or with your download address), the best you can do is to try again. However, if the situation is that you have a list of downloads, and simply want to skip over failed downloads instead of exiting, then:
logf = open("download.log", "w")
for download in download_list:
try:
# code to process download here
except Exception as e: # most generic exception you can catch
logf.write("Failed to download {0}: {1}n".format(str(download), str(e)))
# optional: delete local version of failed download
finally:
# optional clean up code
pass
Things to note:
(1) Use of the “logging” module, as suggested by ~unutbu, gives you much more flexibility and power with your logging output, including time stamp, simultaneously writing to different channels (e.g., stderr, file) depending on error levels, etc. etc.
(2) You might consider implementing the above logic using the “with” construct.
This catches everything.
But it is much, much better to catch the exact exception.
python <= 2.7
while True:
try:
doStuff()
except Exception, e:
f = open('log.txt', 'w')
f.write('An exceptional thing happed - %s' % e)
f.close()
This will write your error to a log file and continue to run the code.
import traceback
#This line opens a log file
with open("log.txt", "w") as log:
try:
# some code
# Below line will print any print to log file as well.
print("Creating DB Connection", file = log)
except Exception:
traceback.print_exc(file=log)
continue
Add a continue statement in the except block. The continue statement will continue the loop at the next item, bypassing any remaining code within the current loop iteration.
for p in query:
try:
# The thing that might fail
except IOError as error:
print(error)
continue
For more information, see: How To Use Break, Continue, and Pass Statements when Working with Loops in Python 3