Problem with Logging Module in Google Colab

Question

I have a python script with an error handling using the logging module. Although this python script works when imported to google colab, it doesn’t log the errors in the log file.

As an experiment, I tried this following script in google colab just to see if it writes log at all

import logging
logging.basicConfig(filename="log_file_test.log",
                            filemode='a',
                            format='%(asctime)s,%(msecs)d %(name)s %(levelname)s %(message)s',
                            datefmt='%H:%M:%S',
                            level=logging.DEBUG)

logging.info("This is a test log ..")

To my dismay, it didn’t even create a log file named log_file_test.log. I tried running the same script locally and it did produce a file log_file_test.log with the following text

13:20:53,441 root INFO This is a test log ..

What is it that I am missing here?
For the time being, I am replacing the error logs with print statements, but I assume that there must be a workaround to this.

Asked By: Afsan Abdulali Gujarati

||

Source

Answer 1

Perhaps you’ve reconfigured your environment somehow? (Try Runtime menu -> Reset all runtimes…) Your snippets works exactly as written for me —

Answered By: Bob Smith

Answer 2

logging.basicConfig can be run just once*

Any subsequent call to basicConfig is ignored.

* unless you are in Python 3.8 and use the flag force=True

logging.basicConfig(filename='app.log',
                    level=logging.DEBUG,
                    force=True, # Resets any previous configuration
                    )

Workarounds (2)

(1) You can easily reset the Colab workspace with this command

exit

Wait for it to come back and try your commands again.

(2) But, if you plan to do the reset more than once and/or are learning to use logging, maybe it is better to use %%python magic to run the entire cell in a subprocess. See photo below.

What is it that I am missing here?

Deeper understanding of how logging works. It is a bit tricky, but there are many good webs explaining the gotchas.

Answered By: Rub

Answer 3

[This answer][1] cover the issue.
You have to:

Clear your log handlers from the environment with logging.root.removeHandler
Set log level with logging.getLogger(‘RootLogger’).setLevel(logging.DEBUG).

Setting level with logging.basicConfig only did not work for me.

Answered By: Евгений Сеннер

Problem with Logging Module in Google Colab

Question:

Answers: