Removing a specific logger object in python logging
Question:
I am trying to suppress all the elasticsearch logging from my output and keep all the other logging in my code. I have tried setting elasticsearch logging level to warning as it is explained here How to set the logging level for the elasticsearch library differently to my own logging? , and Python logging, how to filter a specific logger , but it is not working for me.
Alternatively, if there was an option to remove certain logger objects it would have fixed my issue, like this one here: https://bugs.python.org/issue34199
I want to remove all these POST status messages in this output:
out
import logging
path = os.getcwd()
file_path = os.path.join(path, "operations.log")
tracer = logging.getLogger('elasticsearch')
tracer.setLevel(logging.WARNING)
logging.basicConfig(level=logging.INFO, encoding="utf-8", format="%(asctime)s: %(levelname)s: %(message)s", handlers=[logging.FileHandler(file_path), logging.StreamHandler()])
I am expecting a method like logging.deleteLogger("logger object name")
Answers:
Your code seems right. Rather than delete a logger, you want to set its value so that log messages from that logger are suppressed. This is what you are doing.
Here’s a simple test of your code that shows that it’s working as expected:
import logging
import os
path = "/tmp"
file_path = os.path.join(path, "operations.log")
tracer = logging.getLogger('elasticsearch')
tracer.setLevel(logging.WARNING)
logging.basicConfig(level=logging.INFO, encoding="utf-8", format="%(name)s %(asctime)s: %(levelname)s: %(message)s", handlers=[logging.FileHandler(file_path), logging.StreamHandler()])
tracer.error("ERROR")
tracer.debug("DEBUG")
tracer.warning("WARNING")
Result:
elasticsearch 2022-12-08 21:52:36,261: ERROR: ERROR
elasticsearch 2022-12-08 21:52:36,261: WARNING: WARNING
Note that the DEBUG log message is suppressed
I can think of two things to have you check…
-
tracer.setLevel(logging.WARNING) means you’ll still get WARNING and ERROR messages for the elasticsearch logger (as demonstrated above). You’ll only be suppressing INFO and DEBUG messages. Maybe that’s what you want, but you don’t state that clearly. When I want to suppress all logging levels, I set the level of the logger to SEVERE. Nobody seems to use that level, so it is in effect saying to suppress all levels. I don’t know of a formal way to suppress all logging levels. You usually don’t want to suppress even ERROR messages anyway, so I often use either ERROR or WARNING.
-
Make sure that "elasticsearch" is the name of the logger you want to suppress. It could be something different, like "com.elasticsearch" or some other variation. Note how I added the %(name)s format specifier to the logging format so that the logger name is displayed. You can add this to your own format specifier at least temporarily to be sure that you are suppressing the right logger.
I am trying to suppress all the elasticsearch logging from my output and keep all the other logging in my code. I have tried setting elasticsearch logging level to warning as it is explained here How to set the logging level for the elasticsearch library differently to my own logging? , and Python logging, how to filter a specific logger , but it is not working for me.
Alternatively, if there was an option to remove certain logger objects it would have fixed my issue, like this one here: https://bugs.python.org/issue34199
I want to remove all these POST status messages in this output:
out
import logging
path = os.getcwd()
file_path = os.path.join(path, "operations.log")
tracer = logging.getLogger('elasticsearch')
tracer.setLevel(logging.WARNING)
logging.basicConfig(level=logging.INFO, encoding="utf-8", format="%(asctime)s: %(levelname)s: %(message)s", handlers=[logging.FileHandler(file_path), logging.StreamHandler()])
I am expecting a method like logging.deleteLogger("logger object name")
Your code seems right. Rather than delete a logger, you want to set its value so that log messages from that logger are suppressed. This is what you are doing.
Here’s a simple test of your code that shows that it’s working as expected:
import logging
import os
path = "/tmp"
file_path = os.path.join(path, "operations.log")
tracer = logging.getLogger('elasticsearch')
tracer.setLevel(logging.WARNING)
logging.basicConfig(level=logging.INFO, encoding="utf-8", format="%(name)s %(asctime)s: %(levelname)s: %(message)s", handlers=[logging.FileHandler(file_path), logging.StreamHandler()])
tracer.error("ERROR")
tracer.debug("DEBUG")
tracer.warning("WARNING")
Result:
elasticsearch 2022-12-08 21:52:36,261: ERROR: ERROR
elasticsearch 2022-12-08 21:52:36,261: WARNING: WARNING
Note that the DEBUG log message is suppressed
I can think of two things to have you check…
-
tracer.setLevel(logging.WARNING)means you’ll still getWARNINGandERRORmessages for theelasticsearchlogger (as demonstrated above). You’ll only be suppressingINFOandDEBUGmessages. Maybe that’s what you want, but you don’t state that clearly. When I want to suppress all logging levels, I set the level of the logger toSEVERE. Nobody seems to use that level, so it is in effect saying to suppress all levels. I don’t know of a formal way to suppress all logging levels. You usually don’t want to suppress evenERRORmessages anyway, so I often use eitherERRORorWARNING. -
Make sure that "elasticsearch" is the name of the logger you want to suppress. It could be something different, like "com.elasticsearch" or some other variation. Note how I added the
%(name)sformat specifier to the logging format so that the logger name is displayed. You can add this to your own format specifier at least temporarily to be sure that you are suppressing the right logger.