Python Logging Handler Filter Unable to Filter

Question

Log file contain many rows with the string "[websockets.client.read_frame:1152]" from websockets.client. I want to filter them out and tried the following code:

class LoggingFilter(logging.Filter):
    def filter(self, record):
        def __init__(self):
            logging.Filter.__init__(self)
        
        needle = "[websockets.client.read_frame:1152]"
        if needle in record.msg:
            logger.debug(f"Needle {needle} found in haystack: {record}")
            return 0
        return 1

daily_rotating_filehandler = DailyRotatingFileHandler(sys.path[0] + 'logs', width=4, maxBytes=100000, backupCount=999, encoding='utf-8')
daily_rotating_filehandler.addFilter(LoggingFilter())

logging.basicConfig(
        #handlers=[RotatingFileHandler('./logs/{:%Y%m%d-%H%M}.log'.format(datetime.now()), maxBytes=100000, backupCount=100, encoding='utf-8')],
        #handlers=[DailyRotatingFileHandler(sys.path[0] + 'logs', width=4, maxBytes=100000, backupCount=999, encoding='utf-8')],
        handlers=[daily_rotating_filehandler],
        level=my_global.log_level,
        format="[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s",
        datefmt='%Y-%m-%dT%H:%M:%S')

# Creating an object
logger = logging.getLogger()
#logger.addHandler(daily_rotating_filehandler)

# Change logging levels for other packages
logging.getLogger('websockets.protocol').setLevel(logging.WARNING)
logging.getLogger('websockets.client').addFilter(LoggingFilter())
logging.getLogger('websockets').addFilter(LoggingFilter())

Unfortunately, said log entries still make it through. What am I missing?

Update:

Thanks to @Mark’s answer below, the modification needed to make the code work was:

class LoggingFilter(logging.Filter):

    def __init__(self):
        logging.Filter.__init__(self)
            
    def filter(self, record):
        if record.name == "websockets.client" and record.funcName == "read_frame":
            return 0
        return 1

Asked By: Thomas

||

Source

Answer 1

The problem with your filter is that you expect the value in the log to match the record msg. That is, you are searching the record.msg for "[websockets.client.read_frame:1152]". However, that string is generated by other fields in the logging record. You can figure it out from your format:

"[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s"

So, the string ""[websockets.client.read_frame:1152]" is really generated by this portion of the format string:

[%(name)s.%(funcName)s:%(lineno)d]

However, you could change the code to eliminate the logging of line 1152 on the funcName of "read_frame" like this (I hope):

    if record.lineno == 1152 and record.funcName == "read_fname":
        logger.debug(f"Needle {needle} found in haystack: {record}")
        return False
    return True

I haven’t tested the code above (for instance, I assume that record.lineno is an int, so you’ll need to work a bit with that.

Also, I’ll point that the field you are using is msg, which is really the string passed into the logging call. For instance,

logger.info("Here's a message")

In the example above, msg == "Here's a message". msg does not contain all of the other values (like the logger name or the level or the lineno).

Answered By: Mark

Python Logging Handler Filter Unable to Filter

Question:

Answers: