install filter on logging level in python using dictConfig

Question

I cannot install a filter on a logging handler using dictConfig() syntax. LoggingErrorFilter.filter() is simply ignored, nothing happens.

I want to filter out error messages so that they do not appear twice in the log output. So I wrote LoggingErrorFilter class and overrode filter().

My configuration:

class LoggingErrorFilter(logging.Filter):
  def filter(self, record):
    print 'filter!'
    return record.levelno == logging.ERROR or record.levelno == logging.CRITICAL

config = {
      'version': 1,
      'disable_existing_loggers' : False,
      'formatters' : {
        'standard' : {
          'format' : '%(asctime)s %(levelname)s %(name)s::%(message)s',
        },
      },
      'handlers' : {
        'console': {
          'class' : 'logging.StreamHandler',
          'level' : level,
          'formatter' : 'standard',
          'stream' : 'ext://sys.stdout',
        },
        'errorconsole': {
          'class' : 'logging.StreamHandler',
          'level' : 'ERROR',
          'formatter' : 'standard',
          'stream' : 'ext://sys.stderr',
          'filters'  :['errorfilter',],
        },
      },
      'filters': {
        'errorfilter': {
          'class' : 'LoggingErrorFilter',
        }
      },
      'loggers' : {
        '' : {
          'handlers' : ['errorconsole','console',],
          'level' : level,
          'propagate' : True,
        },
        name : {
          'handlers' : ['errorconsole','console',],
          'level' : level,
          'propagate' : False,
        },
      },
  }
  logging.config.dictConfig(config)

What am I doing wrong here? Why is my filter ignored?

Asked By: Pavlo Dyban

||

Source

Answer 1

Actually, Tupteq‘s answer is not correct in general. The following script:

import logging
import logging.config
import sys

class MyFilter(logging.Filter):
    def __init__(self, param=None):
        self.param = param

    def filter(self, record):
        if self.param is None:
            allow = True
        else:
            allow = self.param not in record.msg
        if allow:
            record.msg = 'changed: ' + record.msg
        return allow

LOGGING = {
    'version': 1,
    'filters': {
        'myfilter': {
            '()': MyFilter,
            'param': 'noshow',
        }
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'filters': ['myfilter']
        }
    },
    'root': {
        'level': 'DEBUG',
        'handlers': ['console']
    },
}

if __name__ == '__main__':
    print(sys.version)
    logging.config.dictConfig(LOGGING)
    logging.debug('hello')
    logging.debug('hello - noshow')

When run, produces the following output:

$ python filtcfg.py 
2.7.5+ (default, Sep 19 2013, 13:48:49) 
[GCC 4.8.1]
changed: hello

which shows that you can configure filters using dictConfig().

Answered By: Vinay Sajip

Answer 2

You can specify a class name, but it is done with the strangely named () key, and it has to include the module name. E.g.:

 'filters': {
    'errorfilter': {
      '()' : '__main__.LoggingErrorFilter',
    }
  },

See 16.7.2.4. User-defined objects in the documentation.

Answered By: Daniel Darabos

Answer 3

if wanted to load from a json file. this is how it should be

{
  "version": 1,
  "grey": "/x1b[38;21m",
  "yellow": "/x1b[33;21m",
  "red": "/x1b[31;21m",
  "bold_red": "/x1b[31;1m",
  "reset": "/x1b[0m",
  "filters": {
    "audit_filter": {
      "()": "app.logs.AuditFilter.AuditFilter",
      "param": "audit"
    },
    "ignore_audit_filter": {
      "()": "app.logs.AuditFilter.IgnoreAuditFilter",
      "param": "audit"
    }
  },
  "formatters": {
    "package_formatter": {
      "format": "[%(asctime)s] - [%(levelname)s] - [%(name)s] : %(message)s"
    }
  },
  "handlers": {
    "console": {
      "class": "logging.StreamHandler",
      "level": "DEBUG",
      "formatter": "package_formatter"
    },
    "development_file_handler": {
      "class": "logging.FileHandler",
      "filename": "../debug.log",
      "level": "DEBUG",
      "formatter": "package_formatter",
      "filters": [
        "ignore_audit_filter"
      ]
    },
    "audit_file_handler": {
      "class": "logging.FileHandler",
      "filename": "../audit.log",
      "level": "DEBUG",
      "formatter": "package_formatter",
      "filters": [
        "audit_filter"
      ]
    }
  },
  "loggers": {
    "audit": {
      "handlers": [

        "audit_file_handler"
      ],
      "level": "INFO"
    }
  },
  "root": {
    "level": "INFO",
    "handlers": [
      "console",
      "development_file_handler"
    ]
  }
}

initialise and then attach logger and run the logger
use app.logger = logging.getlogger("audit")

run the application

Answered By: vishal

install filter on logging level in python using dictConfig

Question:

Answers: