Logging and raising an exception

Question

I’d like to keep a solid logging system going, but it’s also necessary to raise exceptions. This code accomplishes what I’m going for, but it looks clunky and not very Pythonic. What’s a better option?

import logging

if not condition_met:
    missing = set_one - set_two
    logging.error('Missing keys: {}'.format(missing))
    raise ValueError('Missing keys: {}'.format(missing))

Asked By: questionable_code

||

Source

Answer 1

you could catch the exception and log the error at this time, so if another exception occurs you can log it as well, and propagate the exception upstream.

try:
   # some code

    if not condition_met:
        missing = set_one - set_two
        raise ValueError('Missing keys: {}'.format(missing))   

except Exception as e:  # or ValueError to narrow it down
    logging.error(str(e))
    raise   # propagate the exception again

note than logging an exception without logging the traceback leaves something unfinished, specially if the exception is caught and handled upstream. It’s very likely that you’re never going to fix that particular error.

Answered By: Jean-François Fabre

Answer 2

You can use logger‘s exception() function:

from logger import exception

try:
    . . .
except Exception as error:
    exception(msg="Your message")

so that all of the stack will be logged.

You can read an interesting article about this here.

Answered By: PIERPAOLO MASELLA

Answer 3

Another elegant approach is to define custom exceptions for your application that serve the purpose of clarifying lower-level exceptions such as KeyError as well as centralizing error logic. These custom exceptions can be defined on a separate file to make maintenance and updates easier. custom exceptions are derived from a base Error class to inherit global settings which itself is derived from the built-in Exception class.

exceptions.py

from utils import log

class Error(Exception):
  """base class for errors"""


class EnvironmentAttributeError(Error):
  """
  Exception raised when environment variables are empty strings

  Attributes:
    key_attribute -- environment variable
  """

  def __init__(self, environment_variable):
    self.environment_variable = environment_variable
    self.message = f"Environment variable value for key {environment_variable} was not assigned."
    self.log = log.logger.error(f"Environment variable value for key {environment_variable} was not assigned.")
    super().__init__(self.message)


class EnvironmentKeyError(Error):
  """
  Exception raised when the environment variables dict does not have required keys

  Attributes:
    key_attribute -- environment variable
  """

  def __init__(self, vars):
    self.environment_variable = vars
    self.message = f"Environment variable {vars} was not declared."
    self.log = log.logger.error(f"Environment variable {vars} was not declared.")
    super().__init__(self.message)

Notice that the exceptions.py file imports a log utility. That way all you need to do elsewhere in your code is raise the right custom code errors and everything gets logged for you. You can then update these errors in a single place for your entire project.

log.py

import logging

# instantiate a logger object writing to connected-apps.log
logging.basicConfig(
  format='%(asctime)s,%(msecs)d %(levelname)-8s [%(filename)s:%(lineno)d] %(message)s',
  datefmt='%Y-%m-%d:%H:%M:%S',
  level=logging.DEBUG,
  filename='logs/connected_apps.log'
)    
# logger object named after module: https://docs.python.org/3/howto/logging.html#advanced-logging-tutorial
logger = logging.getLogger(__name__)

The logger in the log.py file has been formatted in such a way that logs are both descriptive and readable. You can even define different loggers with different formats and levels.

Here is a simple use of the custom exceptions defined above. Environment variables obtained from .env are sent to this validate() function to verify that the right keys and attributes are available. Notice that we just needed to import exceptions and not logs:

environment.py

from utils import exceptions


def validate(env_dict, env_vars):
  # check that each environment variable has been declared and assigned
  for vars in env_vars:
    try:
      # check the local dictionary pulled from os.environ
      env_dict[vars]

      # check that key value length is non-zero
      if len(env_dict[vars]) == 0:
        raise exceptions.EnvironmentAttributeError(vars)

    except KeyError as error:
      # raises error if an environment variable has not been declared
      raise exceptions.EnvironmentKeyError(vars)

Answered By: stelloprint

Logging and raising an exception

Question:

Answers: