How to make SMTPHandler not block

Most probably you need to write your own logging handler that would do the sending of the email in the background.

You could use QueueHandler and QueueListener. Taken from the docs:

Along with the QueueListener class, QueueHandler can be used to let
handlers do their work on a separate thread from the one which does
the logging. This is important in Web applications and also other
service applications where threads servicing clients need to respond
as quickly as possible, while any potentially slow operations (such as
sending an email via SMTPHandler) are done on a separate thread.

Alas they are only available from Python 3.2 onward.

A thing to keep in mind when coding in Python is the GIL (Global Interpreter Lock). This lock prevents more than one process from happening at the same time. there are many number of things that are ‘Blocking’ activities in Python. They will stop everything until they completed.

Currently the only way around the GIL is to either push off the action you are attempting to an outside source like aix and MattH are suggesting, or to implement your code using the multiprocessing module (http://docs.python.org/library/multiprocessing.html) so that one process is handling the sending of messages and the rest is being handled by the other process.

The simplest form of asynchronous smtp handler for me is just to override emit method and use the original method in a new thread. GIL is not a problem in this case because there is an I/O call to SMTP server which releases GIL. The code is as follows

class ThreadedSMTPHandler(SMTPHandler):
    def emit(self, record):
        thread = Thread(target=SMTPHandler.emit, args=(self, record))
        thread.start()

As the OP pointed out, QueueHandler and QueueListener can do the trick! I did some research and adapted code found on this page to provide you with some sample code:

# In your init part,
# assuming your logger is given by the "logger" variable
# and your config is storded in the "config" dictionary

logging_queue = Queue(-1)
queue_handler = QueueHandler(logging_queue)
queue_handler.setLevel(logging.ERROR)
queue_handler.setFormatter(logging_formatter)
logger.addHandler(queue_handler)

smtp_handler = SMTPHandler(mailhost=(config['MAIL_SERVER'], config['MAIL_PORT']),
                           fromaddr=config['MAIL_SENDER'],
                           toaddrs=[config['ERROR_MAIL']],
                           subject='Application error',
                           credentials=(config['MAIL_USERNAME'], config['MAIL_PASSWORD']),
                           secure=tuple())
smtp_handler.setLevel(logging.ERROR)
smtp_handler.setFormatter(logging_formatter)

queue_listener = QueueListener(logging_queue, smtp_handler)
queue_listener.start()

# Let's test it. The warning is not mailed, the error is.
logger.warning('Test warning')
logger.error('Test error')

What I am not sure about is whether it is necessary to use setLevel and setFormatter twice, probably not.

Here’s the implementation I’m using, which I based on Jonathan Livni code.

import logging.handlers
import smtplib
from threading import Thread

# File with my configuration
import credentials as cr

host = cr.set_logSMTP["host"]
port = cr.set_logSMTP["port"]
user = cr.set_logSMTP["user"]
pwd = cr.set_logSMTP["pwd"]
to = cr.set_logSMTP["to"]



def smtp_at_your_own_leasure(
    mailhost, port, username, password, fromaddr, toaddrs, msg
):
    smtp = smtplib.SMTP(mailhost, port)
    if username:
        smtp.ehlo()  # for tls add this line
        smtp.starttls()  # for tls add this line
        smtp.ehlo()  # for tls add this line
        smtp.login(username, password)
    smtp.sendmail(fromaddr, toaddrs, msg)
    smtp.quit()


class ThreadedTlsSMTPHandler(logging.handlers.SMTPHandler):
    def emit(self, record):
        try:
            # import string  # <<<CHANGE THIS>>>

            try:
                from email.utils import formatdate
            except ImportError:
                formatdate = self.date_time
            port = self.mailport
            if not port:
                port = smtplib.SMTP_PORT
            msg = self.format(record)
            msg = "From: %srnTo: %srnSubject: %srnDate: %srnrn%s" % (
                self.fromaddr,
                ",".join(self.toaddrs),  # <<<CHANGE THIS>>>
                self.getSubject(record),
                formatdate(),
                msg,
            )
            thread = Thread(
                target=smtp_at_your_own_leasure,
                args=(
                    self.mailhost,
                    port,
                    self.username,
                    self.password,
                    self.fromaddr,
                    self.toaddrs,
                    msg,
                ),
            )
            thread.start()
        except (KeyboardInterrupt, SystemExit):
            raise
        except:
            self.handleError(record)


# Test
if __name__ == "__main__":
    logger = logging.getLogger()

    gm = ThreadedTlsSMTPHandler((host, port), user, to, "Error!:", (user, pwd))
    gm.setLevel(logging.ERROR)

    logger.addHandler(gm)

    try:
        1 / 0
    except:
        logger.exception("Test ZeroDivisionError: division by zero")