Python using basicConfig method to log to console and file

Try this working fine(tested in python 2.7) for both console and file

# set up logging to file
logging.basicConfig(
     filename='log_file_name.log',
     level=logging.INFO, 
     format= '[%(asctime)s] {%(pathname)s:%(lineno)d} %(levelname)s - %(message)s',
     datefmt='%H:%M:%S'
 )

# set up logging to console
console = logging.StreamHandler()
console.setLevel(logging.DEBUG)
# set a format which is simpler for console use
formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
console.setFormatter(formatter)
# add the handler to the root logger
logging.getLogger('').addHandler(console)

logger = logging.getLogger(__name__)

In the example below, you can specify the log destination based on its level. For example, the code below lets all logs over the INFO level go to the log file, and all above ERROR level goes to the console.

import logging
logging.root.handlers = []
logging.basicConfig(format='%(asctime)s : %(levelname)s : %(message)s', level=logging.INFO , filename='ex.log')

# set up logging to console
console = logging.StreamHandler()
console.setLevel(logging.ERROR)
# set a format which is simpler for console use
formatter = logging.Formatter('%(asctime)s : %(levelname)s : %(message)s')
console.setFormatter(formatter)
logging.getLogger("").addHandler(console)

logging.debug('debug')
logging.info('info')
logging.warning('warning')
logging.error('error')
logging.exception('exp')

Another technique using the basicConfig is to setup all your handlers in the statement and retrieve them after the fact, as in…

import logging

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(levelname)s %(module)s %(funcName)s %(message)s',
                    handlers=[logging.FileHandler("my_log.log", mode='w'),
                              logging.StreamHandler()])
stream_handler = [h for h in logging.root.handlers if isinstance(h , logging.StreamHandler)][0]
stream_handler.setLevel(logging.INFO)

More sensibly though is to construct your stream handler instance outside and configure them as standalone objects that you pass to the handlers list as in…

import logging

stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging.INFO)

logging.basicConfig(level=logging.DEBUG,
                    format='%(asctime)s %(levelname)s %(module)s %(funcName)s %(message)s',
                    handlers=[logging.FileHandler("my_log.log", mode='w'),
                              stream_handler])

WOOAH!

I just spent about 20 minutes being baffled by this.

Eventually I worked out that the StreamHandler was outputting to stderr, not stdout (in a ‘Doze DOS screen these have the same font colour!).

This resulted in me being able to run the code perfectly OK and get sensible results, but in a pytest function things going awry. Until I changed from this:

out, _ = capsys.readouterr()
assert 'test message check on console' in out, f'out was |{out}|'

to this:

_, err = capsys.readouterr()
assert 'test message check on console' in err, f'err was |{err}|'

NB the constructor of StreamHandler is

class logging.StreamHandler(stream=None)

and, as the docs say, "If stream is specified, the instance will use it for logging output; otherwise, sys.stderr will be used."

NB it seems that supplying the level keyword does not run setLevel on the handlers: you’d need to iterate on the resulting handlers and run setLevel on each, if it matters to you.

This is a ValueError if FileHandler and StreamHandler both are present in BasicConfig function

https://docs.python.org/3/library/logging.html#logging.basicConfig

See image below:

import logging as log
from datetime import datetime

outfile = "./out/out_" + str(datetime.now()) +".log"
log.basicConfig(
    level=log.INFO,
    format="%(asctime)s [%(levelname)s] %(message)s",
    handlers=[
        log.FileHandler(outfile),
        log.StreamHandler()
    ]
)