Python console interference between logging streamHandler() and input()

Question:

I have encountered a minor nuisance in my console output when using a StreamHandler() output from the logging module together with an input() line.

I configure my logger with the following function:

def configure_logger(filename='log'):
    """Configures a logger which outputs at level INFO and above to both
    the console and a file called 'filename'.
    """
    
    file_handler = logging.FileHandler(filename, mode='w')
    stream_handler = logging.StreamHandler()
    file_formatter = logging.Formatter(datefmt="%d/%m/%Y %H:%M:%S",
                                       fmt='%(levelname)-8s %(asctime)s '
                                           '%(name)-50s - %(message)s')
    stream_formatter = logging.Formatter(fmt='%(levelname)-8s %(name)-50s - '
                                             '%(message)s')
    file_handler.setFormatter(file_formatter)
    stream_handler.setFormatter(stream_formatter)
    logging.basicConfig(handlers=(file_handler, stream_handler),
                        level=logging.INFO)

Later I call the following function twice in a row to create two directories:

def create_directory(directory):
    """Creates a directory, prompting the user to confirm overwrite if
    it already exists
    """
    
    logger = logging.getLogger(f'{__name__}.create_directory')
    logger.info(f'Creating directory {directory}')
    
    try:
        os.mkdir(directory)
    except FileExistsError:
        overwrite = input(f'Directory {directory} already '
                          f'exists. Overwrite? (y/n): ')
        overwrite.lower()
        while overwrite not in ['y', 'yes', 'n', 'no']:
            overwrite = input(f'Enter yes/y or no/n?: ')
            overwrite.lower()
        if overwrite in ['y', 'yes']:
            logger.warning('Overwriting existing directory')
            shutil.rmtree(directory)
            os.mkdir(directory)
        else:
            logger.info("Exiting")
            sys.exit()

If the directory I am trying to create already exists, I prompt the user and ask them to confirm they want to overwrite it using the input() function.
Following a ‘yes’ response, I log a warning ‘Overwriting existing directory’.

After calling create_directory() twice in a row, I get the following in my console:

INFO     sowfatools.create_directory                        - Creating directory time_histories
Directory time_histories already exists. Overwrite? (y/n): >? y
Directory time_history_plots already exists. Overwrite? (y/n): WARNING  sowfatools.create_directory                        - Overwriting existing directory
INFO     sowfatools.create_directory                        - Creating directory time_history_plots

>?

The second input line (which comes from the second function call) is printed to the console BEFORE the warning from the first function call. In fact it comes BEFORE the first logging message in the second call as well.

Why might this be? and how do I fix it? It doesn’t make my code unusable, but it is a nuisance. Is the logging StreamHandler just too slow to keep up with the rest of the code?

Asked By: Not Dr Jeff

||

Answers:

input writes its prompts to stdout, while logging goes to stderr. The two streams are independent and can interleave unpredictably.

Either:

  1. Replace logging calls with print,
  2. Configure your logging to go to stdout instead of stderr, or
  3. Replace input(prompt) with print(prompt, file=sys.stderr, end=''); input() in a reusable function.
Answered By: Alex Hall
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.