Suppress newline in Python logging module

Question

I’m trying to replace an ad-hoc logging system with Python’s logging module. I’m using the logging system to output progress information for a long task on a single line so you can tail the log or watch it in a console. I’ve done this by having a flag on my logging function which suppresses the newline for that log message and build the line piece by piece.

All the logging is done from a single thread so there’s no serialisation issues.

Is it possible to do this with Python’s logging module? Is it a good idea?

Asked By: Peter Graham

||

Source

Answer 1

Let’s start with your last question: No, I do not believe it’s a good idea.
IMO, it hurts the readability of the logfile in the long run.

I suggest sticking with the logging module and using the ‘-f’ option on your ‘tail’ command to watch the output from the console. You will probably end up using the FileHandler. Notice that the default argument for ‘delay’ is False meaning the output won’t be buffered.

If you really needed to suppress newlines, I would recommend creating your own Handler.

Answered By: jonEbird

Answer 2

The new line, n, is inserted inside the StreamHandler class.

If you’re really set on fixing this behaviour, then here’s an example of how I solved this by monkey patching the emit(self, record) method inside the logging.StreamHandler class.

A monkey patch is a way to extend or modify the run-time code of dynamic languages without altering the original source code. This process has also been termed duck punching.

Here is the custom implementation of emit() that omits line breaks:

def customEmit(self, record):
    # Monkey patch Emit function to avoid new lines between records
    try:
        msg = self.format(record)
        if not hasattr(types, "UnicodeType"): #if no unicode support...
            self.stream.write(msg)
        else:
            try:
                if getattr(self.stream, 'encoding', None) is not None:
                    self.stream.write(msg.encode(self.stream.encoding))
                else:
                    self.stream.write(msg)
            except UnicodeError:
                self.stream.write(msg.encode("UTF-8"))
        self.flush()
    except (KeyboardInterrupt, SystemExit):
        raise
    except:
        self.handleError(record)

Then you would make a custom logging class (in this case, subclassing from TimedRotatingFileHandler).

class SniffLogHandler(TimedRotatingFileHandler):
    def __init__(self, filename, when, interval, backupCount=0,
                 encoding=None, delay=0, utc=0):

        # Monkey patch 'emit' method
        setattr(StreamHandler, StreamHandler.emit.__name__, customEmit)

        TimedRotatingFileHandler.__init__(self, filename, when, interval,
                                          backupCount, encoding, delay, utc)

Some people might argue that this type of solution is not Pythonic, or whatever. It might be so, so be careful.

Also, be aware that this will globally patch SteamHandler.emit(...), so if you are using multiple logging classes, then this patch will affect the other logging classes as well!

Check out these for further reading:

Hope that helps.

Answered By: pjama

Answer 3

If you wanted to do this you can change the logging handler terminator. I’m using Python 3.4. This was introduced in Python 3.2 as stated by Ninjakannon.

handler = logging.StreamHandler()
handler.terminator = ""

When the StreamHandler writes it writes the terminator last.

Answered By: justengel

Answer 4

I encountered a need to log a certain section in a single line as I iterated through a tuple, but wanted to retain the overall logger.

Collected the output first into a single string and later sent it out to logger once I was out of the section. Example of concept

for fld in object._fields: 
  strX = (' {} --> {} ').format(fld, formattingFunction(getattr(obj,fld)))
debugLine += strX
logger.debug('{}'.format(debugLine))

Answered By: Rembau Times

Answer 5

Python 3.5.9

class MFileHandler(logging.FileHandler):
  """Handler that controls the writing of the newline character"""

  special_code = '[!n]'

  def emit(self, record) -> None:

    if self.special_code in record.msg:
      record.msg = record.msg.replace( self.special_code, '' )
      self.terminator = ''
    else:
      self.terminator = 'n'

    return super().emit(record)

Then

fHandler = MFileHandler(...)

Example:

# without n
log.info( 'waiting...[!n]' )
 ...
log.info( 'OK' )

# with n
log.info( 'waiting...' )
 ...
log.info( 'OK' )

log.txt:

waiting...OK
waiting...
OK

Answered By: kernelplv

Suppress newline in Python logging module

Question:

Answers: