python logging – how do I truncate the pathname to just the last few characters or just the filename?

Question

I’m using python logging and I have a formatter that looks like the following:

formatter = logging.Formatter(
    '%(asctime)s - %(pathname)86s - %(lineno)4s - %(message)s', '%d %H:%M'
    )

As you can see, I like the information in my log files to line up neatly in columns. The reason I have 86 spaces reserved for the pathname is because the full paths to some of the files used in my program are that long. However, all I really need is the actual file name, not the full path. How can I get the logging module to give me just the file name? Better yet, since I have some long filenames, I’d like the first 3 characters of the filename, followed by ‘~’, then the last 16 characters. So

/Users/Jon/important_dir/dev/my_project/latest/testing-tools/test_read_only_scenarios_happily.py

should become

tes~arios_happily.py

Asked By: tadasajon

||

Source

Answer 1

Like this:

import os

def shorten_filename(filename):
    f = os.path.split(filename)[1]
    return "%s~%s" % (f[:3], f[-16:]) if len(f) > 19 else f

Answered By: David Robinson

Answer 2

You’ll have to implement your own Formatter subclass that truncates the path for you; the formatting string cannot do this.

Since you want to use just the filename you’d want to use the filename parameter as the base, which we can then truncate as needed.

For the size-limited version, I’d actually use a new record attribute for, leaving the existing filename to be the full name. Lets call it filename20.

import logging

class FilenameTruncatingFormatter(logging.Formatter):
    def format(self, record):
        # truncate the filename if it is longer than 20 characters
        filename = record.filename
        if len(filename) > 20:
            filename = '{}~{}'.format(filename[:3], filename[-16:])
        record.filename20 = filename
        return super(PathTruncatingFormatter, self).format(record)

Use this class instead of the normal logging.Formatter instance, and reference the filename20 field in the formatting string:

formatter = FilenameTruncatingFormatter(
    '%(asctime)s - %(filename20)20s - %(lineno)4s - %(message)s', '%d %H:%M'
)

Note that I also shortened the space needed for the field to 20 characters: %(...)20s.

If you have to specify the formatter in a ConfigParser-style configuration file, create a named formatter via a [formatter_NAME] section, and give that section a class key:

[formatter_truncated]
class=some.module.FilenameTruncatingFormatter
format=%(asctime)s - %(filename20)20s - %(lineno)4s - %(message)s
datefmt=%d %H:%M

and then you can use formatter=truncated in any handler sections you define.

For the dictionary-schema style configuration, the same applies: specify the formatter class via the class key in any formatter definition.

Answered By: Martijn Pieters

Answer 3

The following is the Python 3 version of the same which @Martijn had written. As @Nick pointed out, the record parameter passed is a LogRecord object in Python 3.

import logging
import os

class PathTruncatingFormatter(logging.Formatter):
    def format(self, record):
        if 'pathname' in record.__dict__.keys():
            # truncate the pathname
            filename = os.path.basename(record.pathname)
            if len(filename) > 20:
                filename = '{}~{}'.format(filename[:3], filename[-16:])
            record.pathname = filename
        return super(PathTruncatingFormatter, self).format(record)

Answered By: Sriram Ragunathan

Answer 4

You can simply use filename instead of pathname. Refer https://docs.python.org/3.1/library/logging.html#formatter-objects for more details

Answered By: Kamesh Jungi

Answer 5

Python 3: Same but different, but thought I would share, I simply wanted the last two things in the path, The directory and the file name.

class MyFormatter(logging.Formatter):
    def format(self, record):
        if 'pathname' in record.__dict__.keys():
            sections = record.pathname.split("/")
            sec_len = len(sections)
            record.pathname = "{}/{}".format(sections[sec_len - 2], sections[sec_len - 1])
        return super(MyFormatter, self).format(record)

Answered By: Glen Bizeau

Answer 6

For python 3.8 and above :

Same than response from @Glen Bizeau but in a more compact and tuneable way, allowing to set the number of maximum parent folders displayed :

class FileFormatter(logging.Formatter):
    MAX_FOLDERS = 4

    def format(self, record):
        if 'pathname' in record.__dict__.keys():
            if (sec_len := len(sections := record.pathname.split(os.path.sep))) > self.MAX_FOLDERS :
                record.pathname = "(...)" + os.path.sep.join([sections[sec_len - i] for i in reversed(range(1,self.MAX_FOLDERS+1))])
        return super().format(record)

Yields: (...)__packages__Inflowiosload.py
instead of: C:UsersMyNameDocumentsPython__packages__Inflowiosload.py

Answered By: Osamoele

python logging – how do I truncate the pathname to just the last few characters or just the filename?

Question:

Answers:

For python 3.8 and above :