How to implement custom indentation when pretty-printing with the JSON module?

A bodge, but once you have the string from dumps(), you can perform a regular expression substitution on it, if you’re sure of the format of its contents. Something along the lines of:

s = json.dumps(data_structure, indent=2)
s = re.sub('s*{s*"(.)": (d+),s*"(.)": (d+)s*}(,?)s*', r'{"1":2,"3":4}5', s)

You could try:

• mark lists that shouldn’t be indented by replacing them with NoIndentList:

  class NoIndentList(list):
      pass
  

• override json.Encoder.default method to produce a non-indented string representation for NoIndentList.

  You could just cast it back to list and call json.dumps() without indent to get a single line

It seems the above approach doesn’t work for the json module:

import json
import sys

class NoIndent(object):
    def __init__(self, value):
        self.value = value

def default(o, encoder=json.JSONEncoder()):
    if isinstance(o, NoIndent):
        return json.dumps(o.value)
    return encoder.default(o)

L = [dict(x=x, y=y) for x in range(1) for y in range(2)]
obj = [NoIndent(L), L]
json.dump(obj, sys.stdout, default=default, indent=4)

It produces invalid output (the list is serialized as a string):

[
    "[{"y": 0, "x": 0}, {"y": 1, "x": 0}]", 
    [
        {
            "y": 0, 
            "x": 0
        }, 
        {
            "y": 1, 
            "x": 0
        }
    ]
]

If you can use yaml then the method works:

import sys
import yaml

class NoIndentList(list):
    pass

def noindent_list_presenter(dumper, data):
    return dumper.represent_sequence(u'tag:yaml.org,2002:seq', data,
                                     flow_style=True)
yaml.add_representer(NoIndentList, noindent_list_presenter)


obj = [
    [dict(x=x, y=y) for x in range(2) for y in range(1)],
    [dict(x=x, y=y) for x in range(1) for y in range(2)],
    ]
obj[0] = NoIndentList(obj[0])
yaml.dump(obj, stream=sys.stdout, indent=4)

It produces:

- [{x: 0, y: 0}, {x: 1, y: 0}]
-   - {x: 0, y: 0}
    - {x: 0, y: 1}

i.e., the first list is serialized using [] and all items are on one line, the second list uses one line per item.

This solution is not so elegant and generic as the others and you will not learn much from it but it’s quick and simple.

def custom_print(data_structure, indent):
    for key, value in data_structure.items():
        print "n%s%s:" % (' '*indent,str(key)),
        if isinstance(value, dict):
            custom_print(value, indent+1)
        else:
            print "%s" % (str(value)),

Usage and output:

>>> custom_print(data_structure,1)

 layer1:
  layer2:
   layer3_2: string
   layer3_1: [{'y': 7, 'x': 1}, {'y': 4, 'x': 0}, {'y': 3, 'x': 5}, {'y': 9, 'x': 6}]

The following solution seems to work correctly on Python 2.7.x. It uses a workaround taken from Custom JSON encoder in Python 2.7 to insert plain JavaScript code to avoid custom-encoded objects ending up as JSON strings in the output by using a UUID-based replacement scheme.

class NoIndent(object):
    def __init__(self, value):
        self.value = value


class NoIndentEncoder(json.JSONEncoder):
    def __init__(self, *args, **kwargs):
        super(NoIndentEncoder, self).__init__(*args, **kwargs)
        self.kwargs = dict(kwargs)
        del self.kwargs['indent']
        self._replacement_map = {}

    def default(self, o):
        if isinstance(o, NoIndent):
            key = uuid.uuid4().hex
            self._replacement_map[key] = json.dumps(o.value, **self.kwargs)
            return "@@%s@@" % (key,)
        else:
            return super(NoIndentEncoder, self).default(o)

    def encode(self, o):
        result = super(NoIndentEncoder, self).encode(o)
        for k, v in self._replacement_map.iteritems():
            result = result.replace('"@@%s@@"' % (k,), v)
        return result

Then this

obj = {
  "layer1": {
    "layer2": {
      "layer3_2": "string", 
      "layer3_1": NoIndent([{"y": 7, "x": 1}, {"y": 4, "x": 0}, {"y": 3, "x": 5}, {"y": 9, "x": 6}])
    }
  }
}
print json.dumps(obj, indent=2, cls=NoIndentEncoder)

produces the follwing output:

{
  "layer1": {
    "layer2": {
      "layer3_2": "string", 
      "layer3_1": [{"y": 7, "x": 1}, {"y": 4, "x": 0}, {"y": 3, "x": 5}, {"y": 9, "x": 6}]
    }
  }
}

---

It also correctly passes all options (except indent) e.g. sort_keys=True down to the nested json.dumps call.

obj = {
    "layer1": {
        "layer2": {
            "layer3_1": NoIndent([{"y": 7, "x": 1, }, {"y": 4, "x": 0}, {"y": 3, "x": 5, }, {"y": 9, "x": 6}]),
            "layer3_2": "string",
        }
    }
}    
print json.dumps(obj, indent=2, sort_keys=True, cls=NoIndentEncoder)

correctly outputs:

{
  "layer1": {
    "layer2": {
      "layer3_1": [{"x": 1, "y": 7}, {"x": 0, "y": 4}, {"x": 5, "y": 3}, {"x": 6, "y": 9}], 
      "layer3_2": "string"
    }
  }
}

---

It can also be combined with e.g. collections.OrderedDict:

obj = {
    "layer1": {
        "layer2": {
            "layer3_2": "string",
            "layer3_3": NoIndent(OrderedDict([("b", 1), ("a", 2)]))
        }
    }
}
print json.dumps(obj, indent=2, cls=NoIndentEncoder)

outputs:

{
  "layer1": {
    "layer2": {
      "layer3_3": {"b": 1, "a": 2}, 
      "layer3_2": "string"
    }
  }
}

UPDATE: In Python 3, there is no iteritems. You can replace encode with this:

def encode(self, o):
    result = super(NoIndentEncoder, self).encode(o)
    for k, v in iter(self._replacement_map.items()):
        result = result.replace('"@@%s@@"' % (k,), v)
    return result

Here’s a post-processing solution if you have too many different types of objects contributing to the JSON to attempt the JSONEncoder method and too many varying types to use a regex. This function collapses whitespace after a specified level, without needing to know the specifics of the data itself.

def collapse_json(text, indent=12):
    """Compacts a string of json data by collapsing whitespace after the
    specified indent level

    NOTE: will not produce correct results when indent level is not a multiple
    of the json indent level
    """
    initial = " " * indent
    out = []  # final json output
    sublevel = []  # accumulation list for sublevel entries
    pending = None  # holder for consecutive entries at exact indent level
    for line in text.splitlines():
        if line.startswith(initial):
            if line[indent] == " ":
                # found a line indented further than the indent level, so add
                # it to the sublevel list
                if pending:
                    # the first item in the sublevel will be the pending item
                    # that was the previous line in the json
                    sublevel.append(pending)
                    pending = None
                item = line.strip()
                sublevel.append(item)
                if item.endswith(","):
                    sublevel.append(" ")
            elif sublevel:
                # found a line at the exact indent level *and* we have sublevel
                # items. This means the sublevel items have come to an end
                sublevel.append(line.strip())
                out.append("".join(sublevel))
                sublevel = []
            else:
                # found a line at the exact indent level but no items indented
                # further, so possibly start a new sub-level
                if pending:
                    # if there is already a pending item, it means that
                    # consecutive entries in the json had the exact same
                    # indentation and that last pending item was not the start
                    # of a new sublevel.
                    out.append(pending)
                pending = line.rstrip()
        else:
            if pending:
                # it's possible that an item will be pending but not added to
                # the output yet, so make sure it's not forgotten.
                out.append(pending)
                pending = None
            if sublevel:
                out.append("".join(sublevel))
            out.append(line)
    return "n".join(out)

For example, using this structure as input to json.dumps with an indent level of 4:

text = json.dumps({"zero": ["first", {"second": 2, "third": 3, "fourth": 4, "items": [[1,2,3,4], [5,6,7,8], 9, 10, [11, [12, [13, [14, 15]]]]]}]}, indent=4)

here’s the output of the function at various indent levels:

>>> print collapse_json(text, indent=0)
{"zero": ["first", {"items": [[1, 2, 3, 4], [5, 6, 7, 8], 9, 10, [11, [12, [13, [14, 15]]]]], "second": 2, "fourth": 4, "third": 3}]}
>>> print collapse_json(text, indent=4)
{
    "zero": ["first", {"items": [[1, 2, 3, 4], [5, 6, 7, 8], 9, 10, [11, [12, [13, [14, 15]]]]], "second": 2, "fourth": 4, "third": 3}]
}
>>> print collapse_json(text, indent=8)
{
    "zero": [
        "first",
        {"items": [[1, 2, 3, 4], [5, 6, 7, 8], 9, 10, [11, [12, [13, [14, 15]]]]], "second": 2, "fourth": 4, "third": 3}
    ]
}
>>> print collapse_json(text, indent=12)
{
    "zero": [
        "first", 
        {
            "items": [[1, 2, 3, 4], [5, 6, 7, 8], 9, 10, [11, [12, [13, [14, 15]]]]],
            "second": 2,
            "fourth": 4,
            "third": 3
        }
    ]
}
>>> print collapse_json(text, indent=16)
{
    "zero": [
        "first", 
        {
            "items": [
                [1, 2, 3, 4],
                [5, 6, 7, 8],
                9,
                10,
                [11, [12, [13, [14, 15]]]]
            ], 
            "second": 2, 
            "fourth": 4, 
            "third": 3
        }
    ]
}

As a side note, this website has a built-in JavaScript that will avoid line feeds in JSON strings when lines are shorter than 70 chars:

http://www.csvjson.com/json_beautifier

(was implemented using a modified version of JSON-js)

Select “Inline short arrays”

Great for quickly viewing data that you have in the copy buffer.

This yields the OP’s expected result:

import json

class MyJSONEncoder(json.JSONEncoder):

  def iterencode(self, o, _one_shot=False):
    list_lvl = 0
    for s in super(MyJSONEncoder, self).iterencode(o, _one_shot=_one_shot):
      if s.startswith('['):
        list_lvl += 1
        s = s.replace('n', '').rstrip()
      elif 0 < list_lvl:
        s = s.replace('n', '').rstrip()
        if s and s[-1] == ',':
          s = s[:-1] + self.item_separator
        elif s and s[-1] == ':':
          s = s[:-1] + self.key_separator
      if s.endswith(']'):
        list_lvl -= 1
      yield s

o = {
  "layer1":{
    "layer2":{
      "layer3_1":[{"y":7,"x":1},{"y":4,"x":0},{"y":3,"x":5},{"y":9,"x":6}],
      "layer3_2":"string",
      "layer3_3":["aaanbbb","cccnddd",{"aaanbbb":"cccnddd"}],
      "layer3_4":"aaanbbb",
    }
  }
}

jsonstr = json.dumps(o, indent=2, separators=(',', ':'), sort_keys=True,
    cls=MyJSONEncoder)
print(jsonstr)
o2 = json.loads(jsonstr)
print('identical objects: {}'.format((o == o2)))

Indeed, one of things YAML is better than JSON.

I can’t get NoIndentEncoder to work…, but I can use regex on JSON string…

def collapse_json(text, list_length=5):
    for length in range(list_length):
        re_pattern = r'[' + (r's*(.+)s*,' * length)[:-1] + r']'
        re_repl = r'[' + ''.join(r'{}, '.format(i+1) for i in range(length))[:-2] + r']'

        text = re.sub(re_pattern, re_repl, text)

    return text

The question is, how do I perform this on a nested list?

Before:

[
  0,
  "any",
  [
    2,
    3
  ]
]

After:

[0, "any", [2, 3]]

Best performance code (10MB text costs 1s):

import json
def dumps_json(data, indent=2, depth=2):
    assert depth > 0
    space = ' '*indent
    s = json.dumps(data, indent=indent)
    lines = s.splitlines()
    N = len(lines)
    # determine which lines to be shortened
    is_over_depth_line = lambda i: i in range(N) and lines[i].startswith(space*(depth+1))
    is_open_bracket_line = lambda i: not is_over_depth_line(i) and is_over_depth_line(i+1)
    is_close_bracket_line = lambda i: not is_over_depth_line(i) and is_over_depth_line(i-1)
    # 
    def shorten_line(line_index):
        if not is_open_bracket_line(line_index):
            return lines[line_index]
        # shorten over-depth lines
        start = line_index
        end = start
        while not is_close_bracket_line(end):
            end += 1
        has_trailing_comma = lines[end][-1] == ','
        _lines = [lines[start][-1], *lines[start+1:end], lines[end].replace(',','')]
        d = json.dumps(json.loads(' '.join(_lines)))
        return lines[line_index][:-1] + d + (',' if has_trailing_comma else '')
    # 
    s = 'n'.join([
        shorten_line(i)
        for i in range(N) if not is_over_depth_line(i) and not is_close_bracket_line(i)
    ])
    #
    return s

UPDATE:
Here’s my explanation:

First we use json.dumps to get json string has been indented.
Example:

>>>  print(json.dumps({'0':{'1a':{'2a':None,'2b':None},'1b':{'2':None}}}, indent=2))
[0]  {
[1]    "0": {
[2]      "1a": {
[3]        "2a": null,
[4]        "2b": null
[5]      },
[6]      "1b": {
[7]        "2": null
[8]      }
[9]    }
[10] }

If we set indent=2 and depth = 2, then too depth lines start with 6 white-spaces

We has 4 types of line:

1. Normal line
2. Open bracket line (2,6)
3. Exceed depth line (3,4,7)
4. Close bracket line (5,8)

We will try to merge a sequence of lines (type 2 + 3 + 4) into one single line.
Example:

[2]      "1a": {
[3]        "2a": null,
[4]        "2b": null
[5]      },

will be merged into:

[2]      "1a": {"2a": null, "2b": null},

NOTE: Close bracket line may has trailing comma

Answer for me and Python 3 users

import re

def jsonIndentLimit(jsonString, indent, limit):
    regexPattern = re.compile(f'n({indent}){{{limit}}}(({indent})+|(?=(}}|])))')
    return regexPattern.sub('', jsonString)

if __name__ == '__main__':
    jsonString = '''{
  "layer1": {
    "layer2": {
      "layer3_1": [
        {
          "x": 1,
          "y": 7
        },
        {
          "x": 0,
          "y": 4
        },
        {
          "x": 5,
          "y": 3
        },
        {
          "x": 6,
          "y": 9
        }
      ],
      "layer3_2": "string"
    }
  }
}'''
    print(jsonIndentLimit(jsonString, '  ', 3))

'''print
{
  "layer1": {
    "layer2": {
      "layer3_1": [{"x": 1,"y": 7},{"x": 0,"y": 4},{"x": 5,"y": 3},{"x": 6,"y": 9}],
      "layer3_2": "string"
    }
  }
}'''