How to limit the number of float digits JSONEncoder produces?

Here’s something that you may be able to use that’s based on my answer to the question:

    Write two-dimensional list to JSON file.

I say may because it requires “wrapping” all the float values in the Python dictionary (or list) before JSON encoding it with dump().

(Tested with Python 3.7.2.)

from _ctypes import PyObj_FromPtr
import json
import re


class FloatWrapper(object):
    """ Float value wrapper. """
    def __init__(self, value):
        self.value = value


class MyEncoder(json.JSONEncoder):
    FORMAT_SPEC = '@@{}@@'
    regex = re.compile(FORMAT_SPEC.format(r'(d+)'))  # regex: r'@@(d+)@@'

    def default(self, obj):
        return (self.FORMAT_SPEC.format(id(obj)) if isinstance(obj, FloatWrapper)
                else super(MyEncoder, self).default(obj))

    def iterencode(self, obj, **kwargs):
        for encoded in super(MyEncoder, self).iterencode(obj, **kwargs):
            # Check for marked-up float values (FloatWrapper instances).
            match = self.regex.search(encoded)
            if match:  # Get FloatWrapper instance.
                id = int(match.group(1))
                float_wrapper = PyObj_FromPtr(id)
                json_obj_repr = '%.7f' % float_wrapper.value  # Create alt repr.
                encoded = encoded.replace(
                            '"{}"'.format(self.FORMAT_SPEC.format(id)), json_obj_repr)
            yield encoded


d = dict()
d['val'] = FloatWrapper(5.78686876876089075543)  # Must wrap float values.
d['name'] = 'kjbkjbkj'

with open('float_test.json', 'w') as file:
    json.dump(d, file, cls=MyEncoder, indent=4)

Contents of file created:

{
    "val": 5.7868688,
    "name": "kjbkjbkj"
}

Update:

As I mentioned, the above requires all the float values to be wrapped before calling json.dump(). Fortunately doing that could be automated by adding and using the following (minimally tested) utility:

def wrap_type(obj, kind, wrapper):
    """ Recursively wrap instances of type kind in dictionary and list
        objects.
    """
    if isinstance(obj, dict):
        new_dict = {}
        for key, value in obj.items():
            if not isinstance(value, (dict, list)):
                new_dict[key] = wrapper(value) if isinstance(value, kind) else value
            else:
                new_dict[key] = wrap_type(value, kind, wrapper)
        return new_dict

    elif isinstance(obj, list):
        new_list = []
        for value in obj:
            if not isinstance(value, (dict, list)):
                new_list.append(wrapper(value) if isinstance(value, kind) else value)
            else:
                new_list.append(wrap_type(value, kind, wrapper))
        return new_list

    else:
        return obj


d = dict()
d['val'] = 5.78686876876089075543
d['name'] = 'kjbkjbkj'

with open('float_test.json', 'w') as file:
    json.dump(wrap_type(d, float, FloatWrapper), file, cls=MyEncoder, indent=4)

Doesn’t answer this question, but for the decoding side, you could do something like this, or override the hook method.

To solve this problem with this method though would require encoding, decoding, then encoding again, which is overly convoluted and no longer the best choice. I assumed Encode had all the bells and whistles Decode did, my mistake.

# d = dict()
class Round7FloatEncoder(json.JSONEncoder): 
    def iterencode(self, obj): 
        if isinstance(obj, float): 
            yield format(obj, '.7f')


with open('test.json', 'w') as f:
    json.dump(d, f, cls=Round7FloatEncoder)

It is still possible to monkey-patch json in Python 3, but instead of FLOAT_REPR, you need to modify float. Make sure to disable c_make_encoder just like in Python 2.

import json

class RoundingFloat(float):
    __repr__ = staticmethod(lambda x: format(x, '.2f'))

json.encoder.c_make_encoder = None
if hasattr(json.encoder, 'FLOAT_REPR'):
    # Python 2
    json.encoder.FLOAT_REPR = RoundingFloat.__repr__
else:
    # Python 3
    json.encoder.float = RoundingFloat

print(json.dumps({'number': 1.0 / 81}))

Upsides: simplicity, can do other formatting (e.g. scientific notation, strip trailing zeroes etc). Downside: it looks more dangerous than it is.

Here is a python code snippet that shows how to quantize json output to the specified number of digits:

#python example code, error handling not shown

#open files
fin  = open(input_file_name)
fout = open(output_file_name, "w+")

#read file input (note this could be done in one step but breaking it up allows more flexibilty )
indata = fin.read()

# example quantization function
def quant(n):
    return round((float(n) * (10 ** args.prec))) / (
        10 ** args.prec
    )  # could use decimal.quantize

# process the data streams by parsing and using call back to quantize each float as it parsed
outdata = json.dumps(json.loads(indata, parse_float=quant), separators=(",", ":"))

#write output
fout.write(outdata)

The above is what the jsonvice command-line tool uses to quantize the floating-point json numbers to whatever precision is desired to save space.

https://pypi.org/project/jsonvice/

This can be installed with pip or pipx (see docs).

pip3 install jsonvice

Disclaimer: I wrote this when needing to test quantized machine learning model weights.

I found the above options within the python standard library to be very limiting and cumbersome, so if you’re not strictly limited to the python standard lib, pandas has a json module that includes a dumps method which has a double_precision parameter to control the number of digits in a float (default 10):

import json
import pandas.io.json

d = {
  'val': 5.78686876876089075543,
  'name': 'kjbkjbkj',
}

print(json.dumps(d))
print(pandas.io.json.dumps(d))
print(pandas.io.json.dumps(d, double_precision=5))

gives:

{"val": 5.786868768760891, "name": "kjbkjbkj"}
{"val":5.7868687688,"name":"kjbkjbkj"}
{"val":5.78687,"name":"kjbkjbkj"}

Inspired by this answer, here is a solution that works for Python >= 3.6 (tested with 3.9) and that allows customization of the format on a case by case basis. It works for both json and simplejson (tested with json=2.0.9 and simplejson=3.17.6).

Note however that this is not thread-safe.

from contextlib import contextmanager


class FormattedFloat(float):
    def __new__(self, value, fmt=None):
        return float.__new__(self, value)

    def __init__(self, value, fmt=None):
        float.__init__(value)
        if fmt:
            self.fmt = fmt
    
    def __repr__(self):
        if hasattr(self, 'fmt'):
            return f'{self:{self.fmt}}'
        return float.__repr__(self)


@contextmanager
def formatted_floats():
    c_make_encoder = json.encoder.c_make_encoder
    json_float = json.encoder.float
    json.encoder.c_make_encoder = None
    json.encoder.float = FormattedFloat
    try:
        yield
    finally:
        json.encoder.c_make_encoder = c_make_encoder
        json.encoder.float = json_float

Example

x = 12345.6789
d = dict(
    a=x,
    b=FormattedFloat(x),
    c=FormattedFloat(x, '.4g'),
    d=FormattedFloat(x, '.08f'),
)

>>> d
{'a': 12345.6789, 'b': 12345.6789, 'c': 1.235e+04, 'd': 12345.67890000}

Now,

with formatted_floats():
    out = json.dumps(d)

>>> out
'{"a": 12345.6789, "b": 12345.6789, "c": 1.235e+04, "d": 12345.67890000}'

>>> json.loads(out)
{'a': 12345.6789, 'b': 12345.6789, 'c': 12350.0, 'd': 12345.6789}

Note that the original json.encoder attributes are restored by the context manager, so:

>>> json.dumps(d)
'{"a": 12345.6789, "b": 12345.6789, "c": 12345.6789, "d": 12345.6789}'