Why does a json file behave like this?

Question

There is a function (rui) that takes the number num and returns the tenth power of num. Task: to write a decorator that will cache the outputs from this function so that if a num that has already been submitted is fed to the input, the decorator will output the value from the hash table, so as not to perform the function itself. If the number is fed to the function (rui) for the first time, then the decorator will simply add num and rui(num) to the hash table. I store cache data in a json file.

import json

def decor(func):
    def wraps(num):

        with open('inf.json') as json_file:
            dictionary = json.load(json_file)

        if num not in dictionary:
            with open('inf.json', 'w') as json_file:
                dictionary.update({num : func(num)})
                json.dump(dictionary, json_file)
        return dictionary[num]
    return wraps




@decor
def rui(num):
    return num**10

Problem:
The json file is now empty. Execute:

rui(10)
rui(11)
rui(10)

According to the idea, the json file should now be like this:

{"10" : 10000000000, "11" : 25937424601}

But the json file is like this:

{"10": 10000000000, "11": 25937424601, "10": 10000000000}

Execute again:

rui(11)
rui(10)
rui(10)
rui(15)
rui(15)

json file:

{"10": 10000000000, "11": 25937424601, "15": 576650390625, "15": 576650390625}

Asked By: xSAMx

||

Source

Answer 1

OK, here’s the problem: when you serialize to JSON, the keys must be strings. However, your inputs are int objects. I had never noticed this before, but something like {'10': 'x', 10:'y'} will get serialized to'{"10": "x", "10": "y"}'!

>>> import json
>>> json.dumps({'10': 'x', 10:'y'})
'{"10": "x", "10": "y"}'

To me, this just seems like aberrant behavior, but it seems to be a consequence of the fact that Python allows any hashable objects as a key, but JSON only accepts strings, and the JSON implementation just naively converts to string whatever key is there.

My suggestion is to simply use another serialization format, e.g. pickle, or alternatively, so some extra work around serialization:

import json

def decor(func):
    def wraps(num):

        with open('inf.json') as json_file:
            dictionary = dict(json.load(json_file))

        if num not in dictionary:
            with open('inf.json', 'w') as json_file:
                dictionary[num] = func(num)
                json.dump(list(dictionary.items()), json_file)
        return dictionary[num]
    return wraps

As an aside, you may want to reconsider whether the deserialization/serialization should happen on each decorated function call, or rather, only once.

Answered By: juanpa.arrivillaga

Answer 2

Simple, try as this, e.g. a list/dict comp to convert keys of the int to a str type:

>>> import json
>>> obj = {"10": 10000000000, "11": 25937424601, 10: 10000000000}
>>> json.dumps(obj)
'{"10": 10000000000, "11": 25937424601, "10": 10000000000}'
>>> # oh no, YIKES!
>>> obj = {str(k): v for k: v in obj.items()}
  File "<stdin>", line 1
    obj = {str(k): v for k: v in obj.items()} 
                          ^
SyntaxError: invalid syntax
>>> obj = {str(k): v for k, v in obj.items()}
>>> json.dumps(obj)
'{"10": 10000000000, "11": 25937424601}'
>>> # oh, SNAP!

For the data that is to be deeply nested, use a recursive function would be a good as the below shows it:

import json

def dump_it_plis(o):
    return json.dumps(inner_dump_it(o))
def inner_dump_it(o):
    return {str(k): inner_dump_it(v) if isinstance(v, dict)
    else [inner_dump_it(e) for e in v] if isinstance(v, list)
    else v
     for k, v in o.items()}

obj = {"lst": [{"10": 10000000000, "11": 25937424601, 10: 10000000000}]}
print(json.dumps(obj))
# this is print:
#  {"lst": [{"10": 10000000000, "11": 25937424601, "10": 10000000000}]}
print(dump_it_plis(obj))
# this is print:
#  {"lst": [{"10": 10000000000, "11": 25937424601}]}

Answered By: rv.kvetch

Why does a json file behave like this?

Question:

Answers: