Pymongo: Cannot encode object of type decimal.Decimal?

Question

After trying to insert_one into a collection. I receive this error:

bson.errors.InvalidDocument: cannot encode object: Decimal('0.16020'), of type: <class 'decimal.Decimal'>

The code runs fine when the JSON does not include the decimal.Decimal object. If there is a solution can you kindly consider coding it in a recursive manner to made the whole of the python dictionary json_dic compatible to be inserted into MongoDB (as there is more than once instance of the class decimal.Decimal in the json.dic entries).

EDIT 1: Here is the JSON I am dealing with

import simplejson as json
from pymongo import MongoClient

json_string = '{"A" : {"B" : [{"C" : {"Horz" : 0.181665435,"Vert" : 0.178799435}}]}}'

json_dict = json.loads(json_string)
this_collection.insert_one(json_dict)

This produces
bson.errors.InvalidDocument: cannot encode object: Decimal('0.181665435'), of type: <class 'decimal.Decimal'>

EDIT 2: Unfortunately my example above simplified my existing JSON too much and the answer provided by @Belly Buster (despite working fine with the Json above) thows an error:

AttributeError: 'decimal.Decimal' object has no attribute 'items'

with my actual JSON, so I am providing the full JSON here hopefully to find out what is wrong (also as a screen-shot):

json_string = 
'
{
  "Setting" : {
    "GridOptions" : {
      "Student" : "HighSchool",
      "Lesson" : 1,
      "Attended" : true
    },
    "Grades" : [
      80,
      50.75
    ],
    "Count" : 2,
    "Check" : "Coursework",
    "Passed" : true
  },
  "Slides" : [
    {
      "Type" : "ABC",
      "Duration" : 1.5
    },
    {
      "Type" : "DEF",
      "Duration" : 0.5
    }
  ],
  "Work" : {
    "Class" : [
      {
        "Time" : 123456789,
        "Marks" : {
          "A" : 50,
          "B" : 100
        }
      }
    ],
    "CourseWorkDetail" : [
      {
        "Test" : {
          "Mark" : 0.987654321
        },
        "ReadingDate" : "Feb162006",
        "Reading" : 300.001,
        "Values" : [
          [
            0.98765
          ],
          [
            -0.98765
          ]
        ]
      },
      {
        "Test" : {
          "Mark" : 0.123456789
        },
        "ReadingDate" : "Jan052010",
        "Reading" : 200.005,
        "Values" : [
          [
            0.12345
          ],
          [
            -0.12345
          ]
        ]
      }
    ]
  },
  "listing" : 5
}
'

Edit 3:
Complimentary to the answer below, you can iterate recursively in a dictionary like this and use the function from the answer

def iterdict(dict_items, debug_out):
    for k, v in dict_items.items():
        if isinstance(v):
            iterdict(v)
        else:
            dict_items[k] = convert_decimal(v)
    return dict_items

Asked By: Mohammad

||

Source

Answer 1

Pymongo doesn’t recognize Decimal – that’s why you are getting the error.

The correct pymongo insert is coll.insert_one({"number1": Decimal128('8.916')}).

You’ll also need the import – from bson import Decimal128

Now, if you want to process your JSON file without changing Decimal to Decimal128`, you could modify the import statement.

from bson import Decimal128 as Decimal

coll.insert_one({"number1": Decimal('8.916')})

Answered By: DaveStSomeWhere

Answer 2

EDIT:

The convert_decimal() function will perform the conversion of Decimal to Decimal128 within a complex dict structure:

import simplejson as json
from pymongo import MongoClient
from decimal import Decimal
from bson.decimal128 import Decimal128

def convert_decimal(dict_item):
    # This function iterates a dictionary looking for types of Decimal and converts them to Decimal128
    # Embedded dictionaries and lists are called recursively.
    if dict_item is None: return None

    for k, v in list(dict_item.items()):
        if isinstance(v, dict):
            convert_decimal(v)
        elif isinstance(v, list):
            for l in v:
                convert_decimal(l)
        elif isinstance(v, Decimal):
            dict_item[k] = Decimal128(str(v))

    return dict_item

db = MongoClient()['mydatabase']
json_string = '{"A" : {"B" : [{"C" : {"Horz" : 0.181665435,"Vert" : 0.178799435}}]}}'
json_dict = json.loads(json_string, use_decimal=True)
db.this_collection.insert_one(convert_decimal(json_dict))
print(db.this_collection.find_one())

gives:

{'_id': ObjectId('5ea743aa297c9ccd52d33e05'), 'A': {'B': [{'C': {'Horz': Decimal128('0.181665435'), 'Vert': Decimal128('0.178799435')}}]}}

ORIGINAL:

To convert a decimal to a Decimal128 that MongoDB will be happy with, convert it to a string and then to a Decimal128. This snippet may help:

from pymongo import MongoClient
from decimal import Decimal
from bson.decimal128 import Decimal128

db = MongoClient()['mydatabase']
your_number = Decimal('234.56')
your_number_128 = Decimal128(str(your_number))
db.mycollection.insert_one({'Number': your_number_128})
print(db.mycollection.find_one())

gives:

{'_id': ObjectId('5ea6ec9b52619c7b39b851cb'), 'Number': Decimal128('234.56')}

Answered By: Belly Buster

Answer 3

from bson.decimal128 import Decimal128, create_decimal128_context
from decimal import localcontext

decimal128_ctx = create_decimal128_context()
with localcontext(decimal128_ctx) as ctx:
    horiz_val = Decimal128(ctx.create_decimal("0.181665435"))
    vert_val = Decimal128(ctx.create_decimal("0.178799435"))

doc = { 'A': { 'B': [ { 'C': { 'Horiz': horiz_val, 'Vert': vert_val } } ] } }
result = collection.insert_one(doc)
# result.inserted_id

pprint.pprint(list(collection.find()))

[ {'A': {'B': [{'C': {'Horiz': Decimal128('0.181665435'),
                      'Vert': Decimal128('0.178799435')}}]},
  '_id': ObjectId('5ea79adb915cbf3c46f5d4ae')} ]

NOTES:

PyMongo’s decimal128 – Support for BSON Decimal128
Python’s decimal
module
provides support for decimal floating point arithmetic.

From the PyMongo’s decimal128 documentation:

To ensure the result of a calculation can always be stored as BSON
Decimal128 use the context returned by create_decimal128_context() (NOTE: as shown in the example code above).

Answered By: prasad_

Answer 4

I would recommend just adding a codec to automatically convert the data types at insertion. If you recursively change the data types to use the Decimal128 object you might break compatibility with existing code.

You can follow the tutorial to create a simple decimal.Decimal codec in the pymongo docs here

Answered By: Ruan Steenkamp

Pymongo: Cannot encode object of type decimal.Decimal?

Question:

Answers: