Peewee ORM JSONField for MySQL

Question

I have a peewee model like so:

class User(peewee.Model):
    name = peewee.CharField(unique=True)
    some_json_data = peewee.CharField()
    requested_at = peewee.DateTimeField(default=datetime.now())

I know that peewee doesn’t support a JSONField for a MySQL DB, but anyway, I though if I could just convert it to a string format and save to db, I can retrieve it as is.

Let’s say, for example, this is my JSONField that I am writing to the DB:

[
  {
    'name': 'abcdef',
    'address': 'abcdef',
    'lat': 43176757,
    'lng': 42225601
  }
]

When I fetch this (JSONField) data, the output is like so:

u'[{u'name': u'abcdef',  u'address': u'abcdef', u'lat': 43176757, u'lng': 42225601'}]'

Trying a simplejson load of this is giving me an error like so:

JSONDecodeError: Expecting property name enclosed in double quotes:
line 1 column 3 (char 2)

I’ve tried json dumps of the json data before entering it to the DB and seeing if something would work, but still I have no luck with that.

I am looking for a solution that involves peewee’s custom field options and I want to stick my MySQL. Can someone guide me?

Asked By: Varun Verma

||

Source

Answer 1

What’s probably happening in your code is Peewee is calling str() (or unicode()) on the value instead of dumping it to JSON, so the Python string representation is being saved to the database. To do JSON manually, just import json and then call json.dumps(obj) when you’re setting the field and json.loads(db_value) when you fetch the field.

It looks like there’s a Peewee playhouse extension defined for certain databases (SQLite, PostgreSQL?) that defined a JSONField type — see JSONField docs here.

Alternatively, I don’t think it’d be hard to define a custom JSONField type which does the json loads/dumps automatically. There’s a simple example of this in playhouse/kv.py:

class JSONField(TextField):
    def db_value(self, value):
        return json.dumps(value)

    def python_value(self, value):
        if value is not None:
            return json.loads(value)

Answered By: Ben Hoyt

Answer 2

Why not using the JSONField field from Peewee’s playhouse?

from playhouse.sqlite_ext import *
db = SqliteExtDatabase(':memory:')
class KV(Model):
     key = TextField()
     value = JSONField()
     class Meta:
         database = db

KV.create_table()

It takes care of converting Python objects to JSON, and viceversa:

KV.create(key='a', value={'k1': 'v1'})
KV.get(KV.key == 'a').value # print {'k1': 'v1'}

You can query using the JSON keys:

KV.get(KV.value['k1'] == 'v1').key # print 'a'

You can easily update JSON keys:

KV.update(value=KV.value.update({'k2': 'v2', 'k3': 'v3'})).execute() # add keys to JSON

Answered By: coccoinomane

Peewee ORM JSONField for MySQL

Question:

Answers: