Building a REST API using a Python based open source framework and NoSQL backend
Question:
I would like to build a REST API using a NoSQL backend with a Python based open source framework to build the API. This API would run in a cloud environment and the goal is for it to be cloud agnostic and have the ability to be deployed anywhere. It must have some abstraction for the backend database technology. I found that Django REST Framework is exactly what I’m looking for, but the Django ORM only supports RDBMS. In an attempt to enable NoSQL support with Django, it seems a few open source packages have been developed but those projects have been abandoned.
I know it’s technically possible to use Amazon DynamoDB or Azure Cosmos DB with Django REST Framework, but as it’s not officially supported, it sounds like it would require custom code and deviating from standard configurations to get it to work.
- Does anyone have an API running for Production use using a NoSQL backend with Django REST framework?
- With Django REST framework, is it possible to abstract the backend database connections to support different NoSQL database types?
- Is a framework like Flask better suited for creating a REST API using this type of backend?
- Are there other REST frameworks available which may provide the functionality required if Django REST framework cannot meet these requirements?
Answers:
I would go with flask + mongodb
It is possible to run Django + MongoDB but it is not a good idea.
The main reason is that Django does not have an official support for Mongodb. There is a package called Djongo in which is very unreliable and unfinished that allows you to use Django’s ORM but with a lot of flaws. You can check the number of issues into this github repository.
Conversely, you can use PyMongo to connect django with mongodb but bear in mind that you will have to do the queries rawly.
In addition, in terms of dealing with different databases in django, I had done such thing but with Postgres and mongodb where you will have to create a Router class to manage the connections.
In my opinion I would go using Flask and have a look also into FastAPI that is almost similar to Flask but with a better approach to create APIs obviously.
So, Django doesn’t support abstraction for NOSQL databases. In fact, it has an internal Object Relation Mapper (ORM) library to interpret objects in real code an SQL database.
What you would need is an Object Data Mapper (ODM), as NOSQL databases are composed of raw data, like MongoDB which is document based.
In my experience, a few months ago I used this python library (https://github.com/MatteoGuadrini/nosqlapi – https://nosqlapi.readthedocs.io/en/latest/) to build my library with a certain abstraction precisely for working with databases in the cloud.
In your case, you should build your module based on document databases, such as MongoDB or CouchDB and then deploy it to your cloud accounts.
Here you can find an example (test) of a document database based on MongoDB: https://github.com/MatteoGuadrini/nosqlapi/blob/main/tests/test_docdb.py
If you follow me, you start creating these classes:
import nosqlapi.docdb
# MongoDB like database
class MongoConnection(nosqlapi.docdb.DocConnection):...
class MongoSession(nosqlapi.docdb.DocSession):...
class MongoResponse(nosqlapi.docdb.DocResponse):...
class MongoBatch(nosqlapi.docdb.DocBatch):...
class MongoSelector(nosqlapi.docdb.DocSelector):...
so that you can then use the databases you want:
# Use MongoDB library
conn = MongoConnection(host='server.local', username='admin', password='pass')
sess = conn.connect() # return MongoSession object
# Create a new database
conn.create_database('new_db')
# CRUD operation
C = sess.insert(path='db/doc1', doc={"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 42}) # Create
R = sess.get(path='db/doc1') # Read
U = sess.update(path='db/doc1', doc={"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 43}, rev=2) # Update
D = sess.delete(path='db/doc1', rev=2) # Delete
print(R) # {"_id": "5099803df3f4948bd2f98391", "rev"= 2, "name": "Arthur", "age": 42}
print(type(R)) # <class 'MongoResponse'>
print(isinstance(R, nosqlapi.Response)) # True
# Extended CRUD operations
sess.insert_many(database='db', docs=[{"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 42},
{"_id": "5099803df3f4948bd2f98392", "name": "Arthur", "age": 43}])
sess.update_many(database='db', docs=[{"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 42, "rev": 2},
{"_id": "5099803df3f4948bd2f98392", "name": "Arthur", "age": 43, "rev": 2}])
# Complex select operation
sel = MongoSelector(selector={"name": "Arthur"}, fields=['_id', 'name', 'age'], limit=2)
sess.find(sel)
Also, the library has a nice set of ODM objects to map your code to the database: https://nosqlapi.readthedocs.io/en/latest/build.html#odm-classes
such as Mongo documents and PermissionDocuments …
In my opinion it is the best way.
Addition: furthermore, by searching on google, I saw that discussions are underway to make this library a real standard: https://discuss.python.org/t/request-for-comment-making-pep-for-nosql-databases/14772
I would like to build a REST API using a NoSQL backend with a Python based open source framework to build the API. This API would run in a cloud environment and the goal is for it to be cloud agnostic and have the ability to be deployed anywhere. It must have some abstraction for the backend database technology. I found that Django REST Framework is exactly what I’m looking for, but the Django ORM only supports RDBMS. In an attempt to enable NoSQL support with Django, it seems a few open source packages have been developed but those projects have been abandoned.
I know it’s technically possible to use Amazon DynamoDB or Azure Cosmos DB with Django REST Framework, but as it’s not officially supported, it sounds like it would require custom code and deviating from standard configurations to get it to work.
- Does anyone have an API running for Production use using a NoSQL backend with Django REST framework?
- With Django REST framework, is it possible to abstract the backend database connections to support different NoSQL database types?
- Is a framework like Flask better suited for creating a REST API using this type of backend?
- Are there other REST frameworks available which may provide the functionality required if Django REST framework cannot meet these requirements?
I would go with flask + mongodb
It is possible to run Django + MongoDB but it is not a good idea.
The main reason is that Django does not have an official support for Mongodb. There is a package called Djongo in which is very unreliable and unfinished that allows you to use Django’s ORM but with a lot of flaws. You can check the number of issues into this github repository.
Conversely, you can use PyMongo to connect django with mongodb but bear in mind that you will have to do the queries rawly.
In addition, in terms of dealing with different databases in django, I had done such thing but with Postgres and mongodb where you will have to create a Router class to manage the connections.
In my opinion I would go using Flask and have a look also into FastAPI that is almost similar to Flask but with a better approach to create APIs obviously.
So, Django doesn’t support abstraction for NOSQL databases. In fact, it has an internal Object Relation Mapper (ORM) library to interpret objects in real code an SQL database.
What you would need is an Object Data Mapper (ODM), as NOSQL databases are composed of raw data, like MongoDB which is document based.
In my experience, a few months ago I used this python library (https://github.com/MatteoGuadrini/nosqlapi – https://nosqlapi.readthedocs.io/en/latest/) to build my library with a certain abstraction precisely for working with databases in the cloud.
In your case, you should build your module based on document databases, such as MongoDB or CouchDB and then deploy it to your cloud accounts.
Here you can find an example (test) of a document database based on MongoDB: https://github.com/MatteoGuadrini/nosqlapi/blob/main/tests/test_docdb.py
If you follow me, you start creating these classes:
import nosqlapi.docdb
# MongoDB like database
class MongoConnection(nosqlapi.docdb.DocConnection):...
class MongoSession(nosqlapi.docdb.DocSession):...
class MongoResponse(nosqlapi.docdb.DocResponse):...
class MongoBatch(nosqlapi.docdb.DocBatch):...
class MongoSelector(nosqlapi.docdb.DocSelector):...
so that you can then use the databases you want:
# Use MongoDB library
conn = MongoConnection(host='server.local', username='admin', password='pass')
sess = conn.connect() # return MongoSession object
# Create a new database
conn.create_database('new_db')
# CRUD operation
C = sess.insert(path='db/doc1', doc={"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 42}) # Create
R = sess.get(path='db/doc1') # Read
U = sess.update(path='db/doc1', doc={"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 43}, rev=2) # Update
D = sess.delete(path='db/doc1', rev=2) # Delete
print(R) # {"_id": "5099803df3f4948bd2f98391", "rev"= 2, "name": "Arthur", "age": 42}
print(type(R)) # <class 'MongoResponse'>
print(isinstance(R, nosqlapi.Response)) # True
# Extended CRUD operations
sess.insert_many(database='db', docs=[{"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 42},
{"_id": "5099803df3f4948bd2f98392", "name": "Arthur", "age": 43}])
sess.update_many(database='db', docs=[{"_id": "5099803df3f4948bd2f98391", "name": "Arthur", "age": 42, "rev": 2},
{"_id": "5099803df3f4948bd2f98392", "name": "Arthur", "age": 43, "rev": 2}])
# Complex select operation
sel = MongoSelector(selector={"name": "Arthur"}, fields=['_id', 'name', 'age'], limit=2)
sess.find(sel)
Also, the library has a nice set of ODM objects to map your code to the database: https://nosqlapi.readthedocs.io/en/latest/build.html#odm-classes
such as Mongo documents and PermissionDocuments …
In my opinion it is the best way.
Addition: furthermore, by searching on google, I saw that discussions are underway to make this library a real standard: https://discuss.python.org/t/request-for-comment-making-pep-for-nosql-databases/14772