ZODB In Real Life
Question:
Writing an app in Python, and been playing with various ORM setups and straight SQL. All of which are ugly as sin.
I have been looking at ZODB as an object store, and it looks a promising alternative… would you recommend it? What are your experiences, problems, and criticism, particularly regarding developer’s perspectives, scalability, integrity, long-term maintenance and alternatives? Anyone start a project with it and ditch it? Why?
Whilst the ideas behind ZODB, Pypersyst and others are interesting, there seems to be a lack of enthusiasm around for them 🙁
Answers:
ZODB has been used for plenty of large databases
Most ZODB usage is/was probably Zope users who migrated away if they migrate away from Zope
Performance is not so good as relatonal database+ORM especially if you have lots of writes.
Long term maintenance is not so bad, you want to pack the database from time to time, but that can be done live.
You have to use ZEO if you are going to use more than one process with your ZODB which is quite a lot slower than using ZODB directly
I have no idea how ZODB performs on flash disks.
With pickling you should be able to use any key value database in a similar fashion.
Compared to “any key-value store”, the key features for ZODB would be automatic integration of attribute changes with real ACID transactions, and clean, “arbitrary” references to other persistent objects.
The ZODB is bigger than just the FileStorage used by default in Zope:
- The RelStorage backend lets you put your data in an RDBMS which can be backed up, replicated, etc. using standard tools.
- ZEO allows easy scaling of appservers and off-line jobs.
- The two-phase commit support allows coordinating transactions among multiple databases, including RDBMSes (assuming that they provide a TPC-aware layer).
- Easy hierarchy based on object attributes or containment: you don’t need to write recursive self-joins to emulate it.
- Filesystem-based BLOB support makes serving large files trivial to implement.
Overall, I’m very happy using ZODB for nearly any problem where the shape of the data is not obviously “square”.
I’ve used ZODB for more than ten years now, in Zope and outside. It’s great if your data is hierarchical. The largest data store a customer operates has maybe. I don’t know, 100GB in it? Something on that order of magnitude anyway.
Here is a performance comparison against Postgres.
If you’re writing a WSGI web app, these packages may be useful:
I would recommend it.
I really don’t have any criticisms. If it’s an object store your looking for, this is the one to use. I’ve stored 2.5 million objects in it before and didn’t feel a pinch.
Writing an app in Python, and been playing with various ORM setups and straight SQL. All of which are ugly as sin.
I have been looking at ZODB as an object store, and it looks a promising alternative… would you recommend it? What are your experiences, problems, and criticism, particularly regarding developer’s perspectives, scalability, integrity, long-term maintenance and alternatives? Anyone start a project with it and ditch it? Why?
Whilst the ideas behind ZODB, Pypersyst and others are interesting, there seems to be a lack of enthusiasm around for them 🙁
ZODB has been used for plenty of large databases
Most ZODB usage is/was probably Zope users who migrated away if they migrate away from Zope
Performance is not so good as relatonal database+ORM especially if you have lots of writes.
Long term maintenance is not so bad, you want to pack the database from time to time, but that can be done live.
You have to use ZEO if you are going to use more than one process with your ZODB which is quite a lot slower than using ZODB directly
I have no idea how ZODB performs on flash disks.
With pickling you should be able to use any key value database in a similar fashion.
Compared to “any key-value store”, the key features for ZODB would be automatic integration of attribute changes with real ACID transactions, and clean, “arbitrary” references to other persistent objects.
The ZODB is bigger than just the FileStorage used by default in Zope:
- The RelStorage backend lets you put your data in an RDBMS which can be backed up, replicated, etc. using standard tools.
- ZEO allows easy scaling of appservers and off-line jobs.
- The two-phase commit support allows coordinating transactions among multiple databases, including RDBMSes (assuming that they provide a TPC-aware layer).
- Easy hierarchy based on object attributes or containment: you don’t need to write recursive self-joins to emulate it.
- Filesystem-based BLOB support makes serving large files trivial to implement.
Overall, I’m very happy using ZODB for nearly any problem where the shape of the data is not obviously “square”.
I’ve used ZODB for more than ten years now, in Zope and outside. It’s great if your data is hierarchical. The largest data store a customer operates has maybe. I don’t know, 100GB in it? Something on that order of magnitude anyway.
Here is a performance comparison against Postgres.
If you’re writing a WSGI web app, these packages may be useful:
I would recommend it.
I really don’t have any criticisms. If it’s an object store your looking for, this is the one to use. I’ve stored 2.5 million objects in it before and didn’t feel a pinch.