Implementation of Repository Pattern in Python?

Question:

Mainly out of curiosity, I’m looking for a Python framework or example for the Repository Pattern of decoupling persistence logic from domain logic.

The name “Repository Pattern” appears in the post “Untangle Domain and Persistence Logic with Curator” (Ruby), idea comes from a section of the “Domain-Driven Design” book and Martin Fowler. The model class contains no persistence logic, rather the app declares repository subclasses whose instances act like in-memory collections of model instances. Each repository persists the model in different ways, for example to SQL (various schema conventions), to Riak or other noSQL and to memory (for caching). Framework conventions mean repository subclasses typically require minimal code: just declaring “WidgetRepository” subclass of SQLRepository would provide a collection that persists the model Widget to the DB table named “widgets” and match columns to Widget attributes.

Differences from other patterns:

Active Record Pattern: for example, Django ORM. The application defines just the model class with domain logic and some metadata for persistence. The ORM adds persistence logic to the model class. This mixes domain and persistence in one class (undesirable according to the post).

Thanks to @marcin I see that when Active Record supports diverse backends and .save(using=”other_database”) function, that gives the multi-backend benefit of the Repository Pattern.

So in a sense Repository Pattern is just like Active Record with the persistence logic moved to a separate class.

Data Mapper Pattern: for example, SQLAlchemy’s Classical Mappings. The app defines additional classes for database table(s), and data mapper(s) from model to table(s). Thus model instance can be mapped to tables in multiple ways e.g. to support legacy schemas. Don’t think SQLAlchemy provides mappers to non-SQL storage.

Asked By: Graham

||

Answers:

You might want to have a good look at James Dennis’ DictShield project

“DictShield is a database-agnostic modeling system. It provides a way to model, validate and reshape data easily. All without requiring any particular database.”

Answered By: Jordan Dimov

Out of my head:

I define two example domains, User and Animal, an base storage class Store and two specialized Storage classes UserStore and AnimalStore. Use of context manager closes db connection (for simplicity I use sqlite in this example):

import sqlite3

def get_connection():
    return sqlite3.connect('test.sqlite')

class StoreException(Exception):
    def __init__(self, message, *errors):
        Exception.__init__(self, message)
        self.errors = errors


# domains

class User():
    def __init__(self, name):
        self.name = name


class Animal():
    def __init__(self, name):
        self.name = name


# base store class
class Store():
    def __init__(self):
        try:
            self.conn = get_connection()
        except Exception as e:
            raise StoreException(*e.args, **e.kwargs)
        self._complete = False

    def __enter__(self):
        return self

    def __exit__(self, type_, value, traceback):
        # can test for type and handle different situations
        self.close()

    def complete(self):
        self._complete = True

    def close(self):
        if self.conn:
            try:
                if self._complete:
                    self.conn.commit()
                else:
                    self.conn.rollback()
            except Exception as e:
                raise StoreException(*e.args)
            finally:
                try:
                    self.conn.close()
                except Exception as e:
                    raise StoreException(*e.args)


# store for User obects
class UserStore(Store):

    def add_user(self, user):
        try:
            c = self.conn.cursor()
            # this needs an appropriate table
            c.execute('INSERT INTO user (name) VALUES(?)', (user.name,))
        except Exception as e:
            raise StoreException('error storing user')


# store for Animal obects
class AnimalStore(Store):

    def add_animal(self, animal):
        try:
            c = self.conn.cursor()
            # this needs an appropriate table
            c.execute('INSERT INTO animal (name) VALUES(?)', (animal.name,))
        except Exception as e:
            raise StoreException('error storing animal')

# do something
try:
    with UserStore() as user_store:
        user_store.add_user(User('John'))
        user_store.complete()

    with AnimalStore() as animal_store:
        animal_store.add_animal(Animal('Dog'))
        animal_store.add_animal(Animal('Pig'))
        animal_store.add_animal(Animal('Cat'))
        animal_store.add_animal(Animal('Wolf'))
        animal_store.complete()
except StoreException as e:
    # exception handling here
    print(e)
Answered By: laurasia

I have written a python repository implementation using SqlAlchemy as the backend. I was looking for one, and couldn’t find one, so I decided to make my own.

But I think there is one useful feature of a repository which you only touched on, particularly in the context of DDD, and with respect to the Active Record pattern, which is that the Repository pattern enables the model’s interface to be couched in domain language, which means you can talk about it without thinking or knowing about the implementation. The Repository pattern helps keep the model’s interface aligned with how the concepts are actually used by domain experts.

Let’s say your model is a Car. Now, Cars can probably drive(), they can probably steer(), and so on. What they probably can’t do, is save(). save(), and the concept of saving, are things that belong in a different abstraction to do with persistence.

It may seem like a small thing, but it can be really useful to keep the model’s interface well aligned with the domain language, because it means you can have easy and clear conversations with clients, without worrying about the implementation details.

Answered By: Obol

I have been working on a repository pattern library. It is still under development but as of the time of writing, it currently supports In-memory lists (useful for unit tests), SQL databases (via SQLAlchemy), MongoDB and HTTP(s) REST APIs. It’s actually quite handy for things where simple CRUD operations are enough. It is also quite useful for plugging in APIs.

Installation

pip install redbird

Some initiation examples

Repository in temporal memory

from redbird.repos import MemoryRepo
repo = MemoryRepo()

Repository for SQL

from sqlalchemy import create_engine
from redbird.repos import SQLRepo

engine = create_engine("sqlite://")
repo = SQLRepo.from_engine(engine=engine, table="my_table")

Repository for MongoDB

from redbird.repos import MongoRepo

repo = MongoRepo.from_uri(uri="mongodb://localhost:27017", database="my_db", collection="my_collection")

Repository for HTTP(s) REST API

from redbird.repos import RESTRepo

repo = RESTRepo(url="https://example.com/items", headers={"Authorization": "Bearer 123456"})

You may also specify a Pydantic model to have more structure and better documentation. Also, an ID field is supported. See more from https://red-bird.readthedocs.io/en/latest/repos/index.html

Usage

# Insert items
repo.add({"id": "a", "color": "red", "car_type": "van"})
repo.add({"id": "b", "color": "red", "car_type": "truck"})

# Read
list(repo) # Get all cars in the repo

repo.filter_by(color="red").all() # Get all red cars
repo.filter_by(color="red").first() # Get first red car
repo.filter_by(color="red").last() # Get last red car

# Update
repo.filter_by(car_type="van").update(color="red")

# Delete
repo.filter_by(car_type="van").delete()

The read operations return a dict by default but will be Pydantic instance if model was specified in the repository.

Note that it is still under development (so significant changes may come) but it is already quite useful and fully functional.

Links:

Answered By: miksus

I’ve written an article about implementing the repository pattern with SQLAlchemy as a backend. It follows a data-mapper approach.

Answered By: pgorecki
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.