Separation of business logic and data access in django

I usually implement a service layer in between views and models. This acts like your project’s API and gives you a good helicopter view of what is going on. I inherited this practice from a colleague of mine that uses this layering technique a lot with Java projects (JSF), e.g:

models.py

class Book:
   author = models.ForeignKey(User)
   title = models.CharField(max_length=125)

   class Meta:
       app_label = "library"

services.py

from library.models import Book

def get_books(limit=None, **filters):
    """ simple service function for retrieving books can be widely extended """
    return Book.objects.filter(**filters)[:limit]  # list[:None] will return the entire list

views.py

from library.services import get_books

class BookListView(ListView):
    """ simple view, e.g. implement a _build and _apply filters function """
    queryset = get_books()

Mind you, I usually take models, views and services to module level and
separate even further depending on the project’s size

Django is designed to be easely used to deliver web pages. If you are not confortable with this perhaps you should use another solution.

I’m writting the root or common operations on the model (to have the same interface) and the others on the controller of the model. If I need an operation from other model I import its controller.

This approach it’s enough for me and the complexity of my applications.

Hedde’s response is an example that shows the flexibility of django and python itself.

Very interesting question anyway!

Django employs a slightly modified kind of MVC. There’s no concept of a “controller” in Django. The closest proxy is a “view”, which tends to cause confusion with MVC converts because in MVC a view is more like Django’s “template”.

In Django, a “model” is not merely a database abstraction. In some respects, it shares duty with the Django’s “view” as the controller of MVC. It holds the entirety of behavior associated with an instance. If that instance needs to interact with an external API as part of it’s behavior, then that’s still model code. In fact, models aren’t required to interact with the database at all, so you could conceivable have models that entirely exist as an interactive layer to an external API. It’s a much more free concept of a “model”.

First of all, Don’t repeat yourself.

Then, please be careful not to overengineer, sometimes it is just a waste of time, and makes someone lose focus on what is important. Review the zen of python from time to time.

Take a look at active projects

• more people = more need to organize properly
• the django repository they have a straightforward structure.
• the pip repository they have a straigtforward directory structure.

• the fabric repository is also a good one to look at.

  • you can place all your models under yourapp/models/logicalgroup.py
• e.g User, Group and related models can go under yourapp/models/users.py
• e.g Poll, Question, Answer … could go under yourapp/models/polls.py
• load what you need in __all__ inside of yourapp/models/__init__.py

More about MVC

• model is your data
  • this includes your actual data
  • this also includes your session / cookie / cache / fs / index data
• user interacts with controller to manipulate the model
  • this could be an API, or a view that saves/updates your data
  • this can be tuned with request.GET / request.POST …etc
  • think paging or filtering too.
• the data updates the view
  • the templates take the data and format it accordingly
  • APIs even w/o templates are part of the view; e.g. tastypie or piston
  • this should also account for the middleware.

Take advantage of middleware / templatetags

• If you need some work to be done for each request, middleware is one way to go.
  • e.g. adding timestamps
  • e.g. updating metrics about page hits
  • e.g. populating a cache
• If you have snippets of code that always reoccur for formatting objects, templatetags are good.
  • e.g. active tab / url breadcrumbs

Take advantage of model managers

• creating User can go in a UserManager(models.Manager).
• gory details for instances should go on the models.Model.
• gory details for queryset could go in a models.Manager.
• you might want to create a User one at a time, so you may think that it should live on the model itself, but when creating the object, you probably don’t have all the details:

Example:

class UserManager(models.Manager):
   def create_user(self, username, ...):
      # plain create
   def create_superuser(self, username, ...):
      # may set is_superuser field.
   def activate(self, username):
      # may use save() and send_mail()
   def activate_in_bulk(self, queryset):
      # may use queryset.update() instead of save()
      # may use send_mass_mail() instead of send_mail()

Make use of forms where possible

A lot of boilerplate code can be eliminated if you have forms that map to a model. The ModelForm documentation is pretty good. Separating code for forms from model code can be good if you have a lot of customization (or sometimes avoid cyclic import errors for more advanced uses).

Use management commands when possible

• e.g. yourapp/management/commands/createsuperuser.py
• e.g. yourapp/management/commands/activateinbulk.py

if you have business logic, you can separate it out

• django.contrib.auth uses backends, just like db has a backend…etc.
• add a setting for your business logic (e.g. AUTHENTICATION_BACKENDS)
• you could use django.contrib.auth.backends.RemoteUserBackend
• you could use yourapp.backends.remote_api.RemoteUserBackend
• you could use yourapp.backends.memcached.RemoteUserBackend
• delegate the difficult business logic to the backend
• make sure to set the expectation right on the input/output.
• changing business logic is as simple as changing a setting 🙂

backend example:

class User(db.Models):
    def get_present_name(self): 
        # property became not deterministic in terms of database
        # data is taken from another service by api
        return remote_api.request_user_name(self.uid) or 'Anonymous' 

could become:

class User(db.Models):
   def get_present_name(self):
      for backend in get_backends():
         try:
            return backend.get_present_name(self)
         except: # make pylint happy.
            pass
      return None

more about design patterns

• there’s already a good question about design patterns
• a very good video about practical design patterns
• django’s backends are obvious use of delegation design pattern.

more about interface boundaries

• Is the code you want to use really part of the models? -> yourapp.models
• Is the code part of business logic? -> yourapp.vendor
• Is the code part of generic tools / libs? -> yourapp.libs
• Is the code part of business logic libs? -> yourapp.libs.vendor or yourapp.vendor.libs
• Here is a good one: can you test your code independently?
  • yes, good 🙂
  • no, you may have an interface problem
  • when there is clear separation, unittest should be a breeze with the use of mocking
• Is the separation logical?
  • yes, good 🙂
  • no, you may have trouble testing those logical concepts separately.
• Do you think you will need to refactor when you get 10x more code?
  • yes, no good, no bueno, refactor could be a lot of work
  • no, that’s just awesome!

In short, you could have

• yourapp/core/backends.py
• yourapp/core/models/__init__.py
• yourapp/core/models/users.py
• yourapp/core/models/questions.py
• yourapp/core/backends.py
• yourapp/core/forms.py
• yourapp/core/handlers.py
• yourapp/core/management/commands/__init__.py
• yourapp/core/management/commands/closepolls.py
• yourapp/core/management/commands/removeduplicates.py
• yourapp/core/middleware.py
• yourapp/core/signals.py
• yourapp/core/templatetags/__init__.py
• yourapp/core/templatetags/polls_extras.py
• yourapp/core/views/__init__.py
• yourapp/core/views/users.py
• yourapp/core/views/questions.py
• yourapp/core/signals.py
• yourapp/lib/utils.py
• yourapp/lib/textanalysis.py
• yourapp/lib/ratings.py
• yourapp/vendor/backends.py
• yourapp/vendor/morebusinesslogic.py
• yourapp/vendor/handlers.py
• yourapp/vendor/middleware.py
• yourapp/vendor/signals.py
• yourapp/tests/test_polls.py
• yourapp/tests/test_questions.py
• yourapp/tests/test_duplicates.py
• yourapp/tests/test_ratings.py

or anything else that helps you; finding the interfaces you need and the boundaries will help you.

In Django, MVC structure is as Chris Pratt said, different from classical MVC model used in other frameworks, I think the main reason for doing this is avoiding a too strict application structure, like happens in others MVC frameworks like CakePHP.

In Django, MVC was implemented in the following way:

View layer is splitted in two. The views should be used only to manage HTTP requests, they are called and respond to them. Views communicate with the rest of your application (forms, modelforms, custom classes, of in simple cases directly with models).
To create the interface we use Templates. Templates are string-like to Django, it maps a context into them, and this context was communicated to the view by the application (when view asks).

Model layer gives encapsulation, abstraction, validation, intelligence and makes your data object-oriented (they say someday DBMS will also). This doesn’t means that you should make huge models.py files (in fact a very good advice is to split your models in different files, put them into a folder called ‘models’, make an ‘__init__.py’ file into this folder where you import all your models and finally use the attribute ‘app_label’ of models.Model class). Model should abstract you from operating with data, it will make your application simpler. You should also, if required, create external classes, like “tools” for your models.You can also use heritage in models, setting the ‘abstract’ attribute of your model’s Meta class to ‘True’.

Where is the rest? Well, small web applications generally are a sort of an interface to data, in some small program cases using views to query or insert data would be enough. More common cases will use Forms or ModelForms, which are actually “controllers”. This is not other than a practical solution to a common problem, and a very fast one. It’s what a website use to do.

If Forms are not enogh for you, then you should create your own classes to do the magic, a very good example of this is admin application: you can read ModelAmin code, this actually works as a controller. There is not a standard structure, I suggest you to examine existing Django apps, it depends on each case. This is what Django developers intended, you can add xml parser class, an API connector class, add Celery for performing tasks, twisted for a reactor-based application, use only the ORM, make a web service, modify the admin application and more… It’s your responsability to make good quality code, respect MVC philosophy or not, make it module based and creating your own abstraction layers. It’s very flexible.

My advice: read as much code as you can, there are lots of django applications around, but don’t take them so seriously. Each case is different, patterns and theory helps, but not always, this is an imprecise cience, django just provide you good tools that you can use to aliviate some pains (like admin interface, web form validation, i18n, observer pattern implementation, all the previously mentioned and others), but good designs come from experienced designers.

PS.: use ‘User’ class from auth application (from standard django), you can make for example user profiles, or at least read its code, it will be useful for your case.

I’m mostly agree with chosen answer (https://stackoverflow.com/a/12857584/871392), but want to add option in Making Queries section.

One can define QuerySet classes for models for make filter queries and so on. After that you can proxy this queryset class for model’s manager, like build-in Manager and QuerySet classes do.

Although, if you had to query several data models to get one domain model, it seems more reasonable to me to put this in separate module like suggested before.

An old question, but I’d like to offer my solution anyway. It’s based on acceptance that model objects too require some additional functionality while it’s awkward to place it within the models.py. Heavy business logic may be written separately depending on personal taste, but I at least like the model to do everything related to itself. This solution also supports those who like to have all the logic placed within models themselves.

As such, I devised a hack that allows me to separate logic from model definitions and still get all the hinting from my IDE.

The advantages should be obvious, but this lists a few that I have observed:

• DB definitions remain just that – no logic “garbage” attached
• Model-related logic is all placed neatly in one place
• All the services (forms, REST, views) have a single access point to logic
• Best of all: I did not have to rewrite any code once I realised that my models.py became too cluttered and had to separate the logic away. The separation is smooth and iterative: I could do a function at a time or entire class or the entire models.py.

I have been using this with Python 3.4 and greater and Django 1.8 and greater.

app/models.py

....
from app.logic.user import UserLogic

class User(models.Model, UserLogic):
    field1 = models.AnyField(....)
    ... field definitions ...

app/logic/user.py

if False:
    # This allows the IDE to know about the User model and its member fields
    from main.models import User

class UserLogic(object):
    def logic_function(self: 'User'):
        ... code with hinting working normally ...

The only thing I can’t figure out is how to make my IDE (PyCharm in this case) recognise that UserLogic is actually User model. But since this is obviously a hack, I’m quite happy to accept the little nuisance of always specifying type for self parameter.

I would have to agree with you. There are a lot of possibilities in django but best place to start is reviewing Django’s design philosophy.

1. Calling an API from a model property would not be ideal, it seems like it would make more sense to do something like this in the view and possibly create a service layer to keep things dry. If the call to the API is non-blocking and the call is an expensive one, sending the request to a service worker (a worker that consumes from a queue) might make sense.

2. As per Django’s design philosophy models encapsulate every aspect of an “object”. So all business logic related to that object should live there:

Include all relevant domain logic

Models should encapsulate every aspect of an “object,” following Martin Fowler’s Active Record design pattern.

3. The side effects you describe are apparent, the logic here could be better broken down into Querysets and managers. Here is an example:

   models.py

   import datetime
   
   from djongo import models
   from django.db.models.query import QuerySet
   from django.contrib import admin
   from django.db import transaction
   
   
   class MyUser(models.Model):
   
       present_name = models.TextField(null=False, blank=True)
       status = models.TextField(null=False, blank=True)
       last_active = models.DateTimeField(auto_now=True, editable=False)
   
       # As mentioned you could put this in a template tag to pull it
       # from cache there. Depending on how it is used, it could be
       # retrieved from within the admin view or from a custom view
       # if that is the only place you will use it.
       #def get_present_name(self):
       #    # property became non-deterministic in terms of database
       #    # data is taken from another service by api
       #    return remote_api.request_user_name(self.uid) or 'Anonymous'
   
       # Moved to admin as an action
       # def activate(self):
       #     # method now has a side effect (send message to user)
       #     self.status = 'activated'
       #     self.save()
       #     # send email via email service
       #     #send_mail('Your account is activated!', '…', [self.email])
   
       class Meta:
           ordering = ['-id']  # Needed for DRF pagination
   
       def __unicode__(self):
           return '{}'.format(self.pk)
   
   
   class MyUserRegistrationQuerySet(QuerySet):
   
       def for_inactive_users(self):
           new_date = datetime.datetime.now() - datetime.timedelta(days=3*365)  # 3 Years ago
           return self.filter(last_active__lte=new_date.year)
   
       def by_user_id(self, user_ids):
           return self.filter(id__in=user_ids)
   
   
   class MyUserRegistrationManager(models.Manager):
   
       def get_query_set(self):
           return MyUserRegistrationQuerySet(self.model, using=self._db)
   
       def with_no_activity(self):
           return self.get_query_set().for_inactive_users()
   

   admin.py

   # Then in model admin
   
   class MyUserRegistrationAdmin(admin.ModelAdmin):
       actions = (
           'send_welcome_emails',
       )
   
       def send_activate_emails(self, request, queryset):
           rows_affected = 0
           for obj in queryset:
               with transaction.commit_on_success():
                   # send_email('welcome_email', request, obj) # send email via email service
                   obj.status = 'activated'
                   obj.save()
                   rows_affected += 1
   
           self.message_user(request, 'sent %d' % rows_affected)
   
   admin.site.register(MyUser, MyUserRegistrationAdmin)
   

Most comprehensive article on the different options with pros and cons:

1. Idea #1: Fat Models
2. Idea #2: Putting Business Logic in Views/Forms
3. Idea #3: Services
4. Idea #4: QuerySets/Managers
5. Conclusion

Source:
https://sunscrapers.com/blog/where-to-put-business-logic-django/