Growing your Business Logic with Django

Last month we looked at Locality of Behaviour, and how focusing on that allows you to reduce churn as the complexity of your application grows. It’s time to apply that to the perennial topic of where you should keep your business logic in Django.

Django turned 19 this summer. Over those last two decades, it’s become quite clear: Django does a great job of solving the web problem quickly. Django has a grain. It wants you to do things a certain way. My contention is, by going with that, by leaning into Django, you can take advantage of Django’s strengths whilst maintaining the appropriate level of abstraction for your application.

I talk about Growing your business logic with Django because we’re always walking a line between not enough and too much structure, between chaos and over-engineering. As always, it’s a balance. It’s one that changes as our application grows, and so we must grow with it. The challenge is not, not really, where we put our logic, but rather how do we evolve that as our application grows? How do we make sure we can scale our code into the medium term? That’s our topic today.

§

Fat Models, Thin Views

Let’s start with some necessary background. This isn’t a new topic in the Django world, and I want to be able to assume a shared foundation.

Not new? Coming to write this, I realise the topic has been on my backlog since 2019, when Will Vincent began a Forum thread on Where to Put Business Logic in Django?

The options mentioned there cover the full gamete: everything from Fat Models to separate service layers to even separate architectures entailing ”probably don’t use Django”. But ultimately that’s not very helpful. The discussion is quite a long way removed from the actual business of writing your app. What are you going to do?

A starting point for me has long-been a 2014 article from Tom Christie on the DapApps blog: Django models, encapsulation and data integrity¹

Herein, Tom identifies the essential problem:

Model instances lack any real encapsulation. As codebases grow it becomes difficult to make any cast-iron guarantees that you really are enforcing application-level data integrity.

The example he gives is of an Account object that needs to be created with a status flag correctly set. We might do this in our signup view, but then our logic becomes spread-out between view and model, and there’s no guarantee that our rule is enforced along some other code path that’s added as our application grows.

Controlling this is what leads to the Django mantra to have Fat models, thin views — we move our business logic into the model layer to maintain encapsulation there — but Tom seeks to add some flesh to those bones:

What constitutes a 'fat model'? How much logic is okay in view code? Does the 'fat models' convention still hold if we split business logic into nicely defined utility functions?

I would rephrase this more strictly:

Never write to a model field or call save() directly. Always use model methods and manager methods for state changing operations.

In the signup example, we add a Account.objects.create_trial() manager method to create new accounts, and an Account.signup() instance method to progress beyond the trial, and so on. We use these to control all the model state changes, and in so doing ensure that are business rules are enforced.

Go and read the article.

Combined with custom QuerySet methods, it states the essence of what I think is the received wisdom in the community about how you can manage your application logic, all the way from “It’s starting to get gnarly” to “Pretty Damn Big”.

Service Layers

Following the forum thread, there was a lot of discussion about the (supposed) need for a service layer: that encapsulating logic in managers and so-on isn’t sufficient. Rather, that you need to move your business logic wholesale into a layer of utility functions that sit above the ORM. Indeed, even, that the only way to scale your application was to use those different architectures that entail ”probably don’t use Django”.

The essential essays here are a pair by James Bennett, Against service layers in Django and More on service layers in Django, and a lovely skit by Luke Plant, Evolution of a Django Repository pattern.

These push-back against the supposed need for a service layer, and argue that in adopting one, you’re not only then battling your application’s complexity, but “developing and maintaining something close to your own private ORM” as well.

Again, the Django approach is to lean into the model layer. James from the first Against service layers essay:

In most well-designed Django applications, the models — and potentially associated utility code, like custom Manager or QuerySet subclasses — are the API exposed to other code. Which in turn means that they are the place where the “business logic” should be implemented.

Again, go give all these ones a read.

Managing Complexity

The underlying issue here is complexity. The bigger your program, the more people that work on it, the harder it gets. That’s just how it is. It’s the battle we’re all facing. Always.

One thought is that moving your logic into a separate layer can help with that. And of course it can, but not on its own.

James from the More on service layers essay:

Taking that over-complex method and just moving it, as-is, into a different place in your code won’t make it stop being over-complex, and won’t make it OK for it to be that complex.

Indeed, unless you change something else along the way, adding an additional indirection layer leaves you with strictly more complexity and nothing to show for it.

Nonetheless, if you can tease out separate responsibilities — validating that some invariants hold (i.e. that some business rules hold) — then you can give yourself a separate working space to reason about that by isolating that logic in its own method.

That though is independent of the service layer question per se. Separating responsibilities into their own subroutines is just good engineering. You’ll drown in complexity, and won’t even get as far as worrying about all this, if you can’t do even that. Exactly where those routines live, or how they’re imported and called is very much secondary to how you factored your application logic at a much deeper level than (merely) that.²

Implicit though in much of the discussion is a kind of strawman, ”Oh, if you don’t see the need for a service layer, you’re just not really dealing with (enough) complexity”. Tacit in that is this idea – that ”probably don’t use Django” — though not necessarily from the person who said it — that it’s the ORM, and not the inherent size and complexity of the application, that’s at fault.

Scaling Django

This is the old ”Does Django scale?” chestnut.

For years we’ve pointed to Instagram, who all these years later still use the core of Django’s request-response handlers, to answer “Yes, Django is fast enough for you”.

For the ORM, Kraken, part of the Octopus Energy Group, gives the perfect example. They have 1000 developers working their Django monolith, that is deployed 250 times a day. You’re not going to get that big. Your application is not that complex.

See Frederike Jaeger’s Spreading our tentacles taking a Django app global keynote from DjangoCon Europe 2021 and Çağıl Uluşahin Sönmez’s Layered Django project structure for large-scale collaboration from DjangoCon Europe 2024 for great insights into how Kraken in fact scales their Django application.

They do separate their logic into various layers. They employ so-called thin-models in their base Data Layer, responsible for the actual persistence to the database with the ORM, and have a separate Domain Layer above that enforcing the business logic rules. But underneath it all, is the ORM.

A Napkin Plan for Growing your Django app

So, all of that is background.

We know Django solves the web problem quickly. We know the standard ORM patterns will get us a long way. We know we’ll likely never reach it, but we can’t help ourselves from dreaming about scale. We’re perfectionists. We’d like to be sure we can adapt if we ever reach that point.

But we’ve got deadlines. It’s Django remember.

We need to do the right thing now, leaving open the pathway for then.

We need what Frank Wiles called a Napkin Plan. His target there was scaling:

A plan for how you will handle [the dreamed of future]. What tech or technique you will move to when you run out of road on your current path.

The key point is that it’s just a plan:

But for God’s sake, don’t build it.

What’s on the Napkin Plan is definitely not what we need to start with:

Premature optimization and too many layers of abstractions are defense mechanisms we use to protect ourselves from these unknowns.

This is us, as we wonder about whether we need a service layer.

We can’t shake the feeling. So let’s sooth it with a Napkin Plan.

Statistically speaking, you’ll never need it, but you have it if you do.

A story

I have a simple Django Bookmark model:

from django.db import models

class Bookmark(models.Model):
    url = models.URLField(unique=True)
    title = models.CharField(max_length=255)
    note = models.TextField(blank=True)
    favourite = models.BooleanField(default=False)

I want to get it on the page quickly so I use Neapolitan’s CRUDView:

# urls.py
from neapolitan.views import CRUDView
from .models import Bookmark


class BookmarkView(CRUDView):
    model = Bookmark
    fields = ["url", "title", "note"]
    filterset_fields = [
        "favourite",
    ]

That gets me started and I can happily work on my UI until a new requirement comes in that we need to associate bookmarks to individual users.

To do this we add a foreign key to our Bookmark model:

class Bookmark(models.Model):
    owner = models.ForeignKey(
       User,
       related_name="bookmarks",
    )
    ...

The question is, how do we go about associating the user when creating the bookmark?

This example uses a foreign key, but it’s no different really from the status field example earlier. We need to make sure that our model is configured in line with our business rules, and the question is where to do that?

The standard form_valid implementation looks like this:

def form_valid(self, form):
    self.object = form.save()
    return HttpResponseRedirect(
        self.get_success_url()
    )

We call form.save() and then redirect to the detail view.

But if we want to attach a reference to the current user we’re going to need to do something different.

The usual approach here is to call form.save with the commit=False keyword argument, so that the instance isn’t actually saved, and then we can configure it ourselves:

def form_valid(self, form):
    self.object = form.save(commit=False)
    self.object.user = self.request.user
    self.object.save()
    # Probably need this too...
    form.save_m2m()
    return HttpResponseRedirect(
        self.get_success_url()
    )

Here we get the Bookmark instance unsaved, we add the user reference and save the model ourselves.

Not for our bookmark model in this example but, in general, we also need to save any many-to-many fields on the model with the special save_m2m() method that’s added to the form. (Forgetting this is a constant foot-gun.)

Now that’s OK. Like it works. But it makes a mess of our view.

More though, the view’s job is about turning requests into responses. By having the user set here, it’s responsible for the business logic that bookmark needs a reference to its parent user.

What we want to do is move that into the form.

Custom Forms

If we create a custom BookmarkForm class, we can override the __init__() method to take an optional reference to the parent user:

class BookmarkForm(forms.ModelForm):
    def __init__(self, *args, user=None, **kwargs):
        super().init(*args, **kwargs)
        if not kwargs.get("instance"):
            assert user, (
                f"{self.class.name} "
                "requires either an instance "
                "(for updates) "
                "or a User (for creation)."
            )
            self.instance.user = user

We check there, if we didn’t pass an instance — so if we’re creating a new bookmark rather than updating an existing one — then we require that we got passed the parent user instead. We raise a clear error if not.

Then, finally, we set the user on the form’s bookmark instance here ourselves.

If we go back to our view, we tell our view class to use the custom form, and then in get_form(), if we’re creating a new bookmark, we make sure to pass the current user in the form kwargs:

class BookmarkView(CRUDView):
    model = Bookmark
    form_class = BookmarkForm
    # ...

    def get_form(self, data=None, files=None, **kwargs):
        cls = self.get_form_class()
        if self.role == Role.CREATE:
            kwargs["user"] = self.request.user
        form = cls(
          data=data, files=files, **kwargs
        )
        return form

We no longer need to do anything custom in form_valid(), and the responsibility for associating the bookmark to its parent user lives in form layer, not in the view.

The form layer then becomes the first locus of enforcing business logic in a single place. As long as you make sure all model updates go via a form you know your business logic will be applied.

That’s just one example, but you can generalise. Where possible you want to push business logic into the form layer, so the view does no more that call is_valid() and save(), and branches accordingly.

The parent object pattern comes up almost for every model — generally your models form some sort of tree, each referring to the one above. I want to generalise this pattern and make it fully declarative in Neapolitan, but there’s nothing Neapolitan specific about it. You can do the same with Django’s create and update views just as well.

The constraint here is that you need to use the form. Folks will often bypass this but you shouldn’t. Forms are Django’s data sanitation layer, and all input should go through them. (Folks are often surprised that model validation isn’t automatically applied on Model.save(); they found that out skipping the form.)

You’ve got a management command. Are you sure you entered the data right? You’re importing CSV. Are you sure that’s well formatted? Of course you’re not. The form layer is your defence against that. Skip it at your peril.

Manager Methods

Now I’ll often stick there. Custom QuerySet methods for querying, plus forms making the wrapper layer for updates. And often that’s enough. It’s clean. It’s tight. It gets you a long way.

But let’s say you want to move to the custom manager methods, as talked about by others above. (This would assume that you’re inside your data validation layer.)

Here, then, you don’t touch your views but, rather, have your form’s save() method use the manager method instead. Bar introspection to generate the form ModelForm essentially adds only the save() method to the Form API. You can override save() and pass cleaned_data, together with the User reference in this example, just as you always would.

You’ve then moved your business logic into the manager method in a way that’s totally transparent to your existing code.

It goes similar for bulk edits. You start with a FormSet, making sure to pass any dependencies as keyword args to each child form. Your view (blissfully) calls is_valid() and save() just as if it were dealing with a single form. Meanwhile, the formset handles looping over all of its children.

In the same way that you can do so on the form, you can override save() on the formset to create the individual instances and use Bookmark.objects.bulk_create() (or any custom manager methods you’ve defined). Again, it’s transparent to your existing code.

To a service layer

And so we finally, maybe, get to the point where we want to move our logic off of our manager classes. We want the clinical space of a separate layer.

Well, moving a method is no big thing.

Taking this (too) simple example:

class BookmarkManager(models.Manager):
    def create_for_user(self, user, **cleaned_data):
        bookmark = self.model(
            user=user,
            **cleaned_data
        )
        bookmark.save()
        return bookmark

We need to change the self parameter, and then we can put that anywhere we like:

def create_bookmark_for_user(Model, user, **cleaned_data):
    bookmark = Model(
        user=user,
        **cleaned_data
    )
    bookmark.save()
    return bookmark

We can even proxy our old method so nothing breaks while we’re updating:

class BookmarkManager(models.Manager):
    def create_for_user(self, user, **cleaned_data):
        return create_bookmark_for_user(
            self.model,
            user,
            **cleaned_data,
        )

Of course nothing here has changed. This is precisely the “just moving it, as-is” that James Bennett was arguing solves nothing earlier.

But what we’ve done, for an (again) too simple example, of associating a bookmark to its parent user, is to move business logic from the view where it began, into our forms, into custom manager methods, and then into a separate utility or service layer, all without breaking anything.

The problem of managing the complexity in our application remains absolutely untouched. That’s the job. That’s still what we have to do as software engineers. Long may it continue.

But that (real) problem is totally separate from where we put our business logic. Our Napkin Plan shows that we don’t really have to think about that bit at all. It’s a red-herring. Likely it always was.