Multi-tenancy is one of the decisions that is hardest to change after the fact, and most teams do not make it deliberately at the start. This post covers the approach we use on production Django SaaS applications: how to choose the right isolation model, how to implement tenant context safely, how to handle tenant-specific configuration, and how to instrument the system so you can actually operate it at scale.

Choosing Your Isolation Model

There are three approaches to multi-tenancy in a relational database. The choice you make determines your data model, your migration strategy, your backup approach, and your compliance posture for the lifetime of the application.

Approach 1: Separate Databases Per Tenant

Each tenant gets their own database. Complete data isolation at the infrastructure level. Simple query logic — no risk of cross-tenant data leakage in queries because queries physically cannot reach another tenant's data.

When to choose this: Regulated industries (healthcare, finance, legal) where data residency requirements demand physical separation. Large enterprise customers who contractually require database isolation. Multi-region deployments where different tenants have different data residency requirements.

When not to choose this: Most SaaS applications. The operational overhead — separate database servers, separate migration runs, separate backup policies, separate connection pools — scales linearly with tenant count. At 100 tenants this is manageable with automation. At 10,000 tenants it is a significant operational burden.

Approach 2: Shared Database, Separate Schemas

One database, one schema per tenant. PostgreSQL schemas provide namespace separation — tenant_a.users, tenant_b.users — with shared infrastructure. The django-tenant-schemas and django-tenants packages implement this pattern for Django.

When to choose this: Good isolation with lower operational overhead than separate databases. Works well for mid-market SaaS with tenants that have significantly different data volumes. Schema-level separation makes it natural to give large tenants dedicated resources.

When not to choose this: Schema proliferation at large tenant counts creates migration complexity. Running a migration across 5,000 schemas sequentially takes time. Cross-tenant analytics (for the platform operator) requires either cross-schema queries or a separate analytics database.

Approach 3: Shared Database, Row-Level Isolation

One database, one schema, all tenant data in shared tables with a tenant_id foreign key. The simplest infrastructure, the lowest operational overhead, the most natural Django data model.

When to choose this: Most SaaS applications, particularly early-stage through mid-scale. Migrations are simple — one migration, all tenants. Cross-tenant analytics is straightforward. Operational overhead is minimal.

When not to choose this: Regulated industries requiring physical isolation. Customers who contractually require it. Applications where the query complexity of always including tenant_id filters creates unacceptable risk of cross-tenant data leakage.

The rest of this post covers the row-level isolation approach in depth, as it is the most common choice for Django SaaS applications and the one with the most implementation nuance.

The Tenant Model

Start with a clean Tenant model that serves as the root of all tenant-specific data:

# tenants/models.py
import uuid
from django.db import models
from django.contrib.auth.models import AbstractUser
 
 
class Tenant(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    name = models.CharField(max_length=255)
    slug = models.SlugField(unique=True)
    subdomain = models.CharField(max_length=100, unique=True)
    is_active = models.BooleanField(default=True)
    plan = models.CharField(
        max_length=50,
        choices=[("starter", "Starter"), ("growth", "Growth"), ("enterprise", "Enterprise")],
        default="starter"
    )
    created_at = models.DateTimeField(auto_now_add=True)
    settings = models.JSONField(default=dict)  # Tenant-specific configuration
 
    class Meta:
        indexes = [
            models.Index(fields=["subdomain"]),
            models.Index(fields=["slug"]),
        ]
 
    def __str__(self):
        return self.name
 
 
class TenantUser(AbstractUser):
    """User model that always belongs to a tenant."""
    tenant = models.ForeignKey(
        Tenant,
        on_delete=models.CASCADE,
        related_name="users"
    )
 
    class Meta:
        indexes = [
            models.Index(fields=["tenant", "email"]),
        ]
        # Enforce email uniqueness per tenant, not globally
        unique_together = [("tenant", "email")]

Note the unique_together on TenantUser: email uniqueness is scoped to the tenant, not global. The same email address can belong to different tenants. This is almost always the right behaviour for SaaS — a user can have accounts with multiple customers on your platform.

Tenant Context: Thread-Local Storage Done Right

The central challenge in row-level multi-tenancy is ensuring that every database query automatically filters by the current tenant. The naive approach — pass tenant as an argument to every service function — works but creates enormous boilerplate and is easy to forget.

The standard Django approach uses thread-local storage to hold the current tenant context, set by middleware on every request:

# tenants/context.py
import threading
from typing import Optional
from .models import Tenant
 
_local = threading.local()
 
 
def set_current_tenant(tenant: Optional[Tenant]) -> None:
    _local.tenant = tenant
 
 
def get_current_tenant() -> Optional[Tenant]:
    return getattr(_local, "tenant", None)
 
 
def clear_current_tenant() -> None:
    if hasattr(_local, "tenant"):
        del _local.tenant

# tenants/middleware.py
from django.http import Http404
from django.core.cache import cache
from .models import Tenant
from .context import set_current_tenant, clear_current_tenant
import logging
 
logger = logging.getLogger(__name__)
 
 
class TenantMiddleware:
    """Identifies the current tenant from the request and sets the thread-local context."""
 
    CACHE_TTL = 300  # 5 minutes
 
    def __init__(self, get_response):
        self.get_response = get_response
 
    def __call__(self, request):
        tenant = self._identify_tenant(request)
 
        if tenant is None:
            # No tenant found — could be the marketing site, admin, or invalid subdomain
            set_current_tenant(None)
        elif not tenant.is_active:
            raise Http404("Tenant not found")
        else:
            set_current_tenant(tenant)
 
        try:
            response = self.get_response(request)
        finally:
            # Always clear — never leave tenant context set between requests
            clear_current_tenant()
 
        return response
 
    def _identify_tenant(self, request) -> Optional["Tenant"]:
        host = request.get_host().lower()
 
        # Check cache first
        cache_key = f"tenant:host:{host}"
        cached = cache.get(cache_key)
        if cached is not None:
            try:
                return Tenant.objects.get(id=cached)
            except Tenant.DoesNotExist:
                cache.delete(cache_key)
 
        # Resolve from subdomain
        subdomain = self._extract_subdomain(host)
        if not subdomain:
            return None
 
        try:
            tenant = Tenant.objects.get(subdomain=subdomain, is_active=True)
            cache.set(cache_key, str(tenant.id), self.CACHE_TTL)
            return tenant
        except Tenant.DoesNotExist:
            logger.warning("Unknown subdomain", extra={"subdomain": subdomain, "host": host})
            return None
 
    def _extract_subdomain(self, host: str) -> Optional[str]:
        # Assumes format: subdomain.yourdomain.com
        # Adjust for your domain structure
        parts = host.split(".")
        if len(parts) >= 3:  # subdomain.domain.tld
            return parts[0]
        return None

The finally: clear_current_tenant() is not optional. Without it, if an exception occurs during request processing, the tenant context remains set and could leak into the next request processed by the same thread. In a threaded server like Gunicorn, threads are reused across requests.

The TenantQuerySet: Automatic Filtering at the Model Layer

Thread-local context is only useful if models actually use it. A custom manager enforces tenant scoping at the ORM level:

# tenants/managers.py
from django.db import models
from .context import get_current_tenant
 
 
class TenantQuerySet(models.QuerySet):
    def for_current_tenant(self):
        tenant = get_current_tenant()
        if tenant is None:
            raise ValueError(
                "No tenant in context. Ensure TenantMiddleware is active "
                "or set the tenant context explicitly."
            )
        return self.filter(tenant=tenant)
 
 
class TenantManager(models.Manager):
    def get_queryset(self):
        return TenantQuerySet(self.model, using=self._db)
 
    def for_current_tenant(self):
        return self.get_queryset().for_current_tenant()

# Example tenant-scoped model
# myapp/models.py
from django.db import models
from tenants.models import Tenant
from tenants.managers import TenantManager
 
 
class Project(models.Model):
    tenant = models.ForeignKey(Tenant, on_delete=models.CASCADE, related_name="projects")
    name = models.CharField(max_length=255)
    description = models.TextField(blank=True)
    created_at = models.DateTimeField(auto_now_add=True)
 
    # Custom manager — always filter by tenant
    objects = TenantManager()
 
    # Unscoped manager for admin and cross-tenant operations
    all_objects = models.Manager()
 
    class Meta:
        indexes = [
            models.Index(fields=["tenant", "created_at"]),
            models.Index(fields=["tenant", "name"]),
        ]
        # Name uniqueness scoped to tenant
        unique_together = [("tenant", "name")]
 
    def __str__(self):
        return f"{self.tenant.slug}/{self.name}"

Usage in views and services is clean:

# In a view or service — automatically scoped to current tenant
projects = Project.objects.for_current_tenant().order_by("-created_at")
 
# Cross-tenant (admin, analytics) — explicit opt-in
all_projects = Project.all_objects.all()

The all_objects manager is the escape hatch for legitimate cross-tenant operations — admin views, platform-level analytics, data exports. Making it explicit creates a natural audit trail in code review: any use of all_objects should be reviewed to ensure cross-tenant access is intentional.

Tenant-Specific Configuration

SaaS applications almost always need tenant-specific configuration: feature flags, branding, integration credentials, plan limits. The settings JSONField on the Tenant model handles most of this, but you need a safe accessor:

# tenants/config.py
from typing import Any
from .context import get_current_tenant
 
# Default configuration — applies to all tenants unless overridden
TENANT_DEFAULTS = {
    "max_projects": 10,
    "max_users": 5,
    "ai_features_enabled": False,
    "custom_branding": False,
    "sso_enabled": False,
    "api_rate_limit_per_minute": 60,
    "data_retention_days": 365,
}
 
 
def get_tenant_config(key: str, default: Any = None) -> Any:
    """
    Get a configuration value for the current tenant.
    Falls back to TENANT_DEFAULTS, then to the provided default.
    """
    tenant = get_current_tenant()
    if tenant is None:
        return TENANT_DEFAULTS.get(key, default)
 
    # Tenant settings override defaults
    if key in tenant.settings:
        return tenant.settings[key]
 
    return TENANT_DEFAULTS.get(key, default)
 
 
def get_plan_limits(tenant=None) -> dict:
    """Get the usage limits for the current or specified tenant's plan."""
    t = tenant or get_current_tenant()
    if t is None:
        return PLAN_LIMITS["starter"]
    return PLAN_LIMITS.get(t.plan, PLAN_LIMITS["starter"])
 
 
PLAN_LIMITS = {
    "starter":    {"max_projects": 5,   "max_users": 3,   "api_calls_per_day": 1000},
    "growth":     {"max_projects": 50,  "max_users": 25,  "api_calls_per_day": 10000},
    "enterprise": {"max_projects": 500, "max_users": 250, "api_calls_per_day": 100000},
}

# Usage in a view
from tenants.config import get_tenant_config, get_plan_limits
 
def create_project(request):
    limits = get_plan_limits()
    current_count = Project.objects.for_current_tenant().count()
 
    if current_count >= limits["max_projects"]:
        return JsonResponse(
            {"error": f"Plan limit reached: {limits['max_projects']} projects maximum"},
            status=403
        )
 
    # Proceed with creation

Testing Multi-Tenant Applications

Testing requires explicit tenant context management. We use a pytest fixture:

# conftest.py
import pytest
from tenants.models import Tenant
from tenants.context import set_current_tenant, clear_current_tenant
 
 
@pytest.fixture
def tenant(db):
    return Tenant.objects.create(
        name="Test Tenant",
        slug="test-tenant",
        subdomain="test",
        plan="growth"
    )
 
 
@pytest.fixture
def tenant_context(tenant):
    """Sets and clears tenant context for the duration of a test."""
    set_current_tenant(tenant)
    yield tenant
    clear_current_tenant()
 
 
@pytest.fixture
def another_tenant(db):
    return Tenant.objects.create(
        name="Another Tenant",
        slug="another-tenant",
        subdomain="another",
        plan="starter"
    )

# test_projects.py
import pytest
from myapp.models import Project
 
 
def test_project_isolation(tenant_context, another_tenant, db):
    """Projects from one tenant must not be visible to another."""
    # Create project for current tenant
    project = Project.objects.create(
        tenant=tenant_context,
        name="Tenant A Project"
    )
 
    # Create project for another tenant
    other_project = Project.objects.create(
        tenant=another_tenant,
        name="Tenant B Project"
    )
 
    # Query with tenant context active — should only see current tenant's project
    results = Project.objects.for_current_tenant()
    assert results.count() == 1
    assert results.first().id == project.id
    assert other_project.id not in results.values_list("id", flat=True)

Cross-tenant isolation tests are the most important tests in a multi-tenant application. Run them on every relevant model. A single missing filter(tenant=tenant) anywhere in your codebase is a data leak.

Observability at the Tenant Level

Standard application monitoring is not enough for a multi-tenant SaaS. You need visibility at the tenant level to identify which tenants are responsible for performance issues, which are approaching plan limits, and which are at risk of churning based on usage patterns.

Add tenant context to structured logs:

# middleware — extend TenantMiddleware to add tenant to log context
import structlog
 
def __call__(self, request):
    tenant = self._identify_tenant(request)
    set_current_tenant(tenant)
 
    # Add tenant to structlog context for all logs in this request
    structlog.contextvars.bind_contextvars(
        tenant_id=str(tenant.id) if tenant else None,
        tenant_slug=tenant.slug if tenant else None,
    )
 
    try:
        response = self.get_response(request)
    finally:
        clear_current_tenant()
        structlog.contextvars.unbind_contextvars("tenant_id", "tenant_slug")
 
    return response

With tenant context in every log line, you can query your log aggregation system for all slow queries from a specific tenant, all errors affecting a specific tenant, or the request volume breakdown by tenant — without adding explicit tenant filtering to every log statement.

For a complete guide to production-ready Django deployment covering ASGI, horizontal scaling, database pooling, and full observability, read Lycore's production Django guide.

Lycore builds production Django SaaS applications for businesses that need multi-tenant architecture that scales. We architect, develop, and deliver custom backend systems, AI integrations, and full-stack applications. Get in touch.

That gap between prototype and production is where most AI projects quietly fail. Not with a dramatic error, but with a slow erosion of trust as outputs drift, edge cases multiply, and monitoring gaps leave you discovering problems from user complaints rather than your own systems.

This post is about what we have learned from taking AI features through that transition. The things that matter in a demo, the things that matter in production, and why they are different.

What a prototype optimises for

A prototype has one job: demonstrate that the AI behaviour is achievable. It answers the question "can we do this?" not "can we run this?"

In prototype mode you accept a lot of shortcuts that feel harmless but accumulate into problems:

• Prompts are tuned on a small, representative sample of inputs

• Error handling is minimal because the demo path always succeeds

• Latency is acceptable because you are running it manually

• Cost is invisible because you are not paying per request at scale

• Evaluation is subjective — it looks right to the person who built it

None of these are problems for a prototype. All of them are problems for a production system.

What changes when you ship

Your input distribution shifts immediately

The inputs you designed for are not the inputs you will receive. Users rephrase, abbreviate, use terminology you did not expect, submit edge cases your prompts were not written to handle, and occasionally try to break the system deliberately.

The first week after launch typically surfaces more edge cases than the entire prototype phase. A document summariser built on clean PDFs encounters scanned images, password-protected files, and 400-page legal documents. A support classifier tested on English tickets receives tickets in three languages. An onboarding chatbot encounters users who ask questions about features that do not exist yet.

The fix is not to handle every edge case before launch — that is impossible. The fix is to have the infrastructure to detect them quickly and iterate on prompts and logic without redeploying the application.

Latency becomes a user experience problem

A five-second AI response is fine in a demo. It is painful in a product. Users lose context, perceive the application as broken, and abandon tasks.

For synchronous AI features visible to end users, we target under two seconds for the full round trip. Achieving this requires:

Streaming responses. Return tokens as they are generated rather than waiting for the full completion. Users perceive streaming output as fast even when total generation time is similar.

# services/ai/summariser.py
def summarise_streaming(self, title: str, content: str):
    """Generator that yields summary tokens as they arrive."""
    messages = self._build_messages(title, content)

    with self.client.chat.completions.stream(
        model=self.MODEL,
        messages=messages,
        max_tokens=self.MAX_TOKENS,
    ) as stream:
        for chunk in stream:
            delta = chunk.choices[0].delta.content
            if delta:
                yield delta

# views.py
from django.http import StreamingHttpResponse

def summarise_document_stream(request, doc_id):
    doc = Document.objects.get(id=doc_id)
    summariser = get_summariser()

    def generate():
        for token in summariser.summarise_streaming(doc.title, doc.content):
            yield f"data: {token}\n\n"
        yield "data: [DONE]\n\n"

    return StreamingHttpResponse(generate(), content_type="text/event-stream")

Async for non-blocking features. AI calls that do not need to block the UI — generating a report, processing a batch of records, enriching a newly created object — should run as background Celery tasks. The user gets an immediate response and the result appears when it is ready.

Caching for repeated queries. As covered in detail in our post on cutting LLM costs, exact and semantic caching can eliminate API calls for common inputs entirely.

Evaluation becomes an engineering problem

In prototype mode, evaluation is a human looking at outputs and deciding if they are good. This does not scale.

In production, you need automated evaluation. Not perfect automated evaluation — that is not achievable for open-ended generation tasks — but enough signal to know when quality has degraded.

For classification tasks, this is straightforward: log predictions with a confidence flag and periodically sample low-confidence outputs for human review.

def classify_with_evaluation(self, subject: str, body: str) -> dict:
    result = self.classify(subject, body)

    if not result["confident"]:
        EvaluationQueue.objects.create(
            feature="ticket_classification",
            input={"subject": subject, "body": body[:500]},
            output=result,
            reason="low_confidence"
        )

    return result

For generation tasks, you can use a second LLM call as an evaluator:

# services/ai/evaluator.py
def evaluate_summary(original: str, summary: str) -> dict:
    """Use a cheap model to spot-check summary quality."""
    response = get_openai_client().chat.completions.create(
        model="gpt-4o-mini",
        messages=[{
            "role": "user",
            "content": f"""Rate this summary on two dimensions (1-5 each):
1. Accuracy: Does it correctly represent the original?
2. Completeness: Does it cover the key points?

Original (first 1000 chars): {original[:1000]}

Summary: {summary}

Respond only with JSON: {{"accuracy": N, "completeness": N, "flag": true/false}}
Flag true if either score is below 3."""
        }],
        max_tokens=60,
        temperature=0.0
    )
    import json
    return json.loads(response.choices[0].message.content)

Run this evaluation on a sampled subset (5–10%) of production outputs. It adds cost but provides the quality signal you cannot get any other way.

Prompt management becomes version control

In prototype mode, prompts live in code as strings. When you change a prompt, you commit it and redeploy. This works until you need to:

• Roll back a prompt change that degraded quality

• A/B test two prompt variants on live traffic

• Change a prompt without a code deployment

• Know exactly which prompt version generated a specific output

We store prompts in the database for production features, with versioning:

# models.py
class PromptVersion(models.Model):
    feature = models.CharField(max_length=100)
    version = models.PositiveIntegerField()
    system_prompt = models.TextField()
    user_template = models.TextField()
    is_active = models.BooleanField(default=False)
    created_at = models.DateTimeField(auto_now_add=True)
    notes = models.TextField(blank=True)

    class Meta:
        unique_together = ('feature', 'version')

    @classmethod
    def get_active(cls, feature: str) -> 'PromptVersion':
        return cls.objects.get(feature=feature, is_active=True)

With this model, a prompt change is a database write, not a code deployment. You can roll back by flipping is_active. You can log prompt_version_id alongside every AI output for complete traceability.

Observability is not optional

A production AI feature that you cannot observe is a liability. You need to know:

• How many requests are succeeding vs. failing vs. hitting fallback paths?

• What is the p50, p95, and p99 latency?

• What is the token count distribution, and is it drifting?

• What is the cache hit rate?

• Which inputs are being flagged for review?

We emit a structured log event on every AI call:

import structlog

log = structlog.get_logger()

def log_ai_call(feature, model, input_tokens, output_tokens,
                latency_ms, cached, flagged, prompt_version):
    log.info(
        "ai_call_complete",
        feature=feature,
        model=model,
        input_tokens=input_tokens,
        output_tokens=output_tokens,
        estimated_cost_usd=calculate_cost(model, input_tokens, output_tokens),
        latency_ms=latency_ms,
        cached=cached,
        flagged=flagged,
        prompt_version=prompt_version,
    )

These events flow into whatever monitoring stack the client uses. Over time they build a picture of system health that makes degradation visible before users notice it.

The deployment checklist we use

Before any AI feature goes to production, we work through the following:

Inputs and edge cases

• What is the maximum input size, and what happens when it is exceeded?

• What happens with empty or malformed input?

• What languages or input formats might appear that were not in the test set?

Fallback and graceful degradation

• What does the feature do when the LLM API is unavailable?

• What does it do when response quality is below the confidence threshold?

• Is there a human fallback path for edge cases?

Performance

• Has the feature been tested at expected peak load?

• Are streaming or async patterns in place for user-facing latency?

• Is caching implemented and tested?

Evaluation

• Is there automated quality sampling in place?

• Are low-confidence outputs being routed to a review queue?

• Is there a baseline quality measurement to compare against after prompt changes?

Observability

• Are structured logs emitting on every call?

• Are cost, latency, and token count being tracked?

• Is there an alert for error rate spikes?

Prompt management

• Is the active prompt version stored and logged with each output?

• Is there a rollback mechanism that does not require a code deployment?

What this looks like in practice

Most of the above is not complex engineering — it is disciplined engineering. A streaming view, a Celery task, a database model for prompt versions, a structured log event. None of these take long to build. The teams that skip them are not saving meaningful time in the short term. They are creating a maintenance burden that compounds for as long as the feature runs.

The AI features that stay valuable over time are the ones that were built to be observed, evaluated, and iterated on from day one — not retrofitted with monitoring after something went wrong.

If you are building trading systems, fintech tools, or other high-stakes AI applications where reliability and auditability are non-negotiable, take a look at how Lycore approaches AI for trading and financial services. The same production discipline described in this post applies — with even stricter requirements around audit trails and risk controls.

Lycore builds production AI systems for businesses — LLM integrations, RAG pipelines, agents, and custom AI applications on Django, React, Flutter, and .NET. Get in touch if you want to talk through your use case.

When you first integrate an LLM into a product, cost is not the priority. Getting it working is. Then you ship it, usage grows, and the OpenAI invoice arrives. Suddenly cost matters a lot.

We have been through this cycle enough times to know where the money goes and how to stop it leaking. This post covers the specific optimisations that cut our clients' LLM API costs significantly — with the techniques ranked by impact.

Where the money actually goes

Before optimising, you need to understand the cost model. Most LLM providers charge per token — input tokens and output tokens, usually at different rates. GPT-4o (as of early 2026) charges roughly 4–5x more for output tokens than input tokens depending on the tier.

This means:

• Sending large prompts costs less than generating large responses

• Verbose system prompts are cheaper than verbose completions

• Asking the model to "think step by step" in the output is expensive — you are paying for every reasoning token

With that in mind, here is what we changed.

1. Model routing — highest impact

The single biggest lever. Not every task needs GPT-4o.

We classify LLM tasks into three tiers:

Tier 1 — Simple extraction and classification Single-label classification, yes/no decisions, structured data extraction from short documents, sentiment analysis. These tasks do not require reasoning depth. gpt-4o-mini handles them at 95%+ of GPT-4o quality for about 15x less cost.

Tier 2 — Standard generation Summarisation, drafting, moderate-complexity Q&A, RAG responses. gpt-4o is appropriate here when quality matters, but gpt-4o-mini is worth testing first — it often performs comparably on well-scoped tasks.

Tier 3 — Complex reasoning Multi-step agent decisions, novel problem solving, tasks where accuracy has high business cost (legal, financial, medical contexts). Use the best available model. Do not try to save money here.

Here is the routing logic we use:

# services/ai/model_router.py
from enum import Enum
from dataclasses import dataclass

class TaskComplexity(Enum):
    SIMPLE = "simple"       # Classification, extraction, yes/no
    STANDARD = "standard"   # Summarisation, generation, RAG
    COMPLEX = "complex"     # Agents, reasoning, high-stakes decisions

@dataclass
class ModelConfig:
    model: str
    max_tokens: int
    temperature: float

MODEL_CONFIGS = {
    TaskComplexity.SIMPLE: ModelConfig(
        model="gpt-4o-mini",
        max_tokens=100,
        temperature=0.0
    ),
    TaskComplexity.STANDARD: ModelConfig(
        model="gpt-4o",
        max_tokens=800,
        temperature=0.2
    ),
    TaskComplexity.COMPLEX: ModelConfig(
        model="gpt-4o",
        max_tokens=2000,
        temperature=0.2
    ),
}

def get_model_config(complexity: TaskComplexity) -> ModelConfig:
    return MODEL_CONFIGS[complexity]

On one e-commerce client, routing support ticket classification (Tier 1) away from GPT-4o to GPT-4o-mini reduced that feature's monthly cost by 87% with no measurable quality degradation.

2. Aggressive caching — second highest impact

LLM calls are expensive. Identical or near-identical inputs should never hit the API twice.

Exact cache — for deterministic outputs:

# services/ai/cache.py
import hashlib
import json
from django.core.cache import cache

def cache_key(model: str, messages: list, **kwargs) -> str:
    payload = json.dumps({"model": model, "messages": messages, **kwargs},
                          sort_keys=True)
    return f"llm:{hashlib.sha256(payload.encode()).hexdigest()}"

def get_cached_response(model: str, messages: list, **kwargs):
    key = cache_key(model, messages, **kwargs)
    return cache.get(key)

def set_cached_response(model: str, messages: list, response: dict,
                         ttl: int = 3600, **kwargs):
    key = cache_key(model, messages, **kwargs)
    cache.set(key, response, timeout=ttl)

# In your AI module
def classify(self, subject: str, body: str) -> dict:
    messages = self._build_messages(subject, body)
    cached = get_cached_response(self.MODEL, messages, temperature=0.0)
    if cached:
        return {**cached, "cached": True}

    response = self.client.chat.completions.create(
        model=self.MODEL,
        messages=messages,
        temperature=0.0,
        max_tokens=20
    )
    result = self._parse_response(response)
    set_cached_response(self.MODEL, messages, result, ttl=86400)
    return {**result, "cached": False}

For classification tasks with temperature=0.0, caching is safe — the same input will always produce the same output. On support ticket classification, we saw a 40–60% cache hit rate within 30 days of deployment because support teams ask similar questions repeatedly.

Semantic cache — for near-identical inputs:

For RAG queries where users ask similar but not identical questions, exact hashing misses most cache opportunities. Semantic caching stores embeddings of previous queries and returns cached responses when a new query is sufficiently similar.

# services/ai/semantic_cache.py
from pgvector.django import CosineDistance
from .embeddings import get_embedding
from ..models import CachedLLMResponse

SIMILARITY_THRESHOLD = 0.05  # Cosine distance — lower = more similar

def get_semantic_cache(query: str):
    query_embedding = get_embedding(query)
    match = (
        CachedLLMResponse.objects
        .annotate(distance=CosineDistance('query_embedding', query_embedding))
        .filter(distance__lt=SIMILARITY_THRESHOLD)
        .order_by('distance')
        .first()
    )
    return match.response if match else None

def set_semantic_cache(query: str, response: dict):
    CachedLLMResponse.objects.create(
        query=query,
        query_embedding=get_embedding(query),
        response=response
    )

On a document Q&A feature, semantic caching reduced API calls by a further 30% on top of the exact cache.

3. Prompt compression — steady ongoing savings

System prompts accumulate. A prompt that started as three sentences grows to fifteen as edge cases are handled, examples are added, and requirements evolve. Every token in that prompt is charged on every single API call.

We audit system prompts quarterly and apply the following:

Remove redundant instructions. "Please", "thank you", and elaborately polite phrasing add tokens without improving outputs. "Return only the category name, lowercase." is better than "Could you please provide just the category name in lowercase letters? Thank you."

Remove examples that are no longer needed. Few-shot examples are expensive — typically 50–200 tokens each. Once a model is performing well on a task, test whether the examples are still necessary. Often they are not.

Use structured output instead of prose instructions. Telling the model to return JSON removes the need for lengthy instructions about response format.

Before (148 tokens):

You are a helpful customer support assistant. Your job is to classify 
incoming support tickets into one of the following categories: billing 
issues, technical problems, account management, feature requests, or 
general enquiries. Please read the ticket carefully and select the most 
appropriate category. Return only the category name. Be as accurate as 
possible. If you are unsure, select general enquiries.

After (41 tokens):

Classify the ticket into one category: billing, technical, account, 
feature_request, general. Return only the category name.

Same output quality. 72% fewer tokens per call.

4. Output token control

Most LLM calls do not need long responses. Setting max_tokens appropriately prevents the model from generating verbose output you are not going to use.

# Bad — allows up to 4096 tokens for a single-word classification
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages
)

# Good — classify tasks need at most 10-20 tokens
response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    max_tokens=20
)

For structured JSON outputs, estimate the maximum token count of the JSON you expect and set max_tokens to 120–150% of that. For summarisation tasks, match max_tokens to your actual display limit — if you show summaries in a 300-word UI component, there is no value in generating 600-word summaries.

5. Batching async requests

For non-real-time workloads — nightly document processing, batch classification, scheduled report generation — the OpenAI Batch API offers 50% cost reduction with a 24-hour turnaround window.

# services/ai/batch_processor.py
import json
from openai import OpenAI
from django.conf import settings

client = OpenAI(api_key=settings.OPENAI_API_KEY)

def submit_batch_classification(tickets: list[dict]) -> str:
    """
    Submit a batch of tickets for classification.
    Returns a batch_id to poll for results.
    """
    requests = []
    for ticket in tickets:
        requests.append({
            "custom_id": str(ticket["id"]),
            "method": "POST",
            "url": "/v1/chat/completions",
            "body": {
                "model": "gpt-4o-mini",
                "messages": [
                    {"role": "system", "content": "Classify the ticket: billing, technical, account, feature_request, general. Return only the category."},
                    {"role": "user", "content": f"{ticket['subject']}\n\n{ticket['body'][:500]}"}
                ],
                "max_tokens": 20,
                "temperature": 0.0
            }
        })

    # Write to JSONL
    jsonl_content = "\n".join(json.dumps(r) for r in requests)
    batch_file = client.files.create(
        file=("batch.jsonl", jsonl_content.encode(), "application/jsonl"),
        purpose="batch"
    )

    batch = client.batches.create(
        input_file_id=batch_file.id,
        endpoint="/v1/chat/completions",
        completion_window="24h"
    )

    return batch.id


def retrieve_batch_results(batch_id: str) -> list[dict]:
    batch = client.batches.retrieve(batch_id)
    if batch.status != "completed":
        return []

    content = client.files.content(batch.output_file_id).text
    results = []
    for line in content.strip().split("\n"):
        item = json.loads(line)
        results.append({
            "ticket_id": item["custom_id"],
            "category": item["response"]["body"]["choices"][0]["message"]["content"].strip()
        })
    return results

For a client processing 10,000 support tickets per day, switching nightly batch classification to the Batch API saved $180/month on that feature alone.

6. Monitoring cost per feature

None of the above optimisations are durable without visibility into what each feature is spending. We add cost tracking to every AI call:

# Approximate cost calculation (update rates periodically)
GPT4O_INPUT_COST_PER_1K = 0.0025
GPT4O_OUTPUT_COST_PER_1K = 0.010
GPT4O_MINI_INPUT_COST_PER_1K = 0.00015
GPT4O_MINI_OUTPUT_COST_PER_1K = 0.0006

COST_TABLE = {
    "gpt-4o": (GPT4O_INPUT_COST_PER_1K, GPT4O_OUTPUT_COST_PER_1K),
    "gpt-4o-mini": (GPT4O_MINI_INPUT_COST_PER_1K, GPT4O_MINI_OUTPUT_COST_PER_1K),
}

def calculate_cost(model: str, input_tokens: int, output_tokens: int) -> float:
    input_rate, output_rate = COST_TABLE.get(model, (0, 0))
    return (input_tokens / 1000 * input_rate) + (output_tokens / 1000 * output_rate)

def log_ai_call(feature: str, model: str, usage, cached: bool = False):
    cost = 0.0 if cached else calculate_cost(
        model, usage.prompt_tokens, usage.completion_tokens
    )
    logger.info("ai_call", extra={
        "feature": feature,
        "model": model,
        "input_tokens": usage.prompt_tokens,
        "output_tokens": usage.completion_tokens,
        "estimated_cost_usd": cost,
        "cached": cached
    })

With structured logs flowing into your monitoring stack, you can build a dashboard showing cost per feature per day. This makes it immediately visible when a new feature is unexpectedly expensive, when cache hit rates drop, or when a prompt change caused token count to spike.

The numbers in practice

On a SaaS client with four AI-powered features (ticket classification, document summarisation, a RAG-based search, and an onboarding chatbot), applying all of the above over a two-month period:

These overlap — total reduction was approximately 61% of baseline monthly spend, not the sum of the above. The model routing and caching changes delivered most of the savings. Prompt compression and token limits are lower effort and worth doing, but do not expect them to move the needle as dramatically.

What not to do

Do not sacrifice quality to save tokens on high-stakes tasks. If a classification error costs a customer service team 20 minutes of rework, the cost of that error vastly exceeds any token savings from using a cheaper model.

Do not cache with long TTLs on tasks where freshness matters. A cached RAG response from six months ago may be confidently wrong about something that changed. Match TTL to the staleness tolerance of the use case.

Do not optimise before you measure. Know your baseline cost per feature before you start. Otherwise you cannot quantify what worked and you will not notice if a change made things worse.

Lycore builds production AI systems for businesses — LLM integrations, RAG pipelines, agents, and custom AI applications on Django, React, Flutter, and .NET. Get in touch if you want to talk through your use case.

Adding AI to a Django app is easy. Adding it in a way that doesn't turn into a maintenance nightmare six months later is harder.

We have integrated LLMs, RAG pipelines, and AI agents into a range of Django applications over the past year and a half. Early on, we made the same mistakes teams typically make: AI calls scattered across views, prompts hardcoded as strings in random places, no clear boundary between AI logic and application logic. The codebase worked. It was not pleasant to work in.

This post is about the structure we have converged on — how we organise Django projects that include AI features, where each piece lives, and why.

The problem with putting AI in views

The fastest way to add an AI feature to a Django app is to drop an OpenAI call directly into a view:

# views.py — the fast way, not the right way
from openai import OpenAI
from django.http import JsonResponse

client = OpenAI()

def summarise_document(request, doc_id):
    doc = Document.objects.get(id=doc_id)
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": "Summarise this document concisely."},
            {"role": "user", "content": doc.content}
        ]
    )
    return JsonResponse({"summary": response.choices[0].message.content})

This works fine until:

• You need to call the same AI logic from a Celery task

• You want to write a test without hitting the OpenAI API

• You need to swap GPT-4o for Claude on one feature

• You want to log every AI call with input/output for debugging

• Your prompt needs to evolve independently of your request handling

None of these are hypothetical — all of them come up in production within the first few months. The view has become a load-bearing wall for logic that should not live there.

The structure we use

Here is the folder layout for a Django app that includes AI features:

myapp/
├── models.py
├── views.py
├── urls.py
├── serializers.py
├── tasks.py                  # Celery async tasks
├── services/
│   ├── __init__.py
│   ├── document_service.py   # Business logic
│   └── ai/
│       ├── __init__.py
│       ├── client.py         # LLM client setup
│       ├── prompts.py        # Prompt templates
│       ├── summariser.py     # Summarisation feature
│       ├── classifier.py     # Classification feature
│       └── rag.py            # RAG pipeline
└── tests/
    ├── test_views.py
    ├── test_services.py
    └── test_ai/
        ├── test_summariser.py
        └── test_classifier.py

The key principle is that services/ai/ is a self-contained module. Views call services. Services call AI modules. AI modules talk to LLM providers. Nothing leaks sideways.

The client layer

A single place to configure and access LLM clients:

# services/ai/client.py
from openai import OpenAI
from anthropic import Anthropic
from django.conf import settings
import logging

logger = logging.getLogger(__name__)

_openai_client = None
_anthropic_client = None

def get_openai_client() -> OpenAI:
    global _openai_client
    if _openai_client is None:
        _openai_client = OpenAI(api_key=settings.OPENAI_API_KEY)
    return _openai_client

def get_anthropic_client() -> Anthropic:
    global _anthropic_client
    if _anthropic_client is None:
        _anthropic_client = Anthropic(api_key=settings.ANTHROPIC_API_KEY)
    return _anthropic_client

Lazy singletons mean the client is only instantiated when first used, not at import time. This matters for tests — you can patch get_openai_client at the module boundary without the client being created during import.

The prompts layer

Prompts are first-class citizens, not inline strings:

# services/ai/prompts.py
from dataclasses import dataclass
from typing import Optional

@dataclass
class PromptTemplate:
    system: str
    user_template: str

    def render_user(self, **kwargs) -> str:
        return self.user_template.format(**kwargs)


DOCUMENT_SUMMARISER = PromptTemplate(
    system="""You are a precise document summariser. Produce a concise summary
of the key points in the document provided. Use plain language.
Return only the summary — no preamble, no meta-commentary.""",
    user_template="Document title: {title}\n\nContent:\n{content}"
)

SUPPORT_TICKET_CLASSIFIER = PromptTemplate(
    system="""Classify the support ticket into exactly one of these categories:
billing, technical, account, feature_request, other.
Respond with only the category name, lowercase.""",
    user_template="Ticket subject: {subject}\n\nTicket body:\n{body}"
)

CONTRACT_RISK_ANALYSER = PromptTemplate(
    system="""You are a contract risk analyst. Review the contract excerpt and
identify any clauses that represent unusual risk. For each risk found, return:
- clause_reference: the section or clause number
- risk_level: low / medium / high
- description: plain language explanation of the risk

Return a JSON array. If no risks are found, return an empty array [].""",
    user_template="Contract excerpt:\n\n{content}"
)

Treating prompts as structured objects — not bare strings — makes them easier to version, test, and review. When a prompt needs to change, the change is in one place and the diff is clean.

The feature modules

Each AI capability gets its own module. Here is the summariser:

# services/ai/summariser.py
import logging
from .client import get_openai_client
from .prompts import DOCUMENT_SUMMARISER

logger = logging.getLogger(__name__)

class DocumentSummariser:
    MODEL = "gpt-4o"
    MAX_TOKENS = 500
    TEMPERATURE = 0.2

    def __init__(self):
        self.client = get_openai_client()

    def summarise(self, title: str, content: str) -> dict:
        if not content.strip():
            raise ValueError("Cannot summarise empty content")

        truncated = content[:12000]  # ~3000 tokens, leave room for response
        user_message = DOCUMENT_SUMMARISER.render_user(
            title=title,
            content=truncated
        )

        logger.info(
            "AI summarise request",
            extra={"title": title, "content_length": len(content)}
        )

        response = self.client.chat.completions.create(
            model=self.MODEL,
            messages=[
                {"role": "system", "content": DOCUMENT_SUMMARISER.system},
                {"role": "user", "content": user_message}
            ],
            max_tokens=self.MAX_TOKENS,
            temperature=self.TEMPERATURE
        )

        summary = response.choices[0].message.content.strip()
        tokens_used = response.usage.total_tokens

        logger.info(
            "AI summarise complete",
            extra={"title": title, "tokens_used": tokens_used}
        )

        return {
            "summary": summary,
            "tokens_used": tokens_used,
            "model": self.MODEL,
            "truncated": len(content) > 12000
        }

And the classifier:

# services/ai/classifier.py
import logging
from .client import get_openai_client
from .prompts import SUPPORT_TICKET_CLASSIFIER

logger = logging.getLogger(__name__)

VALID_CATEGORIES = {"billing", "technical", "account", "feature_request", "other"}

class TicketClassifier:
    MODEL = "gpt-4o-mini"  # Simpler task, cheaper model
    TEMPERATURE = 0.0       # Zero temp for classification — we want determinism

    def __init__(self):
        self.client = get_openai_client()

    def classify(self, subject: str, body: str) -> dict:
        user_message = SUPPORT_TICKET_CLASSIFIER.render_user(
            subject=subject,
            body=body[:3000]
        )

        response = self.client.chat.completions.create(
            model=self.MODEL,
            messages=[
                {"role": "system", "content": SUPPORT_TICKET_CLASSIFIER.system},
                {"role": "user", "content": user_message}
            ],
            max_tokens=20,
            temperature=self.TEMPERATURE
        )

        raw = response.choices[0].message.content.strip().lower()
        category = raw if raw in VALID_CATEGORIES else "other"

        if raw not in VALID_CATEGORIES:
            logger.warning(
                "Unexpected classifier output",
                extra={"raw_output": raw, "subject": subject}
            )

        return {
            "category": category,
            "raw_output": raw,
            "confident": raw in VALID_CATEGORIES,
            "model": self.MODEL
        }

Two things worth noting here: TicketClassifier uses gpt-4o-mini instead of gpt-4o. Classification is a simple task — using the full model wastes money and adds latency. Choose the model per task, not per project. Also, temperature is 0.0 for classification — we want the most deterministic output the model will give, not creative variation.

The service layer

Business logic lives here. AI modules are called here, not in views:

# services/document_service.py
from ..models import Document, DocumentSummary
from .ai.summariser import DocumentSummariser
import logging

logger = logging.getLogger(__name__)

_summariser = None

def get_summariser() -> DocumentSummariser:
    global _summariser
    if _summariser is None:
        _summariser = DocumentSummariser()
    return _summariser


def summarise_document(document_id: int) -> DocumentSummary:
    doc = Document.objects.select_related('owner').get(id=document_id)

    # Check if we already have a fresh summary
    existing = DocumentSummary.objects.filter(
        document=doc,
        model=DocumentSummariser.MODEL
    ).order_by('-created_at').first()

    if existing and not doc.content_changed_since(existing.created_at):
        logger.info(f"Returning cached summary for document {document_id}")
        return existing

    # Generate new summary
    result = get_summariser().summarise(
        title=doc.title,
        content=doc.content
    )

    summary = DocumentSummary.objects.create(
        document=doc,
        summary=result["summary"],
        model=result["model"],
        tokens_used=result["tokens_used"],
        was_truncated=result["truncated"]
    )

    return summary

The service layer owns caching logic, model selection fallbacks, and the boundary between AI output and Django models. Views and tasks call this — they do not touch AI modules directly.

The view

Clean:

# views.py
from django.http import JsonResponse
from django.views.decorators.http import require_POST
from .services import document_service
import logging

logger = logging.getLogger(__name__)

@require_POST
def summarise_document_view(request, doc_id):
    try:
        summary = document_service.summarise_document(doc_id)
        return JsonResponse({
            "summary": summary.summary,
            "model": summary.model,
            "cached": False
        })
    except Document.DoesNotExist:
        return JsonResponse({"error": "Document not found"}, status=404)
    except Exception as e:
        logger.error(f"Summarise failed for doc {doc_id}: {e}")
        return JsonResponse({"error": "Summarisation failed"}, status=500)

The view has no knowledge of OpenAI, prompts, or token counts. It handles HTTP concerns and delegates everything else.

Testing without hitting the API

Because AI logic is isolated in modules with injected clients, mocking is clean:

# tests/test_ai/test_summariser.py
from unittest.mock import MagicMock, patch
from myapp.services.ai.summariser import DocumentSummariser

def test_summarise_returns_expected_structure():
    mock_response = MagicMock()
    mock_response.choices[0].message.content = "This is a test summary."
    mock_response.usage.total_tokens = 150

    with patch("myapp.services.ai.summariser.get_openai_client") as mock_client_fn:
        mock_client = MagicMock()
        mock_client.chat.completions.create.return_value = mock_response
        mock_client_fn.return_value = mock_client

        summariser = DocumentSummariser()
        result = summariser.summarise(title="Test Doc", content="Some content here.")

    assert result["summary"] == "This is a test summary."
    assert result["tokens_used"] == 150
    assert result["truncated"] is False


def test_summarise_raises_on_empty_content():
    summariser = DocumentSummariser()
    with pytest.raises(ValueError, match="empty content"):
        summariser.summarise(title="Test", content="   ")

No API calls, no credentials needed in CI. The mock patches at the client boundary, so the prompt rendering, truncation logic, and response parsing all get tested for real.

A few rules we follow

One AI module per feature, not per model. summariser.py is about summarisation. If you ever swap from GPT-4o to Claude for summarisation, the change is contained to one file. If you named the file openai_summariser.py, you have already made the architecture brittle.

Log inputs and outputs at INFO level. AI system behaviour drifts. Without logs, you cannot diagnose why a summary or classification started producing unexpected results. We log the content length in, the token count out, and the model used on every call.

Store the model name alongside every AI output. When GPT-4o gets updated or you change models, you need to know which outputs were generated by which version. tokens_used and model fields on your AI-derived models are not optional.

Never interpolate user input directly into system prompts. User content goes in the user message role only. System prompts are controlled strings that define behaviour — mixing user input into them opens injection risks.

What this gets you

A codebase where:

• AI features are testable without API calls

• Prompts are versioned and reviewable as first-class code

• Swapping providers or models is a one-file change

• Logging and observability is consistent across all AI features

• Views stay thin and focused on HTTP

The structure adds maybe thirty minutes of setup at the start of a project. It saves hours of untangling per month once the project is in production.

Lycore builds production AI systems for businesses — LLM integrations, RAG pipelines, agents, and custom AI applications on Django, React, Flutter, and .NET. Get in touch if you want to talk through your use case.