Deploying (Multiple) Django Apps to a Single Server with Kamal 2

Shared servers for the win!

— 9 min read

I just migrated two of my production Django apps (Place Card Me and Translation Creator) to a single, shared Hetzner VPS using Kamal.

If you haven’t heard of it, Kamal is a tool built by the Rails community to deploy applications to any server. They recently released Kamal 2, which makes it easy to deploy multiple apps to the same server, which really helps cut down on costs.

Overall the process was quite smooth! And there is still plenty of headroom on the $10/month machine for several more projects.

I did the migration in two steps:

  1. Deploy the apps to a new server.
  2. Migrate the data and flip DNS.

If you’re deploying a new application, you can just do step 1 and you should be done. If you’re migrating from an existing installation you can also do step 2, or do the migration in place (scarier).

Here’s the detailed steps I followed:

Provision the server

As I mentioned already, I use Hetzner for my VPS hosting these days. They have the most affordable hosting out there. But you could do this on Digital Ocean, Linode, or wherever.

I provisioned a 4GB, shared CPU machine on Ubuntu 24 which, at the time of this writing, costs €7.05 ($7.71) / month. I also threw in automatic backups—since I’m hosting my own database on the machine—bringing the monthly total to a whopping $9.80 a month.

Remarkably, if you use Kamal 2’s defaults, this is the only setup step required! There are no other steps needed to deploy the app.

Create a production Dockerfile

Kamal uses Docker to deploy your apps, so you need a Dockerfile to tell it how to build and run your app.

I build all my applications with my SaaS Pegasus Django boilerplate, which ships with a production Dockerfile. But the gist of it is basically setting up the project and running gunicorn.

Pegasus uses a multi-stage Dockerfile to improve performance and caching. Here’s a slightly simplified version of the file:

# Dockerfile.web

# Multi-stage Docker buildfile
# See https://docs.docker.com/build/guide/multi-stage/

# Stage 1: Build the python dependencies
FROM python:3.11-slim-bookworm as build-python
RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=locked \
    --mount=target=/var/cache/apt,type=cache,sharing=locked \
    rm -f /etc/apt/apt.conf.d/docker-clean && \
    apt-get update \
    # dependencies for building Python packages
    && apt-get install -y build-essential libpq-dev

COPY ./requirements /requirements
RUN --mount=type=cache,target=/root/.cache \
    pip wheel --no-deps --wheel-dir /wheels \
    -r /requirements/requirements.txt

# Stage 2: Build the front end files
FROM node:20-bookworm-slim AS build-node
RUN nodejs -v && npm -v
WORKDIR /code
COPY *.json *.js .babelrc /code/
COPY assets /code/assets/
RUN npm install

COPY . /code
RUN npm run build

# Stage 3: Build the final image
# This copies the python dependencies from the first stage
# and the front end files from the second stage.
FROM python:3.11-slim-bookworm
ENV PYTHONUNBUFFERED 1
ENV DEBUG 0

RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=locked \
    --mount=target=/var/cache/apt,type=cache,sharing=locked \
    rm -f /etc/apt/apt.conf.d/docker-clean && \
    apt-get update \
    && apt-get install -y \
    curl \
    # psycopg2 dependencies
    libpq-dev \
    # Translations dependencies
    gettext \
    # cleaning up unused files
    && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false

RUN addgroup --system django \
    && adduser --system --ingroup django django

COPY --from=build-python /wheels /wheels
COPY ./requirements /requirements
RUN --mount=type=cache,target=/root/.cache \
    pip install --no-index --find-links=/wheels \
    -r /requirements/requirements.txt \
    && rm -rf /wheels

WORKDIR /code

COPY --chown=django:django . /code
COPY --from=build-node /code/static /code/static

# collect static files for whitenoise
RUN DEBUG=False python /code/manage.py collectstatic --noinput --settings=translationcreator.settings_production
RUN chown django:django -R static_root

USER django

COPY --chown=django:django deploy/docker_startup.sh /start
RUN chmod +x /start
CMD /start

Dockerfile for building and running a Django application.

Staticfiles are collected during the Docker build process and served by whitenoise. The docker_startup.sh file that gets called at the end does our migrations and then starts gunicorn:

# docker_startup.sh

#!/bin/bash

PORT=${PORT:-8000}

echo "Django migrate"
python manage.py migrate --noinput
echo "Run Gunicorn"
gunicorn --bind 0.0.0.0:$PORT --workers 1 --threads 8 --timeout 0 translationcreator.asgi:application -k uvicorn.workers.UvicornWorker

Startup script to run database migrations and start gunicorn.

Create a Kamal deploy configuration

Next we need to create a Kamal deploy configuration file, which looks like this:

# deploy.yml

service: translationcreator

image: translationcreator

env:
  # This is the minimal set of environment variables needed to run the app
  # in production. You can add more here and in the .env file as needed.
  clear:
    DJANGO_SETTINGS_MODULE: translationcreator.settings_production
    USE_S3_MEDIA: 'True'
    AWS_STORAGE_BUCKET_NAME: translationcreator-media
    ALLOWED_HOSTS: www.translationcreator.com
    # ... other clear variables here
  secret:
    - POSTGRES_PASSWORD
    - SECRET_KEY
    - DATABASE_URL
    - REDIS_URL
    - AWS_ACCESS_KEY_ID
    - AWS_SECRET_ACCESS_KEY
    # ... other secrets here        

# https://kamal-deploy.org/docs/configuration#using-different-roles-for-servers
servers:
  web:
    hosts:
      - 5.161.204.158
    options:
      expose: "8000"

# See https://kamal-deploy.org/docs/configuration/builders/ for config options
builder:
  arch: amd64
  dockerfile: './Dockerfile.web'

# See https://kamal-deploy.org/docs/configuration/docker-registry/ for config options
# This example uses Amazon ECR
registry:
  server: redacted.dkr.ecr.us-east-1.amazonaws.com
  username: AWS
  password: <%= %x(aws ecr get-login-password) %>

accessories:
  postgres:
    image: postgres
    host: 5.161.204.158
    directories:
      - data:/var/lib/postgresql/data
    env:
      clear:
        POSTGRES_USER: translationcreator
        POSTGRES_DB: translationcreator
      secret:
        - POSTGRES_PASSWORD
  redis:
    image: redis
    host: 5.161.204.158

# See https://kamal-deploy.org/docs/configuration/proxy/ for config options
proxy:
  ssl: true
  host: www.translationcreator.com
  app_port: 8000
  healthcheck:
    path: /kamal/up/
    interval: 2
    timeout: 2

Kamal’s deploy.yml configuration.

A few notable changes from the default Kamal configuration:

  1. We’ve added postgres and redis accessories for our DB and cache.
  2. We’ve made some modifications to make the app work on port 8000.
  3. We’ve set up a custom healthcheck endpoint (more on that later).

Finally, you will need to set any secrets you need in .kamal/secrets.

# .kamal/secrets

POSTGRES_PASSWORD=your-secret-postgres-password

# Django settings
SECRET_KEY=some-super-long-key
DATABASE_URL=postgres://translationcreator:your-secret-postgres-password@translationcreator-postgres:5432/translationcreator
REDIS_URL=redis://translationcreator-redis:6379
# ...more secrets here

Kamal’s secrets file for storing…well, secrets.

One thing to note is the use of translationcreator-postgres and translationcreator-redis as the hostnames for your database connection strings. These are determined by your service name (translationcreator) and accessory type (postgres/redis) as described here.

This should be everything you need to deploy your application!

Finally, run:

$ kamal setup

Which will magically do everything it needs to deploy your app! And then… fail.

The failure has to do with the interplay between Kamal’s health checks and Django’s security features.

Here’s how to deal with that.

Workaround health checks and Django’s security settings

Kamal runs “health checks” against your application during deploy. These will hit a predefined URL in your app and check for a 200 response code. If it doesn’t get a 200 back, it fails the deploy (and rolls it back if it’s not the first time). Health checks are configured in the proxy section of your deploy.yml file:

proxy:
  ssl: true
  host: www.translationcreator.com
  app_port: 8000
  healthcheck:
    path: /kamal/up/
    interval: 2
    timeout: 2

Here we are saying “hit the /kamal/up/ URL on port 8000 every 2 seconds until it returns ‘OK’”.

However, there are two problems with this in a Django environment which are both related to security settings.

The first is related to the ALLOWED_HOSTS setting. Kamal makes the request from inside the Docker network using the container’s ID as the hostname. So the request will look like it was made to something like http://7c7168a662c1:8000, which results in Django throwing an “Invalid HTTP_HOST header” error if you are using ALLOWED_HOSTS

The second is a similar problem with Django’s SECURE_SSL_REDIRECT setting. The request comes in over http, and Django immediately tries to redirect it to https, but https is not set up at this low a level (it is set from outside the proxy / Docker network), so the redirect hangs.

The workaround in both cases is to bypass the security middleware for just the health check URL. This is documented in this Github issue. Basically you can create a middleware that handles the URL Kamal will check and return a 200 response.

Here’s the code:

# <project>/middleware.py

from django.http import HttpResponse


class HealthCheckMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        if request.path == "/kamal/up/":
            # Or perform any "real" health checking, if needed
            response = HttpResponse("OK")
        else:
            response = self.get_response(request)

        return response

Then, add it to the very beginning of your settings.MIDDLEWARE and your health checks will pass!

After that, kamal setup and kamal deploy worked like a charm.

Migrating data

If you are migrating from Kamal 1 to Kamal 2 then you aren’t quite done. In my case, I found it safer to re-deploy my application on a new server, then backup/restore the Postgres database and flip DNS. This was fine for me as I could tolerate ~10 minutes of the app being in readonly mode.

That process entailed backing up and restoring the database as outlined in the Pegasus docs here. And finally rerouting my DNS from the old server to the new server.

Kamal also provides a kamal upgrade command which purports to upgrade your environment in-place, but I was too worried about extended downtime if something went wrong to try it that way.

Deploying a second app

After going through this process with one app, I did it again with a second app and it was… exactly the same. Kamal is smart enough to see if there’s already an instance of the proxy server running and just ues it if it already exists. And it routes traffic to the right place depending on the passed in hostname from the URL.

This makes it super easy to run many micro-apps on a single server, which is really nice if, like me, you have a lot of Django projects.

Closing thoughts

Overall this process was quite smooth! Excepting the blip with the health checks, everything worked exactly as advertised. Also Kamal 2 deployments feel substantially faster than Kamal 1 (especially the health checks, ironically, given the issues with them).

I just updated yesterday, so I don’t know if I’ll run into any other issues, but so far I’m finding this a great way to deploy my Django projects to a shared server.

Subscribe for Updates

Hear about posts whenever I publish them. Usually about once a month. Unsubscribe at any time.

or use RSS