Deploy a FastAPI LLM App with Docker and Nginx on Your Own Server

You have a FastAPI app that calls an LLM and it works on your laptop. To put it on your own server, you need four things: a container, a place for the API key that isn't the image, a health check so a crashed process gets noticed, and a reverse proxy that terminates TLS and passes streamed tokens through. This post gives you each as a small file, and tells you exactly which parts I ran and which I couldn't.

Here's the split, up front. I ran the app and its secret loading, validated the compose file with docker compose config, and ran the nginx server block through nginx -t and a real HTTPS request on nginx 1.24.0 with a self-signed certificate. I could not build the Docker image: Docker Hub returned 429 Too Many Requests when I tried to pull the Python base image, so the Dockerfile is unrun. I also did not run certbot, bring up the full compose stack, or deploy to a real server.

Already have a server? Harden it first with the VPS hardening checklist. Choosing a proxy? See nginx vs Caddy vs Traefik. For the application code itself, FastAPI + Claude API production patterns covers auth, rate limiting and error handling, and this post doesn't repeat it.

What does the deployment look like?

internet -> nginx (80, 443, TLS) -> app (8000, internal only) -> LLM API
                 |                         |
          /etc/letsencrypt          /run/secrets/anthropic_api_key

Only nginx is public. The app is reachable only on the compose network. The key is a file mounted into the app container, not an environment variable baked into the image or the compose file.

Project layout:

llm-app/
  app/main.py
  requirements.txt
  Dockerfile
  .dockerignore
  compose.yaml
  nginx/default.conf
  secrets/anthropic_api_key.txt    # never committed
  certbot-webroot/

The app: health, readiness and the secret

This is deliberately small. The chat route streams a stub (a loop standing in for your model call) so the example runs without a key or a network. Swap the loop for your SDK's stream. I ran this app under uvicorn 0.53.0 with FastAPI 0.143.0.

import asyncio
import os
from pathlib import Path

from fastapi import FastAPI, HTTPException
from fastapi.responses import StreamingResponse
from pydantic import BaseModel

app = FastAPI(title="llm-app")


def load_api_key() -> str | None:
    """Prefer a mounted secret file, fall back to an env var for local dev."""
    path = Path(os.environ.get("API_KEY_FILE", "/run/secrets/anthropic_api_key"))
    if path.is_file():
        return path.read_text().strip() or None
    return os.environ.get("ANTHROPIC_API_KEY") or None


API_KEY = load_api_key()


class ChatIn(BaseModel):
    prompt: str


@app.get("/health")
async def health():
    return {"status": "ok"}          # liveness: the process answers


@app.get("/ready")
async def ready():
    if not API_KEY:                  # readiness: what the app cannot work without
        raise HTTPException(status_code=503, detail="api key not configured")
    return {"status": "ready"}


@app.post("/chat")
async def chat(body: ChatIn):
    if not API_KEY:
        raise HTTPException(status_code=503, detail="api key not configured")

    async def gen():
        for word in f"echo: {body.prompt}".split():   # replace with your model stream
            yield f"data: {word}\n\n"
            await asyncio.sleep(0.1)

    return StreamingResponse(gen(), media_type="text/event-stream")

Why two endpoints: if the health check also required the key, a missing secret would make Docker mark the container unhealthy and, with a restart policy, loop it forever. Liveness and readiness answer different questions. I tested both with the secret file absent, using FastAPI's test client:

503 {'detail': 'api key not configured'}
{'status': 'ok'}

The first line is /ready, the second is /health. With the key file present, /ready returned {"status":"ready"} through nginx.

The Dockerfile

FROM python:3.12-slim

ENV PYTHONDONTWRITEBYTECODE=1 \
    PYTHONUNBUFFERED=1

RUN useradd --system --uid 10001 --no-create-home appuser

WORKDIR /srv
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app ./app
USER appuser

EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--forwarded-allow-ips", "*"]

requirements.txt pins what I ran: fastapi==0.143.0 and uvicorn==0.53.0. Add your SDK and pin it too.

Choices worth knowing about:

• Copy requirements.txt first. Docker caches layers, so dependency installs are skipped when only your code changed.
• Non-root user. A bug in your app shouldn't hand an attacker root inside the container.
• PYTHONUNBUFFERED=1. Logs appear immediately in docker compose logs instead of waiting for a buffer to fill.
• --forwarded-allow-ips "*". This makes uvicorn trust the X-Forwarded-* headers from any sender, so the app sees real client IPs and https from nginx. It's only safe because port 8000 is never published. If you publish it, anyone can forge those headers.
• .dockerignore. Exclude .git, .env, secrets/ and virtualenvs so a key can't be copied into an image layer by accident.

This file is unrun, as explained above. I checked that docker build --check reached the base-image lookup, and it stopped there with the 429. Build it on your own machine before trusting it.

The compose file

services:
  app:
    build: .
    restart: unless-stopped
    expose:
      - "8000"
    secrets:
      - anthropic_api_key
    read_only: true
    cap_drop:
      - ALL
    security_opt:
      - no-new-privileges:true
    healthcheck:
      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=3)"]
      interval: 15s
      timeout: 5s
      retries: 3
      start_period: 10s

  nginx:
    image: nginx:1.28
    restart: unless-stopped
    depends_on:
      app:
        condition: service_healthy
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
      - /etc/letsencrypt:/etc/letsencrypt:ro
      - ./certbot-webroot:/var/www/certbot:ro

secrets:
  anthropic_api_key:
    file: ./secrets/anthropic_api_key.txt

docker compose config --quiet (Compose v5.6.0) accepted this file with no output, which is how it reports success. That checks syntax and structure, not that the stack runs. Two things I couldn't confirm by running: the pull itself (rate-limited), and that the healthcheck command passes inside the container. A later check of Docker Hub's tag API on 2026-10-09 showed that both the nginx:1.28 and python:3.12-slim tags exist, but pin versions you have actually pulled. I ran the same Python one-liner on the host against the running app and it exited cleanly with status 200.

Line by line:

• expose, not ports, on the app. Docker's published ports bypass ufw, a trap covered in the hardening checklist. Here only nginx publishes anything.
• secrets with a file. Compose mounts it at /run/secrets/anthropic_api_key, which is the default path in load_api_key(). Env vars are visible to every process in the container and tend to leak into logs; a file mount is a modest step up, not a vault. One catch from Docker's Compose reference: for a file: secret, Compose uses a bind mount, so the file keeps its host owner and mode, and the uid, gid and mode options are silently ignored. The container runs as uid 10001, so a root-owned file with mode 600 is unreadable to the app. Either chown 10001 the file (then chmod 400) or accept a group-readable mode on a directory only trusted users can enter. I tested only with a world-readable file, so test this on your server: /ready should return 200.
• read_only, cap_drop, no-new-privileges. Cheap hardening. If your app writes temp files, add a tmpfs: /tmp rather than dropping read_only. I didn't run the container, so I haven't verified uvicorn starts under read_only in this image.
• depends_on with service_healthy. Nginx starts only after the app's health check passes. Without it, nginx starts first and fails DNS lookup of app at boot if the app isn't up yet.
• restart: unless-stopped. Containers come back after a crash or a server reboot, unless you stopped them yourself.

The nginx config

This is the server block. The streaming and timeout lines are explained in my proxy comparison, so here's only what's specific to this stack:

server_tokens off;

server {
    listen 80;
    server_name ai.example.com;

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl;
    server_name ai.example.com;

    ssl_certificate     /etc/letsencrypt/live/ai.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/ai.example.com/privkey.pem;
    ssl_protocols       TLSv1.2 TLSv1.3;

    client_max_body_size 1m;

    location / {
        proxy_pass http://app:8000;
        proxy_http_version 1.1;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;

        proxy_buffering off;
        proxy_read_timeout 300s;
    }
}

Replace ai.example.com with your domain. proxy_pass http://app:8000 uses the compose service name, which Docker's DNS resolves inside the network.

What I ran: a copy of this config with three substitutions, because it couldn't run verbatim outside Docker. The certificate paths pointed to a throwaway self-signed certificate, the upstream was 127.0.0.1:8000 instead of app:8000, and the ports were 8080 and 8443. nginx -t on nginx 1.24.0 reported:

nginx: the configuration file /tmp/P10/o2/nginx_test/nginx.conf syntax is ok
nginx: configuration file /tmp/P10/o2/nginx_test/nginx.conf test is successful

Then three requests, with the app running under uvicorn using the real secret-file code path:

GET  http://ai.example.com:8080/health   -> 301, Location: https://ai.example.com:8443/health
GET  https://ai.example.com:8443/ready   -> 200  {"status":"ready"}  (Server: nginx, no version)
POST https://ai.example.com:8443/chat    -> data: echo: / data: hello / data: from / data: tls

So the redirect, TLS termination, proxying and stream passthrough work, and server_tokens off removed the version from the Server header. I did not check token timing in this run; the earlier proxy post measured that. The image will run a newer nginx than the 1.24.0 I tested and directive syntax can differ between versions, so re-run nginx -t inside the image.

How do you get the certificate and bring it up?

Nginx won't start the 443 block if the certificate files are missing, so the order matters:

1. Point your domain's DNS A record at the server and wait for it to resolve.
2. Issue the certificate with certbot on the host before the first docker compose up, writing to /etc/letsencrypt. A standalone issuance needs port 80 free, which it is before nginx runs.
3. Put the API key in secrets/anthropic_api_key.txt, owned by uid 10001 with mode 400 (see the secrets note above), and keep it out of git.
4. docker compose up -d --build.
5. Check docker compose ps for healthy, then curl https://your-domain/ready.

I haven't run certbot, so I'm not giving you its exact flags or renewal setup; its own documentation does. Renewal is the part people forget: a standalone challenge needs port 80, which nginx now holds, so plan a webroot-style renewal (that's what the certbot-webroot mount is for) and reload nginx after renewal. If this feels like a lot, Caddy obtains and renews certificates itself, and the proxy comparison shows its config.

Where do you run it?

Any Ubuntu VPS with Docker works, and nothing here depends on a provider. If you don't have one yet, Hostinger's KVM plans are one option (plans here), and Hetzner, DigitalOcean and Linode are the usual alternatives. Some links on this page are affiliate links; they cost you nothing and I only link tools I'd mention anyway. I haven't sized this particular app's memory use, so check docker stats under your own load rather than trusting a plan's RAM number. For a longer Hostinger walkthrough with a different stack, see deploying Next.js on a Hostinger VPS or the Hostinger VPS review.

A pre-flight checklist

• Container runs as non-root, and the image contains no secrets (check with docker history).
• Only nginx publishes ports; docker compose ps shows the app with no host port.
• /health is used by Docker and /ready by your monitoring.
• The key is in a file readable only by the container user (not world-readable), outside git, with a spend limit set at the provider. /ready returns 200.
• nginx -t passes inside the same image you run, not only on your laptop's version.
• Streaming works through the proxy: curl -N the chat route and watch tokens arrive one by one.
• Certificate renewal is tested before the certificate expires.

When you change the code later, redeploy without dropping requests using the zero-downtime Docker Compose deploys post, which measures what a plain docker compose up -d --build costs you.