Back Up and Restore an AI Server with restic: A Tested VPS Setup

A backup you've never restored is a hope, and AI servers make that hope more expensive than usual. The server holds a database of embeddings, an automation tool whose saved credentials are encrypted with a key that lives in an env file, and a pile of compose configuration you tuned over a weekend. Lose the disk and all three go together.

This is a working restic setup for a single VPS: what to include, a nightly script that dumps Postgres and snapshots the stack, retention, integrity checks, and a restore drill. I ran the restic and Postgres steps on 2026-10-09 in a sandbox against a local repository (restic 0.16.4 from Ubuntu 24.04's apt, Docker Engine 29.8.2, the Postgres 18 pgvector container from the pgvector setup post). Remote backends are labelled as unrun, because I had no remote storage to point at. For securing the host itself, see the VPS hardening checklist.

What should you back up on an AI server?

Sort everything on the box into three buckets before writing a script.

The Qdrant RAG stack and Vaultwarden and Uptime Kuma post cover the tool-specific export steps for those stacks. This post is about the part they share: getting those files off the box, encrypted, versioned and restorable.

Why restic?

restic keeps encrypted, deduplicated snapshots in a repository you can put on a local disk, another server over SFTP, an S3-compatible bucket, Backblaze B2, or anything rclone can reach. The restic documentation page on preparing a repository (read 2026-10-09) shows these URL forms:

sftp:user@host:/srv/restic-repo
s3:s3.us-east-1.amazonaws.com/bucket_name
b2:bucketname:path/to/repo
rclone:foo:bar

S3 and B2 take credentials from environment variables (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY and B2_ACCOUNT_ID / B2_ACCOUNT_KEY in the docs). Pick whichever storage you already trust. Alternatives with the same shape are Borg and Kopia; this post doesn't compare them.

Install on Ubuntu with sudo apt install restic. In my sandbox that gave version 0.16.4, which is what every output below comes from. Newer versions may print slightly different text.

How do you create a repository?

The repository password encrypts everything. Put it in a file readable only by root:

sudo install -d -m 700 /etc/restic
printf '%s\n' 'a-long-random-passphrase' | sudo tee /etc/restic/pw.txt > /dev/null
sudo chmod 600 /etc/restic/pw.txt
restic -r /srv/restic-repo --password-file /etc/restic/pw.txt init

(The shell lines are templates. In my sandbox I ran the last command with scratch paths.) Output:

created restic repository 61c06acb85 at /tmp/H2/restic/repo

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

I tried a different password file against that repository and got Fatal: wrong password or no key found. restic won't let you in, and neither will anyone else. Save the passphrase in your password manager now, before the first backup, and not on the same server alone.

A local repository on the same disk protects you from deleting files, not from losing the disk. For real use, point RESTIC_REPOSITORY at a different machine or a bucket.

What does the nightly backup script look like?

Create /etc/restic/env (mode 600) with the repository settings:

export RESTIC_REPOSITORY=sftp:backup@other-host:/srv/restic-repo
export RESTIC_PASSWORD_FILE=/etc/restic/pw.txt

Create /etc/restic/excludes.txt:

**/node_modules
**/.cache
**/*.tmp
/srv/ai/ollama-models

Then the script. It dumps Postgres first, snapshots the whole stack folder, prunes old snapshots, and verifies a slice of the repository:

#!/usr/bin/env bash
# Nightly backup for a small AI server: Postgres dump + stack files -> restic.
set -euo pipefail

source "${RESTIC_ENV:-/etc/restic/env}"   # exports RESTIC_REPOSITORY, RESTIC_PASSWORD_FILE (+ backend keys)

STACK="${STACK:-/srv/ai}"
COMPOSE_FILE="${COMPOSE_FILE:-$STACK/stack/compose.yaml}"
EXCLUDES="${EXCLUDES:-/etc/restic/excludes.txt}"

mkdir -p "$STACK/backups"

# 1. Consistent database dump (custom format), written to a temp name first.
docker compose -f "$COMPOSE_FILE" exec -T db pg_dump -U app -d appdb -Fc > "$STACK/backups/appdb.dump.tmp"
docker compose -f "$COMPOSE_FILE" exec -T db pg_restore --list < "$STACK/backups/appdb.dump.tmp" > /dev/null
mv "$STACK/backups/appdb.dump.tmp" "$STACK/backups/appdb.dump"

# 2. Snapshot the stack directory (compose files, env files, the dump).
restic backup "$STACK" --exclude-file "$EXCLUDES" --tag ai-server

# 3. Keep 7 daily, 4 weekly, 6 monthly snapshots; remove the rest.
restic forget --tag ai-server --keep-daily 7 --keep-weekly 4 --keep-monthly 6 --prune

# 4. Verify structure and a random 10% of the data.
restic check --read-data-subset=10%

bash -n accepted the syntax. I ran it twice against a scratch stack folder and a local repository, overriding the paths with environment variables (STACK, COMPOSE_FILE, EXCLUDES, RESTIC_ENV). First run:

no parent snapshot found, will read all files

Files:           4 new,     0 changed,     0 unmodified
Dirs:            7 new,     0 changed,     0 unmodified
Added to the repository: 216.407 KiB (210.729 KiB stored)

processed 4 files, 212.599 KiB in 0:00
snapshot ca9ed631 saved
Applying Policy: keep 7 daily, 4 weekly, 6 monthly snapshots
keep 1 snapshots:
...
no errors were found

The second run found one changed file:

using parent snapshot ca9ed631

Files:           0 new,     1 changed,     3 unmodified
Dirs:            0 new,     6 changed,     1 unmodified
Added to the repository: 109.091 KiB (105.978 KiB stored)

processed 4 files, 212.599 KiB in 0:00
snapshot 46fb6598 saved
Applying Policy: keep 7 daily, 4 weekly, 6 monthly snapshots
keep 2 snapshots:
...
read 10.0% of data packs
[0:00] 100.00%  1 / 1 packs

no errors were found

Unchanged files (compose file, secrets file, the older dump) cost nothing the second time. The fresh appdb.dump was added in full: 109 KiB again.

Why didn't the database dump deduplicate?

I tested it, because dump files are the usual surprise. I ran pg_dump twice back to back, in two formats, straight into restic with --stdin. Both times the second run added the whole file again (custom format: 106.266 KiB both times; plain SQL: 307.584 KiB both times). Diffing two plain dumps showed why for the plain case: the only differences were two lines, \restrict <random token> and \unrestrict <random token>, which pg_dump 18.6 writes with a fresh random value each time. For the custom format I didn't find the cause; it is already compressed, which tends to defeat deduplication.

The caveat is size: these files are tiny, so a one-line change rewrites the whole file. I didn't measure how well a multi-gigabyte dump deduplicates between nights. Budget for each night's dump being stored in full until you've measured your own repository (restic stats and the "Added to the repository" line tell you).

How do you restore from a restic snapshot?

List what's in a snapshot, then restore. Paths are restored under the target with their full original path:

restic ls latest
restic restore latest --target /srv/restore-test

Real output of the restore (scratch paths):

restoring <Snapshot 46fb6598 of [/tmp/H2/restic/srv/ai] at 2026-10-09 07:14:39 ...> to /tmp/H2/restic/restored-all
Summary: Restored 11 files/dirs (212.599 KiB) in 0:00

Then I compared the restored tree with the original using diff -r. The only difference was the .cache folder I had put there on purpose to test the exclude rule:

Only in /tmp/H2/restic/srv/ai/stack: .cache

You can restore a single file with --include. I restored only the dump:

restic restore latest --target /srv/restore-test --include /srv/ai/backups/appdb.dump

That printed Restored 7 / 1 files/dirs (105.981 KiB / 105.981 KiB). The 7 counts the parent directories restic recreates.

Prove the database restore works

A file that restores byte for byte can still be a bad backup. So the last step of the drill is loading the dump into a database and counting rows. I copied the restored dump to a second Postgres container (same image, empty volume), created a database, and ran:

docker compose exec -T db createdb -U app restoretest
docker compose exec -T db pg_restore -U app -d restoretest --no-owner /backups/from-restic.dump
docker compose exec -T db psql -U app -d restoretest -c "SELECT count(*) FROM chunks"

The count came back 5001, the same as the source. If you run pgvector, restore into the same image or the extension statement fails; the pgvector post shows that error.

How do you schedule it and know it ran?

Cron is enough (unrun template; adjust the path and the user):

23 2 * * * RESTIC_ENV=/etc/restic/env /usr/local/bin/backup-ai-server.sh >> /var/log/backup-ai-server.log 2>&1

Cron will not tell you when the job stops. Add a dead-man check: have the script call a push monitor URL on success, and alert when it doesn't arrive. The Uptime Kuma section of the Vaultwarden post shows how to watch the server; note that a monitor on the same VPS dies with the VPS, so a free external uptime service or a second small machine is a better home for this one.

A systemd timer is the other option, and the trade-offs between systemd and Docker for background jobs are in the systemd vs Docker post.

What can go wrong?

• Backing up a running database's files. Copying the Postgres data directory while it runs can give you a backup that won't start. Use the dump.
• Backing up the key but not the data, or the data but not the key. Back up both, and keep the key somewhere separate too.
• Everything on one disk. A repository on the same VPS disappears with the VPS. Use another host or a bucket.
• A password only on the server. If the server is the only place the restic passphrase exists, a dead server means a dead backup.
• Pruning blindly. forget --prune deletes data. Run it with --dry-run first when you change a policy. I ran forget --keep-last 1 --dry-run in an earlier pass and it listed the snapshot it would remove and removed nothing.
• Never testing. Put a calendar reminder for the restore drill. Rehearse after any stack change.

What did I not verify?

• Remote backends (SFTP, S3, B2, rclone). I used a local repository; the URL forms come from restic's docs page.
• Restore speed and deduplication on gigabyte-sized data.
• Backup of tool-specific volumes such as n8n, Open WebUI or Qdrant. Follow each tool's own docs.
• The cron line and the behaviour of newer restic versions than 0.16.4.

Checklist

• List of what to back up, what to skip, and which keys to store separately.
• restic repository on a different machine or bucket than the server.
• Repository passphrase stored in a password manager and one other place.
• Postgres dumped with pg_dump inside the container, not by copying files.
• Script run by cron or a timer, with a success ping to something off the server.
• Retention set and tried with --dry-run.
• restic check --read-data-subset in the nightly job.
• Full restore drill done once, including loading the database and running a real query.