Mock LLM Responses in pytest With VCR Record and Replay

You run the test suite and it spends real money. Or it passes on Tuesday and fails on Wednesday because the model phrased something differently. Or it doesn't run at all on a train because there's no network.

The fix for all three is the same: record one real HTTP exchange with the model API, save it to a file, and replay it in every later run. In Python that's vcrpy plus pytest. This post shows a complete, working setup, including the part most tutorials skip: making sure a changed prompt fails the test instead of silently passing against an old answer.

Everything below was run on 2026-10-30 with Python 3.14, pytest 9.2.0, vcrpy 9.2.0 and httpx 0.28.1. The "model" is a tiny local HTTP stub I wrote, not a real LLM, so you can reproduce it offline. I did not run it against a live vendor API; the live-call step is labelled where it appears.

What does record and replay do for LLM tests?

You record once, replay forever. On the first run, vcrpy lets the real request through and writes the request and response to a YAML file (a cassette). On later runs it intercepts the HTTP call and answers from the file. Your code can't tell the difference, and no network is needed.

The trade-off is honest and worth stating: a cassette proves your code handles that one recorded reply. It does not prove the model still behaves that way. Record and replay tests your plumbing. Judging model quality belongs to evals; see agent evals from scratch and prompt regression tests in CI for that side.

Step 1: the code under test

A small function that calls a chat-style endpoint and returns the reply text. Note it reads the API key from the environment and sends it in an Authorization header, because that's exactly what will try to leak into your cassette.

# app.py
import os

import httpx


def summarize(text: str, base_url: str) -> str:
    resp = httpx.post(
        f"{base_url}/v1/chat",
        headers={"Authorization": f"Bearer {os.environ.get('LLM_API_KEY', 'test-key')}"},
        json={
            "model": "stub-model",
            "messages": [{"role": "user", "content": f"Summarize in one line: {text}"}],
        },
        timeout=5,
    )
    resp.raise_for_status()
    return resp.json()["reply"].strip()

The URL shape and the reply field are my stub's, not a vendor's. Your real provider's path and response schema will differ; the testing pattern doesn't.

Step 2: a local stub to record from

For the recording step I used a 30-line http.server stub that returns a fixed reply and counts how many requests it served. In your project, the recording step is where you'd point at the real API with a real key. That step is unrun here.

# stub_server.py (trimmed)
import json, threading
from http.server import BaseHTTPRequestHandler, HTTPServer


class Stub:
    def __init__(self, port=8765):
        self.hits = 0
        stub = self

        class Handler(BaseHTTPRequestHandler):
            def do_POST(self):
                body = json.loads(self.rfile.read(int(self.headers["Content-Length"])))
                stub.hits += 1
                text = body["messages"][-1]["content"]
                data = json.dumps({"reply": f" STUB SUMMARY of {len(text)} chars "}).encode()
                self.send_response(200)
                self.send_header("Content-Type", "application/json")
                self.send_header("Content-Length", str(len(data)))
                self.end_headers()
                self.wfile.write(data)

            def log_message(self, *a):
                pass

        self.server = HTTPServer(("127.0.0.1", port), Handler)
        self.thread = threading.Thread(target=self.server.serve_forever, daemon=True)

    def start(self):
        self.thread.start()
        return self

    def stop(self):
        self.server.shutdown()
        self.server.server_close()

Step 3: configure VCR to match on the prompt, not the port

Defaults are the trap. Out of the box vcrpy matches on method, scheme, host, port, path and query, but not the request body. That means two different prompts to the same URL would get the same recorded answer, which is the opposite of what you want for an LLM call.

Match on the method, the path and the body. Leave host and port out so the cassette still works when the base URL changes between your laptop, CI and a gateway.

# test_replay.py
import os
from pathlib import Path

import pytest
import vcr
from vcr.errors import CannotOverwriteExistingCassetteException

from app import summarize
from stub_server import Stub

CASSETTES = Path(__file__).parent / "cassettes"
RECORDING = bool(os.environ.get("RECORD"))

my_vcr = vcr.VCR(
    cassette_library_dir=str(CASSETTES),
    match_on=["method", "path", "body"],
    filter_headers=["authorization"],
    record_mode="once" if RECORDING else "none",
)

Two choices carry the weight here:

• filter_headers=["authorization"] stops vcrpy writing the header to disk.
• record_mode is none unless you set RECORD=1. In none mode a request with no matching recording is an error, never a live call. Your default pytest run can then never spend money or hit the network by accident.

Step 4: the tests

def test_summarize_replays_from_cassette(monkeypatch):
    monkeypatch.setenv("LLM_API_KEY", "sk-this-must-not-be-saved")
    stub = Stub().start() if RECORDING else None   # server only exists while recording
    try:
        with my_vcr.use_cassette("summarize.yaml"):
            out = summarize("The quarterly numbers were fine.", "http://127.0.0.1:8765")
        assert out == "STUB SUMMARY of 55 chars"
    finally:
        if stub:
            stub.stop()


@pytest.mark.skipif(RECORDING, reason="replay-only check")
def test_changed_prompt_fails_loudly():
    with pytest.raises(CannotOverwriteExistingCassetteException):
        with my_vcr.use_cassette("summarize.yaml"):
            summarize("A different input than the one recorded.", "http://127.0.0.1:8765")


def test_cassette_has_no_secret():
    text = (CASSETTES / "summarize.yaml").read_text()
    assert "sk-this-must-not-be-saved" not in text
    assert "authorization" not in text.lower()

Record once:

RECORD=1 pytest -k replays

Real output:

.                                                                        [100%]
1 passed, 2 deselected in 0.63s

Then run everything with the stub server not running, which proves the replay doesn't touch the network:

test_replay.py::test_summarize_replays_from_cassette PASSED [ 33%]
test_replay.py::test_changed_prompt_fails_loudly PASSED [ 66%]
test_replay.py::test_cassette_has_no_secret PASSED [100%]

============================== 3 passed in 0.10s ===============================

Port 8765 was closed on that run, so the first test could only have passed from the cassette. The recorded file contains the request body, the stub's reply and the headers, with no authorization entry.

What happens when the prompt changes?

This is the check that makes the cassette trustworthy. I changed the input text and ran with record_mode="none". vcrpy refused and told me why:

CannotOverwriteExistingCassetteException
Can't play back this request: no recorded interaction matches it (the closest differ in body).
record_mode 'none' doesn't record new requests.

  POST http://127.0.0.1:8765/v1/chat
  body: {"model":"stub-model","messages":[{"role":"user","content":"Summarize in one line: A different input."}]}

Cassette: /tmp/Q1/p1/cassettes/summarize.yaml
  record_mode: none
  match_on: method, path, body
  interactions: 1 recorded, 0 played

Closest recorded requests:
  1. POST http://127.0.0.1:8765/v1/chat
     body differs:
       --- recorded
       +++ this request
       @@ -3,3 +3,3 @@
            {
       -      "content": "Summarize in one line: The quarterly numbers were fine.",
       +      "content": "Summarize in one line: A different input.",
              "role": "user"

That diff is the whole point. Edit a prompt template and every test that depended on the old wording fails with a readable explanation, instead of passing against a stale answer. The fix is to re-record on purpose (new_episodes adds the new exchange; all with drop_unused_requests=True rebuilds the file) and then read the cassette diff in your pull request.

One gotcha I hit: I first ran my error-printing script with a relative cassette_library_dir, from a different working directory. vcrpy then reported that "the cassette doesn't exist", which is a misleading way to say "I looked in the wrong folder". Use an absolute path built from __file__, as above.

Which requests should never be in a cassette?

Decide before the first recording:

Rule of thumb: record from fake data you wrote for the purpose. If you must record from real data, scrub it in a hook and review every cassette like source code.

Keeping cassettes from going stale

Cassettes rot in two ways. The request changes (caught, as shown above), or the provider changes its reply format and your recording keeps passing against a shape that no longer exists. The second one is invisible to replay.

Two cheap defences:

1. Run a small live smoke test on a schedule, not on every commit. Mark it @pytest.mark.live, skip it unless an env var is set, and keep it to one or two calls. This step is unrun in this post because it needs a real key.
2. Validate recorded replies against the same schema your production code expects. If you ask for structured outputs, assert the recorded reply parses with the real parser.

A cassette that was recorded three months ago and never refreshed is a claim about the past. Put the recording date in the filename or a comment if that helps your team notice.

Where this fits with the rest of your tests

• Pure logic (parsers, routers, formatters): plain unit tests with hand-written strings. See writing unit tests with AI and checking them with mutation testing.
• Request plumbing (headers, retries, timeouts): cassettes, as here. If your code retries on 429s, hand-write a fake that returns a 429 first; a cassette recorded from a healthy API won't contain one. The retry logic itself is covered in rate limits, retries and backoff for LLM APIs.
• Model behaviour: evals on a golden set. See building evaluation datasets and golden test sets.

Checklist

• Default test run uses record_mode="none", so it can't call out.
• match_on includes body; host and port are excluded.
• Auth headers filtered, and a test asserts the secret is not in the file.
• Cassettes recorded from synthetic input only.
• Re-recording is a deliberate command, and the cassette diff is reviewed.
• One scheduled live smoke test (unrun here) catches provider drift.

I also installed pytest-recording, a pytest plugin that wraps vcrpy with a marker and command-line flags, but I did not exercise it, so I can't vouch for its details. The plain vcrpy setup above is the part I ran.