The pull request merged on Tuesday. The release notes are due Thursday, the docs page still describes the old behaviour, and the only explanation of what changed is a PR description that says "refactor shipping calc, fix edge cases." You open the diff and the function is 40 lines with three new branches.
This is where AI prompts for technical writers earn their keep, and where they quietly cause damage. A model can turn code into a parameter table in seconds. It can also add a currency parameter that doesn't exist, because shipping functions usually have one. The fix isn't a better prompt alone. It's a prompt that forbids invention plus a check you run against the source.
These are untested templates, and the code sample below is invented. What I did run is a small Python script that compares a docs draft with the function it documents. Its real output is shown. For reviewing the code itself rather than documenting it, see AI code review prompts that actually catch bugs.
How do I draft docs from code without the model inventing things?
Give the model the code and a closed world. The key instruction is that every sentence must be supported by something in the pasted source, and that unsupported things are flagged, not smoothed over.
You are drafting reference documentation for a developer audience.
SOURCE (the only truth; do not use general knowledge of similar APIs):
[PASTE the function/class, with signatures and docstrings]
AUDIENCE: [who, what they already know]
FORMAT: [heading, one-sentence summary, parameter table with Name |
Default | Description, return value, raised errors, one example]
Rules:
- Document only parameters, defaults, return values and errors visible
in the source. Do not add any.
- Defaults must be copied exactly from the signature.
- For each raised error, say the condition from the code.
- Every example call must be one a reader can run. Mark expected output
as [RUN TO CONFIRM] unless you can derive it from the code by hand;
show your derivation if you do.
- If the source does not explain WHY something is the way it is, write
[NOT IN CODE: ask the author] instead of guessing a reason.
- After the doc, list questions for the author.
The [NOT IN CODE] flag gives you the author interview. A function that multiplies by 1.5 when express is true leaves unanswered whether that is a published surcharge or a placeholder. Public docs shouldn't guess.
How do I check a docs draft against the code?
Reading carefully is not enough. People skim tables they didn't write. A cheap mechanical check catches three common errors: a parameter in the docs that isn't in the code (or the reverse), a wrong default, and an example whose output is wrong.
Here's the invented sample I used, a 14-line function:
def shipping_cost(weight_kg, zone="domestic", express=False, insured_value=0.0):
if weight_kg <= 0:
raise ValueError("weight_kg must be positive")
if zone not in ("domestic", "regional", "international"):
raise KeyError(zone)
base = {"domestic": 4.0, "regional": 7.5, "international": 15.0}[zone]
cost = base + 1.2 * weight_kg
if express:
cost *= 1.5
cost += insured_value * 0.01
return round(cost, 2)
And a docs draft of the kind a model might hand you (I wrote the three errors in on purpose, to see whether the check finds them):
| Parameter | Default | Description |
|---|---|---|
| weight_kg | required | Parcel weight in kilograms |
| zone | "domestic" | One of domestic, regional, international |
| express | True | Applies the express surcharge |
| insured_value | 0.0 | Declared value to insure |
| currency | "USD" | Currency code |
Raises ValueError if weight_kg is not positive. Raises KeyError for an
unknown zone.
>>> shipping_cost(2)
6.4
>>> shipping_cost(2, express=True)
9.6
>>> shipping_cost(2, zone="regional")
11.0
The checker is about 60 lines: it loads the module, reads the signature with inspect, parses the docs table, finds raise statements with ast, and re-evaluates each >>> call against the stated result. The core of it:
sig = inspect.signature(func)
actual = {n: ("required" if p.default is inspect.Parameter.empty else repr(p.default))
for n, p in sig.parameters.items()}
# documented = {name: default} parsed from the markdown table
for name in actual:
if name not in documented:
problems.append(f"UNDOCUMENTED parameter: {name}")
for name, d in documented.items():
if name not in actual:
problems.append(f"DOCUMENTED but not in code: {name}")
elif d.replace('"', "'") != actual[name].replace('"', "'"):
problems.append(f"DEFAULT mismatch for {name}: docs say {d}, code says {actual[name]}")
# raised exception types found via ast.walk are checked against the docs text
# each ">>> call" is eval'd against the module and compared with the next line
Output on the draft above (real run, 2026-10-09):
checked 5 documented params, 3 examples, 2 raised exception types
FAIL: DEFAULT mismatch for express: docs say True, code says False
FAIL: DOCUMENTED but not in code: currency
FAIL: EXAMPLE mismatch: shipping_cost(2, zone="regional") -> docs 11.0, actual 9.9
3 problem(s)
All three planted errors found. After fixing the table (default False, currency row removed) and the example (9.9), the same script prints checked 4 documented params, 3 examples, 2 raised exception types then OK.
Be clear about the limits. This check proves the docs agree with the code. It does not prove the code is right, and it can't check prose such as "applies the express surcharge." It only handles a plain function with a simple table. A real project needs an adapter per language, or you lean on tools that already extract signatures (OpenAPI validators, doctest, typed-docs generators). The principle transfers: make a machine compare the claims that a machine can compare, and spend human attention on the rest. I did not test this script on other languages or on classes with decorators.
One more trick. Run the model's example calls yourself before trusting any output it shows. In this draft the wrong 11.0 looked entirely reasonable (7.5 + 2.4 = 9.9, so someone had rounded up or guessed).
How do I turn a PR into release notes and a changelog entry?
A commit message tells you what a developer did. A changelog tells a user what to do. These are different documents. The Keep a Changelog convention (version 1.1.0, which I read on 2026-10-09 at keepachangelog.com; it also recommends ISO dates like 2026-10-09) puts it plainly: changelogs are for humans, not machines; there should be an entry for every version; similar changes are grouped; the latest version comes first; and the release date is shown. It names the groups Added, Changed, Deprecated, Removed, Fixed and Security. That's a convention, not a standard your project must follow, but it's a sensible default.
Write a changelog entry for version [X.Y.Z], dated [DATE], from the
material below. Use the groups Added, Changed, Deprecated, Removed,
Fixed, Security, omitting empty ones.
MATERIAL (only source): PR titles and descriptions, linked issues, and
the diff summary:
[PASTE]
Rules:
- Write for the user of the software, not the developer. Say what is
different for them and, if action is needed, what to do.
- One line per change, plus a link to the PR in the form [#123].
- Anything that breaks existing usage goes first under Changed or
Removed and starts with "Breaking:". Explain the migration in one
sentence, only if the PR describes it; otherwise write
[MIGRATION NOT DESCRIBED: ask the author].
- Do not claim a performance gain, fix or security impact that the PR
text does not state. If a PR is unclear, list it under "Needs
clarification" with the question.
- Do not include purely internal refactors unless they affect users.
The "Breaking:" rule and the clarification list do the heavy work. The model tends to merge a breaking change into a friendly "Improved shipping calculation," which is the single worst changelog sentence you can ship. Use the model for drafting; the engineer who wrote the change signs off the breaking-change wording.
What can I do with a doc review checklist?
Use the model as a reviewer with a rubric, on docs written by humans or other models. Keep the checks concrete.
Review this documentation page against the checklist. For each item
answer PASS, FAIL or CANNOT TELL, with the quoted sentence.
1. First paragraph says what this is for and who it is for.
2. Prerequisites are listed before the first step.
3. Every step is one action, in order, starting with a verb.
4. Every command or code block is complete and copyable (no "...").
5. Every term used before it is defined, or links to a definition.
6. Expected result is shown after each major step.
7. Error cases the code can produce are documented with a fix.
8. Version or date the page applies to is stated.
9. No sentence that describes behaviour not shown in the source I
gave (list each suspect).
Do not rewrite. Do not mark PASS if you cannot tell from the text.
PAGE: [PASTE]
SOURCE FOR FACT-CHECKING (optional): [PASTE]
Item 4 and 6 often fail on pages that "look done." And CANNOT TELL is an honest outcome for anything that needs the running product. A new teammate following the page cold, with a timer, finds more than any checklist, so schedule that test for anything important.
How do I keep the prompt and the docs in sync with the project?
Put a short context block at the top of every docs prompt, so you don't re-explain your style each time. The same idea applies to coding assistants; see AGENTS.md vs CLAUDE.md vs .cursorrules.
STYLE: second person, present tense, active voice. Sentence-case headings.
Code identifiers in backticks. No "simply", "just", "easily".
Terms: use "workspace" not "project"; "API key" not "token".
Do not describe future plans or unreleased features.
Keep a glossary of your own terms in it. Models drift toward the generic word, and a docs set that calls the same thing three names is harder to search.
A worked example (illustrative)
An invented PR: "Add insured_value to shipping_cost; fix negative weight handling." Diff shows a new insured_value=0.0 parameter adding 1% of its value to cost, and a new ValueError for non-positive weight, where previously a zero weight returned the base price.
A correct changelog draft shape:
## [2.4.0] - [DATE]
### Added
- `shipping_cost` accepts `insured_value` (default 0.0); the cost increases
by 1% of the value you declare. [#212]
### Changed
- Breaking: `shipping_cost` now raises `ValueError` when `weight_kg` is
zero or negative. Before, zero weight returned the base price for the
zone. Validate weight before calling. [#212]
Needs clarification: Is 1% a published insurance rate or a placeholder?
The last line is the model correctly refusing to describe business meaning that is not in the diff.
Where this goes wrong
| Failure | What it looks like | Fix |
|---|---|---|
| Invented parameter or default | A currency row that isn't in the code | Closed-world prompt; signature comparison script |
| Plausible wrong example output | 11.0 where the code gives 9.9 | Run every example; do not accept derived numbers |
| Breaking change hidden | "Improved validation" | "Breaking:" rule; engineer signs off |
| Reasons guessed | "Defaults to False for performance" | [NOT IN CODE] flag; ask the author |
| Docs describe unreleased code | Branch ahead of release | State version; paste the tagged source |
| Style drift | Three names for one concept | Style and glossary block in every prompt |
| Secrets in the prompt | Keys, internal hosts in pasted code | Strip them; approved tools only |
What not to automate
- Decisions about what the product promises. Stability, deprecation and support statements are the owner's call.
- Security and compliance wording. An engineer or security lead must approve statements about what is safe.
- Release notes for breaking changes. The author who knows the migration signs the wording.
- The first-run test. Someone following the docs cold, with a clean environment, finds what no prompt will.
- Truth that is not in the code. Behaviour in production, performance numbers and customer impact come from measurement, not generation.
For the principle behind the closed-world rule, read avoiding hallucinations, and XML tags and delimiters helps when you paste code, rules and a draft in one prompt. The prompt library has more templates to copy.



