Someone opens your repo with ninety seconds to spare. If the README opens with an installation block and a list of technologies, they learn how you built it and nothing about whether it works. This AI engineer project README template reorders things so the answers a reviewer wants come first, and gives you a section for the thing most repos leave out: how it fails.
The template is a structure I designed around what a reader needs to judge a project quickly; I did not survey reviewers. It is not taken from any employer's hiring guide, and I have no data on what raises callback rates. For project ideas to put it on, see AI portfolio projects that get interviews. This post is the document that goes with them.
What should the top of an AI project README say?
Three lines, before any heading about setup:
- What the project does, for whom, in one sentence.
- One result with its denominator.
- How to try it in under two minutes.
Compare two openings. Weak: "A chatbot built with Python, LangChain and a vector database." Stronger: "Answers questions about the (fictional) Acme refund policy from 12 PDFs. Retrieval finds the right page in the top 4 for 17 of 20 test questions. make demo runs it locally with a stub model, no API key needed."
The stronger version names a scope, states a count you can check against a committed file, and removes the key as a barrier. The numbers in that example are placeholders to show the shape, not real results.
The copy-paste template
Replace every square bracket. If you cannot fill a bracket honestly, delete the line instead of writing something vague.
# [Project name]: [what it does, who for, in one sentence]
[Screenshot or 30-60 second recording]
**Status:** [prototype | working demo | used by N people since DATE]
**Last verified to run:** [date] on [OS, Python/Node version]
## Result at a glance
| Metric | Value | n | How measured |
|---|---|---|---|
| [Retrieval hit rate@4] | [x of n] | [n] | [`eval/questions.jsonl`, `python eval/run.py`] |
| [Answer correct] | [x of n] | [n] | [rubric in `eval/RUBRIC.md`, graded by me / model + spot-checked] |
| [Correct refusals on out-of-scope] | [x of n] | [n] | [`eval/out_of_scope.jsonl`] |
| [Median latency / cost per query] | [value] | [n runs] | [how you timed or counted tokens] |
## Run it
1. `cp .env.example .env` (add [KEY_NAME], or set `LLM_MODE=stub` to run without a key)
2. `[install command]`
3. `[run command]`
4. Expected output: [paste a short real output]
## What it does
[3 to 5 sentences. Inputs, outputs, the main flow. A diagram if it helps.]
## How it works
- Data: [what documents or tables, how many, where from, license]
- Chunking / schema: [decision and size, and why]
- Model(s): [name the models you actually called]
- Prompt files: [path]
- Guardrails: [what you validate, what you refuse]
## Evaluation
[How the test set was built, who labelled it, what it does not cover.
Link the files. Report what changed between versions.]
| Version | Change | Hit rate@4 | Answer correct |
|---|---|---|---|
| v1 | [baseline] | [x of n] | [x of n] |
| v2 | [one change] | [x of n] | [x of n] |
## Known failures
1. [Input that breaks it] -> [what happens] -> [why, as far as you know]
2. [...]
3. [...]
## Decisions and trade-offs
- [Chose X over Y because Z. What would make me switch.]
## Not done / next
- [Honest list. Reviewers read this as judgment.]
## Which parts were AI-assisted
[One or two sentences: what tools, what you wrote yourself, what you changed.]
## Security and cost notes
- Secrets: `.env` is git-ignored; [key has a spend cap of $X]
- Data: [no personal data committed; samples are synthetic/public]
## License and credits
Why each section is there
Status and last verified. Dependencies rot. A date tells a reviewer whether "it runs" is a current claim.
Result at a glance. The n column is the point. A hit rate without a denominator is not evidence. If your set has 20 questions, say 20, and say what it does not cover. A tiny set is fine; a hidden one is not.
Run it, with a stub mode. Reviewers rarely have your API key and won't set up billing to look at a portfolio project. A stub mode that returns canned model outputs lets the pipeline run end to end. The stub must be labelled as a stub in the README so nobody mistakes canned output for model output.
Evaluation. The before/after table is your best evidence of engineering habit. It shows you changed one thing and measured it. The approach is in building evaluation datasets. Report retrieval hit rate separately from answer correctness so a reader can see which stage failed, and see chunking strategies compared for what to vary between versions.
Known failures. Write three. The real value is the "why": "scanned PDFs return empty text because there is no OCR step" shows you diagnosed it. If you cannot produce three, you have not tested hard enough. Feed those failures into your interview answers: the story structure is in how to explain a RAG project in an interview.
Which parts were AI-assisted. Most engineers use coding assistants now. What matters is that you can defend every file. A short honest line beats a reviewer guessing.
Repo layout that makes the README true
A README that points to things that are not there is worse than a short one. A layout that supports the template:
project/
README.md
.env.example
Makefile # make demo, make eval
src/ # app code, small modules
prompts/ # prompt files, versioned
eval/
questions.jsonl # committed test set
RUBRIC.md # how answers were graded
run.py # one command to reproduce the table
results/ # dated outputs, not overwritten
data/sample/ # synthetic or public sample only
docs/design.md # long-form decisions
Two rules make this work. Every number in the README must be reproducible from a command in the repo, or else flagged as measured by hand on a specific date. And eval/results/ keeps old runs so the before/after table can be checked against raw output.
A pre-publish checklist
Run through it with the repo open in a private browser window, logged out, as a stranger would see it.
- First screen answers: what, for whom, one result with
n. - Clone into a fresh folder and follow "Run it" exactly. Fix every step that failed.
- No secrets in the working tree or git history. Search before pushing: a key that was ever committed should be treated as leaked and rotated, because deleting the file does not remove it from history.
- No personal data, client data or scraped content you lack rights to.
- Every number traces to a file or command.
- Known failures has at least three entries.
- Spend cap set on any key used by a deployed demo.
- You can explain each file aloud in one sentence.
Fill it in with a model, then edit hard
A prompt that drafts the skeleton from your own notes. It is an untested template, so read the output critically. It must not invent numbers.
Below are my raw notes about a project and the files in the repo.
Draft a README using the section order: result at a glance, run it,
what it does, how it works, evaluation, known failures, decisions,
not done, AI-assisted note.
Rules:
- Use ONLY facts and numbers present in my notes. Where a number is
missing, write [MISSING: what I need to measure] instead of guessing.
- Never use the words "robust", "scalable", "production-ready" or
"state-of-the-art".
- Each known failure needs: input, behaviour, probable cause.
NOTES:
[paste]
FILE TREE:
[paste]
Every [MISSING: ...] it prints is a task. That list tells you what to measure before you publish. Then check the draft against the repo; a README that claims a feature the code lacks will be found.
After publishing
Pin only your best two or three repos. Rehearse a two-minute walkthrough of each, and run the mock interview script with your README as the project block: if you stumble on your own table, the table needs simplifying.
