"Tell me about a project you've built" is the easiest question to prepare for and the one most people answer badly. They list technologies ("I used Python, LangChain, FAISS and OpenAI") in the order they installed them. The interviewer learns the stack and nothing about judgment.
This post gives you a five-part structure to explain a RAG project in an interview, a script template you can fill in, a worked example that is clearly fictional, and a drill for the follow-ups that come next. It assumes you have built something, even small. If you have not, build the beginner RAG chatbot first and break it; you need real stories to tell, and no structure replaces them.
I haven't sat on a hiring panel for this post, and I'm not reporting what any company asks. The structure is a communication pattern, built to answer the questions anyone evaluating a system would ask: what problem, what decisions, how you know it works, where it breaks.
What is the best structure to explain a RAG project?
Five parts, about two minutes total. Call it PDMFL: Problem, Design, Measurement, Failure, Lesson.
| Part | Time | Answers the interviewer's question | Contains |
|---|---|---|---|
| Problem | 20 s | Why does this exist? | Who asks, what documents, what a wrong answer costs |
| Design | 40 s | What did you decide? | 2 or 3 decisions with a reason each, not a tool list |
| Measurement | 25 s | How do you know it works? | Test set size, retrieval vs answer numbers |
| Failure | 25 s | What went wrong? | Symptom, cause, fix, verification |
| Lesson | 10 s | What would you do differently? | One concrete change |
The order matters. Opening with the problem makes the design choices make sense. Putting measurement before failure lets the failure land as a continuation of the evidence instead of a confession.
The script template
Fill the brackets with your own facts. If you cannot, that gap is what to build next.
PROBLEM
"[Who] needed to ask questions about [document set, N docs] and a wrong
answer would [consequence]. So the goal was [grounded answers with
citations / saying 'not found' when unsure]."
DESIGN
"Three decisions mattered. First, [chunking choice] because [reason I
observed]. Second, [retrieval choice] because [reason]. Third,
[generation choice, e.g. refusal rule] because [reason]."
MEASUREMENT
"I wrote [N] questions with the expected source passage. Retrieval put
the right chunk in the top [k] for [x of N]. Answers were correct for
[y of N], graded by [me / rubric / model plus spot checks]."
FAILURE
"The worst failure was [symptom]. I found the cause by [looking at the
retrieved chunks / ...]: [cause]. I fixed it by [change], and
re-running the same [N] questions moved [metric] from [a] to [b]. I
added [the case] to the test set."
LESSON
"If I did it again I would [change], because [evidence]."
Rules for filling it:
- Reasons beat names. "I split on headings because fixed 500-character cuts separated rule numbers from their text" shows you looked at your data. "I used a recursive text splitter" does not.
- Two numbers, kept separate. Retrieval and answer correctness are different measurements. Saying both shows you can tell which component failed. The evaluation metrics interview guide shows how to compute recall@k and related numbers, and the AI engineer question list has the evaluation questions themselves.
- Denominators always. "17 of 20", not "85%". Small sets are fine when you say they are small.
- First person, honest. "I" for what you decided. "My teammate" for what they did. If AI tools wrote parts of the code, say so if asked, and be able to explain those parts.
A worked example (fictional)
The project below is invented to show the shape. It is not a real project, the numbers are made up, and you must not reuse it as your own. Replace every fact with yours.
Fictional project: "PolicyPal", a Q&A bot over a made-up company's 40-page HR policy PDFs.
Problem. "A small company's staff kept asking HR the same leave and reimbursement questions. A wrong answer about leave could cause a dispute, so the goal was answers with a page citation, and 'I can't find that' when the policy didn't say."
Design. "Three decisions mattered. I split on section headings instead of fixed lengths, because in my first version a reimbursement limit got separated from the sentence naming the role it applied to. I used hybrid search, keyword plus embeddings, because staff typed exact policy codes like 'L-4' and embeddings handled those poorly. And I made the prompt refuse unless a retrieved chunk supported the answer."
Measurement. "I wrote 30 questions with the page that answers each, plus 8 out-of-scope questions. The right page was in the top 4 results for 26 of 30. Answers were correct for 23 of 30, graded by me against a one-page rubric. It declined 6 of 8 out-of-scope questions correctly."
Failure. "The worst failure was a confident wrong answer about carry-over leave. Looking at the retrieved chunks, the right section was retrieved, but a superseded 2021 policy chunk ranked above it and the model used that. The cause was stale data, not the prompt. I added the document date to chunk metadata, filtered out superseded files, and re-ran the 30 questions: answers correct went from 23 to 25. I added that case to the test set."
Lesson. "I'd write the test set before building, not after. I tuned chunking by feel for two days before I had any numbers to tell me whether it helped."
Read it aloud: about two minutes, and every claim has a reason or number attached. Notice what is absent: no framework names beyond what mattered, no adjectives like "robust".
Why this works
The failure story is the strongest part because it contains a diagnosis path (look at retrieved chunks first), a root cause that isn't the obvious one, a verification and a regression test. That is the same debugging order covered in the debugging a wrong RAG answer question. You don't need a dramatic failure. You need a real one told in that order.
Prepare the failure stories
Collect two or three from your own project. For each, write five lines:
- Symptom the user would have seen.
- How I found it (the log, the chunk, the test).
- Cause, stated as a fact you confirmed or a hypothesis you labelled as one.
- Fix, and the metric before and after on the same questions.
- Guard added: a test, a filter, a monitor.
If you can't remember any, run your pipeline against ten awkward questions this evening: a question whose answer spans two documents, one with a typo, one asking about something that isn't in the corpus. Breakage will turn up quickly. Record it in your repo's known-failures list, which the project README template includes a place for.
The follow-ups to rehearse
After your two minutes, expect an interviewer to pick on the weakest or most interesting point. These are patterns, not a leaked list, and you can write answers to each from your own project.
| They might ask | What a prepared answer contains |
|---|---|
| "Why that chunk size?" | What you tried, what you measured, what you'd test next |
| "How did you get the test set?" | Where questions came from, who labelled, what it misses |
| "How would this change at 100x the documents?" | One or two honest bottlenecks (index refresh, reranking latency, cost), not a rewrite |
| "What if two documents conflict?" | Metadata, dates, precedence rules, or surfacing both |
| "How would you stop it leaking restricted documents?" | Access metadata filtered at retrieval time, not in the prompt |
| "Did you use AI to write this?" | A direct, specific answer, and understanding of every file |
| "What doesn't it do?" | Your known failures list |
When you don't know, say what you would measure to find out. That reads better than guessing, and the AI engineer question list flags the same pattern.
Practise it with a model, without letting it flatter you
Paste your filled template into a chat and use this untested prompt.
Act as a skeptical interviewer. Here is my two-minute project story.
1. List every claim that has no number, reason or evidence behind it.
2. Ask the three follow-up questions you think are hardest, one at a
time, and wait for my answer to each.
3. After the third answer, tell me which of my answers contained a
claim I could not support, quoting the sentence.
Do not praise. Do not suggest facts about my project that I have not
given you.
Then run the timing yourself: record the answer on your phone and check it lands near two minutes. For a full run including a design question and a score sheet, use the mock interview script. The design-question side is covered in the RAG system design walkthrough.
Before the interview
- Know your own numbers cold: corpus size, chunk size, test set size, the two metrics.
- Have one failure story ready in the five-line shape.
- Open your repo and make sure the README matches what you will say.
- Decide your one-sentence answer for "what would you change".
Say less, back it with a number, and let them ask.
