"Design an AI agent that handles customer refund requests." The weak answers start with a framework and a diagram of five boxes labelled "planner", "memory" and "tools". The strong ones start by asking whether this needs an agent, what the worst action it could take is, and how anyone would find out it went wrong.
This post is a practice walkthrough for the AI agent design interview: tools, memory, loops, guardrails, evals and human-in-the-loop, in an order you can follow aloud. It is a practice structure, not a set of questions from any real company. For rapid-fire Q&A, use AI engineer interview questions on RAG, agents and evals. The patterns underneath are in AI agent design patterns, and tool craft gets its own treatment in how to design tools for AI agents.
I'll use one invented scenario throughout: a support agent for an online store that can look up orders and, with approval, issue refunds. The details are mine, for practice.
Step 1: decide whether it should be an agent
Say this first, because it signals judgment: "An agent is a loop where the model chooses the next action. I'd only use one if the path can't be written down in advance."
A refund request often can be written down: look up the order, check the policy window, check the amount, refund or escalate. That is a workflow with a model doing extraction and a model drafting the reply. It's cheaper, faster and testable step by step. The case for an agent is the messy tail: customers who combine three problems, orders split across shipments, ambiguous policy. A sensible design answer is "workflow for the 80 percent, agent loop for the rest, with a router in front".
Then pin the contract:
- Inputs: the customer message and their authenticated customer ID (never take identity from the message text).
- Allowed actions: read orders, read policy, draft reply, request refund.
- Forbidden: anything outside that customer's own orders.
- Cost of error: a wrong refund loses money; a wrong denial annoys a customer. These are not symmetric, which decides where the human goes.
Step 2: design the tools
Tools are the agent's whole interface to the world, so spend time here. Four tools for the scenario:
| Tool | Kind | Notes |
|---|---|---|
get_order(order_id) | read | Returns only that customer's orders; error if not theirs |
search_policy(query) | read | Retrieval over the refund policy |
request_refund(order_id, amount, reason) | write | Does not refund; creates a pending request |
escalate_to_human(summary) | write | Always available |
Points to say out loud:
- Few tools, sharply different. If two tools overlap, the model will pick wrongly some of the time. Merge them or sharpen the descriptions.
- Descriptions are prompts. State when to use the tool and when not to. Put the confusing parameter's format in the description with an example.
- Make the write tool a request, not an action.
request_refundcreating a pending record is what makes the approval gate possible without redesign. - Return only what the model needs, and return structured errors. An error like
{"ok": false, "error": "order_not_found", "hint": "order IDs look like A100"}lets the model recover. A stack trace wastes tokens and confuses it. - Idempotency keys on writes. A retry must not refund twice.
- Authorization in the tool, not the prompt. The tool checks that the order belongs to the authenticated customer. A prompt instruction is not access control.
Step 3: loops and stop conditions
An agent without stop conditions is a billing incident. The conditions to list: a maximum step count, a token or money budget, repeated-call detection, and a wall-clock timeout. Also say what each stop returns: a status the caller can act on, such as "escalated: step limit", never silence.
I wrote a small skeleton and ran it so the claims about behavior are real. The model is a scripted stub (a hard-coded list of decisions), so this shows the control flow, not model behavior. The listing is abridged: the script definitions, import json, run_tool and the wall-clock timeout are left out. Python, run on 2026-10-08 and re-run on 2026-10-09 with identical output:
MAX_STEPS = 6
MAX_TOKENS = 5_000
MAX_REPEATS = 2
NEEDS_APPROVAL = {"issue_refund"}
def run(script, approver):
seen, used, log = {}, 0, []
for step, move in enumerate(script, start=1):
if step > MAX_STEPS:
return "STOP max_steps", log
used += move["tokens"]
if used > MAX_TOKENS:
return "STOP token_budget", log
if "final" in move:
return f"DONE {move['final']}", log
key = json.dumps([move["tool"], move["args"]], sort_keys=True)
seen[key] = seen.get(key, 0) + 1
if seen[key] > MAX_REPEATS:
return f"STOP repeated_call {move['tool']}", log
if move["tool"] in NEEDS_APPROVAL and not approver(move):
log.append(f"step {step}: {move['tool']} DENIED by human")
return "STOP human_denied", log
result = run_tool(move["tool"], move["args"])
log.append(f"step {step}: {move['tool']} -> {result}")
return "STOP script_ended", log
Output of the three scenarios:
== looping agent: STOP repeated_call lookup_order
step 1: lookup_order -> {'ok': True, 'order_id': 'A100', 'status': 'delivered', 'total': 499}
step 2: lookup_order -> {'ok': True, 'order_id': 'A100', 'status': 'delivered', 'total': 499}
== refund, human approves: DONE Refund of 499 issued for A100.
step 1: lookup_order -> {'ok': True, 'order_id': 'A100', 'status': 'delivered', 'total': 499}
step 2: issue_refund -> {'ok': True, 'refund_id': 'R-1'}
== refund, human denies: STOP human_denied
step 1: lookup_order -> {'ok': True, 'order_id': 'A100', 'status': 'delivered', 'total': 499}
step 2: issue_refund DENIED by human
The looping agent was cut off on its third identical call. That check is cheap and catches the most common real loop: the model calling the same tool with the same arguments because the result didn't change its mind. A stronger version also detects near-repeats, such as the same tool with trivially different arguments, and feeds a message back ("you already called this; try a different approach or escalate") before hard-stopping.
Step 4: memory
Split memory into three, and name which you need:
- Working context: the messages and tool results of this run. It grows every step, and cost grows with it, since the whole history is re-sent. Trim old tool results and keep decisions.
- Run summary or scratchpad: for long tasks, a compact running state ("order A100 verified, in policy window, awaiting approval") that survives context trimming.
- Durable memory across sessions: customer preferences, past issues. Only add it if a user-visible feature needs it.
The rules worth stating: write to durable memory on purpose, with a source, a timestamp and an expiry; don't save raw conversation by default; and retrieve it as a knowledge source. Two risks to name: stale memory ("customer prefers email", changed last month) and memory poisoning, where a hostile message gets written into memory and later steers the agent. Treat anything retrieved from memory as untrusted input, the same as web content. Longer treatments: AI agent memory architectures and context window budget.
For our refund agent I'd say: no durable memory in version one. The order system is the source of truth, and the extra surface isn't earning its keep.
Step 5: guardrails
Layer them, and say which layer catches what:
- Input: authenticate the user; strip or flag instruction-like text in untrusted fields (order notes, emails the agent reads).
- Tool layer: authorization, argument validation, limits (refund cap, one pending request per order), idempotency.
- Output: check the reply doesn't contain data from another customer, and that any amount quoted matches the tool result.
- Policy: refunds above a threshold always need a human.
The key concept is what Simon Willison named the lethal trifecta: access to private data, exposure to untrusted content, and the ability to communicate externally. When all three are present, prompt injection becomes a real data-theft path. Cut one. (The refund agent can also act, not just send data out, which makes the write path worth guarding the same way.) In the refund agent, reading an order note written by a customer is untrusted content, so the write path must not depend on the model's judgment alone. See prompt injection in tool-using agents and production guardrails for AI agents.
Step 6: human in the loop
Don't say "add a human" as a reflex. Say where, with a rule:
- Approve before irreversible or costly actions: refunds over a threshold, account changes, anything that emails a customer on the company's behalf in version one.
- Review the trace, not just the answer. Show the human the order data, the policy excerpt that applied, and the proposed action, so approval takes seconds and isn't a rubber stamp.
- Sample audits for the auto-approved path. Review a fraction of approved low-risk actions so drift is visible.
- Escalation as a first-class outcome. If the agent escalates 40 percent of cases, that is data about the tools or policy, not a failure of the model.
- Loosen over time, with evidence. Raise the auto-approve threshold only after the eval set and the audit sample support it.
Step 7: evals for a multi-step system
You grade two things. Outcome: is the final state right (correct refund or correct denial, correct reply)? Path: right tool, right arguments, no wasted steps, no policy violation. Spell out how to build the set: 50 to 100 tasks from realistic (anonymized) cases, including the ugly ones: wrong order ID, expired window, a customer pasting an injection attempt in the note, a tool timing out.
Track per task: pass or fail, step count, tokens, cost, and which stop condition fired. Run it on every prompt, tool or model change, and add each production failure as a new test. Because runs are non-deterministic, run each task a few times and report the pass rate rather than a single result. More depth in agent evals from scratch and AI agent evaluation frameworks.
Observability supports all of this: log every step with tool name, arguments, result and latency, tied to a trace ID. Without traces you can't answer "why did it do that".
Common wrong answers
| Wrong answer | Why it loses points | Better |
|---|---|---|
| "Use a multi-agent system with a planner, critic and executor" | Complexity before a measured need | Single agent or fixed workflow; split only when context or tools clearly conflict |
| "Give it access to everything so it's flexible" | Large tool surface, higher error rate, bigger blast radius | Minimal tools, scoped per task |
| "The prompt tells it not to refund over 500" | Prompt is not enforcement | Cap in the tool, approval gate in code |
| "Memory: store the whole conversation in a vector DB" | Noise, staleness, poisoning risk | Deliberate writes with source and expiry |
| "We'll evaluate by trying it a few times" | No regression protection | Fixed task set, pass rate, cost per task |
| "Add a human reviewer for everything" | Kills the value, humans rubber-stamp | Risk-tiered approval with trace shown |
| "Loop until the task is done" | Infinite loops and runaway cost | Step, budget, repeat and time limits |
| "Retry on error" with no change | Same failure, more cost | Structured errors, bounded retries, escalate |
Putting it together aloud
A compressed answer for the last two minutes of the interview:
"Router sends clear refund cases to a fixed workflow, ambiguous ones to a single agent with four tools. Reads are scoped to the authenticated customer; refunds are requests that a human approves above a threshold; the tool layer enforces caps and idempotency. The loop stops on step count, budget, repeated calls or timeout and always returns a status. No durable memory in v1. Quality is measured on a task set for outcome and path, with traces logged on every run. I'd widen autonomy only as the audit sample shows it's safe."
Before you ship a real one, the AI agent production checklist is a good second pass. And if you can't explain your design in two minutes, the design is probably too big.
One caveat on this post: the scenario, the thresholds and the task-set sizes are invented teaching numbers, and the loop skeleton runs against a scripted stub, not a live model. Tune all of them against your own data.
