AI Take-Home Assignment: How to Approach It and What to Submit

The short answer to "how do I approach an AI take-home assignment" is: read the brief twice, write down what question it's really asking, build the smallest end-to-end version first, measure it on a handful of cases you wrote yourself, and spend the last hour on the README. The code is only part of what the reviewer reads.

I haven't seen your assignment, and I don't know any company's grading rubric. Everything here is practical advice based on how these tasks are usually structured and on the checks you can run yourself. Treat the time splits as a starting suggestion. For the interview that may follow, AI engineer interview questions on RAG, agents and evals is a good companion, and so is how to explain a RAG project in an interview.

What is the brief really asking?

Before you open an editor, write three lines at the top of a notes file:

1. The decision the reviewer wants to see. Most AI take-homes are a disguised question such as "can you build a retrieval pipeline and tell whether it works?" or "can you make an LLM extract fields reliably?" The word to find is usually "evaluate", "compare" or "explain".
2. The explicit constraints. Time budget, language, allowed libraries, whether external APIs and AI coding tools are allowed, what format to submit.
3. The ambiguities. Anything the brief leaves open. Ask about the ones that would change your design; for the rest, pick a default and write it into the README as an assumption.

Sending one short email with two or three precise questions is normal. Asking ten is a sign you haven't read it.

How should you split the time?

A suggested split for a task with a budget of about eight hours. Scale it down proportionally for shorter ones. This is my suggestion, not a standard:

A trap worth avoiding is spending most of the time on the first pipeline and almost none on checking whether it works. A reviewer can run your code but can't see your effort, so the evaluation table and the README are what carry your judgement.

What should the thin version look like?

Aim for the dullest design that answers the brief. Examples:

• RAG task: load documents, split them in simple fixed-size chunks, embed, retrieve the top few, answer with citations. Skip re-ranking, hybrid search and agents until you have a baseline number. If you need a refresher on building one, see building a RAG chatbot over PDFs.
• Extraction task: one prompt, a schema, validation of the output in code, and a retry on invalid output.
• Agent task: a plain loop with two or three tools and a hard step limit before reaching for a framework. The trade-offs are covered in plain Python agent loop versus a framework.

Make the model call replaceable behind one function. If the API key stops working on the reviewer's machine, a stub or a cached-response mode lets them still run your evaluation and see the structure.

How do you show that it works?

This is the part that separates submissions. A model that "seems good on two examples" is not evidence. Build a small test set:

• Write 10 to 30 cases yourself, each with an input and what a correct output looks like. Include at least a few awkward ones: missing information, ambiguous questions, inputs in the wrong format.
• Decide the metric before you run it. For retrieval, whether the right document appears in the top k. For extraction, per-field exact match. For free-text answers, a short rubric and a manual read, optionally with a model judge whose weaknesses you note (see LLM-as-judge pitfalls).
• Report the result honestly as a small table, including failures. For a method to build such a set, see building evaluation datasets and golden test sets.
• Say how small the set is. "17 of 20 correct" is fine as long as you also say that 20 cases can't distinguish a 0.85 system from a 0.75 one.

If you make one improvement, pick the failure that your own table shows most clearly, change one thing, and rerun the same cases. A before and after comparison on the same data is worth more than three features without measurements.

What goes in the README?

Assume a reviewer opens it first, and possibly reads nothing else. Keep it under about 800 words with these sections, in this order:

# Project name: one-line purpose

## Run it
3 to 5 commands, tested from a clean checkout. State Python version and
which environment variables are needed (names only, never values).

## What I built
One paragraph and, if useful, a small diagram of the data flow.

## Decisions
Bulleted: choice, alternative considered, why. Include model, chunking,
prompt format, and what you deliberately did not do.

## Results
The evaluation table, how many cases, how the cases were made.

## Limitations
What fails, what you would not trust, what you did not test.

## If I had more time
Two or three specific next steps, ordered.

Time spent: N hours. Tools used: (including any AI coding assistants).

If you want a fuller template for portfolio projects, there's one in AI engineer project portfolio README template. The honest limitations section is the highest-value part. It shows you know where the system breaks, which is what a follow-up interview will probe.

Can a script check the boring things for you?

Yes, and it's worth it, because the boring things are cheap to check and embarrassing to miss: a committed key, an unpinned dependency, no tests, a README with no run instructions. Here is a checker I wrote and ran on 2026-10-09. It's mechanical only. It can't tell whether your work is good.

"""Pre-submission checker for a take-home repo. Usage: python3 submit_check.py <repo_dir>"""
import re
import sys
from pathlib import Path

root = Path(sys.argv[1])
results = []

def check(name, ok, detail=""):
    results.append((name, ok, detail))

readme = next((p for p in root.glob("README*")), None)
text = readme.read_text(errors="ignore") if readme else ""
words = len(text.split())
check("README exists", readme is not None)
check("README under 1200 words", 0 < words <= 1200, f"{words} words")
for section in ["run", "decisions", "limitations", "next"]:
    hit = re.search(rf"^#+\s.*{section}", text, re.I | re.M) is not None
    check(f"README has a '{section}' heading", hit)
check("README states time spent", re.search(r"\b\d+(\.\d+)?\s*(hours|hrs|h)\b", text, re.I) is not None)

reqs = [p for p in root.glob("requirements*.txt")] + [p for p in root.glob("pyproject.toml")]
check("dependency file present", bool(reqs), ", ".join(p.name for p in reqs))
if reqs and reqs[0].suffix == ".txt":
    lines = [l for l in reqs[0].read_text().splitlines() if l.strip() and not l.startswith("#")]
    unpinned = [l for l in lines if "==" not in l]
    check("requirements pinned with ==", not unpinned, f"unpinned: {unpinned}" if unpinned else "")

tests = list(root.glob("tests/**/test_*.py")) + list(root.glob("test_*.py"))
check("at least one test file", bool(tests), f"{len(tests)} found")
check("eval/sample-output file present",
      any(root.glob("eval*")) or any(root.glob("results*")) or any(root.glob("outputs*")))

check(".gitignore present", (root / ".gitignore").exists())
env_committed = (root / ".env").exists()
check("no .env in repo", not env_committed)
secret_re = re.compile(r"(sk-[A-Za-z0-9]{16,}|api[_-]?key\s*=\s*['\"][^'\"]{12,})", re.I)
leaks = []
for p in root.rglob("*"):
    if p.is_file() and p.suffix in {".py", ".md", ".txt", ".json", ".yaml", ".yml", ".env"} and ".git" not in p.parts:
        if secret_re.search(p.read_text(errors="ignore")):
            leaks.append(str(p.relative_to(root)))
check("no key-shaped strings", not leaks, ", ".join(leaks))

big = [str(p.relative_to(root)) for p in root.rglob("*") if p.is_file() and p.stat().st_size > 1_000_000]
check("no files over 1 MB", not big, ", ".join(big))

for name, ok, detail in results:
    print(f"{'PASS' if ok else 'FAIL'}  {name}" + (f"  ({detail})" if detail else ""))
print(f"\n{sum(o for _, o, _ in results)}/{len(results)} checks passed")

To test it, I made a deliberately weak fixture repo: a 34-word README with only "run" and "decisions" sections, a requirements.txt with one unpinned package, one file containing a made-up key-shaped string, no tests and no .gitignore. The output:

PASS  README exists
PASS  README under 1200 words  (34 words)
PASS  README has a 'run' heading
PASS  README has a 'decisions' heading
FAIL  README has a 'limitations' heading
FAIL  README has a 'next' heading
FAIL  README states time spent
PASS  dependency file present  (requirements.txt)
FAIL  requirements pinned with ==  (unpinned: ['numpy'])
FAIL  at least one test file  (0 found)
FAIL  eval/sample-output file present
FAIL  .gitignore present
PASS  no .env in repo
FAIL  no key-shaped strings  (app.py)
PASS  no files over 1 MB

7/15 checks passed

Caveats on the script: the key-shaped pattern is crude and will miss some secrets and flag some harmless strings, so it's no replacement for looking through your git history before you push. If you ever committed a real key, rotate it; deleting the file doesn't remove it from history. The heading checks assume plain Markdown headings that contain the words "run", "decisions", "limitations" and "next". Adjust the list to your own README.

What should you cut when time is short?

Decide cuts before you're tired. A rough order from cut-first to keep-always:

1. Deployment, front-end polish and Docker unless the brief asks for them.
2. Extra models or frameworks compared "for completeness".
3. A second retrieval or prompting technique before the first one has a measured number.
4. Unit tests for glue code. Keep tests for the parsing, scoring and validation functions, since those are cheap and catch real bugs.
5. Never cut: a runnable core, the evaluation table, the limitations section.

If you're wondering whether to use an existing framework, the question to ask yourself is whether you can explain what it does for you. A 40-line loop you understand beats 10 imports you can't explain on the follow-up call.

What happens after you submit?

Many processes include a call where you walk through the work. Rehearse it. Prepare a two-minute tour: the problem as you understood it, the design in one picture, one decision you'd defend, one thing that surprised you, and one thing you'd change. Practise answering "why did you choose X over Y?" for each item in your Decisions list. Expect behavioural questions about your decisions too: see the STAR story template for AI engineers. The mock interview script and score sheet and AI interview prep prompts can help you rehearse with a friend or a model playing interviewer.

Final checklist before you press send

• A clean clone runs with the README's commands. Actually try it in a fresh directory.
• No secrets in the repo or history; .env.example lists variable names only.
• The evaluation table matches the code that's in the repo, at its current commit.
• Limitations and next steps are specific, not "could be improved".
• Time spent and any AI tools used are stated, in line with the brief's rules.
• The submission format matches what was asked: link, zip, size limits.

What I could not verify

I haven't seen any real company's take-home brief or grading rubric, so I make no claim about what any employer values; the time split and cut order are my suggestions. The checker script ran on my own small fixture only, not on real submissions. I did not measure how reviewers read READMEs, so the "read first and sometimes only" line is a general expectation, not a finding.