If you spend your days in SQL, spreadsheets and dashboards, the AI engineer job descriptions can look like a different profession. Python services, vector stores, evals, deployment. The good news is that you already own more of the list than you think. The bad news is that the missing part is concentrated in one place, and vague self-assessment ("I need to learn more AI") hides that.
This data analyst to AI engineer skills gap checklist has 30 skills in six blocks. You score each 0 to 3 against a proof task, total the blocks, and get a study order from your own numbers. Nothing here depends on hiring statistics, and I don't quote any. It's a self-audit, and the scoring bands are my own design, not an industry standard.
Related reading: the career roadmap for SDETs and backend developers covers the same move from a different starting point, and prompts for data analysts shows what to delegate to a model in your current job while you retrain.
How does the scoring work?
For each skill, use the proof task. Score what you can show, not what you recognise.
| Score | Meaning |
|---|---|
| 0 | I haven't done this, or I'd need a tutorial open to start |
| 1 | I've done it once with help or by copying an example |
| 2 | I've done it alone on a real or realistic task, and could explain the choices |
| 3 | I could teach it, and I've handled a failure or edge case in it |
Be harsh. A 2 means someone could ask "why did you do it that way" and you'd answer without guessing. Each block has five skills, so each block maxes at 15. The grand total maxes at 90.
Use the table at the end to turn block totals into a study order. Copy the checklist into a note and fill it in. Re-score after each project, since the second score is the one that shows progress.
Block A: What you probably already have (data and analysis)
| # | Skill | Proof task |
|---|---|---|
| A1 | Write joins, window functions and CTEs without looking up syntax | Answer a ranking-within-group question in SQL from a blank editor |
| A2 | Define a metric precisely and spot when two definitions disagree | Write the definition of "active user" and two ways it could be gamed |
| A3 | Clean messy data and document what you dropped | Take a dirty CSV, clean it, and log row counts at each step |
| A4 | Sanity-check a number against a second source | Reconcile one dashboard figure to raw data |
| A5 | Explain a finding to a non-technical stakeholder in five sentences | Write the summary and have someone repeat it back |
Why it matters here: evaluation is analysis. Writing a labelled test set, defining what counts as a correct answer, and noticing that two graders disagree are the same muscles as A2 to A4. If this block is strong, say so in interviews and in your README, because it is a genuine strength that software-only backgrounds often have to build deliberately.
Block B: Python as software, not scripts
This block is the likeliest gap if your Python so far lives in notebooks. Notebook Python and production Python differ.
| # | Skill | Proof task |
|---|---|---|
| B1 | Structure code into modules and functions with clear inputs and outputs | Refactor a notebook into a package with a main entry point |
| B2 | Use git: branches, meaningful commits, resolve a conflict | Make a feature branch, break something, revert it |
| B3 | Write unit tests with pytest | Test a function with normal, empty and bad input |
| B4 | Manage dependencies and environments | Recreate your project in a fresh virtualenv from a requirements file |
| B5 | Handle errors and logging | Make a function fail loudly with a useful message and log it |
If you score under 7 here, the plan in this post is simple: do not start with AI concepts. Build one small tool in plain Python with tests first. Then move into FastAPI patterns for Claude API apps when you are ready.
Block C: Services and plumbing
| # | Skill | Proof task |
|---|---|---|
| C1 | Call a REST API with authentication, and handle errors | Fetch paginated data and survive a 429 |
| C2 | Build a small HTTP endpoint | Expose a function as POST /predict and test it with curl |
| C3 | Use environment variables and keep secrets out of git | Show a .env.example and a .gitignore that covers .env |
| C4 | Read and write JSON, validate it against a schema | Validate input with Pydantic or a similar library |
| C5 | Use Docker or an equivalent to run the service | Build an image and run it with one command |
Skip C5 at first if you must; it matters for deployment, not for your first project. The rest are daily requirements for anything that calls a model, and rate limits and retries will make more sense once C1 is solid.
Block D: LLM application skills
| # | Skill | Proof task |
|---|---|---|
| D1 | Explain tokens, context window and why outputs vary between runs | Explain to a colleague why the same prompt gave two answers |
| D2 | Write a prompt with role, task, constraints and output format | Get valid JSON out of a model 20 times in a row, and record failures |
| D3 | Use structured outputs and validate them | Parse model output into a schema and handle a malformed response |
| D4 | Build retrieval over documents: chunk, embed, search | Answer questions from 20 documents with source citations |
| D5 | Use tool calling for one tool | Let a model call a function you wrote, and handle a bad argument |
For D2 and D3, structured outputs with Pydantic is the practical starting point. For D4, the beginner RAG chatbot tutorial gets you to a first answer. D1 is mostly reading and experimenting, and the prompt engineering vs AI engineering post helps you see where the jobs split.
Block E: Evaluation and quality (your edge)
| # | Skill | Proof task |
|---|---|---|
| E1 | Build a labelled test set for an LLM task | 40 questions with expected answers or source passages |
| E2 | Separate retrieval errors from generation errors | Report hit rate and answer correctness as two numbers |
| E3 | Choose and justify a metric | Explain why accuracy is misleading for your task |
| E4 | Notice and handle grader disagreement | Have two people (or a model and you) grade 20 answers and compare |
| E5 | Catch regressions after a prompt or model change | Re-run your set after a change and report what moved |
E1 to E3 should feel familiar, because defining metrics and checking labels is already analyst work. E4 and E5 are newer. The method for E1 is in building evaluation datasets and golden test sets. If you use a model as a grader, read LLM-as-judge pitfalls before trusting its scores. For worked RAG metric arithmetic, see RAG evaluation metrics explained.
Block F: Shipping and judgement
| # | Skill | Proof task |
|---|---|---|
| F1 | Estimate cost per request and per month | Compute from token counts and the provider's pricing page, dated |
| F2 | Measure latency, not only average but a high percentile | Time 50 calls and report median and p95 |
| F3 | Document known failures | A failures.md with three diagnosed cases |
| F4 | Deploy somewhere a stranger can use it | A live link or a one-command local run |
| F5 | Explain trade-offs aloud | Do a 3-minute walkthrough of a project with no notes |
For F1, copy the method from LLM API cost per 1000 requests. F5 is practised with the mock interview script and score sheet.
Turn your scores into a study order
Total each block (0 to 15). Then:
| Block total | Reading | What to do |
|---|---|---|
| 0 to 5 | Real gap | Study this block first, using the proof tasks as your syllabus |
| 6 to 10 | Workable | Close it through your project, not separate courses |
| 11 to 15 | Strength | Mention it in interviews and don't spend study time here |
Order of attack, as a starting rule you can override:
- If Block B is under 7, fix B first. Everything else is built on it.
- Then C (a model call is just an API call), then D.
- Build a first project that exercises D and E together.
- Use F as the finishing pass on that same project.
If your B and C totals are both 10 or more, you can start the project straight away and learn D as you go.
Worked example (fictional)
A fictional analyst, call her Meera, scores her blocks like this:
| Block | A | B | C | D | E | F |
|---|---|---|---|---|---|---|
| Total (of 15) | 14 | 5 | 6 | 3 | 9 | 2 |
Reading it: B is the real gap (5), so her first two weeks go on structuring one small data tool as a tested package. C is a gap too (6), so she adds a small FastAPI endpoint in the same repo. D is low because she hasn't started, which is normal and cheap to fix. E at 9 is her edge and she should plan her project around it. Her first project becomes a document Q&A service where the evaluation is the centrepiece. Meera is invented to show how to read the table; her numbers are not data from real people.
A small script to total your scores
Save your scores as a CSV with columns skill,score (for example A1,2) and run this. I ran it on 2026-10-09 against a five-line sample file (scores for A1, A2, B1, B2, E1) and it printed Block A: 5 of 6, Block B: 3 of 6 and Block E: 3 of 3.
import csv
import sys
from collections import defaultdict
totals = defaultdict(int)
counts = defaultdict(int)
with open(sys.argv[1], newline="") as f:
for row in csv.DictReader(f):
block = row["skill"][0].upper()
totals[block] += int(row["score"])
counts[block] += 1
for block in sorted(totals):
print(f"Block {block}: {totals[block]} of {counts[block] * 3}")
Run it as python3 scores.py my_scores.csv. If a block shows fewer than 15 possible points (the "of" number), you left skills out. Fill them in before reading the totals.
What not to do with your results
- Don't treat a low score as a reason to wait. Pick the lowest block that your project needs and start.
- Don't replace the proof tasks with courses. Certificates are optional; whether AI certifications are worth it in India is a separate decision.
- Don't claim a skill on a CV that you scored 0 or 1. It will come up in the interview, and the mock interview script will find it first if you let it.
- Don't skip your current job as a practice ground. If you already use SQL assistants or spreadsheets daily, the spreadsheets vs Python with AI post helps you decide when a task has outgrown a sheet and is a good first Python project.
Re-score in six weeks. If block totals rose and the repo holds the evidence, you are moving. If they haven't, look at which proof tasks you avoided, since those are usually the ones that matter.
