A resume pipeline that outputs a number invites everyone downstream to treat the number as a decision. So build the opposite: a pipeline that reads resumes, strips identity fields, scores only against a rubric you wrote first, checks itself, and ends in a queue for a human to read.
This post gives you the runnable version, in Python's standard library. Every candidate in it is invented, there are no hiring statistics anywhere, and the bias checks are descriptive checks on the script itself. For the prompt side of HR work see AI prompts for HR and recruiters. For the name-swap method against a live model see LLM bias testing with counterfactual names. This post is the pipeline around them.
What should a resume screening pipeline do, and what should it never do?
Split the work by who is allowed to do it.
| Stage | Who does it | Rule |
|---|---|---|
| Parse text into fields | Script or LLM | Must be validated; failures go to a person, never to "score 0" |
| Redact identity fields | Script | Whitelist what survives, do not blacklist what goes |
| Score against the rubric | Script | Advisory, with per-criterion evidence |
| Audit the pipeline | Script plus a person | Descriptive, on your own system |
| Decide who moves forward | A person | Always, and write down why |
Things it should never do: auto-reject, guess age, gender, caste or religion from a name or photo, score "culture fit", penalise career breaks, or look people up on social media. If a feature needs one of those to work, the feature is the problem.
How do I parse resumes without silently dropping people?
The parser below returns a status for every resume. ok means the dates parsed cleanly. warn means it parsed year-only dates, so tenure is approximate. human_review means it found no dates, and that record is not scored.
The distinction matters because parsers fail on formatting, and formatting isn't merit. In the run below, a resume written as a paragraph ("Spent four years at...") has no date ranges. A naive pipeline gives it zero years of experience and quietly buries it. This one hands it to a person.
If you replace the rule-based parser with an LLM extractor, keep the same contract. The model proposes fields, then a check disposes: every skill it returns must literally appear in the resume text. The script's grounded() function does that, and the last section of the output shows it catching an invented Kubernetes.
What gets redacted before scoring?
Redaction uses a whitelist. The scorer receives only a normalised skills list and years of experience. It never receives the name, email, phone, city, college or company names.
Two things the invented data makes visible:
- College tracks the audit label perfectly in this dataset (by construction: I gave the two groups different college names). Any scorer that could see the college would effectively see the group. Dropping it is not optional.
- The skills alias map is a policy decision. It maps PostgreSQL and MySQL to
sql, and pytest and unittest totesting. Java, Spring and REST are not in it, so they count for nothing in this Python-focused rubric. A different alias map is a different hiring preference, so write it down and review it like the rubric.
How do I write the rubric so it can be checked?
Write it before you read a single resume, and keep it to things the job needs. The rubric in the script is for an invented Python backend role, 10 points: Python 3, SQL 2, any cloud or container skill 1, any testing skill 1, years of experience 3 (1 point from 1 year, 2 from 3 years, 3 from 5 years).
Then look at what it does. In the run, 11 of 13 scored resumes landed in band A (7 or more). A rubric that cannot tell candidates apart at the top hasn't done the human's job, it has only removed the clear mismatches. Treat the score as a way to sort the reading order and surface evidence, then make the person read the top band. Do not shrink the top band by adding arbitrary criteria just to spread the scores.
What bias checks can a script run?
Five, all descriptive. None of them proves fairness, and with 14 resumes none of them can support a statistical claim.
- Redaction leak check. Search the scorer's input for the fields that should be gone. Expect zero. It looks trivial because the whitelist makes it pass, but it is the regression test for the day someone adds a field.
- Name-swap invariance. Three pairs of resumes have identical content and different names. Scores must match. Then a negative control: a "leaky" scorer with a planted penalty, to prove the same check fails when it should. A check you have never seen fail is a check you cannot trust.
- Group table. Counts, mean, median and band counts per audit label, with the human-review bucket included. The labels
AandBare an invented stand-in. In real use, any such labels must be collected lawfully, stored apart from the scoring data, and used only for auditing. - Parse failure by group. The one unscored resume belongs to group B because I wrote that resume as a paragraph. With real data, a parse-failure rate that differs between groups is a finding about your parser, and it is the kind of unfairness a score-only audit never sees.
- Criterion sensitivity. Add a "career gap of 12 months or more costs 2 points" rule and see who it moves. It's a variant, not a recommendation.
In this data, group B has a higher mean (9.50 against 7.86). That's an accident of 14 hand-built resumes, where group A contains a Java developer and an intern. It tells you nothing about people, and it is why you never read one number as a verdict in either direction.
The full script
Save as resume_pipeline.py. It generates the invented resumes, runs every stage and prints each check.
"""Resume parsing, redaction, rubric scoring and a descriptive audit.
All candidates are INVENTED. Standard library only. No LLM is called: the parser is
rule-based so the pipeline is checkable. An LLM extractor would slot in where
parse_resume() is, and must pass the same grounding check.
Run: python3 resume_pipeline.py
"""
import json
import random
import re
import statistics
TODAY = (2026, 10) # fixed so output is reproducible
MONTHS = ["Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec"]
# ---------------------------------------------------------------- invented data
# (id, name, audit_group, city, college, skills, roles, fmt)
# roles: (title, company, (y, m), (y, m) or None for present)
SPECS = [
("C01", "Rahul Verma", "A", "Pune", "Example Institute of Technology", ["Python", "PostgreSQL", "Docker", "pytest"],
[("Backend Developer", "Finlo", (2019, 6), (2022, 8)), ("Senior Backend Developer", "Paynest", (2022, 9), None)], "my"),
("C02", "Priya Nair", "B", "Kochi", "Sample College of Engineering", ["Python", "PostgreSQL", "Docker", "pytest"],
[("Backend Developer", "Finlo", (2019, 6), (2022, 8)), ("Senior Backend Developer", "Paynest", (2022, 9), None)], "my"),
("C03", "Amit Shah", "A", "Ahmedabad", "Example Institute of Technology", ["Python", "SQL", "REST", "AWS"],
[("Software Engineer", "Kiteworks", (2021, 1), (2024, 3)), ("Backend Engineer", "Orbit", (2024, 4), None)], "my"),
("C04", "Anita Rao", "B", "Mysuru", "Sample College of Engineering", ["Python", "SQL", "REST", "AWS"],
[("Software Engineer", "Kiteworks", (2021, 1), (2024, 3)), ("Backend Engineer", "Orbit", (2024, 4), None)], "my"),
("C05", "Vikram Singh", "A", "Jaipur", "Example Institute of Technology", ["Python", "MySQL", "unittest"],
[("Developer", "Tallyfy", (2023, 7), None)], "my"),
("C06", "Sunita Das", "B", "Kolkata", "Sample College of Engineering", ["Python", "MySQL", "unittest"],
[("Developer", "Tallyfy", (2023, 7), None)], "my"),
("C07", "Meera Iyer", "B", "Chennai", "Sample College of Engineering", ["Python", "SQL", "Docker", "pytest", "GCP"],
[("Backend Developer", "Zesta", (2015, 2), (2018, 6)), ("Python Engineer", "Cobalt", (2020, 9), None)], "my"),
("C08", "Karan Mehta", "A", "Mumbai", "Example Institute of Technology", ["Java", "SQL", "Spring"],
[("Java Developer", "Infra Corp", (2018, 1), (2024, 12))], "my"),
("C09", "Divya Pillai", "B", "Thiruvananthapuram", "Sample College of Engineering", ["Python", "PostgreSQL", "Docker", "pytest", "AWS"],
[("Backend Engineer", "Nimbus", (2017, 5), (2019, 2)), ("Backend Engineer", "Nimbus", (2022, 6), None)], "my"),
("C10", "Arjun Reddy", "A", "Hyderabad", "Example Institute of Technology", ["Python", "SQL"],
[("Intern", "Quanta", (2025, 6), (2025, 12))], "my"),
("C11", "Neha Gupta", "B", "Delhi", "Sample College of Engineering", ["Python", "SQL", "pytest"],
[("Backend Developer", "Lumo", (2022, 1), (2025, 12))], "prose"),
("C12", "Sanjay Kulkarni", "A", "Nagpur", "Example Institute of Technology", ["Python", "SQL", "Docker", "pytest"],
[("Python Developer", "Brightloop", (2016, 1), (2021, 1)), ("Tech Lead", "Brightloop", (2021, 1), None)], "year"),
("C13", "Farah Khan", "B", "Lucknow", "Sample College of Engineering", ["Python", "SQL", "AWS", "pytest"],
[("Data Engineer", "Cloudly", (2020, 1), (2024, 1)), ("Backend Engineer", "Cloudly", (2024, 1), None)], "year"),
("C14", "Rohit Jain", "A", "Surat", "Example Institute of Technology", ["Python", "SQL", "Docker"],
[("Backend Developer", "Swiftcart", (2022, 3), (2024, 6)), ("Software Engineer", "Swiftcart", (2024, 7), None)], "my"),
]
def render(spec, rng):
cid, name, _g, city, college, skills, roles, fmt = spec
email = name.lower().replace(" ", ".") + "@example.com"
phone = "+91 00000 %05d" % rng.randint(0, 99999) # obviously fake
lines = [name, f"{email} | {phone} | {city}", f"Education: B.Tech, {college}",
("Skills: " if rng.random() < 0.5 else "Tech: ") + ", ".join(skills), "Experience"]
for title, co, s, e in roles:
if fmt == "my":
end = "Present" if e is None else f"{MONTHS[e[1] - 1]} {e[0]}"
lines += [f"{title}, {co}", f"{MONTHS[s[1] - 1]} {s[0]} - {end}"]
elif fmt == "year":
end = "present" if e is None else str(e[0])
lines += [f"{title}, {co} ({s[0]} - {end})"]
else:
lines += [f"Spent four years at {co} as a {title.lower()} building Python services."]
return "\n".join(lines)
# ---------------------------------------------------------------- parsing
RE_MY = re.compile(r"(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) (\d{4}) - (?:(Present)|(Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec) (\d{4}))")
RE_YR = re.compile(r"\((\d{4}) - (present|\d{4})\)")
RE_EMAIL = re.compile(r"[\w.]+@[\w.]+")
RE_PHONE = re.compile(r"\+91 [\d ]{10,}")
ALIASES = {"postgresql": "sql", "mysql": "sql", "sqlite": "sql", "sql": "sql", "python": "python",
"pytest": "testing", "unittest": "testing", "docker": "cloud", "aws": "cloud", "gcp": "cloud"}
def parse_resume(text):
lines = [l for l in text.splitlines() if l.strip()]
rec = {"name": lines[0], "email": (RE_EMAIL.search(text) or [None])[0], "phone": (RE_PHONE.search(text) or [None])[0],
"raw_skills": [], "ranges": [], "status": "ok", "reason": ""}
for l in lines:
m = re.match(r"(Skills|Tech):\s*(.+)", l)
if m:
rec["raw_skills"] = [s.strip() for s in m.group(2).split(",")]
for m in RE_MY.finditer(text):
s = int(m.group(2)) * 12 + MONTHS.index(m.group(1))
e = TODAY[0] * 12 + TODAY[1] - 1 if m.group(3) else int(m.group(5)) * 12 + MONTHS.index(m.group(4))
rec["ranges"].append((s, e + 1))
if not rec["ranges"]:
for m in RE_YR.finditer(text):
s = int(m.group(1)) * 12
e = TODAY[0] * 12 + TODAY[1] if m.group(2) == "present" else int(m.group(2)) * 12
rec["ranges"].append((s, e))
if rec["ranges"]:
rec["status"], rec["reason"] = "warn", "year-only dates, tenure approximate"
if not rec["ranges"]:
rec["status"], rec["reason"] = "human_review", "no parseable dates"
rec["months"] = sum(e - s for s, e in rec["ranges"])
rs = sorted(rec["ranges"])
rec["gap_months"] = sum(max(0, b[0] - a[1]) for a, b in zip(rs, rs[1:])) if rec["status"] == "ok" else None
return rec
def grounded(extracted_skills, text):
"""Grounding check for ANY extractor (rules or LLM): every skill must appear in the resume text."""
low = text.lower()
return [s for s in extracted_skills if s.lower() not in low]
# ---------------------------------------------------------------- redact + score
def redact(rec):
"""Only job-relevant fields survive. No name, email, phone, college, city, company."""
return {"skills": sorted({ALIASES[s.lower()] for s in rec["raw_skills"] if s.lower() in ALIASES}),
"years": round(rec["months"] / 12, 1)}
def score(red):
parts = {"python": 3 * ("python" in red["skills"]), "sql": 2 * ("sql" in red["skills"]),
"cloud": 1 * ("cloud" in red["skills"]), "testing": 1 * ("testing" in red["skills"]),
"years": 3 if red["years"] >= 5 else 2 if red["years"] >= 3 else 1 if red["years"] >= 1 else 0}
return sum(parts.values()), parts
def band(rec, total):
if rec["status"] == "human_review":
return "H"
return "A" if total >= 7 else "B" if total >= 5 else "C"
def run(texts):
out = {}
for cid, text in texts.items():
rec = parse_resume(text)
red = redact(rec)
total, parts = score(red)
out[cid] = {"rec": rec, "red": red, "total": None if rec["status"] == "human_review" else total, "parts": parts,
"band": band(rec, total)}
return out
if __name__ == "__main__":
rng = random.Random(11)
texts = {s[0]: render(s, rng) for s in SPECS}
group = {s[0]: s[2] for s in SPECS}
res = run(texts)
print("== 1. Parse check against the invented ground truth ==")
print("id truth_yrs parsed_yrs status reason")
for s in SPECS:
cid, roles = s[0], s[6]
truth = sum(((TODAY[0] * 12 + TODAY[1] if e is None else e[0] * 12 + e[1] - 1 + 1) - (st[0] * 12 + st[1] - 1)) for _t, _c, st, e in roles) / 12
r = res[cid]
parsed = "-" if r["rec"]["status"] == "human_review" else f"{r['red']['years']:.1f}"
print(f"{cid} {truth:8.1f} {parsed:>9} {r['rec']['status']:<13} {r['rec']['reason']}")
print("\n== 2. Redaction leak check ==")
leaks = 0
for s in SPECS:
blob = json.dumps(res[s[0]]["red"]).lower()
for secret in (s[1], s[3], s[4], res[s[0]]["rec"]["email"], res[s[0]]["rec"]["phone"], s[6][0][1]):
if secret and secret.lower() in blob:
leaks += 1
print("fields leaked into the scorer input:", leaks)
print("\n== 3. Scores (rubric: python 3, sql 2, cloud 1, testing 1, years 3) ==")
print("id band score parts")
for cid, r in res.items():
print(f"{cid} {r['band']:>4} {str(r['total']):>5} {r['parts'] if r['total'] is not None else 'no score: a human reads this one'}")
print("\n== 4. Name-swap invariance on the three content-identical pairs ==")
for a, b in (("C01", "C02"), ("C03", "C04"), ("C05", "C06")):
print(a, b, "scores", res[a]["total"], res[b]["total"], "equal" if res[a]["total"] == res[b]["total"] else "DIFFERENT")
def leaky(cid): # negative control: a scorer that can see the name and has a planted penalty
return res[cid]["total"] - (1 if group[cid] == "B" else 0)
print("negative control, a scorer that sees names and has a planted -1 for group B:")
print(" C01 vs C02 leaky scores", leaky("C01"), leaky("C02"), "-> the same pair check flags it" if leaky("C01") != leaky("C02") else "-> missed")
print("\n== 5. Descriptive group table (n is tiny; no significance claims) ==")
for g in ("A", "B"):
ids = [c for c in res if group[c] == g]
scored = [res[c]["total"] for c in ids if res[c]["total"] is not None]
bands = {k: sum(res[c]["band"] == k for c in ids) for k in "ABCH"}
print(f"group {g}: n={len(ids)} scored={len(scored)} mean={statistics.mean(scored):.2f} median={statistics.median(scored)} bands={bands}")
print("\n== 6. What a gap penalty would do (variant, not recommended) ==")
for g in ("A", "B"):
ids = [c for c in res if group[c] == g and res[c]["total"] is not None]
pen = [res[c]["total"] - (2 if (res[c]["rec"]["gap_months"] or 0) >= 12 else 0) for c in ids]
gaps = sum((res[c]["rec"]["gap_months"] or 0) >= 12 for c in ids)
print(f"group {g}: candidates with gap>=12 months={gaps}, mean before={statistics.mean(res[c]['total'] for c in ids):.2f}, mean after={statistics.mean(pen):.2f}")
print("\n== 7. Grounding check on a simulated extractor output ==")
fake_llm_skills = ["Python", "SQL", "Kubernetes"] # stub: pretend a model added a skill
print("C14 ungrounded skills:", grounded(fake_llm_skills, texts["C14"]))
What does the output look like?
Real output from running it (seeded; it prints the same thing every time):
== 1. Parse check against the invented ground truth ==
id truth_yrs parsed_yrs status reason
C01 7.4 7.4 ok
C02 7.4 7.4 ok
C03 5.8 5.8 ok
C04 5.8 5.8 ok
C05 3.3 3.3 ok
C06 3.3 3.3 ok
C07 9.6 9.6 ok
C08 7.0 7.0 ok
C09 6.2 6.2 ok
C10 0.6 0.6 ok
C11 4.0 - human_review no parseable dates
C12 10.9 10.8 warn year-only dates, tenure approximate
C13 6.9 6.8 warn year-only dates, tenure approximate
C14 4.7 4.7 ok
== 2. Redaction leak check ==
fields leaked into the scorer input: 0
== 3. Scores (rubric: python 3, sql 2, cloud 1, testing 1, years 3) ==
id band score parts
C01 A 10 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 1, 'years': 3}
C02 A 10 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 1, 'years': 3}
C03 A 9 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 0, 'years': 3}
C04 A 9 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 0, 'years': 3}
C05 A 8 {'python': 3, 'sql': 2, 'cloud': 0, 'testing': 1, 'years': 2}
C06 A 8 {'python': 3, 'sql': 2, 'cloud': 0, 'testing': 1, 'years': 2}
C07 A 10 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 1, 'years': 3}
C08 B 5 {'python': 0, 'sql': 2, 'cloud': 0, 'testing': 0, 'years': 3}
C09 A 10 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 1, 'years': 3}
C10 B 5 {'python': 3, 'sql': 2, 'cloud': 0, 'testing': 0, 'years': 0}
C11 H None no score: a human reads this one
C12 A 10 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 1, 'years': 3}
C13 A 10 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 1, 'years': 3}
C14 A 8 {'python': 3, 'sql': 2, 'cloud': 1, 'testing': 0, 'years': 2}
== 4. Name-swap invariance on the three content-identical pairs ==
C01 C02 scores 10 10 equal
C03 C04 scores 9 9 equal
C05 C06 scores 8 8 equal
negative control, a scorer that sees names and has a planted -1 for group B:
C01 vs C02 leaky scores 10 9 -> the same pair check flags it
== 5. Descriptive group table (n is tiny; no significance claims) ==
group A: n=7 scored=7 mean=7.86 median=8 bands={'A': 5, 'B': 2, 'C': 0, 'H': 0}
group B: n=7 scored=6 mean=9.50 median=10.0 bands={'A': 6, 'B': 0, 'C': 0, 'H': 1}
== 6. What a gap penalty would do (variant, not recommended) ==
group A: candidates with gap>=12 months=0, mean before=7.86, mean after=7.86
group B: candidates with gap>=12 months=2, mean before=9.50, mean after=8.83
== 7. Grounding check on a simulated extractor output ==
C14 ungrounded skills: ['Kubernetes']
How to read it:
- Section 1: eleven of fourteen parse exactly. C12 and C13 are
warnand land within 0.1 year of the truth, because year-only dates lose the months. C11 has no dates, so it becomeshuman_review. - Section 2: zero leaked fields.
- Section 3: C08 (Java, no Python) and C10 (six months of experience) fall to band B. Everyone else with scores is band A, which is the sorting problem described above.
- Section 4: the content-identical pairs score identically, and the negative control shows a 10 versus 9 split when the planted penalty exists.
- Section 6: the gap penalty touches two group B resumes (C07 and C09 each have gaps over 12 months). It lowers the group B mean from 9.50 to 8.83 but does not move anyone out of band A. It would change reading order, and for a reason that has nothing to do with the job. That is what a proxy looks like.
Where does an LLM fit?
At the parse step, as a proposer. This prompt is an untested template: I did not run it against a live model.
Extract fields from the resume below into JSON with exactly these keys:
skills (list of strings), roles (list of objects with title, start, end),
and evidence (for each skill, the exact phrase from the resume that supports it).
Rules: use only text that appears in the resume. If a field is missing, use null.
Do not infer anything about the person (age, gender, nationality, background).
Do not rate or rank the candidate.
Resume:
<resume>
{resume_text}
</resume>
Then run the checks the model cannot talk its way past: parse the JSON strictly, run grounded() on every skill, compare the model's dates to a rule-based parse when both exist, and send any mismatch to a person. For the schema side see structured outputs with Pydantic, and because resumes are untrusted text that can contain instructions, read prompt injection in production systems before wiring a model to real files.
A human review protocol you can adopt
- A person reads every
human_reviewresume in full. - A person reads every band A resume, not just the top five scores.
- A sample of band B and C is read too, to catch rubric blind spots.
- Each decision gets a one-line reason tied to the written rubric.
- The audit runs every time the rubric, alias map or parser changes.
- Applicants are told how their data is used. See the DPDP checklist for the Indian context, and take professional advice for your jurisdiction.
What I did not verify
No LLM was called, so the extraction prompt is untested and the grounding check was shown against a simulated bad output. The resumes are invented, so parse accuracy on real-world resumes (tables, columns, scanned PDFs, other languages) is unknown and will be worse. I did not research hiring law in any jurisdiction, and nothing here is legal advice.
