You can turn a meeting recording into decisions and tasks on your own machine: transcribe with local Whisper, ask a model for JSON where every item carries a verbatim quote, then run a checker script that rejects any item the transcript does not support. The checker is the part most workflows skip, and it is the part that stops an invented owner from reaching your team's inbox.
This post builds on two earlier ones. The Whisper batch script covers installing faster-whisper and transcribing a folder, and the meeting notes workflow covers the extraction prompt and a basic quote check. Here the transcript keeps timestamps, the checker tests more, and I show what it did and did not catch.
What did I actually run?
| Stage | What happened on 2026-10-09 |
|---|---|
| Transcription | transcribe_ts.py (below) run with faster-whisper 1.2.1 and base.en on one 5.4 second synthetic clip from the earlier post. Real output. |
| Meeting transcript | Invented by me: three people, 14 lines, about 2 minutes 44 seconds of dialogue. Not a recording, not Whisper output. |
| Extraction JSON | Written by hand, not produced by a model. One clean file and one with planted mistakes. |
| Checker | check_extraction.py (below) run on all of them. Real output. |
| Extraction prompt and repair prompt | Untested templates. I did not run them on a live model. |
So the checker is tested against mistakes I invented to resemble typical model errors. It has not been tested against real model output, long meetings, accents, or overlapping speech. Treat the numbers as a demonstration, not a benchmark.
What does the pipeline look like end to end?
| Step | Tool | Where the data goes | What can go wrong |
|---|---|---|---|
| 1. Record, with consent | Your meeting app or a phone | Wherever you store it | People did not agree to be recorded. See the recording consent basics. |
| 2. Transcribe with timestamps | transcribe_ts.py | Stays on your machine after the first model download | Names and numbers misheard |
| 3. Add attendees and speaker names | You, by hand | Local | Owners become guesses if you skip this |
| 4. Extract JSON with quotes | A model, local or hosted | Hosted model: the transcript leaves your machine | Invented owners, hedges dropped |
| 5. Check mechanically | check_extraction.py | Local | False comfort if you treat a pass as proof |
| 6. Human read, then send | You | Your team | Skipping it |
Step 2: how do I get a timestamped transcript?
The script below writes one line per Whisper segment as [mm:ss] text. Timestamps matter because they let you jump from any extracted item back to the audio and check it in seconds.
#!/usr/bin/env python3
"""Transcribe one audio file to a timestamped transcript: one '[mm:ss] text' line per segment.
Usage: python transcribe_ts.py AUDIO OUT.txt [--model base.en]
"""
import argparse
from pathlib import Path
from faster_whisper import WhisperModel
def stamp(seconds: float) -> str:
s = int(seconds)
return f"{s // 60:02d}:{s % 60:02d}"
def main() -> None:
ap = argparse.ArgumentParser()
ap.add_argument("audio", type=Path)
ap.add_argument("out", type=Path)
ap.add_argument("--model", default="base.en")
args = ap.parse_args()
model = WhisperModel(args.model, device="cpu", compute_type="int8")
segments, info = model.transcribe(str(args.audio), vad_filter=True, beam_size=5)
lines = [f"[{stamp(seg.start)}] {seg.text.strip()}" for seg in segments]
args.out.write_text("\n".join(lines) + "\n", encoding="utf-8")
print(f"{args.audio.name}: {info.duration:.1f}s audio, {len(lines)} segments -> {args.out}")
if __name__ == "__main__":
main()
Run on the synthetic clip (output paths shortened):
$ python transcribe_ts.py memo_01.wav memo_01_ts.txt
memo_01.wav: 5.4s audio, 1 segments -> memo_01_ts.txt
$ cat memo_01_ts.txt
[00:00] Note to self, move the stand-up to 10.30 on Thursday and ask Priya for the invoice export.
A 5 second clip yields one segment, so this only proves the format works. A real meeting gives you many lines. If the first run fails with a PyAV error about metadata_errors, the earlier post explains the version pin I needed.
Whisper does not say who is talking. Before extraction, add speaker names at each turn change by hand, in the [mm:ss] Name: text shape the invented transcript below uses, or accept unlabelled lines and let owners be UNASSIGNED unless a name is spoken. I did not test a diarisation tool.
The transcript I invented for the test
Everything below runs against this. Names are fictional.
[00:04] Asha: Okay, three things today. The billing export, the onboarding email, and the support backlog.
[00:19] Asha: Billing export first. Dev, where are we?
[00:24] Dev: The export works for September but October fails on the tax column. I'll fix it and send you a sample file by Thursday.
[00:41] Asha: Good. And we agreed the finance team gets it on the 15th, right?
[00:46] Dev: Yes, 15 October, as long as the sample passes their check.
[01:02] Asha: Fine, we go with the 15th, conditional on that check.
[01:15] Meera: On onboarding, I think we should rewrite the second email, but I haven't looked at the open rates properly.
[01:30] Asha: Okay, so that's undecided. Meera, can you pull the open rates?
[01:36] Meera: Sure, I'll have them by Monday.
[01:52] Asha: Someone needs to tell the support leads about the new refund wording.
[02:03] Dev: Yeah, that's a good point.
[02:20] Asha: Last thing, the backlog is at 212 tickets. We need a plan for that.
[02:31] Meera: I'd say we pause new feature work for a week, but that's a bigger call than this meeting.
[02:44] Asha: Agreed, I'll raise it with Rohan next week. Thanks everyone.
It contains the usual traps: a conditional decision (the 15th, but only if a check passes), an undecided idea (rewrite the second email), an ownerless task (tell the support leads), a hedge that sounds like a plan (pause feature work, "a bigger call than this meeting"), and three real commitments with owners and dates.
Step 4: what should the extraction prompt ask for?
Ask for JSON that the checker can read. This is an untested template. It extends the one in the meeting notes post with the attendee list and timestamps.
Extract records from the meeting transcript below. Use only the transcript.
Do not infer, round up or fill gaps.
Return only JSON with this shape:
{
"attendees": ["names exactly as written in the transcript"],
"decisions": [{"decision": str, "conditions": str or null, "evidence": str}],
"action_items": [{"task": str, "owner": a name from attendees or "UNASSIGNED",
"due": "as spoken, or none stated", "evidence": str}],
"open_questions": [{"question": str, "evidence": str}]
}
Rules:
- "evidence" is one exact sentence copied from the transcript, without the timestamp.
- A decision needs agreement. Opinions and suggestions go in open_questions.
- An action item needs a person who commits ("I'll", "I can", or a yes to "can you...?").
"Someone should", "we need to" and "we should" mean owner UNASSIGNED.
- Keep conditions such as "unless" and "as long as" in the conditions field.
- Copy dates and numbers exactly as spoken. Do not convert them.
- An empty array is a valid answer.
TRANSCRIPT:
[paste]
For the idea behind forcing JSON with fixed keys, see the constrained generation lesson; for a plain-language summary prompt without the checker, the library has a meeting summary prompt. Two choices carry most of the weight. Asking for one exact sentence makes the evidence testable. Telling the model that suggestions are open questions removes the pressure to turn every idea into a task.
Step 5: what does the checker test?
The script reads the transcript and the JSON and prints ERROR lines (the item is unsupported) and WARN lines (a human should look). It exits with status 1 if any ERROR appears, so you can call it from a shell script.
#!/usr/bin/env python3
"""Check a model's meeting extraction (JSON) against the transcript it came from.
Usage: python check_extraction.py TRANSCRIPT.txt EXTRACTION.json [--window 2]
Transcript lines look like: [mm:ss] Speaker: text or [mm:ss] text
Exit code 1 if any ERROR is found; WARN lines need a human look.
"""
import argparse
import difflib
import json
import re
import sys
from pathlib import Path
LINE = re.compile(r"^\[(\d{1,2}:\d{2}(?::\d{2})?)\]\s*(.*)$")
COMMIT = re.compile(r"\b(i'll|i will|i can|i'm going to|will do|on it|sure|yes|yep|i'll take)\b", re.I)
DATEWORDS = re.compile(r"\b(monday|tuesday|wednesday|thursday|friday|saturday|sunday|today|tomorrow|"
r"next week|end of (the )?(day|week|month)|eod|\d{1,2}(st|nd|rd|th)?\s+"
r"(jan|feb|mar|apr|may|jun|jul|aug|sep|oct|nov|dec)\w*)\b", re.I)
STRONG = re.compile(r"\b(i'll|i will|i can|i'm going to|will do|on it)\b", re.I)
OWNERLESS = re.compile(r"\b(someone|somebody|we need to|we should|needs to be)\b", re.I)
OPINION = re.compile(r"\b(i think|i'd say|i guess|maybe|probably|not sure|undecided)\b", re.I)
HEDGE = re.compile(r"\b(unless|conditional|as long as|provided that|if|assuming|depends)\b", re.I)
KEYS = {
"decisions": {"decision", "conditions", "evidence"},
"action_items": {"task", "owner", "due", "evidence"},
"open_questions": {"question", "evidence"},
}
def norm(s: str) -> str:
return re.sub(r"\s+", " ", re.sub(r"[^a-z0-9 ]", " ", s.lower())).strip()
def load_transcript(path: Path):
rows = []
for raw in path.read_text(encoding="utf-8").splitlines():
if not raw.strip():
continue
m = LINE.match(raw)
stamp, text = (m.group(1), m.group(2)) if m else ("??:??", raw.strip())
rows.append((stamp, text))
return rows
def locate(quote: str, rows):
"""Return (index, ratio). index is the transcript row containing the quote, or the closest row."""
q = norm(quote)
if not q:
return None, 0.0
for i, (_, text) in enumerate(rows):
if q in norm(text):
return i, 1.0
joined = [norm(rows[i][1] + " " + (rows[i + 1][1] if i + 1 < len(rows) else "")) for i in range(len(rows))]
for i, j in enumerate(joined):
if q in j:
return i, 1.0
best_i, best = None, 0.0
for i, (_, text) in enumerate(rows):
sm = difflib.SequenceMatcher(None, q, norm(text), autojunk=False)
r = sum(b.size for b in sm.get_matching_blocks()) / len(q) # share of the quote found in this line
if r > best:
best_i, best = i, r
return best_i, best
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("transcript", type=Path)
ap.add_argument("extraction", type=Path)
ap.add_argument("--window", type=int, default=2, help="lines either side of the evidence to search")
args = ap.parse_args()
rows = load_transcript(args.transcript)
data = json.loads(args.extraction.read_text(encoding="utf-8"))
attendees = {norm(a) for a in data.get("attendees", [])}
errors, warns = [], []
cited = set()
seen_evidence = {}
for section, required in KEYS.items():
for n, item in enumerate(data.get(section, []), 1):
tag = f"{section}[{n}]"
missing = required - set(item)
if missing:
errors.append(f"{tag}: missing keys {sorted(missing)}")
continue
idx, ratio = locate(item["evidence"], rows)
if ratio < 1.0:
hint = ""
if idx is not None and ratio >= 0.6:
hint = f" (closest is {rows[idx][0]}, {ratio:.0%} of the quote matches, so it was reworded)"
errors.append(f"{tag}: evidence not found verbatim{hint}")
continue
cited.add(idx)
key = norm(item["evidence"])
if key in seen_evidence:
warns.append(f"{tag}: same evidence as {seen_evidence[key]}")
seen_evidence[key] = tag
lo, hi = max(0, idx - args.window), min(len(rows), idx + args.window + 1)
near = norm(" ".join(t for _, t in rows[lo:hi]))
at = rows[idx][0]
if section == "action_items":
owner = item["owner"]
if owner != "UNASSIGNED":
if attendees and norm(owner) not in attendees:
errors.append(f"{tag}: owner '{owner}' is not in attendees")
if norm(owner) not in near:
errors.append(f"{tag}: owner '{owner}' not named within {args.window} lines of {at}")
if OWNERLESS.search(rows[idx][1]):
errors.append(f"{tag}: evidence is ownerless wording (someone / we should / we need to) but owner is '{owner}'")
pair = rows[idx][1] + " " + (rows[idx + 1][1] if idx + 1 < len(rows) else "")
if not COMMIT.search(pair):
warns.append(f"{tag}: no commitment phrase in the evidence line or the reply after {at}")
due = item["due"]
if due and due.lower() != "none stated":
if norm(due) not in near:
errors.append(f"{tag}: due '{due}' not found near {at}")
elif owner != "UNASSIGNED" and DATEWORDS.search(rows[idx][1]):
warns.append(f"{tag}: due is 'none stated' but the evidence line has a date word ({at})")
text_fields = " ".join(str(item.get(k) or "") for k in ("task", "decision", "question", "conditions"))
if section == "decisions":
if not item["conditions"] and HEDGE.search(rows[idx][1]):
warns.append(f"{tag}: conditions is null but the evidence line hedges ({at})")
if OPINION.search(rows[idx][1]):
warns.append(f"{tag}: evidence at {at} is opinion wording ('I think', 'I'd say'); was it agreed?")
whole = norm(" ".join(t for _, t in rows))
for num in re.findall(r"\d[\d.,:]*\d|\d", text_fields):
if not re.search(r"\b" + re.escape(norm(num)) + r"\b", whole):
errors.append(f"{tag}: number '{num}' never appears in the transcript")
for i, (stamp, text) in enumerate(rows):
if i not in cited and STRONG.search(text):
warns.append(f"possible missed commitment at {stamp}: {text[:80]}")
for e in errors:
print("ERROR", e)
for w in warns:
print("WARN ", w)
items = sum(len(data.get(s, [])) for s in KEYS)
print(f"{items} items checked, {len(errors)} errors, {len(warns)} warnings")
return 1 if errors else 0
if __name__ == "__main__":
sys.exit(main())
Here is what each check is for, and where it is blind.
| Check | Catches | Blind to |
|---|---|---|
| Quote must appear in the transcript (case and punctuation ignored) | Invented or reworded evidence | A real quote attached to a wrong claim |
| Owner must be in the attendee list and named within two lines of the quote | Invented owners | A real attendee given someone else's task, if both are named nearby |
| Ownerless wording ("someone", "we should") with a named owner | Suggestions turned into assignments | Ownerless ideas phrased as questions |
| Commitment phrase in the evidence line or the reply after it | Tasks nobody accepted | Sarcasm, or a commitment made three lines later |
| Due phrase must appear near the quote | Wrong or invented dates | Dates that are right words in the wrong order |
| Numbers in task, decision or question must appear in the transcript | Transposed or invented digits | A real number attached to the wrong thing |
| Decision with null conditions where the evidence line hedges | Dropped "unless" clauses | Hedges in a different line |
| Decision whose evidence is opinion wording ("I think") | Ideas promoted to decisions | Opinions with no marker words |
| Strong commitment lines nobody cited | Missed action items | Commitments phrased without "I'll" or "I can" |
What did the checker say on the clean file?
The clean JSON has seven items I wrote by hand from the transcript, including the UNASSIGNED refund-wording task and both open questions.
7 items checked, 0 errors, 0 warnings
No errors and no warnings. That does not prove the file is right. It means nothing the checker knows how to test was wrong.
What did it say when I planted mistakes?
The second JSON file has these planted problems, each resembling a common model error:
- A decision for the 15 October export with the "as long as the sample passes" condition left out.
- A decision that is really Meera's opinion about rewriting the email.
- Dev's task with the quote reworded ("I will" instead of "I'll") and the wrong due day (Wednesday).
- The rewrite suggestion turned into a task for Meera.
- The ownerless refund-wording task handed to Dev.
- A backlog item for "Priyanka", who is not in the meeting, with 221 tickets instead of 212.
- The real commitments from Meera and Asha left out.
Output:
ERROR action_items[1]: evidence not found verbatim (closest is 00:24, 96% of the quote matches, so it was reworded)
ERROR action_items[2]: evidence is ownerless wording (someone / we should / we need to) but owner is 'Meera'
ERROR action_items[3]: evidence is ownerless wording (someone / we should / we need to) but owner is 'Dev'
ERROR action_items[4]: owner 'Priyanka' is not in attendees
ERROR action_items[4]: owner 'Priyanka' not named within 2 lines of 02:20
ERROR action_items[4]: number '221' never appears in the transcript
WARN decisions[1]: conditions is null but the evidence line hedges (00:46)
WARN decisions[2]: evidence at 01:15 is opinion wording ('I think', 'I'd say'); was it agreed?
WARN action_items[2]: no commitment phrase in the evidence line or the reply after 01:15
WARN action_items[3]: no commitment phrase in the evidence line or the reply after 01:52
WARN action_items[4]: no commitment phrase in the evidence line or the reply after 02:20
WARN possible missed commitment at 00:24: Dev: The export works for September but October fails on the tax column. I'll fi
WARN possible missed commitment at 01:36: Meera: Sure, I'll have them by Monday.
WARN possible missed commitment at 02:44: Asha: Agreed, I'll raise it with Rohan next week. Thanks everyone.
6 items checked, 6 errors, 8 warnings
Everything I planted was flagged. Two details are worth knowing. First, problem 3 hid a second problem: when a quote does not match, the checker skips the remaining tests for that item. After I fixed the quote by hand and re-ran, the wrong due day appeared as due 'Wednesday' not found near 00:24. So fix the first error, then run the checker again. Second, the three "possible missed commitment" lines (Dev, Meera, Asha) come from a crude search for phrases like "I'll", so they would also fire on a commitment you deliberately left out as trivial. Read WARN lines as prompts to look, not as a full audit.
What does a pass not prove?
I wrote one more file with a single item: Dev, due Thursday, task "Send the finished export to the finance team", with the real quote "I'll fix it and send you a sample file by Thursday." The task is wrong, because Dev promised a sample file to Asha. The result:
WARN possible missed commitment at 01:36: Meera: Sure, I'll have them by Monday.
WARN possible missed commitment at 02:44: Asha: Agreed, I'll raise it with Rohan next week. Thanks everyone.
1 items checked, 0 errors, 2 warnings
Zero errors. The quote exists, Dev is named, the commitment phrase is there, Thursday is there. Only the meaning is wrong, and a script that matches strings cannot see that. This is why step 6 stays, and why the timestamps matter: for each item you intend to send, read the quoted line in the transcript and confirm the task matches what was said.
What do I do when the checker finds errors?
Send the errors back to the model, not the whole job. This is an untested template:
Your JSON failed these automated checks against the transcript:
[paste ERROR and WARN lines]
Fix only the items named. For each fix, copy an exact sentence from the
transcript as evidence. If an item cannot be supported by the transcript,
remove it. Add an item only if a "possible missed commitment" line names it.
Return the full corrected JSON.
Then run the checker again. Two rounds is a reasonable limit. If errors persist, extract by hand or switch to a stronger model, because a model that keeps inventing owners after being told exactly what failed is not going to improve on round three.
How do I handle a long meeting?
Split the transcript at agenda boundaries and extract per chunk, as the meeting notes post suggests, but run the checker against the full transcript each time. The quote search covers every line, so a chunk-level extraction is still verified against everything that was said. Then run a merge prompt that asks which decisions a later chunk changed. If the whole transcript is too long for one request, the context window budget post covers what to cut.
Where does the data go, and who agreed to it?
The audio stays on your machine through step 3 if you use the local script. At step 4 it depends on the model. A hosted model receives the full transcript, which for a client call or an HR conversation may be exactly what you promised not to share. The local vs cloud privacy guide lays out the options, and the Ollama beginner guide shows how to run a model locally. Separately, recording someone and processing their words with AI are different acts. Tell attendees before you start, and ask a lawyer if the meeting involves another country or a regulated field. I am not giving legal advice here.
When is this pipeline not worth it?
Skip it for stand-ups where nothing was decided, and for meetings you can summarise from memory in three lines. Skip the model entirely if the meeting had one action item: type it. The pipeline earns its keep when a meeting produces many commitments across several people and the cost of a wrong owner is a missed deadline. If you are unsure about transcription quality on your own voices, run the accuracy test protocol before trusting any of this on real meetings.
