You finish a business book, ask a chatbot for a summary, and get three crisp paragraphs that make the book sound simpler and more certain than it was. Two weeks later you repeat one of its claims in a meeting, someone asks where it's from, and you can't find it in the book.
That's the central risk when you try to read nonfiction books with AI: the summary feels like understanding, and the nuance is the first thing to go. This post is a workflow that keeps the nuance in. It's question-first, chapter-by-chapter, with a verification step you can run yourself. The script and its output below are real (run on 2026-10-09). The text it checks is a short chapter I invented for the demo, not a real book, and the prompts are untested templates.
What goes wrong when AI summarises a book?
Four recurring failures, all checkable:
- Hedges vanish. "Walk-aways fell only after the chalkboard went up" becomes "chalkboards reduce walk-aways".
- Scope grows. A result from one town becomes a general rule.
- Quotes are rebuilt, not copied. The model outputs something that reads like a quote but is a paraphrase, or is wholly invented.
- The model fills the gaps from its general knowledge, not from the book, so you can't tell the author's claim from the model's.
Every step below targets one of these. For the general habit of checking AI answers, the fact-checking checklist goes wider; this post is about books.
Step 1: Write your questions before you open the book
Spend five minutes before page one. Write down:
- What do I want this book to settle? (two or three questions)
- What do I already believe about it, and what would change my mind?
- What would make this book not worth finishing?
If you want AI help here, ask it for questions, not answers, and give it only the title, table of contents and back-cover blurb:
I'm about to read a nonfiction book. Here are its title, table of contents and
blurb: [PASTE]. Suggest 8 questions a sceptical reader should hold in mind
while reading. Do not summarise the book or guess its conclusions. Base the
questions only on the structure shown.
Your questions give you a lens, and they give you a baseline: after reading, compare what you expected with what the author actually argued.
Step 2: Read the chapter yourself first
This sounds obvious. It's the step people skip, and it's where the nuance lives. Read the chapter, mark what surprised you, then use AI as a check on your understanding, not a substitute for it. Active recall works the same way for books as for exams; the study prompts post explains the reasoning.
Step 3: Write your own summary, then compare
Write three to five sentences on the chapter's argument, in your own words, before you ask the model anything. Then paste only text you have the right to use (see the copyright section below) and use the prompt that preserves qualifiers:
Using ONLY the chapter text below, list the author's main claims. For each:
1. The claim in one sentence.
2. Every qualifier the author attaches (sample size, place, time period, "may",
"in some cases", stated limitations). Write "none stated" if there are none.
3. A short direct quote that supports it, with the paragraph number.
Do not add anything that is not in the text. If you can't find a quote for a
claim, say "no direct quote".
Then compare with my summary and tell me what I got wrong or left out.
My summary: [PASTE]
CHAPTER TEXT:
[PASTE]
"None stated" and "no direct quote" are doing the work. They give the model a legitimate way to admit a gap instead of filling it.
For long chapters, split the text at section breaks and run the prompt per section; very long inputs are where models lose detail, as covered in working with long documents.
Step 4: Verify the quotes and numbers against the text
Don't eyeball it. If you have the chapter as text, check every quoted claim mechanically. Here's a short script I wrote and ran. It normalises case, whitespace and curly quotes, then looks for each quote in the chapter. If there's no exact match but a close one, it prints the real wording (the signature of a paraphrase posing as a quote). It also flags numbers in the claim that never appear in the chapter.
#!/usr/bin/env python3
"""Check quotes from an AI chapter summary against the chapter text you own.
Usage: python3 check_quotes.py chapter.txt claims.json
Verdicts:
EXACT quote found verbatim (ignoring case, whitespace, curly quotes)
NEAR no exact match, but a very similar passage exists (likely a paraphrase
presented as a quote; print the real wording and fix the quote)
MISSING nothing similar found (possible invention)
Also flags numbers in the claim that never appear in the chapter.
"""
import difflib
import json
import re
import sys
def norm(s):
s = s.replace("’", "'").replace("“", '"').replace("”", '"')
return re.sub(r"\s+", " ", s).strip().lower()
def best_window(text, quote):
words = text.split()
n = len(quote.split())
best, best_r = "", 0.0
for size in (n - 1, n, n + 1, n + 2):
if size < 1:
continue
for i in range(0, max(1, len(words) - size + 1)):
cand = " ".join(words[i:i + size])
r = difflib.SequenceMatcher(None, quote, cand).ratio()
if r > best_r:
best, best_r = cand, r
return best, best_r
def main(chapter_path, claims_path):
raw = open(chapter_path, encoding="utf-8").read()
text = norm(raw)
nums_in_text = set(re.findall(r"\d+", text))
bad = 0
for c in json.load(open(claims_path, encoding="utf-8")):
q = norm(c["quote"])
if q in text:
verdict, note = "EXACT", ""
else:
cand, r = best_window(text, q)
if r >= 0.75:
verdict, note = "NEAR", f' real wording: "{cand}" (similarity {r:.2f})'
else:
verdict, note = "MISSING", f" closest was {r:.2f}"
extra = []
for n in re.findall(r"\b\d+\b", c["claim"]):
if n not in nums_in_text:
extra.append(n)
if extra:
note += f" | number(s) in claim not found in chapter: {extra}"
if verdict != "EXACT" or extra:
bad += 1
print(f"[{c['id']}] {verdict}{note}")
print(f"\n{bad} of the claims need a fix")
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1], sys.argv[2]))
The demo chapter is about an invented baker who counted customers leaving a queue. I wrote six claims-with-quotes in the style of an AI summary, some faithful and some not. The output:
[1] EXACT
[2] EXACT
[3] EXACT
[4] NEAR real wording: "the chalkboard would work in every shop," (similarity 0.89)
[5] NEAR real wording: "that the bread was too expensive." (similarity 0.86)
[6] NEAR real wording: "she tried two changes." (similarity 0.84)
3 of the claims need a fix
Read what each result means:
- Claim 4 quoted "the chalkboard works in every shop". The chapter says the author did not claim the chalkboard would work in every shop. The script found the sentence, and its wording differs. A human reading the real sentence sees the reversal at once.
- Claim 5 quoted "most said the bread was too expensive". The real sentence is the author's first guess, which the chapter then rejects ("It was not."). The quote exists as a fragment but the meaning is the opposite.
- Claim 6 said three changes; the text says two.
Be clear about the limits. This script catches wording and misquotation; it does not understand meaning. Claim 5 shows that a fragment can be real and still be used to say the opposite, which is why the NEAR output tells you to read the surrounding sentence. The number check also has a hole: my claim 6 says "3 changes", and the digit 3 appears in the chapter heading ("Chapter 3"), so the number check did not flag it. The quote check caught it instead. Treat the script as a filter that narrows what you need to read, not a verdict.
For a real book, you need your own text version of the passage. If you can't legitimately get one, you can still check quotes by hand: find the page in the book and read the sentence around it.
Step 5: Ask what the author did not show
After the author's claims, use the model as a sceptical reader, with the same text:
Based only on the chapter text: what evidence does the author give for the main
claim, and what is missing? List (a) sample sizes or sources named, (b)
alternative explanations the author does not address, (c) claims stated more
strongly than the evidence shown. Quote the text for each point. If the text
doesn't allow a judgement, say so.
Then decide yourself. The model can only point out gaps in what was shown. It can't know whether the author is right, and it may add outside knowledge that sounds authoritative. If you want outside evidence, make that a separate step with a separate prompt, and look the sources up.
Step 6: Keep a claims ledger
After each chapter, keep a running note with three columns: the claim (your wording), the page, and your confidence (high, medium, check). Only write a claim into the ledger after you've seen it in the text. Before repeating anything from the book in public, look at its confidence column. Later, you can turn the strongest ledger entries into flashcards using the Anki workflow.
Is it copyright-safe to use a book with AI?
I'm not a lawyer, and this isn't legal advice; the law differs by country. What I did verify, on the US Copyright Office's fair use pages on 2026-10-09 (copyright.gov/fair-use/ and copyright.gov/help/faq/faq-fairuse.html; I re-read both pages directly, but read the originals rather than relying on my paraphrase):
- The Office lists four factors courts weigh: purpose and character of the use, nature of the work, amount and substantiality of the portion used, and the effect on the market for the work.
- The page says there is "no formula to ensure that a predetermined percentage or amount of a work... may be used without permission".
- The FAQ says "There are no legal rules permitting the use of a specific number of words", and that "in cases of doubt, the Copyright Office recommends that permission be obtained". Citing the author is good practice, but I did not find either page saying that credit alone counts as permission, so don't assume it does.
I did not verify Indian, UK or EU rules, or any AI tool's terms about uploaded books. Practical habits that lower your risk regardless:
- Work from your own notes and short excerpts, not whole chapters, where you can.
- Read the tool's terms about uploaded content, and look at whether your plan trains on your inputs.
- Don't strip DRM or use pirated copies. That's a separate legal problem.
- Never publish or share the book's text, a chapter-length summary that closely follows its structure, or a prompt pack that embeds the text.
- Use your own words for what you publish, and cite the book.
- Prefer public-domain or openly licensed books for practising this workflow.
If a tool is built around uploaded sources, such as the one covered in the NotebookLM study guide, the same rules apply to what you upload.
When this workflow is overkill
Skip it for a book you only need a few facts from: search the index and read those pages. Skip it for narrative or literary works, where the form is the point. And skip the AI step entirely when the stakes are low and you're reading for pleasure. The full routine earns its time on books you intend to cite, teach or act on.
The short version
- Write your questions before you read.
- Read the chapter first.
- Write your own summary.
- Ask the model for claims with qualifiers and quotes, using only text you can use.
- Check every quote and number against the text. Read the real sentence around any NEAR result.
- Log claims with page numbers and confidence, and share only your own words.
