PII Redaction Before LLM Calls: Python Regex vs Presidio

A customer pastes a refund request into your support bot. It includes their email, a phone number, a UPI id and an Aadhaar number they typed "for verification". Your code forwards the whole thing to an LLM API. The fix is to detect and replace the identifiers before the call, then restore them in the reply. Regex gets you most structured IDs; a named-entity model gets you names and places; neither gets you everything.

I wanted to see how far each goes, so on 2026-10-09 I built a regex redactor and ran Microsoft Presidio on the same invented text, then counted the misses. All values below are made up. Every line of output is from a real run.

What did I run?

• Python 3.13, presidio-analyzer and presidio-anonymizer 2.2.364, spaCy 3.8.16, model en_core_web_lg 3.8.0, installed in a fresh virtualenv.
• A ten-document test set: Indian-flavoured emails, phones, PAN, Aadhaar, IFSC, UPI, a card number, names, a Hindi sentence, API keys and an address.
• Three redactors: my regex version, Presidio with default recognisers, and Presidio plus custom Indian recognisers.
• The full script is about 200 lines. The key parts are below.

The generated Aadhaar-style number in the test passes the real Verhoeff checksum, so the validator is exercised. It is not a real person's number.

How do you redact structured identifiers with regex?

Order matters: most specific pattern first. Add a checksum where one exists, so a 12-digit SKU doesn't get treated as an Aadhaar number.

import re

def luhn_ok(num: str) -> bool:
    s, alt = 0, False
    for ch in reversed(num):
        n = int(ch)
        if alt:
            n *= 2
            if n > 9:
                n -= 9
        s += n
        alt = not alt
    return s % 10 == 0

# verhoeff_ok(): standard Verhoeff tables, 12-digit Aadhaar check (omitted here for length)

PATTERNS = [  # most specific first
    ("API_KEY", re.compile(r"\b(?:sk-[A-Za-z0-9_\-]{16,}|AKIA[0-9A-Z]{16})\b")),
    ("EMAIL", re.compile(r"\b[\w.+-]+@[\w-]+(?:\.[\w-]+)+\b")),
    ("UPI", re.compile(r"\b[\w.\-]{2,}@(?:ok\w+|ybl|paytm|upi|ibl|axl|sbi|apl)\b")),
    ("PAN", re.compile(r"\b[A-Z]{5}[0-9]{4}[A-Z]\b")),
    ("IFSC", re.compile(r"\b[A-Z]{4}0[A-Z0-9]{6}\b")),
    ("ACCOUNT", re.compile(r"(?<=A/c )\d{9,18}\b")),
    ("CARD", re.compile(r"\b\d(?:[ -]?\d){12,18}\b")),
    ("AADHAAR", re.compile(r"\b[2-9]\d{3}[ -]?\d{4}[ -]?\d{4}\b")),
    ("PHONE", re.compile(r"(?<!\d)(?:\+91[ -]?)?[6-9]\d{4}[ -]?\d{5}(?!\d)")),
]

def regex_redact(text):
    mapping, counts = {}, {}
    def make(kind, value):
        for ph, v in mapping.items():
            if v == value:
                return ph
        counts[kind] = counts.get(kind, 0) + 1
        ph = f"[{kind}_{counts[kind]}]"
        mapping[ph] = value
        return ph
    for kind, pat in PATTERNS:
        def sub(m, kind=kind):
            raw = m.group(0)
            digits = re.sub(r"\D", "", raw)
            if kind == "CARD" and not luhn_ok(digits):
                return raw
            if kind == "AADHAAR" and not (len(digits) == 12 and verhoeff_ok(digits)):
                return raw
            return make(kind, raw)
        text = pat.sub(sub, text)
    return text, mapping

def restore(text, mapping):
    for ph, v in mapping.items():
        text = text.replace(ph, v)
    return text

My first version got two things wrong, and they're typical. The account number 50100123456789 happened to pass the Luhn check (about one in ten random digit strings do), so my card rule claimed it and labelled it [CARD_1]. And the card pattern (?:\d[ -]?){13,19} swallowed the space after the number, producing Card [CARD_1]exp 09/28. The version above puts the account rule first, anchors it to the A/c prefix, and only allows a separator between digits. The lesson: a checksum reduces false positives, it doesn't remove them, so context (the A/c prefix) helps.

How do you set up Presidio for Indian identifiers?

Presidio's analyzer is a framework: NLP engine plus a registry of recognisers. The default set has no Indian identifiers, so I added pattern recognisers, plus a Verhoeff validator for Aadhaar:

from presidio_analyzer import AnalyzerEngine, PatternRecognizer, Pattern
from presidio_analyzer.nlp_engine import NlpEngineProvider
from presidio_anonymizer import AnonymizerEngine

nlp = NlpEngineProvider(nlp_configuration={
    "nlp_engine_name": "spacy",
    "models": [{"lang_code": "en", "model_name": "en_core_web_lg"}],
}).create_engine()
eng = AnalyzerEngine(nlp_engine=nlp, supported_languages=["en"])

class AadhaarRecognizer(PatternRecognizer):
    def validate_result(self, pattern_text):
        digits = re.sub(r"\D", "", pattern_text)
        return len(digits) == 12 and verhoeff_ok(digits)

eng.registry.add_recognizer(PatternRecognizer(
    supported_entity="IN_PAN",
    patterns=[Pattern("pan", r"\b[A-Z]{5}[0-9]{4}[A-Z]\b", 0.85)]))
eng.registry.add_recognizer(AadhaarRecognizer(
    supported_entity="IN_AADHAAR",
    patterns=[Pattern("aadhaar", r"\b[2-9]\d{3}[ -]?\d{4}[ -]?\d{4}\b", 0.85)]))
# ...same for IFSC, UPI, API_KEY, IN_PHONE

results = eng.analyze(text=text, language="en")
print(AnonymizerEngine().anonymize(text=text, analyzer_results=results).text)

The default entity list printed by get_supported_entities("en") on this install:

AGE, CREDIT_CARD, CRYPTO, DATE_TIME, EMAIL, EMAIL_ADDRESS, IBAN_CODE, ID, IP_ADDRESS,
LOCATION, MAC_ADDRESS, MEDICAL_LICENSE, NRP, ORGANIZATION, PERSON, PHONE_NUMBER,
UK_NHS, URL, US_BANK_NUMBER, US_DRIVER_LICENSE, US_ITIN, US_PASSPORT, US_SSN

No PAN, Aadhaar, IFSC or UPI. That matches what happened next.

What did the output look like?

Four of the ten documents, verbatim. "RE" is my regex version, "PD" is Presidio with default recognisers, "PC" is Presidio plus my custom recognisers.

IN : KYC: PAN ABCDE1234F, Aadhaar 2345 6789 0124, IFSC HDFC0001234, A/c 50100123456789, UPI rohan@okhdfcbank.
RE : KYC: PAN [PAN_1], Aadhaar [AADHAAR_1], IFSC [IFSC_1], A/c [ACCOUNT_1], UPI [UPI_1].
PD : KYC: <ORGANIZATION>, Aadhaar 2345 <DATE_TIME>, <ORGANIZATION>, A/c <CREDIT_CARD>, <ORGANIZATION> rohan@okhdfcbank.
PC : KYC: <ORGANIZATION>, Aadhaar <IN_AADHAAR>, <ORGANIZATION>, A/c <CREDIT_CARD>, <ORGANIZATION> <IN_UPI>.

IN : Invoice dated 2026-10-09 total 4,82,500 for SKU 123456789012, pincode 682001.
RE : Invoice dated 2026-10-09 total 4,82,500 for SKU 123456789012, pincode 682001.
PD : Invoice dated <DATE_TIME> total 4,82,500 for SKU <US_BANK_NUMBER>, pincode <US_DRIVER_LICENSE>.

IN : Priya Nair from Kochi met Dr. Anil Kapoor. Later ask priya nair to confirm.
RE : Priya Nair from Kochi met Dr. Anil Kapoor. Later ask priya nair to confirm.
PD : <PERSON> from <LOCATION> met Dr. <PERSON>. Later ask <PERSON> to confirm.

IN : मेरा नाम सुनीता शर्मा है और मेरा नंबर 9876543210 है
RE : मेरा नाम सुनीता शर्मा है और मेरा नंबर [PHONE_1] है
PD : <DATE_TIME> <PERSON> शर्मा है और मेरा नंबर <UK_NHS> है

(PC matched PD on the last three documents, so I've left it out.)

What each result tells you:

• Default Presidio mislabels Indian IDs. The PAN became ORGANIZATION, the IFSC too, the Aadhaar number was split so 2345 survived, and the UPI id was left in the clear. Custom recognisers fixed the Aadhaar and UPI parts, but the PAN and IFSC in that document still show as ORGANIZATION in PC. I didn't dig into why my patterns lost the overlap. The text was redacted either way, but the label is wrong, which matters if you route on entity type.
• Presidio over-redacts. The invoice date, the SKU and the pincode were all replaced. My regex left them alone because the SKU fails the Verhoeff check. In a RAG pipeline, over-redaction costs you answer quality.
• Regex can't see names. Priya Nair and Anil Kapoor went straight through.
• Hindi is a mess. en_core_web_lg is an English model. It labelled "मेरा नाम" ("my name is") as DATE_TIME, tagged the first name as PERSON and left the surname शर्मा in the clear. The regex version caught the phone number but not the name. If your users write in Indian languages, neither tool, as configured here, is enough.
• Presidio produced one more false positive I didn't plan: in the first document, "Reach me at" became <PERSON> me at. The verb was tagged as a name.

What did each approach miss?

I defined 22 strings that must not survive, and 6 that must. Counting is crude: a string counts as leaked if it survives whole or if a four-plus character chunk containing a digit survives.

Don't read that last zero as "perfect". My counter ignores partial leaks of words, and reading the output I can see:

• The Hindi surname शर्मा remained.
• In Flat 12B, Lotus Apartments, MG Road, Kochi, Presidio caught the apartment name as ORGANIZATION and the city as LOCATION, but Flat 12B and MG Road stayed.
• Phone number spoken in words ("nine eight seven six...") was only half replaced, and an email written as "rohan dot mehta at example dot com" kept at example dot com.
• My regex missed the lowercase PAN in "my pan is abcde1234f" because the pattern is uppercase-only, and so did default Presidio. Presidio plus my custom PAN recogniser caught it as <IN_PAN>. As far as I can tell Presidio compiles pattern recognisers case-insensitively by default, which is why, but I didn't confirm that in its source. For a PAN, case-insensitive is arguably what you want.

I had also run the smaller en_core_web_sm model first. It labelled that lowercase PAN as PERSON and missed the lowercase name "priya nair", while lg caught the name but left the PAN. Model size changed results in both directions, so test on your own text.

Real cost of the better model: the en_core_web_lg download was 400 MB.

How do you put the real values back after the model replies?

Use numbered placeholders and keep the mapping server-side. This stub run used no API and no key, so it only shows the mechanics:

SENT TO MODEL : Customer [EMAIL_1] (phone [PHONE_1]) says order 4471 never arrived and asks for a refund to UPI [UPI_1].
MODEL REPLY   : Draft reply: Hello [EMAIL_1], sorry about order 4471. We will refund to [UPI_1] and call you on [PHONE_1] if needed.
RESTORED      : Draft reply: Hello rohan.mehta@example.com, sorry about order 4471. We will refund to rohan@okhdfcbank and call you on +91 98765 43210 if needed.

The "model reply" is hand-written by a stub function, labelled as illustrative. A real model might rewrite or drop a placeholder, so after restoring, check that every placeholder you sent is accounted for and flag the reply if one is missing or invented. Presidio's default <PERSON> labels are not reversible. If you want round trips with Presidio, you'd write a custom operator that numbers entities; I didn't build or test that.

The mapping dictionary holds the real values. Keep it in memory for the request, or encrypt it if you must store it, and put it under the same retention rules as the original data. Under India's data protection law, personal data in a mapping table is still personal data. The DPDP checklist covers where those stores matter.

Which approach should you pick?

• Start with regex plus checksums for structured IDs. It's cheap, deterministic and fast. Unit-test every pattern with positives and near-misses.
• Add NER (Presidio or similar) when names and places matter, and expect over-redaction and gaps. Tune on your own data, in your own languages.
• Reduce what you send. The strongest control is not putting the field in the prompt at all. Redaction is a second line.
• Redact logs too, not just model calls. Traces and error trackers often store the raw prompt.
• Don't promise customers "all PII is removed". Every configuration in my small test missed or mangled something.
• Run it locally if the text can't leave at all. A local model removes the vendor leg entirely; see running LLMs locally with Ollama.

For the wider question of what to keep out of chatbots in the first place, see what not to paste into ChatGPT, and for what each vendor does with data after you send it, the retention and training table.