PII Masking

Customers paste phone numbers, emails, Aadhaar, PAN and card numbers into chats. Mask them before the text goes to the LLM provider, your logs or your eval datasets, and keep a private mapping so you can put real values back in the reply if needed.

PII (Personally Identifiable Information) is any data that can identify a person, alone or combined with other data. Masking (also called redaction or pseudonymisation when reversible) replaces each value with a placeholder like <PHONE_1>. The model rarely needs the real value: to answer "refund to my UPI ID", it only needs to know that a UPI ID was given. This is the principle of data minimisation: every system that does not need the raw value should not receive it. Fewer copies means less to leak, and less to delete when a customer asks.

1. Detect: find PII with patterns (regex for structured IDs) and models (NER for names and addresses).
2. Validate: reduce false positives with checksums, such as the Luhn check digit used by card numbers.
3. Replace: swap each value for a numbered token and store token → value in a short-lived, access-controlled map.
4. Restore: if the reply must show the value back to the same user, unmask just before sending.

import re

def luhn_ok(number):
    digits = [int(d) for d in re.sub(r"\D", "", number)][::-1]
    total = sum(d if i % 2 == 0 else (d * 2 - 9 if d * 2 > 9 else d * 2) for i, d in enumerate(digits))
    return total % 10 == 0

RULES = [  # order matters: longer / more specific first
    ("EMAIL",   r"[\w.+-]+@[\w-]+\.[\w.]+"),
    ("UPI",     r"\b[\w.-]+@(?:okaxis|oksbi|okhdfcbank|okicici|ybl|paytm|ibl|axl|upi)\b"),
    ("CARD",    r"\b(?:\d[ -]?){15}\d\b"),
    ("AADHAAR", r"\b[2-9]\d{3}[ -]?\d{4}[ -]?\d{4}\b"),
    ("PAN",     r"\b[A-Z]{5}\d{4}[A-Z]\b"),
    ("PHONE",   r"(?:\+91[ -]?)?\b[6-9]\d{4}[ -]?\d{5}\b"),
]

def mask(text):
    mapping, counts = {}, {}
    for label, pattern in RULES:
        def repl(m):
            if label == "CARD" and not luhn_ok(m.group()):
                return m.group()
            counts[label] = counts.get(label, 0) + 1
            token = f"<{label}_{counts[label]}>"
            mapping[token] = m.group()
            return token
        text = re.sub(pattern, repl, text)
    return text, mapping

def unmask(text, mapping):
    for token, value in mapping.items():
        text = text.replace(token, value)
    return text

msg = ("I am Priya, [email protected], +91 98765 43210. Refund to priya@okaxis please. "
       "Card 4111 1111 1111 1111, PAN ABCDE1234F, Aadhaar 2345 6789 0123. Order SE1042.")
masked, mapping = mask(msg)
print(masked)
print(sorted(mapping))
print(unmask("We will refund to <UPI_1> within 5 days.", mapping))

I am Priya, <EMAIL_1>, <PHONE_1>. Refund to <UPI_1> please. Card <CARD_1>, PAN <PAN_1>, Aadhaar <AADHAAR_1>. Order SE1042.
['<AADHAAR_1>', '<CARD_1>', '<EMAIL_1>', '<PAN_1>', '<PHONE_1>', '<UPI_1>']
We will refund to priya@okaxis within 5 days.

Every structured ID is replaced, and the order ID SE1042 is left alone because the support flow needs it. The name "Priya" is not masked: regex cannot tell names from ordinary words. The rule order matters too: EMAIL runs before UPI, and priya@okaxis has no dot after the @, so EMAIL skips it and UPI catches it. The last line shows unmasking for the user's own reply.

Regex catches structured IDs well. Names and addresses need an NER model; open-source Microsoft Presidio combines both and is easy to extend with Indian recognisers. Mask logs at the logging layer so no one forgets.

• Mask PII before it reaches the LLM provider, logs or datasets (data minimisation).
• Regex + checksums for structured IDs; NER models for names and addresses.
• Reversible tokens let you restore values for the same user.
• Keep IDs the task needs; review misses regularly.