/

Voices

A Practical Guide to PII Redaction for AI Systems

Redacting PII before it reaches an LLM lowers the stakes of a breach, but redacted text you can swap back is still personal data. Liam Tengelis covers what GDPR, Quebec’s Law 25 and HIPAA ask for, how accurate open redaction tools are today, and an example architecture that sends an LLM only redacted text.

Redacting PII before it reaches an LLM lowers the stakes of a breach, but redacted text you can swap back is still personal data. Liam Tengelis covers what GDPR, Quebec’s Law 25 and HIPAA ask for, how accurate open redaction tools are today, and an example architecture that sends an LLM only redacted text.

Liam Tengelis

AI Architect

An isometric redacted document with a small placeholder map lifted out beside it; a thread runs from each redaction bar to its row in the map.

Redacting PII before it reaches an LLM lowers the stakes of a breach, but redacted text you can swap back is still personal data. Liam Tengelis covers what GDPR, Quebec’s Law 25 and HIPAA ask for, how accurate open redaction tools are today, and an example architecture that sends an LLM only redacted text.

PII (Personally Identifiable Information) requires special handling: We need to be careful about where we send it and how it is stored.

In the context of AI, handling PII safely matters more than ever: businesses are processing more and more unstructured data while new regulation is being written.

In this article we’ll survey the primary concerns from relevant regulations and walk through how an example AI deployment can be architected to best abide by those regulations.

We’ll also share a PII redaction toolkit your team can use to become familiar with the technical points of redaction.

Key takeaways

The limits of redaction and pseudonymization

We advocate for taking the same precautions with redacted data as you would with unredacted data. The argument is two-fold:

  1. If you retain the original PII from redacted text (in order to swap it back in for example), then this is technically classified as pseudonymized, or tokenized, data. Under GDPR and Quebec’s Law 25, pseudonymized data is regarded identically to personal data, for anyone who can “un-redact” it.

  2. It is impossible to guarantee perfect redaction while still preserving a text’s meaning.

Redacted text is still personal data if you keep the originals. The same sentence at three stages: raw text, pseudonymized with a placeholder map kept (still personal data under GDPR and Law 25), and anonymized with nothing kept (outside the law).

Thus while a PII redaction process is an important risk-minimization step, it’s not a cure-all for your privacy compliance.

Why redact

Redaction’s benefits are best seen in the context of a breach.

Legally, redaction has the following benefits:

In general, redacting PII is evidence of the kind of care that regulators look for in a data privacy policy. As good stewards of our customers’ personal data, redaction is a minimum protection we can offer them, even if technically imperfect.

The relevant regulatory concerns

Depending on the regions you operate in, different regulations will legally dictate the specifics of how we must handle personal information. There are key principles behind all such regulations though, and we would encourage thinking in terms of these principles instead of any particular regulation’s criteria.

Regulatory frameworks, in general, have the following concerns:

GDPR, Quebec Law 25 and HIPAA compared across nine concerns: personal data and when redaction ends it, permission to use it, stated purpose, how long to keep it, recipients and transfers, security, transparency and people's rights, breach notification, and accountability.

Sources for this table

EU AI Act

The EU AI Act doesn’t introduce any personal-data specific requirements directly. One potentially relevant aspect however for “high-risk” systems is that logs of decisions made by AI are required to be kept for at least 6 months (Article 19 for providers, Article 26 for deployers). Redaction is basically the only solution to satisfy both this requirement and GDPR requirements for storing personal data .

A survey of redaction technology

We’ve published the code to a ‘PII Toolbox’ that we’ll use to illustrate the relevant technical concerns in PII redaction / tokenization.

How to evaluate a redaction system

Consider the two competing concerns of a redaction system:

  • Utility of the redacted text

  • Redaction coverage

Thus we measure the quality of a redaction system through its:

A) Recall - the % of actual PII that is redacted

and

B) Precision - the % of redacted text that was actually PII

Since it’s generally preferable to err on the side of over-redaction, as a single metric for redaction system quality, we recommend the F2 score of Recall to Precision, which treats Recall as twice as important as Precision.

The F2 score judges a redaction system mostly on the PII it misses. An invented support message with four pieces of PII: the system redacted five things, three of them PII, for recall 75%, precision 60% and F2 0.71.

Structured forms

Structured PII like emails or card numbers are relatively straightforward to detect with deterministic rules and regular expressions. Here are some common structured forms we might need to redact:

  • Bank account numbers

  • Driver’s license numbers

  • Social insurance numbers

  • Phone numbers

  • Email addresses

  • Dates of birth

  • Credit and debit card numbers

  • Health insurance and patient ID numbers

  • Postal codes and addresses

Some of these PII types even have checksums we can use to be almost certain that we have correctly identified them. With that being said, the possibility of typos means that even for such well-structured PII types, we can not guarantee perfect redaction.

In our toolbox, we use Microsoft Presidio for structured PII redaction. You can find example redaction rates and performance rates here.

On two synthetic test sets, Nemotron and Gretel, Presidio found at least 92% of every structured type we tested, except SWIFT/BIC codes, which it almost never found.

Presidio is cheap to run, with a peak memory of 0.45 GiB at a rate of 0.003 seconds per 1,000 characters.

Unstructured forms

Unstructured PII forms include:

  • Names

  • Addresses

  • Occupations

These have no reliable deterministic method of detection, although we can use ‘hints’, like names of all known users of a given system, to improve our redaction rate. Ultimately however, the unpredictability of unstructured data requires us to rely on language models for robust unstructured PII detection.

In our toolbox, we compare the redaction rates and resource requirements of some good small model options. All these options fit comfortably within AWS Lambda’s 10 GB memory limit, peaking at 2.78 GiB at most, but have varying redaction performance, especially when it comes to name detection.

Identity linking

To maximize the usefulness of tokenized text for an LLM, it’s important that references to the same person are still identifiable as the same person in the tokenized text, even if the name isn’t duplicated. For example, “David” might simply be “him” later on in a document. This motivates the use of another language model for the purpose of coreference resolution. The toolbox uses Maverick, a coreference model, followed by a nickname step that links short forms such as “Dave” to “David”.

With a ‘coreference layer’, GLiNER + Presidio correctly resolved differently mentioned identities 80% of the time, vs 3% of the time without. You can find the full results here.

A coreference layer raised correct identity linking from 3% to 80%. David, him and Dave in an example passage all resolve to one placeholder, PERSON_1.

What accuracy rates to expect

Names, places, organisations and dates

We tested four language models on two test sets:

  • TAB is English court judgments from the European Court of Human Rights, whose names are mostly European

  • Worldwide is English newswire from around the world, with many non-Western names

The scores use relaxed matching: a detection counts if it has the right type and covers at least half of the right text.

F2 by type (full table):

F2 score by model and type. spaCy large: names TAB 0.85, names Worldwide 0.81, locations 0.65, organisations 0.48, dates 0.95. spaCy small: 0.74, 0.71, 0.64, 0.46, 0.94. GLiNER: 0.65, 0.73, 0.58, 0.57, 0.77. RoBERTa: 0.85, 0.96, 0.73, 0.51, dates not measured.

Emails, phone numbers, card numbers and bank details

We tested Presidio on two synthetic datasets:

Keep in mind that synthetic data is likely cleaner than real production data.

Share of each type that Presidio found (recall):

Presidio recall on Gretel and Nemotron: email 97% and 99.6%, phone number 96% and 99.6%, card number 98% and 99.8%, valid card numbers 100% and 100%, IBAN 92% (Gretel only), valid IBAN 97% (Gretel only), SWIFT/BIC code 0% and 6%.

Cost

Memory and speed:

Peak memory and time per 1,000 characters: GLiNER 2.78 GiB and 0.112 s, RoBERTa 2.49 GiB and 0.055 s, spaCy large 1.33 GiB and 0.017 s, spaCy small 0.72 GiB and 0.015 s, Presidio 0.45 GiB and 0.003 s. All sit far below AWS Lambda's 10 GB limit.

Example

To make this concrete, let’s consider an example scenario:

Suppose a European online retailer wants an LLM to draft replies to customer support emails. The emails contain names, postal addresses, phone numbers, email addresses and potentially other PII.

Here is a simple architecture that sends only tokenized text to the LLM:

Only redacted text leaves the retailer's systems. A customer email goes to a redaction function in the retailer's EU systems, which writes placeholders and originals to an encrypted placeholder map and sends redacted text only to an LLM on an Azure EU endpoint. The LLM's draft with placeholders goes to a swap step that puts the real details back, then to a support agent who reviews it and replies to the customer.

What still applies after redaction

Because we keep the placeholder map in order to swap names back in, the redacted text is pseudonymised data, and under GDPR pseudonymised data is still personal data. So the following all still apply:

  • A written contract with the provider.

  • Where the data is processed. Use a deployment in an EU region, which keeps prompts and responses in the EU.

  • Purpose limitation and minimisation. Send the model only what it needs to perform its task, and nothing more.

  • Retention. The map should be kept only as long as the ticket needs it.

  • Breach notification. A leak of the redacted text at the provider is still a breach, but with the identifying details replaced, it is far less likely to require telling the regulator or customers. The placeholder map is what would make a leak serious, so if persisted, it should be encrypted.

Conclusion

Redaction lowers the stakes of a breach, but it is not a cure-all for private data handling. Maintain discipline with the handling of redacted data, and continuously evaluate your redaction system in production. We hope this article has given you the tools to do so confidently.

Have an AI priority that has to work?

Have an AI priority that has to work?

Tell us what you're working on.

Senior AI expertise for work that has to perform.

Senior AI expertise for work that has to perform.

Senior product and engineering judgment

for AI that has to work.