Redact PII before it reaches the LLM: a gateway pattern

To keep customer PII out of a third-party model, put a redaction step inside your own network, directly in front of every outbound LLM call. Detect sensitive fields with a layered detector, swap them for reversible placeholder tokens before the prompt leaves your perimeter, then restore the real values in the response your user sees. The model never receives the raw data, and you get an audit trail that proves it.

That is the whole idea. The hard part is where you put it and how many places you forget to put it. A SaaS team shipping an AI feature usually gets the detection library right and the placement wrong.

Why "the vendor redacts it" is not enough

Most hosted AI platforms offer to scrub PII for you. The problem is timing. Their redaction runs after your text has already crossed the wire into their infrastructure. By then the raw email address, the support ticket, and the customer's home address have all left your control. For a SOC 2 or HIPAA boundary, "they promise to delete it" is a weaker claim than "it never arrived."

The distinction that matters for compliance is your perimeter. Redaction has to happen on your side of the network boundary so the sensitive value is gone before the request is serialized and sent. Anything that happens on the provider's side shrinks nothing about your exposure, and it gives your security reviewer nothing to point at.

This is the same reasoning we apply to any outbound dependency in an AI feature. If you already route model traffic through a single control point, as we argued in our note on an AI gateway for production LLM architecture, you already own the right place to enforce redaction. If you do not, the redaction requirement is a good reason to build one.

The redaction gateway pattern

Treat redaction as a mandatory stage on the path out to the model, not as a helper you call when you remember. The gateway owns four responsibilities: detect PII, replace it, forward the clean prompt, and restore values on the way back. Because it sits on the boundary, the rule is uniform across every feature and every engineer who ships one.

Reversible tokenization

Masking (replacing an email with a block of asterisks) destroys information the model may need to do its job. A support-reply drafter that sees five identical asterisk blocks cannot tell which customer it is writing to. Reversible tokenization solves this: replace each detected value with a stable, meaningless placeholder, keep the placeholder-to-value mapping in memory for the duration of the request, send the tokenized prompt, and swap the real values back into the model's response.

A prompt that started as "Refund the order for jane@acme.io shipped to 44 Elm St" becomes "Refund the order for PERSON_EMAIL_1 shipped to ADDRESS_1" on the wire. The model reasons about the structure, returns a draft that references PERSON_EMAIL_1, and your gateway restores the real address before the human agent ever sees it. The customer reads their own data; the provider never did.

Layered detection

No single detector catches everything. Regular expressions are fast and precise for structured formats: emails, phone numbers, card numbers, national IDs, IP addresses. They are useless for a name buried in free text. A named-entity recognition model catches the names, organizations, and locations that regexes miss, at the cost of some latency and the occasional false positive. Run both, union the spans, and prefer over-redaction on the fields you cannot afford to leak.

Open detectors now span dozens of entity types across personal, contact, financial, credential, and government-identifier categories. You do not need to build this from scratch. What you do need is to decide, per field, whether a miss is a compliance incident or a cosmetic annoyance, and to tune recall accordingly. A leaked social security number and a leaked first name are not the same risk.

The agent leak trap: redact at every hop

Here is the failure mode that catches teams who did everything else right. They redact the first user message, feel safe, and ship. But a modern AI feature is rarely one call. A retrieval step pulls documents that contain PII. An agent calls a tool that returns a customer record. A summarization pass runs over raw logs. Every one of those is a fresh outbound model call, and every one is a new chance to leak.

Redacting only the entry point is like locking the front door and leaving every window open. The redaction stage has to sit at each outbound model call as a hard dependency in the request path, not as optional middleware someone can bypass under deadline. This is doubly true for agentic flows, where the number of model calls is decided at runtime and you cannot enumerate them in advance. If you are building approval-gated agents, the same discipline we describe in human-in-the-loop approval for AI agents applies to redaction: it belongs at the boundary the agent cannot route around.

The same boundary thinking protects you from a related risk. Untrusted retrieved content can carry injected instructions as well as PII, which is why we treat the outbound and inbound edges of an agent as security surfaces in our piece on prompt injection in production AI agents. Redaction and injection defense want to live in the same place.

A worked example: a support-desk SaaS

Picture a support-desk product that drafts reply suggestions. Tickets arrive full of PII: customer names, emails, order numbers, shipping addresses, sometimes a partial card number a customer pasted in frustration. The naive build sends the ticket straight to the model API. That is a data-processing agreement problem waiting to happen, and enterprise buyers will ask about it during procurement.

The gateway build looks like this. An inbound ticket hits the redaction stage first. A regex layer tags the email, order number, and any card-shaped digits; an NER layer tags the name and address. Each span becomes a token; the mapping lives in a per-request context object that never touches durable storage. The tokenized ticket, now safe, goes to the model. The draft comes back referencing tokens, the gateway rehydrates it, and the human agent sees a complete, correct suggestion. Total added latency is small, often under fifteen milliseconds for the regex pass plus whatever your NER model costs, which you can cache or run locally.

Two properties fall out of this design for free. First, the audit log. Every redacted request records what was detected and tokenized, exportable for a security or legal review. That turns "we believe customer data stays private" into a record you can hand an auditor. Second, provider portability. Because PII never reaches any specific vendor, switching models or splitting traffic across providers does not reopen the privacy question. Reliability patterns like the ones in reducing hallucinations in production RAG compose cleanly on top, because your retrieval and grounding logic operates on already-clean text.

What redaction does not solve

Redaction is a boundary control, not a correctness control. A tokenized prompt can still produce a wrong or unsafe answer; the model is simply wrong about anonymized data instead of real data. Keep your output validation and guardrails in place regardless, along the lines we cover in AI feature guardrails before launch.

Two more caveats. Reversible tokenization means the mapping exists somewhere for the length of the request; treat that context object as sensitive, keep it in memory, and never log it. And detection is probabilistic on free text, so measure recall on a labeled sample of your real tickets rather than trusting a library's marketing. A detector that catches 99 percent of emails and 80 percent of names has an 80 percent problem you need to know about before an auditor finds it.

Frequently asked questions

Is prompt-level redaction enough for GDPR or HIPAA?

Redacting before data leaves your perimeter is a strong control, but compliance is a program, not a library. You still need a data-processing agreement with your model provider, a retention policy, access controls, and an audit trail. Perimeter redaction makes those claims defensible; it does not replace them.

Does redaction hurt answer quality?

Rarely, if you use reversible tokenization instead of masking. The model reasons over stable placeholders that preserve structure, so a support drafter or classifier behaves almost identically. Quality drops mainly when you blank out fields the task actually needs, which is a tuning problem, not a fundamental limit.

Where should the redaction stage live?

At the same control point you use for model routing, rate limiting, and logging: a gateway every outbound call passes through. Scattering redaction across individual features guarantees someone forgets it. Centralizing it makes the rule uniform and auditable, which is the point of a gateway in the first place.

Can I just rely on my model provider's built-in redaction?

Only if you accept that your raw data reaches their infrastructure first. For many teams that is an acceptable risk; for anyone with a HIPAA, SOC 2, or enterprise-procurement obligation, it usually is not. The safer default is to strip PII on your side so the provider never receives it.