Redaction, in an AI feature, is removing or replacing the fields that identify a person after your server builds the prompt and before the request leaves for OpenAI, Anthropic or another model provider. How to redact personal data before LLM calls is a question of where: one wrapper every call passes through, fed by a list of what each sends.
What redaction before a model call is, and where it sits
Redaction before a model call is four moves on the server: list what each call sends, stop sending what the feature does not need, swap what remains for placeholders the server restores in the response, and record the provider’s terms for whatever still leaves. It belongs in one wrapper around the provider SDK, never in the browser.
The order matters more than the tooling. Cutting comes before masking because a field that is never sent cannot leak, and the record comes last because it describes only what survives the first three moves. It is one control inside the wider job of how to manage SaaS data compliance, and each move gets its own section below.
The wrapper lives on the server, in one module that wraps the provider SDK. Never put it in the browser: a redaction the client performs is a redaction the client can skip, because anyone can call your API route without your front end. One module also gives a CI rule a single place to check, which comes up again in the masking section.
Five things can happen to a field on its way out. The costs in this table are my reading of what each one takes away from the model.
| Technique | What the model loses | When to use it |
|---|---|---|
| Drop the field | Everything about it: a dropped name cannot go in a greeting | The feature does not need it, which is the default for most fields |
| Generalize it | Precision: a city instead of a street address, an age band instead of a birth date | The feature needs the rough value, not the exact one |
| Replace it with a reversible placeholder | The value, not the role: <PERSON_1> can still be greeted, and the server puts the name back in the reply | The reply must contain the value, but the model never needs to read it |
| Hash it | The value; with one salt per request, equal values still give equal hashes, so the model can match records | The model only has to tell records apart or match them |
| Send as is | Nothing | The model must read the value to do the job, and you have written that choice down |
The prompt is not the only exit. The entry on sensitive information disclosure in the OWASP LLM Top 10 names other channels personal data leaves through: tool-call arguments, retrieved chunks, logs and embeddings.
Inventorying the user data that reaches AI providers, redacting what is not needed and recording each provider’s data-processing terms is deliverable 12.7 of the Production Hardening Sprint, because for an AI app the model provider is a second place customer data lives.
What goes wrong without it
Customer data sent to an AI API can get there without anyone deciding it should. These are the five routes I’d check first.
| What the code does | What leaves | Who finds out, and how |
|---|---|---|
Passes the whole user or order record into the prompt (JSON.stringify(user)) | Email, phone, address and internal notes, beside the one field the feature needed | A customer who asks what the provider holds, or anyone reading a logged prompt |
| Sends free text as typed: support tickets, uploaded documents | Whatever the customer wrote, including card numbers, health details and other people’s names | Nobody, until someone reads a captured payload |
| Pulls documents into the context with a search that is not filtered by tenant | Another customer’s documents, inside this customer’s prompt | The customer who sees someone else’s data quoted back |
| Writes prompts and responses in full to application logs, the error tracker or a tracing tool | A second copy of everything sent, kept under each tool’s own retention | Whoever searches those logs for something else |
| Leaves the provider off the sub-processor list and the privacy policy | Nothing new leaves; the record is wrong | The first business customer who asks, for example in a security questionnaire |
Take a founder whose support assistant builds its prompt from the whole customer record: the address and phone number go to the model provider with every ticket, though the reply needs only the first name and the ticket text. When a business customer asks what the provider holds, the honest answer includes OpenAI’s abuse monitoring logs, which its docs say may contain certain customer content, such as prompts and responses, and which are kept for up to 30 days by default, with the exceptions quoted in the provider section below. A field the feature never needed now has to be listed, explained and tracked, and the lesson I take from it is that the cheapest redaction is the field you never send.
The third row is a tenant isolation failure first, an access-control subject of its own; the model is only where it shows. The fix for the fourth row is to keep sensitive data out of logs before the event leaves the process, and the wider setup is a matter of how to do logging with a retention you chose on purpose. The fifth row turns on what is a sub-processor: in my reading, a model provider that receives a business customer’s data to run your feature belongs on the list your privacy policy points to.
At least 5 of the 21 third-party apps exposed PII or PHI. Those 21 are the third-party apps I audited in June and July 2026: 11 public vibe-coded apps audited across all 12 pillars, and a held-out set of 10 more apps audited blind. They are a selected set, not a random sample, and not a rate for AI-built apps in general; the finding does not say how the data was exposed, so I do not tie those apps to model calls. In the same audits, 8 of the 14 AI apps had a live prompt-injection path, with untrusted text flowing straight into the model’s instructions. The AI/LLM-Native Risk pillar averages 38.4 out of 100 across the third-party apps it was scored on, 14 of the 21.
Knowing what is prompt injection matters here for one reason, as I see it: it is one way data already in the model’s context can be pulled back out, which is another argument for keeping that data out of the context at all.
How to redact personal data before LLM calls: trace, cut, mask, record
Personal data is redacted before LLM calls in one place: a wrapper every model call passes through. It detects emails, phone numbers, card numbers and names, replaces each with a typed placeholder, calls the provider, then restores the values in the reply. A CI rule fails the build if the provider SDK is imported anywhere else.
The pattern in this section is my recommendation. What a tool does comes from its own documentation, and provider terms come from the providers’ own pages as read on 3 October 2026. The five parts below run in working order, because each one feeds the next.
Audit what fields each model call sends
A model-call audit lists, for every call site, the fields sent, whether each is personal data, and whether the feature needs it. It covers 6 inputs people forget: the system prompt, conversation history, retrieved documents, tool results, attachments and metadata. Log field names in staging, never values.
- 01 Find every call. Search the repo for the provider SDK imports (
openai,@anthropic-ai/sdk) and for the API hosts (api.openai.com,api.anthropic.com), and include embeddings, moderation, image, speech and file-upload endpoints, plus calls made from background jobs and edge functions. - 02 For each call site, list every input that can carry personal data: the system prompt, the user message, conversation history, retrieved documents, tool and function results, attachments, and metadata such as a user id passed for abuse tracking.
- 03 In staging, log the field names and sizes of the final payload at the point it is sent, never the values, run each feature once, and fill in the table below.
- 04 Mark each field as needed or not needed by the feature, and write its action: drop, generalize, mask, hash or send as is.
- 05 Add each provider, and each field it receives, to the data map.
Anthropic’s docs also list separate packages for its cloud platform routes (@anthropic-ai/bedrock-sdk, @anthropic-ai/vertex-sdk, @anthropic-ai/aws-sdk, @anthropic-ai/foundry-sdk), so a search for one package name can miss a call that reaches the same models another way. The last step feeds the data map GDPR reviewers ask for, so the provider list and the field list end up in one record.
The table template, with one row filled in as an illustration:
| Call site | Feature | Provider and model | Fields sent | Personal data (yes/no, which) | Needed by the feature | Action |
|---|---|---|---|---|---|---|
support/draft-reply.ts | Support reply draft | Your provider and model | Full customer record, ticket thread | Yes: name, email, phone, address, any names in the thread | First name and latest message | Drop the record; mask names in the message |
Cut first: send less than the code does today
I put minimization ahead of masking for one plain reason: a field that is never sent cannot leak, cannot sit in a provider’s logs and never has to be explained to a customer. Most of the cutting is ordinary code. Pass the three fields the prompt uses, never the record. Replace real identifiers with internal ids the model does not need to understand. Trim conversation history to the turns the feature needs. Filter retrieval by tenant and by permission before ranking, not after. Strip signatures, quoted replies and headers from emails before a summarizer sees them.
The GDPR principle behind this is data minimization. EU GDPR Article 5(1)(c) says personal data shall be “adequate, relevant and limited to what is necessary in relation to the purposes for which they are processed”, and the UK GDPR’s 5(1)(c) uses the same words in the GDPR text on legislation.gov.uk.
Three before-and-after examples, illustrative rather than taken from any one app:
| Call | Before | After |
|---|---|---|
| Support reply draft | The whole customer record and the full ticket thread | First name, the latest message and the order status the reply mentions |
| Invoice summarizer | The invoice object, with billing address, tax id and the card’s last four digits | Line items, totals and currency |
| CRM note classifier | The note with the contact’s name, email and phone | The note text with the contact block removed, and an internal contact id kept on the server |
Mask what must stay: patterns, entity detection and reversible placeholders
Masking uses two detector layers: patterns for structured data such as emails, phone numbers and card numbers, and entity recognition for names and places. Each hit becomes a placeholder like <EMAIL_1> that the server maps back afterwards. Free text always leaks something, and under the EU and UK GDPR pseudonymized data is still personal data.
The wrapper is one function. It takes the assembled messages, runs the detectors, swaps each hit for a typed placeholder (<EMAIL_1>, <PERSON_2>), keeps the mapping in memory for that request only, calls the provider, and restores the values in the response before it reaches the user. Here is its shape as pseudocode, with callProvider standing in for your SDK call:
// pseudocode: the only module allowed to call a model provider
export async function callModel(messages, options) {
const map = new Map(); // placeholder -> real value, this request only
const masked = messages.map((m) => ({
...m,
content: maskText(m.content, map), // patterns first, then entity recognition
}));
logShape(masked); // field names and sizes, never values
const reply = await callProvider(masked, options);
return restore(reply, map); // put the real values back before the user sees them
}| Data type | How it is detected | What replaces it |
|---|---|---|
| Email address | A pattern; Presidio lists “Pattern match, context and RFC-822 validation” | <EMAIL_1> |
| Phone number | A pattern with context; Presidio lists “Custom logic, pattern match and context” | <PHONE_1> |
| Payment card number | A pattern plus a checksum; Presidio lists “Pattern match and checksum” | <CARD_1> as a backstop; the field itself stays out of prompts by design (below) |
| IBAN | A pattern plus a checksum; Presidio lists “Pattern match, context and checksum” | <IBAN_1> |
| Person’s name | Entity recognition; Presidio lists “Custom logic and context” | <PERSON_1> |
| Place | Entity recognition; Presidio lists “Custom logic and context” | <LOCATION_1> |
Presidio is an open-source toolkit for both layers, and its pages say it is moving from a Microsoft-owned project to a community-governed one under the Data Privacy Stack organization. Its anonymizer has operators to replace, redact, hash, mask or encrypt each hit, and a replace with no new value writes the entity type in angle brackets, such as <PHONE_NUMBER>. Its own docs carry a warning: “because it is using automated detection mechanisms, there is no guarantee that Presidio will find all sensitive information.”
That matches the limits I’d plan around. Free text is never fully clean. Rare names and names in other languages get missed. A handful of unmasked details, say a job title, a town and a date, can still point to one person.
Masking lowers the risk; it does not change what the data is. EU GDPR Article 4(5) defines pseudonymization as processing so the data “can no longer be attributed to a specific data subject without the use of additional information”, and recital 26 says such data “should be considered to be information on an identifiable natural person”; the UK text reads the same. So the provider stays on your list, and masked output is never called anonymous: whether anonymized data is still personal data is a separate test.
Two groups stay out of prompts by design, not by detector, as my working rule. The first is special category data, whose processing EU GDPR Article 9(1) prohibits unless an exception in 9(2) applies: health, ethnic origin, religious beliefs, sexual orientation and the rest of that list. The second is payment card numbers.
The last piece is a rule in CI. A lint rule or a plain search fails the build when the provider SDK is imported, or the provider’s API host appears, anywhere outside the wrapper. Check both: a raw HTTP call skips an import-only rule.
What the coding assistant and the model API say they do with your data
Model providers can keep API data by default. OpenAI’s docs say abuse monitoring logs are kept for up to 30 days, unless law requires longer or it is reasonably necessary to protect against harm. A coding assistant is a separate flow: it sees the repository, so its privacy question is about code and data left beside it.
OpenAI’s data controls documentation states the default in one sentence: “By default, abuse monitoring logs are generated for all API feature usage and retained for up to 30 days, unless longer retention is required by law, or is reasonably necessary to protect our services or any third party from harm.” Training use, which endpoints keep stored data longer, every other vendor’s terms and what zero data retention covers are set out in zero data retention for your app’s AI features, so this page carries no vendor table.
The coding assistant is a different data flow. It sees the repository, and with it any seed file, .env or production export left there, so GitHub Copilot privacy concerns are about code and whatever sits beside it, not the app’s customers at run time. What Copilot keeps by plan and feature, training on private code and content exclusion are covered in GitHub Copilot security risks, plan by plan. My working rule for both flows: no production data in the repo, and none pasted into an assistant.
AI vendor data privacy checklist, and what GDPR asks of an AI feature
An AI vendor data privacy checklist records 7 things per provider: the fields your app sends, the data processing agreement, default retention and training use with the setting chosen, region and transfer mechanism, the provider’s sub-processors page, how deletion reaches it, and the date verified.
The zero data retention article above already covers the vendor, product surface, control name and approval date to write down. These seven lines are what it leaves to you:
- The fields your app sends to this provider, copied from the audit table.
- A data processing agreement in place, and where the signed copy is kept.
- Retention and training use by default, and the setting you chose.
- The processing region, and the transfer mechanism if data leaves the UK or the EU.
- The link to the provider’s own sub-processors page.
- How a deletion request reaches data held there, if any is held.
- The date this row was verified, and the date of the next check.
For line two, the question of what is a DPA agreement comes first; line six follows the same path as other GDPR delete requests in a small SaaS.
The purpose of GDPR in an AI feature is the same as everywhere else in the app: it governs the processing of personal data, and sending a prompt to a model provider is processing, since Article 4(2) includes “disclosure by transmission”. The table covers the EU GDPR and the UK GDPR. EU wording comes from the text as adopted, and the last column notes where the UK’s current text differs.
| GDPR article | What it asks | What it means for one AI feature | UK GDPR |
|---|---|---|---|
| 5(1)(c) | Data “adequate, relevant and limited to what is necessary” | Send the fields the feature needs and drop the rest | Same wording |
| 13 and 14 | Tell people “the recipients or categories of recipients of the personal data, if any” | The privacy notice names the model provider, or its category | Same wording at 13(1)(e) and 14(1)(e) |
| 28(3) | Processing by a processor governed by “a contract or other legal act” setting out its subject-matter, duration, nature and purpose | A data processing agreement with the provider before customer data flows | ”domestic law” where the EU text says “Union or Member State law” |
| 44 | Transfers to a third country only if the chapter’s conditions “are complied with by the controller and processor” | Know where the provider processes data and which transfer mechanism covers it | Article 44 omitted from 5 February 2026; Article 44A permits a transfer only if it is approved by regulations under Article 45A, made subject to appropriate safeguards (Article 46) or made in reliance on a derogation (Article 49) |
| 35(1) | An impact assessment before processing that “is likely to result in a high risk to the rights and freedoms of natural persons” | Assess the feature before launch if its processing meets that test | Same wording |
The UK regulator’s reading, in the ICO’s guidance on AI and data protection, is that when you buy in AI systems or use ones run by third parties, minimization “should form part of the procurement process due diligence”, and that you “need to be transparent about how you process personal data in an AI system”; the ICO marks that guidance as under review after the Data (Use and Access) Act.
The table rows are legal duties. Masking, the CI rule and the canary test are good practice that none of the table’s articles names. This is not legal advice. Which role you play for each customer, controller or processor, and the general rules sit in GDPR for a small SaaS, in order. The database provider belongs in the same paperwork, as Supabase’s DPA, region and subprocessor list shows for one stack, while cookie consent requirements are a separate control with their own record.
How to verify it
Redaction is verified with a canary user whose fields hold unique markers. Exercise every AI feature in staging, then search the captured payloads, the logs and the error tracker for the markers. Any marker in a field the audit marked to cut or mask, or in a logged prompt, is a fail.
- 01 Create a canary user in staging whose every field holds a unique marker that still looks real: a plausible name, a valid-format email on a domain you control, a phone number in a valid format, and a note that appears nowhere else. A detector skips a string that does not look like a name or an email. Evidence: the marker list.
- 02 Turn on payload logging at the wrapper in staging, after its masking step, writing to a separate capture file so it holds exactly what is sent. Then use every AI feature as that user, including retrieval, attachments and background jobs. Evidence: the dated capture file.
- 03 Search the captured payloads for each marker. A hit in a field the audit table lets through unchanged is expected; a hit in a field marked for cutting or masking is a fail, even when the feature needs that field. Evidence: the search output.
- 04 Search the application logs, the error tracker and any tracing tool for the same markers, leaving out the capture file, while the run is still inside each tool's log retention. Read the retention first, because an empty search of logs already dropped proves nothing. Pass: no logged prompt or model response carries a marker. Evidence: the search output per tool.
- 05 Search the codebase for provider SDK imports and API hosts outside the wrapper. Pass: zero hits, and the CI rule shown failing on a test branch. Evidence: the failed run's link.
- 06 Compare the audit table with the data map and the sub-processor list. Pass: every provider and field appears in both. Evidence: the three documents side by side.
Turn payload logging off afterwards. It runs in staging only, with test data only.
This is also how deliverable 12.7 of the Production Hardening Sprint is verified: trace each model call, list the fields sent, and match them against the inventory.
Where the sprint does this
In the sprint, the result of 12.7, the work completed and its verification evidence go into the production readiness report, deliverable 13.1, which accounts for all 123 IDs, keeps failures visible until resolved and explains genuine non-applicable items. Legal advice and certification are separate services; the sprint implements and documents the technical data-handling controls. Hosting, paid tools and API usage remain in your own accounts, and any required third-party costs are explained before they are enabled. Every deliverable, 12.7 included, is listed in the published scope.
Common questions about personal data and AI features
What are 5 examples of PII?
Name, social security number, biometric data records, date and place of birth, and mother’s maiden name are five examples in NIST’s definition of PII.
The EU and UK GDPR use a wider term, personal data, whose listed identifiers include “a name, an identification number, location data, an online identifier”. I’d add two that an AI feature brings with it: free-text notes and uploaded documents, which carry whatever a person typed or scanned.
What PII must be redacted?
No law gives a list of fields to strip from prompts: the EU and UK GDPR set principles, such as minimization, rather than field lists, as I read them. My working rule is to keep special category data and payment card numbers out by design, and to cut everything the feature does not need.
Masking then covers what is left: the fields the feature needs and the model does not.
Does AI have to comply with GDPR?
Yes, when the AI feature processes personal data and the business falls under the EU or UK GDPR: the law covers the processing, with or without AI.
In my reading, the app owner is usually the controller and the model provider a processor. EU GDPR Article 28(3) sets what the contract between them must contain, and the UK version of that paragraph points to UK law in place of EU law.
Is ChatGPT GDPR-compliant?
It depends on which OpenAI product you use, the agreement in place and what your app sends: no product is compliant or non-compliant on its own. The same answer holds for any model provider asked which AI is GDPR compliant: the GDPR governs your processing, not a product.
For the API, OpenAI’s data controls page sets retention per endpoint, with abuse monitoring logs of 30 days on /v1/responses and “Until deleted” for stored conversations. The answer for your own app comes from your own record, the seven-line checklist above, never from a yes or a no.
The checks in this guide show you where the app is open. The sprint below closes those gaps, tests the result and writes the evidence down.
Built it with AI. Now it has to hold up for real customers.
The Production Hardening Sprint takes the app you already have and builds the production foundation underneath it. Authentication and access rules, payments that stay consistent, error handling, monitoring, backups, automated tests and a documented handover. Our engineers work inside your existing codebase for ten working days. All 123 deliverables are included, and you get the evidence for each one.
See the Production Hardening Sprint → 
$2,500 fixed price · 10 working days · One codebase