Use case · LLMs and RAG

How to keep confidential data out of LLMs in development

Most leaks into LLMs don't start in production. They start when someone indexes a folder of real contracts to try a RAG prototype, pastes a real ticket into a prompt, or fine-tunes on last year's support logs. Here is how to stop that at the source.

Get a 48-hour proof Read the six steps

The scale of it

~40%

of files uploaded to generative AI tools contained personal or card data, and about 18% of enterprise employees paste data into those tools.

LayerX Enterprise AI and SaaS Data Security Report 2025, via eSecurity Planet

Where confidential data gets in

Prototype indexes

A developer points a RAG demo at a real SharePoint folder. Every chunk is now retrievable by anyone with the demo link.

Fine-tuning sets

Support logs and emails used for fine-tuning carry names and account numbers into the model's weights.

Evaluation data and traces

Test prompts are copied from real tickets, and the traces logged during testing keep them for months.

Six steps

How to keep it out

1Build dev indexes from synthetic documentsPrototype on documents that look like yours but describe no real person. Switch to masked real documents only when the pipeline is ready.
2Mask before you chunkRemove personal data from source files before they are split and embedded. Removing it from vectors afterwards isn't reliable.
3Use consistent tokens, not blanks"PERSON_7F3A signed with PERSON_2B10" keeps who-did-what for retrieval. "[REDACTED] signed with [REDACTED]" doesn't.
4Keep the token map out of reachThe table that turns tokens back into real values never goes into the index, the prompt or the logs.
5Plant canariesAdd synthetic values you can search for, such as a test SSN, and alert if one ever appears in an answer or a log.
6Mask what you keepEvaluation sets and traces outlive the test. Mask them, or build them from synthetic data in the first place.

RAG security checklist: the data side

Development and production indexes are separate.
Retrieval checks each user's access to each document.
Personal data is masked at ingestion, not only filtered at answer time.
Canaries are planted and monitored.
Evaluation sets contain no real customers.
Traces have a retention limit and are masked before they are kept.

What Synthesized covers

Masking source documentsEngineer-led

Synthetic documents for dev indexesEngineer-led

Canary valuesFree tool

Masking traces and memoryEngineer-led

Formats: RAG and agent data, documents, chats and tickets.

Questions

Isn't a prompt filter or guardrail enough?

It helps, but it has to catch every leak in every answer. Data that never entered the index or the training set can't leak at all. Use both, and treat the data side as the one you fully control.

Can I fine-tune on masked data?

Yes, and realistic replacements work better than placeholder tags, because the model learns from text that still reads naturally.

What is a canary value?

A unique, fake value planted in your data, such as a never-issued SSN. If it ever shows up in an output or a log, you know exactly which source leaked.

The platform underneath, in production

Documents run on the same platform banks, insurers and health companies already trust with their data.

Documents and text use the same policies, the same engine and the same deployment as these results.

Health · masking20bn+rows masked and subsetted in hours, not weeks, with all manual configuration removed.Global HealthTech leaderCase study

Insurance · security0security waivers across 14 priority applications: 28M+ rows secured, 100% referential integrity, test data 60% faster.Global specialty insurerCase study

Published results from database and synthetic data work on the Synthesized platform.

Give your AI team safe data by default

Masked and synthetic documents, prepared in your environment, ready to index. Start with 50 of your documents, in your environment, in 48 hours, for a fixed fee.

Get a 48-hour proof