Get the whitepaper!
The Test Data Imperative: Towards SAP Autonomous Enterprise

Use case · Document AI

Test document AI on documents you can share

Intelligent document processing, OCR and extraction models get tested on real statements, claims and KYC packs because nothing else looks real enough. Synthetic documents with known values let you measure accuracy field by field without a single real customer.

Get a 48-hour proofTry a free sample
Extraction test · 1,000 statementsScoredIllustration
FieldTrue valueModel read
1Account holderJordan HaleJordan Hale
2Account number3071 88423071 8842
3Statement period1–30 Sep1–30 Sep
4Closing balance£4,182.06£4,182.60×
5Transactions42 rows41 rows×
Field accuracy across 1,000 documents · scale from 90%
Account holder99.6%
Account number99.8%
Statement period99.1%
Closing balance96.4%
Transaction rows93.2%
The true values ship with every generated document, so every miss can be counted.

Why real documents make poor test sets

They can't leave production
Vendors, offshore testers and AI labs can't see them, so the people building the model test it least.
Nobody knows the right answer
Accuracy needs labels. Labelling real documents is slow, expensive, and puts more people in front of personal data.
The hard cases are rare
Skewed scans, handwriting, missing fields and odd layouts are what break extraction, and real samples have few of them.
Some teams still use paper
Testers fill in real forms by hand and photograph them for each release, because nothing else looks real enough.

How it works

1Fix the templatesYour forms, statements or claims, in fixed layouts. Each template is set up once and re-run as often as you like.
2Generate synthetic customersPeople, employers and accounts first, so every document about one person agrees with the others.
3Fill and assembleValues are written into each template and the document is built, with the values kept alongside as the ground truth.
4Score every fieldCompare extraction with the truth, per field and per quality level, and re-run after every model change.

The test matrix

A test set that exercises every rule, not just the happy path

A minimum set per form for any engine that checks incoming documents against expected field states and structural rules.

Every field can becorrectblankmissingwrong valuepartialoverflowwrong format
Golden path
Every field valid, current version, all pages present.
All blank
The structure is there; every data field is empty.
All mismatch
Every field filled, every value wrong.
Conditional triggers
One document per trigger combination, with the dependent field valid and invalid.
Pages
Missing, out-of-order, duplicated and extra pages.
Versions and headers
Current, expired, mixed and malformed version strings.
Signatures and dates
Present, absent, wrong format, in the future or the past.
Repeating sections
Minimum, maximum and mixed-validity entries.
OCR noise
Case, whitespace and misread characters on key fields.

What a good test set has

  • A known true value for every field you extract.
  • Layouts from more than one issuer, and more than one version of each.
  • Clean, scanned and low-quality copies of the same document.
  • Missing, extra and contradictory fields, on purpose.
  • The same customer across document types, so cross-checks can be tested.
  • Nothing that traces back to a real person, and a watermark that says so.

Where it's used

  • Mortgage and loan intake in banks
  • Claims processing in insurers
  • KYC and onboarding checks
  • Accounts payable automation
  • Clinical document coding
  • Form validation and document quality checks
Format: documents and PDFsFree sample document library

Questions

How is this different from downloading sample PDFs?

A handful of sample PDFs is a demo. A test set is thousands of documents with known values, layouts you choose and degraded versions, generated again whenever the model changes.

Can you match our real templates?

Yes. Your templates are the starting point: each one is set up once with our engineers, then your team re-runs it as often as it needs.

Could these documents be used for fraud?

They are designed not to be: every page is watermarked as test data, institutions are fictional, and identifiers come from ranges that are never issued. Testing your fraud checks against them is a good use.

The platform underneath, in production

Synthetic data from Synthesized has been tested against real data by the European Commission.

Documents and text use the same policies, the same engine and the same deployment as these results.

Banking · test data80%less time to provision test data, with compliant data generated through the API inside CI/CD.Deutsche BankCase studySynthetic data · fidelity0.359%modelled losses on Synthesized synthetic data with business rules, against 0.347% on the real data, in the European Commission JRC's evaluation.European Commission JRCReport

Published results from database and synthetic data work on the Synthesized platform.

Measure your extraction against the truth

Start with the free samples, then see it on 50 of your own documents, in your environment, in 48 hours, for a fixed fee.

Get a 48-hour proof