Measured accuracy
We do not ask you to trust us. We show you the measurements.
Every detection change passes automated quality gates. At the deep detection level, used for documents, scans and visual redaction, if even one personal-data item in our evaluation sets escapes unmasked, that release does not ship. For the chat we publish the standard level's accuracy separately.
- 34 — personal data categories
- 2 — languages with full name detection: Greek and English
- 2 — detection levels: standard for the chat, deep for documents
- 2,600+ — automated backend checks on every release
The scorecard, checked by CI
The numbers per entity and detection level
From our gold sets, in hermetic replay, per detection level. The published numbers are compared in CI with the current evaluation run: if they drift, the release is blocked.
What the numbers mean, in plain words
- Recall: Of 100 personal data items actually present in the text, how many we detected. 1.000 means none slipped past us in the set.
- Precision: Of 100 items we masked, how many really were personal data rather than a false alarm. 1.000 means no needless masking.
- F1: The two combined into one number, so that a system that simply hides everything does not look good. 1.000 is perfect.
- Leak recall: How many of the items that had to be hidden were covered at least in part. It is the number we care about most: one uncovered item is a leak.
Every figure refers to our own evaluation sets, not to every possible document. Always review the findings before you send a text.
Greek gold set (hand-authored, held out)
| Entity | Gold | recall (PARTIAL) | F1 (EXACT) | FP |
|---|---|---|---|---|
| AFM | 6 | 1.000 | 1.000 | 0 |
| AMKA | 2 | 1.000 | 1.000 | 0 |
| 2 | 1.000 | 1.000 | 0 | |
| IBAN | 3 | 1.000 | 1.000 | 0 |
| ID_CARD | 5 | 1.000 | 1.000 | 0 |
| LICENSE_PLATE | 2 | 1.000 | 1.000 | 0 |
| NAME | 64 | 0.984 | 0.992 | 0 |
| PHONE | 5 | 1.000 | 1.000 | 0 |
| Total (micro) | 89 | 0.989 | 0.989 | 1 |
82 documents · 89 gold spans · 89 detections · PARTIAL recall 0.989 · leak recall 1.000 · PARTIAL F1 0.989 · EXACT F1 0.989
False findings on entities with no gold spans: 1.
| Entity | Gold | recall (PARTIAL) | F1 (EXACT) | FP |
|---|---|---|---|---|
| AFM | 6 | 1.000 | 1.000 | 0 |
| AMKA | 2 | 1.000 | 1.000 | 0 |
| 2 | 1.000 | 1.000 | 0 | |
| IBAN | 3 | 1.000 | 1.000 | 0 |
| ID_CARD | 5 | 1.000 | 1.000 | 0 |
| LICENSE_PLATE | 2 | 1.000 | 1.000 | 0 |
| NAME | 64 | 1.000 | 1.000 | 0 |
| PHONE | 5 | 1.000 | 1.000 | 0 |
| Total (micro) | 89 | 1.000 | 1.000 | 0 |
82 documents · 89 gold spans · 89 detections · PARTIAL recall 1.000 · leak recall 1.000 · PARTIAL F1 1.000 · EXACT F1 1.000
False findings on entities with no gold spans: 0.
English gold set
| Entity | Gold | recall (PARTIAL) | F1 (EXACT) | FP |
|---|---|---|---|---|
| ADDRESS | 8 | 1.000 | 0.625 | 3 |
| BANK_ACCOUNT | 6 | 1.000 | 1.000 | 0 |
| COMPANY | 1 | 1.000 | 0.500 | 2 |
| DATE | 11 | 1.000 | 1.000 | 0 |
| EIN | 3 | 1.000 | 1.000 | 0 |
| 3 | 1.000 | 1.000 | 0 | |
| IBAN | 1 | 1.000 | 1.000 | 0 |
| ITIN | 1 | 1.000 | 1.000 | 0 |
| NAME | 81 | 0.963 | 0.981 | 0 |
| NINO | 3 | 1.000 | 1.000 | 0 |
| PHONE | 7 | 1.000 | 0.714 | 2 |
| POSTAL_CODE | 8 | 1.000 | 0.842 | 3 |
| REF_CODE | 1 | 1.000 | 1.000 | 0 |
| SSN | 3 | 1.000 | 1.000 | 0 |
| VAT_GB | 3 | 1.000 | 1.000 | 0 |
| Total (micro) | 140 | 0.979 | 0.936 | 10 |
121 documents · 140 gold spans · 142 detections · PARTIAL recall 0.979 · leak recall 0.943 · PARTIAL F1 0.972 · EXACT F1 0.936
False findings on entities with no gold spans: 0.
| Entity | Gold | recall (PARTIAL) | F1 (EXACT) | FP |
|---|---|---|---|---|
| ADDRESS | 8 | 1.000 | 0.375 | 5 |
| BANK_ACCOUNT | 6 | 1.000 | 1.000 | 0 |
| COMPANY | 1 | 1.000 | 1.000 | 0 |
| DATE | 11 | 1.000 | 1.000 | 0 |
| EIN | 3 | 1.000 | 1.000 | 0 |
| 3 | 1.000 | 1.000 | 0 | |
| IBAN | 1 | 1.000 | 1.000 | 0 |
| ITIN | 1 | 1.000 | 1.000 | 0 |
| NAME | 81 | 1.000 | 1.000 | 0 |
| NINO | 3 | 1.000 | 1.000 | 0 |
| PHONE | 7 | 1.000 | 0.714 | 2 |
| POSTAL_CODE | 8 | 1.000 | 1.000 | 0 |
| REF_CODE | 1 | 1.000 | 1.000 | 0 |
| SSN | 3 | 1.000 | 1.000 | 0 |
| VAT_GB | 3 | 1.000 | 1.000 | 0 |
| Total (micro) | 140 | 1.000 | 0.950 | 7 |
121 documents · 140 gold spans · 140 detections · PARTIAL recall 1.000 · leak recall 0.986 · PARTIAL F1 1.000 · EXACT F1 0.950
False findings on entities with no gold spans: 0.
Measurements last updated: 7 November 2026
The other side of the same problem
Not masking what does not need masking
We would rather mask one item too many than miss one (the “recall-first” principle). That carries an obvious cost: a system that masks everything never misses anything, and is useless. In September 2026 we measured and reduced false findings, under the explicit condition that recall was not allowed to drop by a single value. The figures refer to the first version of the adversarial set (594 documents).
- False findings on an adversarial set: 241 → 92 (62% reduction)
- Documents with no false finding at all: 64.1% → 86.9% (+22.8 points)
- Recall on the Greek reference set, deep level: 1.000, unchanged at every step
- F1 (micro) on the Greek reference set, deep level: 0.994
- The byte-exact reference snapshot stayed identical throughout
- Two candidate rules were rejected because the quality gate priced them
Language coverage
Three tiers. For each one we say exactly what holds.
Production depth
Greek · English
Full detection with Greek-specialised models and deterministic validators: AFM checksum, AMKA Luhn, IBAN, land-registry codes, IDs, plates. Name inflection handling and identity linking across documents. Validated on real public documents, not only curated text.
Structured identifiers validated deterministically, names experimental
German · French · Spanish · Italian · Portuguese · Dutch · Polish · Romanian
Dates, postal codes, phone numbers, VAT numbers and IBANs are validated deterministically. Name detection in these languages is experimental: at the deep level it relies on a multilingual model, at the chat level on name lists and surname morphology, which covered 68 of the 72 names in the evaluation set.
Other scripts: flagged before sending
Cyrillic · Arabic · Chinese and other scripts
Names written in Cyrillic, Arabic or Chinese script are not detected yet. When an uploaded document contains them to a significant degree, filterit flags it before anything leaves, so the decision stays with you. Script-independent items such as IBANs, emails, phone numbers and card numbers are detected as usual.
How we measure
The methodology is the product. These are its four instruments.
01 Quality gates that do not relax silently
Labelled evaluation sets per language and data type. At the deep level the recall floor is 100%: a single leak in any set blocks the release. Lowering a threshold requires a recorded rationale in the thresholds file.
02 Real-document validation
Curated sets are not enough. Reality has stamps, scanned forms and captions printed under the values. We test the deterministic validators on 22 real Greek public documents with labelled data, and every finding becomes a permanent rule with a permanent test.
03 Deterministic first
Anything with structure is verified by proof, not statistics: the Greek tax number by its checksum, AMKA by Luhn, IBAN by mod-97. Neural models handle only what cannot be proven, such as names and addresses, and are wrapped in confirmation layers.
04 When we cannot, we say so
Unreadable handwriting? An unsupported script? The product does not pretend. On uploaded documents it shows an explicit warning before anything leaves. Silent failure is the only truly dangerous failure mode in a privacy tool.
What gets detected
34 categories, 11 of them specific to Greek documents
Greek identifiers are validated (check digit, format, context) rather than matched as digit runs, and names are followed across grammatical cases. Tap a category for details.
| Tax ID (ΑΦΜ) | Greek tax identification number with modulo-11 check digit validation, not a bare 9-digit match. | check: modulo 11 |
|---|---|---|
| Social security (ΑΜΚΑ) | Greek social-security number with Luhn and birth-date validation. | check: Luhn |
| ID card number | Greek police ID card, old and new formats (letters and digits). | check: format |
| Vehicle plates | Greek registration plates, restricted to the letters Greek plates actually use. | check: format |
| Court case numbers (ΓΑΚ, ΕΑΚ, ΑΒΜ) | Greek court case and docket numbers, context-gated with NLP. | check: context |
| Business registry (ΓΕΜΗ) | General Commercial Registry number, 12 digits, context-gated to avoid clashing with other numbers. | check: context |
| Cadastre code (ΚΑΕΚ) | National Cadastre property code, 12-digit and full 18-digit forms, with prefecture-code validation. | check: prefecture code |
| myDATA and EFKA registries | AADE myDATA MARK and invoice UID, EFKA employer and insured numbers (ΑΜΕ, ΑΜΟΕ). | check: context |
| Property references (ΑΤΑΚ, ΠΕΑ) | E9 property identifier and energy-performance certificate security numbers. | check: context |
| Tiresias, auctions, BIC | Greek credit-bureau references, e-auction codes and BIC/SWIFT codes. | check: context |
| Greek places | Greek gazetteer of cities, municipalities and regions, in every grammatical case. | gazetteer |
| IBAN | Bank accounts of every EU/EEA country with MOD-97 validation. | check: MOD-97 |
|---|---|---|
| EU and UK VAT numbers | VAT numbers of all 27 EU countries and the UK: per-country format check, check digit for Greece, Italy, Poland, Germany and the UK. | check: VIES |
| National IDs (EU, US, UK) | DNI, PESEL, Codice Fiscale, NIR, SSN/ITIN/EIN, National Insurance and more. | check: national checks |
| Passports | Passport numbers gated by context words in the 24 EU languages (automated tests for 4). | check: context |
| Driving licences | Driving-licence numbers, context-gated. | check: context |
| Phone numbers | Landlines and mobiles for the EU, UK, US/Canada and Switzerland, validated per country. | check: libphonenumber |
| Payment cards | Card numbers with Luhn, together with CVV and expiry. | check: Luhn |
| Chassis number (VIN) | 17-character vehicle identification number per ISO 3779, context-gated. | check: ISO 3779 |
| Digital signatures | E-signature identifiers: DocuSign envelope, Adobe Sign agreement, SHA-1/SHA-256 certificate thumbprints. | check: format |
| Postal codes | Greek postcodes, US ZIP codes and UK postcodes. | check: context |
| Names across grammatical cases | Greek NER that follows inflection, greeklish spellings and links the forms of the same person. | NER |
|---|---|---|
| Companies and organisations | Company and institution names, Greek and international. | NER |
| Email addresses | Email addresses. | check: format |
| Street addresses | Street, number and area, in Greek and Latin-script text. | NER |
| Dates | Birth dates and other identifying dates. | check: format |
| Trademarks | Trademark names from a private list of your own clients. | gazetteer |
| Medical data | Blood type, record, insurance and prescription numbers, ICD-10 codes, and a condition written behind a caption or as the reason for a sick leave. A condition inside a free sentence usually stays readable. | NLP |
|---|---|---|
| Special categories | Gender, citizenship, religion, marital status and profession written in a captioned form field, and the specialty in staff tables. Inside a free sentence these, like political or trade-union references, usually stay readable. | NLP |
| API keys and secrets | API keys, tokens and JWTs, private keys, connection strings, passwords, typed in the placeholder. | check: format |
|---|---|---|
| Reference codes | Prefixed identifiers such as EMP-, INV-, CUST-, ORD-. | check: format |
| MAC addresses | Network device identifiers. | check: format |
| IP addresses | IPv4 and IPv6. | check: format |
| Technical identifiers | File paths carrying a username and internal hostnames. | check: format |
In scanned PDFs and images, signatures and stamps are covered by visual redaction, with a preview before it is applied.
How to read the measurements
The figures on this page come from our evaluation sets and from the real documents we test on, and describe how the system behaves on those. On a new document performance can differ. That is why the privacy panel shows what was masked in every conversation, visual redaction goes through a preview before it is applied, and the system tells you wherever it cannot read the text. The final decision is always yours.
See it on your own document
Upload an invoice or a letter and see what gets masked, before anything leaves.