Records / De-identification rules

The engine is measured on a public benchmark

We measure the engine on two public benchmarks and publish the result for the part of each we did not look at while building: the Text Anonymization Benchmark, court judgments in which legal annotators marked every identifier, with its strict metric; and PrivacyBench, company Slack, email and document exports in which people marked every identifier by hand, scored with its authors' own scorer beside the engines they published. The scripts that measure it ship with the engine, and so do its tests.

Dated 15 Oct 2026Source checked 15 Oct 2026Page out of reach at the last check, 16 Oct 2026Basis: our standard

In the source's own words
“although the standard, mention-level recall seems relatively good at first sight (around 0.7), a closer look a the entity-level recall over direct identifiers ERdi shows a much poorer performance (around 0.45).”
Figures it counts in
  • 93.4%Of direct identifiers masked in every mention
  • 88.3%Of the text it masked needed masking
  • 86.8%F1 on company Slack, email and documents
  • 95.8%Of people's identifiers found in company records
  • 0.46Microsoft Presidio on the same measure
The engine is measured on a public benchmark | AIData.Broker