86.8%

F1 on company Slack, email and documents

Our engine (version 2026-10-18.1) on the 6 exports of PrivacyBench v2.5 (Tonic.ai, revision of 2026-10-15) that people annotated by hand, 5,198 email and Slack messages and 656 document pages, none of them read while the rules were written. Each export's own Slack member list and email headers gave the engine its list of people, as a holder's export does in the tool. Precision 88.1%, recall 85.6% over 25,855 marked identifiers of ten kinds, with the benchmark's overlap rule; the benchmark's own scorer (https://github.com/TonicAI/privacy_bench_metrics) returns the same figures (it did NOT agree on the last run). Of the 9 engines the benchmark's card publishes on the same gold, 5 score higher: Textual Graph Synthesis 92.5%, Textual via SDK 92.1%, Claude Opus 5 88.3%, Claude Sonnet 5 87.0%, GPT-6 astra 86.8%. Measured 2026-10-18 by apps/app/scripts/deid-privacybench.mjs.

Updated 18 Oct 2026, from its newest record.

The record it is computed from

Records behind this figure
Record	As stated	Dated	Source
The engine is measured on a public benchmark	We measure the engine on two public benchmarks and publish the result for the…	15 Oct 2026	Pilán et al., TAB, Table 7 discussion; NIST SP…

The engine is measured on a public benchmark
As stated
We measure the engine on two public benchmarks and publish the result for the…
Dated
15 Oct 2026
Source
Pilán et al., TAB, Table 7 discussion; NIST SP…

Cited in this guide

HIPAA De-Identification for AI Training: Safe Harbor vs Expert Determination

Each record links to its own page, its source and its permanent copy.

The de-identification standard Every figure