An expanded document audit found training matches for 35.2% of test pages
Source check: Source checked. Not independent scientific confirmation.
Finding: An expanded audit of the RVL-CDIP benchmark found training-set matches for 14,076 test pages, or 35.20%, and 14,151 validation pages, or 35.38%. The search used learned image matching and counted near-duplicates that shared a document template as well as copies. A smaller 2023 analysis had already identified substantial duplication.
Technology: CLIP, SuperPoint, LightGlue
Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.
- AI contribution
- Learned CLIP embeddings screen scanned documents for likely matching pairs, and SuperPoint and LightGlue match their visual structure before final duplicate classification.
- Test and result
- Learned CLIP embeddings proposed likely cross-split matches before SuperPoint and LightGlue visual matching and a text-similarity check. The calibration set contained 403 matched pairs among 1,600 candidate pairs, with 82.9% recall at the chosen threshold.
- Limitation / still unknown
- The audit includes same-template pages and may miss matches. The earlier 2023 sample-based estimate of about 32% uses a different method and is not a baseline for measuring a change in duplication.
Evidence sources and annotation context
Evidence context (descriptions follow the cited sources; test type is a catalog classification):
Evidence sources: https://arxiv.org/abs/2606.31446v1 · https://arxiv.org/html/2606.31446v1 · https://arxiv.org/html/2610.32126v1 · https://aclanthology.org/2023.eacl-main.195.pdf
Source checked · editorial details
The source page was checked against this entry. Independent confirmation is not assessed by this label.
Source type: paper. Paper: linked separately.
Review date: 21 Nov 2026. Review scope: June arXiv v1 front matter, §§4.1–4.2 learned matching and Table 3; October v1 extension introduction and stated matching result; EACL 2023 §5 sampled overlap estimate..
No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.
Editorial assessment: Impact 2/5 · Validation 2/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.