AI Science Tracker
EN

An expanded document audit found training matches for 35.2% of test pages

Vanderbilt University · 30 Jun 2026 · computer science

Discovery

An expanded document audit found training matches for 35.2% of test pages

Source check: Source checked. Not independent scientific confirmation.

Finding: An expanded audit of the RVL-CDIP benchmark found training-set matches for 14,076 test pages, or 35.20%, and 14,151 validation pages, or 35.38%. The search used learned image matching and counted near-duplicates that shared a document template as well as copies. A smaller 2023 analysis had already identified substantial duplication.

Technology: CLIP, SuperPoint, LightGlue

Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.

AI contribution
Learned CLIP embeddings screen scanned documents for likely matching pairs, and SuperPoint and LightGlue match their visual structure before final duplicate classification.
Test and result
Learned CLIP embeddings proposed likely cross-split matches before SuperPoint and LightGlue visual matching and a text-similarity check. The calibration set contained 403 matched pairs among 1,600 candidate pairs, with 82.9% recall at the chosen threshold.
Limitation / still unknown
The audit includes same-template pages and may miss matches. The earlier 2023 sample-based estimate of about 32% uses a different method and is not a baseline for measuring a change in duplication.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2606.31446v1 · https://arxiv.org/html/2606.31446v1 · https://arxiv.org/html/2610.32126v1 · https://aclanthology.org/2023.eacl-main.195.pdf

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 21 Nov 2026. Review scope: June arXiv v1 front matter, §§4.1–4.2 learned matching and Table 3; October v1 extension introduction and stated matching result; EACL 2023 §5 sampled overlap estimate..

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 2/5 · Validation 2/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open in the full catalog for filters, charts and related discoveries. Browse computer science discoveries.