Language models trusted incorrect reassurances more than incorrect warnings in a defensive audit
Source check: Source checked. Not independent scientific confirmation.
Finding: In tests of a prerecorded defensive audit dataset, incorrect reassurances moved language-model verdicts toward agreement more than incorrect warnings did. A prespecified check on seven additional model configurations found a mean asymmetry of 0.16, with a 95% interval of 0.10–0.24. This measures verdict behavior under controlled evaluation conditions.
Technology: DeepSeek; Qwen; GLM; Kimi audit models
Family DeepSeek: profile · models page
Publication: Preprint · Test context: direct test. Catalog categories, not an evidence grade.
- AI contribution
- Eleven language model configurations independently evaluated prerecorded audit examples and supplied verdicts that were compared under paired evidence conditions.
- Test and result
- The study analyzed 16,572 evaluations on 916 matched pairs across eleven configurations. The seven-configuration confirmation used prerecorded samples and paired comparisons against valid evidence.
- Limitation / still unknown
- The confirmation was conducted by the same team. The data and decision conditions are controlled and do not measure real-world incidence.
Evidence sources and annotation context
Evidence context (descriptions follow the cited sources; test type is a catalog classification):
Evidence sources: https://arxiv.org/abs/2610.32876v1 · https://arxiv.org/html/2610.32876v1
Source checked · editorial details
The source page was checked against this entry. Independent confirmation is not assessed by this label.
Source type: paper. Paper: linked separately.
Review date: 21 Nov 2026. Review scope: arXiv v1 Methods, experimental setup and §6.4 aggregate defensive behavior.
No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.
Editorial assessment: Impact 2/5 · Validation 3/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.