AI Science Tracker
EN

University of Hull

1 discovery from 2026

Discoveries

Fewer biased answers after model compression often coincided with more abstention

Source check: Source checked. Not independent scientific confirmation.

Finding: On the BBQ question-answering benchmark, 20 of 26 quantization configurations with fewer biased answers also abstained more often on questions they could answer. The comparison covered models from the Qwen3, Gemma 3 and SmolLM2 families.

Technology: Qwen3, Gemma 3, SmolLM2

Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.

AI contribution
The researchers compared outputs from Qwen3, Gemma 3 and SmolLM2 across model sizes and quantization settings, separating abstention from biased or non-biased answers.
Test and result
The authors compared multiple quantization configurations and reported correct, biased, and abstaining answers separately. They also changed prompt wording and removed the abstention option in a separate comparison.
Limitation / still unknown
The 20-of-26 result is conditional on configurations with lower observed bias in this benchmark. It does not measure fairness in real use or establish that all compression increases abstention.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2610.29657 · https://arxiv.org/html/2610.29657v1

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 21 Nov 2026. Review scope: arXiv v1 front matter, §§3–5, exact quantization comparison, prompt variation and forced-choice appendix.

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 2/5 · Validation 2/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open the interactive profile for the timeline, or browse all organizations and model families.