A hidden-state shift can cause a language model to repeat one token
Source check: Source checked. Not independent scientific confirmation.
Finding: Researchers isolated a shift shared across responses during language-model preference training. Inserting the learned shift into an untreated model reproduced a failure in which one token dominated generation. The failure appeared in 14 of 17 tested model lineages under the study’s per-token ranking objective.
Technology: Preference-optimized language models, including Llama-2 and Llama-3.1
Family Llama: profile · models page
Publication: Preprint · Test context: direct test. Catalog categories, not an evidence grade.
- AI contribution
- The researchers isolated a common direction learned during preference training and tested its contribution by transplanting it into an untreated language model.
- Test and result
- The authors isolated the shared offset, compared transplantation with magnitude-matched random directions, and tested masking of the component in the training loss.
- Limitation / still unknown
- The failure depends on model and objective. Related hidden shifts were also observed without generation collapse in tested standard preference-training runs.
Evidence sources and annotation context
Evidence context (descriptions follow the cited sources; test type is a catalog classification):
Evidence sources: https://arxiv.org/abs/2610.29410 · https://arxiv.org/html/2610.29410v1
Source checked · editorial details
The source page was checked against this entry. Independent confirmation is not assessed by this label.
Source type: paper. Paper: linked separately.
Review date: 21 Nov 2026. Review scope: arXiv v1 front matter, §§4–5 and 7, Appendix B.17, and primary intervention controls.
No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.
Editorial assessment: Impact 2/5 · Validation 3/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.