Matching final answers masked different forgetting patterns during fine-tuning
Source check: Source checked. Not independent scientific confirmation.
Finding: Researchers recovered 100% final performance on an old generation task across several replay budgets for a Qwen model. The same training runs nevertheless followed different trajectories in the probability assigned to correct old answers. The finding came from evaluating checkpoints throughout training as well as the final model.
Technology: Qwen and Mistral language-model fine-tuning
Family Qwen: profile · models page
Publication: Preprint · Test context: direct test. Catalog categories, not an evidence grade.
- AI contribution
- Fine-tuned Qwen and Mistral models enabled direct measurement of task outputs and loss at multiple checkpoints under matched training protocols.
- Test and result
- The study varied old-task replay budgets at matched training settings and separately measured old-task generation at the endpoint and negative log-likelihood across training checkpoints.
- Limitation / still unknown
- The result is specific to these models, old-task response tests and replay schedules. Checkpoint behavior cannot establish whether knowledge was removed or merely harder to elicit. The qualified original Qwen retained-response bank contained only 17 examples; its 100% endpoint is not broad preservation of old-task knowledge.
Evidence sources and annotation context
Evidence context (descriptions follow the cited sources; test type is a catalog classification):
Evidence sources: https://arxiv.org/abs/2610.30191 · https://arxiv.org/html/2610.30191v1
Source checked · editorial details
The source page was checked against this entry. Independent confirmation is not assessed by this label.
Source type: paper. Paper: linked separately.
Review date: 21 Nov 2026. Review scope: arXiv v1 controlled replay budgets and objective allocation, Table 7 and endpoint analysis.
No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.
Editorial assessment: Impact 2/5 · Validation 3/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.