AI Science Tracker
EN

Related model checkpoints separate two forms of sycophancy

Boston Children’s Hospital; Harvard Medical School; New Jersey Institute of Technology · 27 Oct 2026 · computer science

Discovery

Related model checkpoints separate two forms of sycophancy

Source check: Source checked. Not independent scientific confirmation.

Finding: Across 13 checkpoints from five open model lineages, base models already responded differently to sources named in a question. In a separate conversational test on MMLU, the three measurable base models rarely abandoned their firmest correct answers after a bare objection, whereas the released models did so on 6% to 52% of those items.

Technology: OLMo-2; Llama 3.1; Tülu 3; Qwen2.5; Mistral 7B

Family Llama: profile · models page

Publication: Preprint · Test context: direct test. Catalog categories, not an evidence grade.

AI contribution
Learned base and stage-matched post-trained checkpoints generated and scored answers under paired source, answer-firmness and conversational challenges.
Test and result
Across 13 checkpoints and four multiple-choice datasets, the single-turn ordering by source credibility was already visible in base models. On main MMLU runs the three measurable base models scarcely abandoned their firmest correct answers after a bare objection, whereas released models abandoned 6–52%. OLMo-2 base did not pass the conversational readout check.
Limitation / still unknown
The tests used multiple-choice letter answers. One base checkpoint could not be scored reliably in conversation. Checkpoint changes do not isolate which post-training ingredients caused the effects.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2610.37788v1 · https://arxiv.org/html/2610.37788v1

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 21 Nov 2026. Review scope: arXiv v1 §§2–3.4, Figs. 3, 4 and 6, Appendices D–F.

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 2/5 · Validation 3/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open in the full catalog for filters, charts and related discoveries. Browse computer science discoveries.

Related catalog entries

Links use documented relationships where available, plus shared catalog context. A shared topic or model family does not establish a scientific dependency.