Language-model safety probes transferred despite shifts in learned directions
Source check: Source checked. Not independent scientific confirmation.
Finding: Across four language-model families, the directions fitted to separate benign and harmful requests shifted by about 17° to 35° after distillation or instruction tuning. Applying the base probes unchanged shifted AUC by −0.011 to +0.027 on XSTest and −0.015 to +0.053 on an independent prompt set.
Technology: Fitted linear activation probes; Qwen2.5, Llama 3.1 and distilled DeepSeek-R1 variants
Family Llama: profile · models page
Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.
- AI contribution
- Fitted activation classifiers estimated directions and classification performance on base and successor models. Their transfer separated change in direction from change in discrimination.
- Test and result
- Linear classifiers trained on the four base models were evaluated unchanged on two matched successors per family. Transfer was compared with the same probe’s performance on the base model.
- Limitation / still unknown
- Similar observed accuracy does not establish equivalence, and XSTest fold intervals are too narrow. On OR-Bench the probes flagged 29% to 42% of benign prompts that sounded harmful.
Evidence sources and annotation context
Evidence context (descriptions follow the cited sources; test type is a catalog classification):
Evidence sources: https://arxiv.org/abs/2610.35586v1 · https://arxiv.org/html/2610.35586v1
Source checked · editorial details
The source page was checked against this entry. Independent confirmation is not assessed by this label.
Source type: paper. Paper: linked separately.
Review date: 21 Nov 2026. Review scope: §§3–6, Appendix B and front matter.
No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.
Editorial assessment: Impact 2/5 · Validation 2/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.