AI Science Tracker
EN

Informatics Institute of Technology, Sri Lanka

1 discovery from 2026

Discoveries

Two GPT-2 circuits showed different preservation after distillation

Source check: Source checked. Not independent scientific confirmation.

Finding: On tested prompts, DistilGPT-2 retained 87.3% accuracy on a number-comparison task versus GPT-2’s 94.0%, but fell to 35.2% on an indirect-object task where GPT-2 achieved 94.8%. Activation interventions found evidence for retained early comparison components and little recovery from the tested indirect-object heads.

Technology: GPT-2; DistilGPT-2; causal activation interventions

Publication: Preprint · Test context: direct test. Catalog categories, not an evidence grade.

AI contribution
Interventions on the learned activations of two language models exposed different functional preservation across the tested circuits.
Test and result
The study compared behavior, activation readouts and matched discovery/evaluation head tests across two previously characterized circuits, and reported activation patching results.
Limitation / still unknown
One teacher–student pair and two tasks are represented. The result does not rule out alternate indirect-object mechanisms or establish what caused differences during the original training.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2610.37157v1 · https://arxiv.org/html/2610.37157v1

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 21 Nov 2026. Review scope: §§3–6, Tables 4–6 and the single model-pair limitation.

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 2/5 · Validation 3/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open the interactive profile for the timeline, or browse all organizations and model families.