AI Science Tracker
EN

Later attention queries rescued selected early-loop retrieval errors

University of British Columbia · 25 Oct 2026 · computer science

Discovery

Later attention queries rescued selected early-loop retrieval errors

Source check: Source checked. Not independent scientific confirmation.

Finding: Moving same-item later-loop attention queries into the first loop corrected 89.8% and 96.9% of selected synthetic two-hop errors in two Ouro models. Queries shuffled between items corrected 23.7% and 20.6%, respectively. The result applies to questions answered incorrectly in the first loop and correctly later.

Technology: Ouro-1.4B; Ouro-2.6B; activation transplantation

Publication: Preprint · Test context: direct test. Catalog categories, not an evidence grade.

AI contribution
The authors intervened on pretrained Ouro attention query activations and compared matched late-loop and shuffled donors.
Test and result
The primary synthetic test included 600 scenes per model, with 245 and 223 qualifying early-loop errors, respectively. Same-item and shuffled queries were compared at aligned positions.
Limitation / still unknown
Natural-question rescue was smaller and did not establish bridge redirection. A positional contribution remains possible. In Ouro-1.4B the rescue changes four-candidate ranking, not greedy generated answers. The intervention supplies a separate query at every layer/head, not one vector; the two model sizes share developmental history.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2610.34639v1 · https://arxiv.org/html/2610.34639v1

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 21 Nov 2026. Review scope: v1 query swap Methods, Table 2 and natural-question caveats.

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 2/5 · Validation 3/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open in the full catalog for filters, charts and related discoveries. Browse computer science discoveries.