AI Science Tracker
EN

Successful human puzzle players voiced qualitative explanations more often than language agents

Nanyang Technological University; Technical University of Darmstadt; Basis Research Institute; Harvard University; Cornell University · 25 Oct 2026 · computer science

Discovery

Successful human puzzle players voiced qualitative explanations more often than language agents

Source check: Source checked. Not independent scientific confirmation.

Finding: In an ARC-AGI-3 trajectory study, qualitative descriptions appeared in 72 of 144 successful human runs and 17 of 119 successful runs from three agent systems. The human data came from think-aloud recordings and the model data from agent traces. The comparison measures recorded descriptions in successfully completed games.

Technology: GPT-5.6 Sol assisted analysis; ARC-AGI-3 agent traces

Family GPT-5: profile · models page

Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.

AI contribution
GPT-5.6 Sol implemented trigger detectors and annotated cognition from aligned transcripts and game traces. Learned agent trajectories supplied matched-task comparison material.
Test and result
The authors collected 144 human trajectories across 64 participants and 25 games, collected 250 publicly released model trajectories, and analyzed successful subsets. They spot-checked parts of the model-assisted annotation.
Limitation / still unknown
Differences in what humans say aloud and what agents write or log limit claims about underlying cognitive representations.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2610.34523v1 · https://arxiv.org/html/2610.34523v1

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 21 Nov 2026. Review scope: v1 annotation workflow, participant selection, Section 4 trajectory results.

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 2/5 · Validation 2/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open in the full catalog for filters, charts and related discoveries. Browse computer science discoveries.

Related catalog entries

Links use documented relationships where available, plus shared catalog context. A shared topic or model family does not establish a scientific dependency.