Coding agents and maintainers surfaced different types of software defects
Source check: Source checked. Not independent scientific confirmation.
Finding: Across 20 open-source repositories, arithmetic and comparison errors made up 18.5% of agent reports and 9.9% of historical maintainer fixes. Missing branches for supported inputs made up 7.7% of agent reports and 18.0% of the fixes. The study compared 14 agent configurations with 2,457 fixes from 18 months of repository history.
Technology: Frontier coding agents including GPT-5.6 Sol, Claude Opus 5 and Gemini 3.5 Flash
Family Claude: profile · models page
Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.
- AI contribution
- Coding agents generated bug reports for historical snapshots. Additional learned classification and matching helped construct and categorize the analysis set.
- Test and result
- The study categorized agent reports from historical snapshots and same-repository fixes using a shared bug taxonomy. Agent percentages were averages across configurations.
- Limitation / still unknown
- The comparison contrasts reported findings with fixes that maintainers chose to complete. Some reports may be invalid, and unreported or unfixed historical bugs are outside the reference set.
Evidence sources and annotation context
Evidence context (descriptions follow the cited sources; test type is a catalog classification):
Evidence sources: https://arxiv.org/abs/2610.35987v1 · https://arxiv.org/html/2610.35987v1
Source checked · editorial details
The source page was checked against this entry. Independent confirmation is not assessed by this label.
Source type: paper. Paper: linked separately.
Review date: 21 Nov 2026. Review scope: §§2–4, Table 4 and taxonomy Appendix D aggregate methods.
No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.
Editorial assessment: Impact 2/5 · Validation 2/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.