AI Science Tracker
EN

Interface-name checks rejected some AI-written code that passed behavioral tests

Independent researcher · 25 Oct 2026 · computer science

Discovery

Interface-name checks rejected some AI-written code that passed behavioral tests

Source check: Source checked. Not independent scientific confirmation.

Finding: In a post hoc analysis of AI-written code, 19 of 36 artifacts rejected solely for dropping leading underscores from parameter names passed the original hidden behavioral checks after only that name-comparison rule was bypassed. The original study kept those artifacts recorded as contract failures. The selection covers one specific failure reason in the study’s coding tasks.

Technology: Claude Opus 5.5; Qwen3-Coder; Nemotron Cascade 2

Family Claude: profile · models page

Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.

AI contribution
Existing language models acting as coding workers generated artifacts assessed under an interface contract and hidden behavioral tests.
Test and result
A secondary offline diagnostic applied selected generated artifacts unchanged to fresh task copies, bypassing only strict parameter-name comparison, then ran the original sealed behavioral checks. No parent review or end-to-end acceptance was replayed.
Limitation / still unknown
This selected subset cannot estimate overall pass rates or establish that contract requirements were unnecessary in deployment. Original saved records were private and not independently auditable; passing these author-supplied checks does not prove general software correctness.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2610.34241v1 · https://arxiv.org/html/2610.34241v1

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 21 Nov 2026. Review scope: v1 §§9.3–9.4 coding delegation and post hoc hidden-test diagnostic.

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 1/5 · Validation 2/5 · Novelty 2/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open in the full catalog for filters, charts and related discoveries. Browse computer science discoveries.

Related catalog entries

Links use documented relationships where available, plus shared catalog context. A shared topic or model family does not establish a scientific dependency.