AI Science Tracker
EN

Manual checks reveal label noise in bug-localization datasets

Affiliation not stated in the reviewed arXiv metadata · 3 Nov 2026 · computer science

Discovery

Manual checks reveal label noise in bug-localization datasets

Source check: Source checked. Not independent scientific confirmation.

Finding: GenTag, an LLM issue classifier, was used to study issue-label quality in five information-retrieval bug-localization datasets. The paper reports label noise after checking a stratified sample against manual annotations and an independently hand-labeled dataset. The noise estimates depend on classifier decisions and the sampled issues.

Technology: GenTag large-language-model issue classifier with manual label checks

Publication: Preprint · Test context: direct test. Catalog categories, not an evidence grade.

AI contribution
GenTag uses an LLM with a fine-grained issue taxonomy to classify bug reports; manual annotations check its labels.
Test and result
A stratified manually labeled sample and an independent hand-labeled dataset were used to assess GenTag; its labels indicated noise across five bug-localization datasets.
Limitation / still unknown
The dataset-noise estimates depend partly on GenTag’s classification decisions and the reviewed samples.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2611.05314v1 · https://arxiv.org/html/2611.05314v1

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: linked separately.

Review date: 22 Nov 2026. Review scope: Primary result-bearing material for the stated method, result and evaluation, with bibliographic version information..

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 2/5 · Validation 3/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open in the full catalog for filters, charts and related discoveries. Browse computer science discoveries.

Related catalog entries

Links use documented relationships where available, plus shared catalog context. A shared topic or model family does not establish a scientific dependency.