AI Science Tracker
EN

Interpretable embeddings of speech explain and enhance the brain encoding performance of audio models

Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, United States of America; Microsoft Research, Redmond, Washington, United States of America · 21 Jul 2025 · computer science

Discovery

Interpretable embeddings of speech explain and enhance the brain encoding performance of audio models

Source check: Source checked. Not independent scientific confirmation.

Finding: The analysis found that speech-model representations contain brain-relevant semantic information beyond lower-level speech features, with the reported signal varying by model scale and context length. Reanalysis of a public ECoG dataset with variance-partitioning and feature controls; no new neural data were collected, and the finding is limited to that dataset and partial explanatory variance. The first result-bearing public version was arXiv v1 on 2025-07-21; the later journal version appeared in 2026.

Technology: Speech foundation-model embeddings (Whisper, HuBERT, and WavLM) were compared with interpretable acoustic and linguistic features in neural encoding models.

Publication: Preprint · Test context: retrospective or benchmark. Catalog categories, not an evidence grade.

AI contribution
Speech foundation-model embeddings (Whisper, HuBERT, and WavLM) were compared with interpretable acoustic and linguistic features in neural encoding models.
Test and result
The analysis found that speech-model representations contain brain-relevant semantic information beyond lower-level speech features, with the reported signal varying by model scale and context length.
Limitation / still unknown
Reanalysis of a public ECoG dataset with variance-partitioning and feature controls; no new neural data were collected, and the finding is limited to that dataset and partial explanatory variance.
Evidence sources and annotation context

Evidence context (descriptions follow the cited sources; test type is a catalog classification):

Evidence sources: https://arxiv.org/abs/2507.16080 · https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1014906

Source checked · editorial details

The source page was checked against this entry. Independent confirmation is not assessed by this label.

Source type: paper. Paper: not linked in this catalog; this does not mean no paper exists.

Review date: 22 Nov 2026. Review scope: Primary source full text Methods and Results; version/date checked against bibliographic record..

No follow-up recorded; this is not a claim that no follow-up exists. Follow-up review scope and date are not recorded. No source-supported relationships recorded.

Editorial assessment: Impact 3/5 · Validation 2/5 · Novelty 3/5. Impact and novelty are editorial judgments; validation follows the annotated test type where one exists. Not an evidence grade. Scoring criteria.

Open in the full catalog for filters, charts and related discoveries. Browse computer science discoveries.