IEEE transactions on bio-medical engineering · 2026
SACM: SEEG-Audio Contrastive Matching for Chinese Speech Decoding.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Wang Hongbin, Jia Zhihong, Shen Yuanzhong, Wang Ziwei, Li Siyang, Tang Zhouping, Shu Kai, Hu Feng, Wu Dongrui
Аннотация
OBJECTIVE: Speech disorders such as dysarthria and anarthria can severely impair patients' ability to communicate verbally. Speech decoding brain-computer interfaces (BCIs) offer a potential alternative by directly translating speech intentions into spoken words, serving as speech neuroprostheses.
METHODS: This paper reports an experimental protocol for Mandarin Chinese speech decoding BCIs and proposes a contrastive learning-based decoding algorithm termed SEEG and Audio Contrastive Matching (SACM). Stereo-electroencephalography (SEEG) and synchronized audio data were collected from ten patients with drug-resistant epilepsy as they performed a word-level reading task. SACM leverages the cross-modal correlation between neural activity and audio signals to decode matched speech segments.
RESULTS: The proposed framework achieved accuracies significantly exceeding random matching in both isolated-word and continuous speech decoding tasks, and outperformed SEEG-only baselines across seven backbone architectures in the isolated-word setting. Electrode- wise analysis revealed that a single ventral sensorimotor cortex electrode achieved performance comparable to that of the full electrode array. Our code is publicly available.
SIGNIFICANCE: To our knowledge, this is the first work on multimodal decoding for tonal speech BCIs.
METHODS: This paper reports an experimental protocol for Mandarin Chinese speech decoding BCIs and proposes a contrastive learning-based decoding algorithm termed SEEG and Audio Contrastive Matching (SACM). Stereo-electroencephalography (SEEG) and synchronized audio data were collected from ten patients with drug-resistant epilepsy as they performed a word-level reading task. SACM leverages the cross-modal correlation between neural activity and audio signals to decode matched speech segments.
RESULTS: The proposed framework achieved accuracies significantly exceeding random matching in both isolated-word and continuous speech decoding tasks, and outperformed SEEG-only baselines across seven backbone architectures in the isolated-word setting. Electrode- wise analysis revealed that a single ventral sensorimotor cortex electrode achieved performance comparable to that of the full electrode array. Our code is publicly available.
SIGNIFICANCE: To our knowledge, this is the first work on multimodal decoding for tonal speech BCIs.