Cross-Hemispheric Spatial-Temporal Attention Network for Decoding Silent Speech From EEG.

OBJECTIVE

Speech, as the core of advanced human cognition, is fundamental to social interaction and daily life. Electroencephalogram (EEG)-based speech brain-computer interface (BCI) offers a novel communication pathway for patients with speech disorders, where deep learning has demonstrated significant advantages. Given the established dominance of the left hemisphere in speech processing, exploring methods to extract speech-related neural features fully is crucial for enhancing decoding performance.

APPROACH: In this study, EEG signals were recorded during a silent speech task involving the articulation of 10 distinct Chinese characters. Leveraging the principle of language function lateralization, we proposed a novel deep learning model, the cross-hemispheric spatial-temporal attention network (CHSTAN), for EEG-based silent speech recognition. A multiscale temporal convolution block was employed to extract the temporal dynamics of EEG signals. A hemispheric spatial convolutional block was designed to independently process spatial information from the left and right hemispheres. Furthermore, the cross-attention mechanism was introduced to enhance inter-hemispheric feature interaction and specifically reinforce left-hemispheric feature representation for the final classification.

RESULTS

We compared CHSTAN with other existing methods using 5-fold cross-validation on the collected dataset. CHSTAN achieved an average classification accuracy of 49.88% and an average F1-score of 48.75% in decoding the 10 Chinese characters, significantly outperforming other methods.

CONCLUSION

The results indicate that the CHSTAN performs effectively in silent speech EEG classification tasks. Notably, the feature patterns learned through its innovative architecture correspond to neural speech processing mechanism.

SIGNIFICANCE: CHSTAN provides valuable insights and practical solutions for improving the performance of EEG-based speech decoding.