Dynamic Multi-modal Fusion for Biosignal-based Motion Sickness Prediction in Vehicles.

With the advent of autonomous vehicles, motion sickness (MS) has emerged as a significant issue, affecting the comfort and safety of drivers and passengers. However, traditional MS research, often confined to simulations or manual data analysis, does not fully capture real-world complexities. Therefore, the significance of multi-biosignal fusion, which can reflect the complexity and utility of data, is underscored. In this study, we propose a novel dynamic multi-modal fusion for the MS classification (DMFMS) framework. DMFMS adaptively focuses on significant samples by evaluating data quality in noisy environments. It includes confidence-aware learning to estimate the reliability of modalities, a dynamic gating mechanism that adjusts based on each modality's contribution to the features, and a spatial-temporal attention module (STAM) that focuses on relevant information while filtering out the extraneous. We conducted extensive experiments on a multi-biosignal dataset from real driving scenarios, involving data from 13 subjects. The results show that DMFMS outperforms conventional MS prediction models, showing that the proposed dynamic fusion approach is the superior solution for detecting MS in a real-world driving environment.