MedElevator Войти в исследования
← Поиск публикаций

DOI 10.1016/j.neunet.2026.109557

Медицинская библиотека · Neural networks : the official journal of the International Neural Network Society · 2026

Modality as heterogeneity: Node splitting and graph rewiring for multimodal graph learning.

Zhang Yihan, Kuruoglu Ercan E

СтатьяПеревод в очереди

Аннотация

Multimodal graphs are gaining increasing attention due to their rich representational power and wide applicability, yet they introduce substantial challenges arising from severe modality confusion. To address this issue, we propose NSG (Node Splitting Graph)-MoE, a multimodal graph learning framework that integrates a node-splitting and graph-rewiring mechanism with a structured Mixture-of-Experts (MoE) architecture, explicitly coupling expert specialization and graph connectivity patterns. It decomposes each node into modality-specific components and assigns relation-aware experts to process heterogeneous message flows, thereby preserving structural information and multimodal semantics while mitigating the undesirable mixing effects commonly observed in general-purpose GNNs. Extensive experiments on three multimodal benchmarks demonstrate that NSG-MoE consistently surpasses strong baselines. Despite incorporating MoE-which is typically computationally heavy-our method achieves competitive training efficiency. Beyond empirical results, we provide a spectral analysis revealing that NSG performs adaptive filtering over modality-specific subspaces, thus explaining its disentangling behavior. Furthermore, an information-theoretic analysis shows that the architectural constraints imposed by NSG reduces mutual information between data and parameters, improving generalization capability.

Для цитирования: Zhang Yihan, Kuruoglu Ercan E. Modality as heterogeneity: Node splitting and graph rewiring for multimodal graph learning.. Neural networks : the official journal of the International Neural Network Society. 2026. DOI: 10.1016/j.neunet.2026.109557.
Подготовка и перевод публикации
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…

История · было → стало

Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.

Тип публикации указан по данным источника. Уровень доказательности автоматически не присваивается.