Multi-label tongue image classification via label semantics-driven feature interaction enhancement.

Multilabel classification of tongue images plays a pivotal role in intelligent diagnosis within Traditional Chinese Medicine (TCM), enabling clinical decision-making through systematic analysis of the intricate relationships between tongue characteristics and pathological conditions. However, existing methods face critical imitations: (1) inadequate exploitation of label semantic information and local feature representations, leading to suboptimal cross-modal interaction efficiency; and (2) neglect of synergistic and complementary mechanism among hierarchical features (e.g., holistic and detailed features), which severely constrains classification performance. To address these challenges, this paper proposes a multilabel tongue image classification framework based on Label Semantic-Driven Feature Interaction Enhancement (LSDFIE). The core objective of this method is to achieve robust recognition by integrating instance-level (local features) and spatial-level (contextual features) representations. The LSDFIE framework comprises two key modules: 1) Cross-Modal Fusion Module which aligns label semantics with local tongue image features using low-rank bilinear attention to generate instance-level representations; 2) Image Modality Attention Enhancement Module which evaluates correlation between spatial-level and instance-level representations to produce instance-level features enriched with contextual information. These representations are subsequently fused via content-guided attention which enhances discriminative features while suppressing irrelevant ones. This approach effectively bridges the semantic gap between modalities and leverages hierarchical feature representations comprehensively. Experimental results demonstrate that the proposed method achieves mean average precision (mAP) scores of 93.02 % and 95.33 % on the MlLTID and Tooth-Marked datasets, respectively. On the ChestX-ray14 dataset, the LSDFIE attains an area under the curve (AUC) of 84.11 %, outperforming the state-of-the-art by 0.36 %. These results validate the method's superior classification accuracy and strong generalization capability across diverse medical imaging tasks in healthcare applications.