An interpretable multimodal machine learning model for predicting malignancy of thyroid nodules in low-resource scenarios.

BACKGROUND

Thyroid nodules (TNs) represent a prevalent clinical issue in endocrinology. The diagnostic process for malignant TNs typically involves a three-stage detection: the function detection, color ultrasound (CU) detection and biopsy. Early identification is crucial for effective management of malignant TNs.

METHODS

This study developed a multimodal network for classifying CU images and thyroid function (TF) test data. Specifically, the PubMedClIP model was employed to extract visual features from CU images, generating a 512-dimensional feature vector. This vector was subsequently concatenated with five indicators of TF tests, as well as gender and age information, to construct a comprehensive representation. The combined representation was then fed into a downstream ML classifier, where we evaluated seven models, including AdaBoost, Random Forest, and Logistic Regression.

RESULTS

Among the seven ML models evaluated, the AdaBoost classifier demonstrated the highest overall performance, surpassing other classifiers in terms of area under the curve (AUC), F1, accuracy, and coordinate attention (CA) metrics. The incorporation of visual features extracted from CU images using PubMedCLIP further enhanced the model’s performance. Feature importance analysis revealed that laboratory indicators such as free thyroxine (FT4), free triiodothyronine (FT3), and clip_feature_184 were the most influential clinical variables. Additionally, the integration of PubMedCLIP significantly improved the model’s capacity to accurately classify data by leveraging both clinical and imaging information.

CONCLUSION

The proposed PubMedCLIP-based multimodal framework, which jointly utilizes ultrasound imaging features and clinical laboratory data, demonstrated superior diagnostic performance in differentiating benign from malignant TNs. This approach offers a promising tool for individualized risk assessment and clinical decision support, potentially facilitating more precise and personalized protocols for patients with TNs. CLINICAL TRIAL NUMBER: Not applicable.