Dual-stream token fusion with Swin Transformer and lesion-aware tokens for gastric metaplasia classification in IoMT-assisted deployment.

BACKGROUND

Gastric intestinal metaplasia (GIM) is often visually inconspicuous on routine endoscopy, while many artificial intelligence systems rely on dense supervision, lack calibrated probabilities, or provide limited evidence of transfer across datasets and devices. We developed a single-frame, four-class endoscopic classifier that jointly models anatomic context and metaplasia status.

METHODS

We propose a dual-stream architecture that combines an RGB Swin Transformer backbone with SHAP-guided lesion-aware multi-scale auxiliary tokens. The two streams are fused through class-token attention to obtain a compact and interpretable representation without relying on video context or pixel-level masks. To address class imbalance, training combines class-balanced focal loss, balanced-softmax/logit adjustment, class-aware sampling, and validation-tuned per-class thresholds. The model was evaluated on an internal four-class cohort of 666 endoscopic still frames and externally assessed, without retuning, on an unseen public endoscopy dataset recast as a binary normal-versus-abnormal task.

RESULTS

On the internal cohort, the proposed model achieved macro-AUROC 0.950, macro-AUPRC 0.920, macro-F1 0.926, and accuracy 0.954, with expected calibration error 0.034 and inference latency of approximately 182 ms per 224 × 224 frame. On the unseen external dataset, the model retained AUROC 0.940, AUPRC 0.900, and F1 0.890 using frozen operating thresholds. Comparative and ablation analyses indicated that lesion-aware tokenization and token fusion contributed more strongly to performance gains than backbone choice alone, while calibration quality also improved.

CONCLUSIONS

A dual-stream, single-frame token-fusion model can provide accurate, calibrated, and interpretable classification of gastric intestinal metaplasia while remaining compatible with low-latency edge-oriented inference. Although broader multicenter validation is still required, the results support the feasibility of deployment-oriented AI assistance for endoscopic GIM triage.