Transformer-based Siamese multimodal imaging model for predicting pathological complete response after neoadjuvant chemoradiotherapy in locally advanced rectal cancer.

BACKGROUND

Accurate prediction of pathological complete response (pCR) following neoadjuvant chemoradiotherapy (nCRT) remains a critical yet unresolved challenge in locally advanced rectal cancer (LARC). Reliable noninvasive biomarkers could facilitate organ-preserving strategies and help avoid unnecessary radical surgery.

METHODS

We retrospectively analyzed 440 patients with LARC from three centers who underwent nCRT and multimodal magnetic resonance imaging (MRI). A total of 1,197 handcrafted radiomic features were extracted using PyRadiomics, while deep learning features were derived from a Med3D-pretrained ResNet101 and subsequently compressed via principal component analysis. To integrate pre- and post-treatment MRI features, we developed a transformer-based Siamese multimodal framework. Model performance was systematically evaluated using receiver operating characteristic (ROC) curves, decision curve analysis, calibration, and survival outcomes.

RESULTS

In the independent validation cohort, the Siamese multimodal transformer achieved the highest predictive performance (AUC = 0.789), outperforming early-fusion machine learning models (AUC = 0.629) and single-timepoint transformer models (AUC = 0.739 and 0.728). Decision curve analysis demonstrated superior net clinical benefit, and calibration curves indicated close agreement between predicted and observed outcomes. Notably, in the training cohort, transformer-based risk stratification was significantly associated with progression-free survival (p < 0.05).

CONCLUSIONS

We propose and validate a transformer-based multimodal imaging framework that effectively integrates Siamese multimodal features to enable individualized prediction of pCR in LARC. These findings underscore the potential of transformer architectures for noninvasive risk stratification and provide a novel approach to inform precision treatment strategies.