Benchmarking Open-Source Pathology Foundation Models for Breast Cancer Biomarker Prediction from H&E Whole-Slide Images.

BACKGROUND/

OBJECTIVES

Breast cancer biomarker detection through immunohistochemistry (IHC) is essential for treatment planning but faces challenges including turnaround time, variability, and laboratory resource constraints. Large open-source vision-language foundation models offer a potential avenue for inferring biomarker status directly from hematoxylin-and-eosin (H&E)-stained whole-slide images (WSIs).

METHODS

We evaluated two open-source pathology foundation models-TITAN (Transformer-based Pathology Image and Text Alignment Network, approximately 48.5 M parameters) and CHIEF (Clinical Histopathology Imaging Evaluation Foundation Model, approximately 1.2 M parameters)-for predicting estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) status from H&E-stained breast cancer WSIs. WSI data were obtained from The Cancer Genome Atlas Breast Invasive Carcinoma collection (TCGA-BRCA) via the NCI Imaging Data Commons, with biomarker labels from the NCI Genomic Data Commons. In total, 937 cases (995 WSIs; 78.3% ER-positive) were evaluated for ER, 934 cases (992 WSIs; 68.4% PR-positive) for PR, and 646 cases (691 WSIs; 21.1% HER2-positive) for HER2. All evaluation was performed under a strict patient-level 50/25/25 split with 10 independent random partitions; metrics are reported as the mean across partitions with percentile-based 95% confidence intervals. Performance was assessed using area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), sensitivity, specificity, and positive predictive value (PPV).

RESULTS

TITAN and CHIEF achieved comparable performance for ER (TITAN AUROC: 0.885 [95% CI: 0.848, 0.921], AUPRC: 0.954 [0.940, 0.964]; CHIEF AUROC: 0.877 [0.831, 0.914], AUPRC: 0.955 [0.938, 0.969]) and PR (TITAN AUROC: 0.799, AUPRC: 0.868; CHIEF AUROC: 0.791, AUPRC: 0.864). At the default 0.5 operating point, ER PPV was 0.90 and PR PPV was 0.79-0.81. For HER2, both models achieved AUROC values of 0.71-0.74 and AUPRC values of 0.41-0.45-well above the prevalence-based random baseline (approximately 0.211)-but default-threshold sensitivity was very low (approximately 0.07-0.08), reflecting class imbalance and the use of an uncalibrated default threshold rather than a categorical absence of morphologic signal.

CONCLUSIONS

Under retrospective evaluation, both models demonstrate strong discriminative performance for ER and moderate performance for PR; HER2 prediction at the default operating point is limited and motivates threshold-calibration and multimodal extensions before any clinical use. AUPRC summarizes precision-recall behavior across thresholds and is distinct from threshold-specific precision (PPV); the two should be reported together for clinical-utility assessment in pathology AI. The findings are hypothesis-generating and motivate prospective external validation across independent institutional cohorts before any clinical deployment is considered.