Journal of oral biosciences · 2026
Multiclass diagnostic performance and error patterns of a multimodal large language model in oral histopathology: A WHO-aligned study.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Hassanein Fatma E A, Hassan Muhammad Talha, Javed Mariya, Hamouda Mai Abdelhalim, Amer Mohamed, Saad Fatma Adel, Madkour Gihane Gharib, Mohamed Mai Hafez, Abou-Bakr Asmaa
Аннотация
OBJECTIVES: To evaluate the diagnostic performance, feature-level agreement, and error patterns of a multimodal large language model (MLLM) across the oral dysplasia-malignancy continuum.
METHODS: In this retrospective diagnostic accuracy study, 300 histopathology images representing normal oral mucosa (n = 100), oral potentially malignant disorders (OPMD) (n = 100), and oral squamous cell carcinoma (OSCC) (n = 100) were evaluated using a standardized two-stage prompting framework consisting of morphological feature extraction followed by structured diagnostic classification. Reference diagnoses were established by three experienced oral pathologists. Interobserver agreement among pathologists was assessed using Fleiss' κ statistics. Primary outcomes included overall accuracy, macro-F1 score, and balanced accuracy. Secondary outcomes included feature-level agreement, confidence-accuracy alignment, and clinically significant error patterns.
RESULTS: Interobserver agreement among the reference pathologists was substantial for the primary three-class diagnosis (Fleiss' κ = 0.81). Overall accuracy was 0.667 (95% confidence interval [CI]: 0.620-0.727), with a macro-F1 score of 0.662 and balanced accuracy of 0.755. Performance varied substantially across diagnostic categories, with high sensitivity for normal mucosa (0.990; 95% CI: 0.968-1.000). Reduced sensitivities were observed for OPMD (0.430; 95% CI: 0.340-0.531) and OSCC (0.580; 95% CI: 0.495-0.688), indicating reduced discriminative ability relative to normal mucosa. Feature-level agreement was generally low, with the weakest agreement for architectural dysplasia features (κ = -0.116 to -0.068), slight agreement for invasion-related features (κ = 0.111-0.120), and the highest agreement was for nuclear hyperchromasia (κ = 0.281). Most misclassifications (84.8%) occurred at the dysplasia-malignancy interface. Proxy calibration analysis demonstrated systematic overconfidence in the histopathological predictions.
CONCLUSIONS: The MLLM demonstrated moderate performance in oral histopathology, with reliable recognition of normal tissue and reduced discriminative ability for OPMD and OSCC relative to normal mucosa, particularly at the dysplasia-malignancy interface. These findings support a potential assistive role under expert supervision while highlighting important limitations in clinically critical diagnostic settings.
METHODS: In this retrospective diagnostic accuracy study, 300 histopathology images representing normal oral mucosa (n = 100), oral potentially malignant disorders (OPMD) (n = 100), and oral squamous cell carcinoma (OSCC) (n = 100) were evaluated using a standardized two-stage prompting framework consisting of morphological feature extraction followed by structured diagnostic classification. Reference diagnoses were established by three experienced oral pathologists. Interobserver agreement among pathologists was assessed using Fleiss' κ statistics. Primary outcomes included overall accuracy, macro-F1 score, and balanced accuracy. Secondary outcomes included feature-level agreement, confidence-accuracy alignment, and clinically significant error patterns.
RESULTS: Interobserver agreement among the reference pathologists was substantial for the primary three-class diagnosis (Fleiss' κ = 0.81). Overall accuracy was 0.667 (95% confidence interval [CI]: 0.620-0.727), with a macro-F1 score of 0.662 and balanced accuracy of 0.755. Performance varied substantially across diagnostic categories, with high sensitivity for normal mucosa (0.990; 95% CI: 0.968-1.000). Reduced sensitivities were observed for OPMD (0.430; 95% CI: 0.340-0.531) and OSCC (0.580; 95% CI: 0.495-0.688), indicating reduced discriminative ability relative to normal mucosa. Feature-level agreement was generally low, with the weakest agreement for architectural dysplasia features (κ = -0.116 to -0.068), slight agreement for invasion-related features (κ = 0.111-0.120), and the highest agreement was for nuclear hyperchromasia (κ = 0.281). Most misclassifications (84.8%) occurred at the dysplasia-malignancy interface. Proxy calibration analysis demonstrated systematic overconfidence in the histopathological predictions.
CONCLUSIONS: The MLLM demonstrated moderate performance in oral histopathology, with reliable recognition of normal tissue and reduced discriminative ability for OPMD and OSCC relative to normal mucosa, particularly at the dysplasia-malignancy interface. These findings support a potential assistive role under expert supervision while highlighting important limitations in clinically critical diagnostic settings.