Vision-weighted diagnostic gain (VWDG) in the diagnosis of oral malignant and potentially malignant lesions: A multimodal comparative analysis.

BACKGROUND

Early diagnosis of oral malignant lesions is essential for improving outcomes in head and neck oncology, yet overlapping clinical features often hinder recognition. Multimodal large language models (LLMs), which integrate text and image inputs, may provide adjunctive diagnostic support. We applied vision-weighted diagnostic gain (VWDG), a novel metric to quantify the added value of visual input, in a comparative evaluation of ChatGPT-5 and Gemini Pro 2.5.

METHODS

A paired diagnostic accuracy study was conducted on 150 biopsy-confirmed oral lesions, comparing ChatGPT-5 (OpenAI) and Gemini Pro 2.5 (Google) with board-certified oral medicine experts. Each vignette included structured clinical data and images. Diagnostic accuracy was measured at Top-1, Top-3, and Top-5 ranks. VWDG was expressed as absolute gain, relative gain, and error reduction, with subgroup analyses by lesion type and difficulty.

RESULTS

ChatGPT-5 showed consistent vision-driven improvements, with Top-1 accuracy increasing from 27.3% (text) to 50.0% (image+text). Gains were most pronounced in oral potentially malignant disorders (+133% relative gain) and in moderate-to-high difficulty cases. Gemini Pro 2.5 demonstrated stable text-only performance but derived minimal or inconsistent benefit from images, including negative gains at higher ranks. Experts achieved the highest Top-1 accuracy (65.3%), but ChatGPT-5 multimodal outputs approached expert-level performance at Top-3 and Top-5.

CONCLUSION

VWDG provided a reproducible framework for quantifying the contribution of visual input in multimodal LLMs. ChatGPT-5 benefited substantially from images, particularly in complex cases, whereas Gemini Pro 2.5 showed limited integration. These findings highlight both the promise and current constraints of multimodal AI in oral oncology.