Diagnostic Accuracy and Pitfalls of Publicly Available Artificial Intelligence Models for Nail Disorders.

BACKGROUND

Artificial intelligence is increasingly applied to dermatology, yet its reliability in clinically high-stakes subspecialty domains such as nail disease remains poorly understood. This study evaluated the ability of widely available Vision Language Models (VLMs), ChatGPT-3.5, ChatGPT-4o, and Google Gemini, in diagnosing common nail disorders using clinical images.

METHODS

A total of 110 clinical images across 11 nail conditions (10 images per diagnosis) were compiled from peer-reviewed sources and confirmed by a board-certified dermatologist. Disorders included infectious, inflammatory, and neoplastic entities. Each VLM was queried with a standardized prompt requesting the top three differential diagnoses and confidence (1–10 scale) for the top choice.

RESULTS

Across the models, Google Gemini achieved 34% top-1 accuracy and 51% top-3 accuracy, followed by ChatGPT-4o (33%; 51%) and ChatGPT-3.5 (31%; 45%). VLMs performed best on onychomycosis (sensitivity 0.80; specificity 0.63), green nail syndrome (sensitivity 0.83; specificity 0.98), and onychocryptosis (sensitivity 0.50; specificity 1.00). Furthermore, onychomycosis was the most common incorrect guess across misclassified cases, often appearing as a default diagnosis across a wide range of unrelated presentations. Overall, current VLMs demonstrate poor performance in diagnosing common nail disorders, with moderate accuracy for select conditions, such as onychomycosis. Notably, periungual warts, while frequently encountered in clinical practice, were routinely missed across all models.

CONCLUSIONS

These findings support the potential role of VLMs as adjunctive tools for pattern recognition but reaffirm that clinical judgment remains essential, especially in cases where diagnostic accuracy has critical implications for patient outcomes. &nbsp.