Large Language Models for Interpreting Standardized Neuropsychological Summaries in Idiopathic Normal Pressure Hydrocephalus A Pilot Comparative Study.

BACKGROUND

Diagnosis and prognostication in idiopathic normal pressure hydrocephalus (iNPH) remain challenging. This study examined whether large language models (LLMs) can classify neuropsychological test (NPT) summaries for identifying iNPH and exploring postoperative shunt responsiveness.

METHODS

This retrospective study included 42 shunt-treated iNPH patients and 53 neurological controls. De-identified NPT summaries with a standardized structure across cognitive domains were evaluated using three LLMs (ChatGPT-5, Gemini 2.5 Flash, DeepSeek) under a standardized zero-shot prompt. Each model generated categorical outputs (iNPH-compatible) and confidence scores (0-100). For analysis, responses labeled "Compatible" were considered AI-positive, whereas "Incompatible" or "Indeterminate" responses were treated as AI-negative; outputs with low confidence were conservatively handled as negative classifications. Diagnostic performance was compared with established clinical diagnoses. Prognostic performance was assessed for postoperative improvement, defined as ΔiNPHGS ≥1.

RESULTS

Diagnostic performance showed the highest accuracy for ChatGPT-5 (78%, 95% CI: 68.6-85.1), followed by Gemini 2.5 Flash (67%, 95% CI: 57.4-75.9) and DeepSeek (63%, 95% CI: 53.1-72.2). Corresponding AUC values were 0.84, 0.67, and 0.63. Calibration metrics for ChatGPT-5 included a Brier score of 0.18 and a calibration slope of 0.94. In the exploratory prognostic analysis conducted in the shunted iNPH cohort (n = 42), postoperative improvement occurred in 37 patients, whereas five patients were classified as non-responders. Prognostic accuracy was 83% (95% CI: 69.8-92.5) for ChatGPT-5, 67% (95% CI: 52.1-78.8) for DeepSeek, and 64% (95% CI: 49.8-76.9) for Gemini 2.5 Flash. Positive predictive values ranged from 87% to 96%. When model confidence was analyzed as a continuous predictor, AUC values for prognostic discrimination were 0.69 for DeepSeek, 0.56 for ChatGPT-5, and 0.53 for Gemini 2.5 Flash. Decision-curve analysis indicated varying net benefit patterns across threshold probabilities.

CONCLUSIONS

LLMs applied to standardized NPT summaries demonstrated measurable diagnostic signal and exploratory prognostic potential in iNPH. These findings support the feasibility of NPT-based LLM assistance, but larger, balanced, externally validated cohorts are required before clinical implementation.