Validation of a Large Language Model Enhanced Frailty Index.

BACKGROUND: Efficient, validated frailty screening tools are needed to identify high-risk older adults. The electronic frailty index (eFI), derived from diagnostic codes, may under-capture functional deficits, such as those documented in physical therapy (PT) notes. OBJECTIVE: This study evaluated the performance of a standard eFI and an enhanced eFI— augmented with deficits extracted from PT notes using a large language model (LLM)—against a geriatrician-performed frailty index based on a comprehensive geriatric assessment (FI-CGA). DESIGN: Retrospective cross-sectional study. PARTICIPANTS: Adults with a PT visit within 30 days prior to FI-CGA at a large health system in Boston, MA between March 2018 and September 2024. MAIN MEASURES: An LLM (GPT4o, Microsoft Azure) was prompted to extract eFI deficits from PT notes. Descriptive statistics were used to report demographics characteristics and performance measures. Discrimination was assessed using AUROC with Delong’s test, and agreement was evaluated using Pearson correlation coefficients. KEY RESULTS: Among 2413 subjects, 1534 (63.6%) were frail (FI-CGA > = 0.25). The AUROCs for the enhanced eFI compared to standard eFI predicting geriatrician assessed frailty (FI-CGA) was 70.1% vs. 61.4% (p < 0.001). Pearson correlations were 0.756 for the eFI vs. enhanced eFI, 0.239 for the eFI vs. FI-CGA, and 0.365 for the enhanced eFI vs. FI-CGA. CONCLUSIONS: An LLM-enhanced eFI incorporating functional deficits from PT notes demonstrated improved discrimination and stronger correlation with FI-CGA compared to the standard eFI. Leveraging unstructured narratives from PT notes may improve identification of vulnerable older adults who benefit from early intervention.