Harnessing Large Language Models in Neonatal Intraventricular Hemorrhage: Exploring Retrieval Augmented Generation Methodology for Prognostic Variable Discovery.

OBJECTIVE

The objective of this study is to evaluate whether large language models (LLMs) can autonomously synthesize existing literature and accurately extract prognostic variables for neonatal intraventricular hemorrhage (IVH) and its outcomes while assessing their capability for clinical feature ranking and risk stratification.

STUDY

DESIGN

This pilot study employed a systematic literature review combined with retrieval-augmented generation (RAG) methodology. GPT 4 (OpenAI) and Claude Sonnet (4.0, Anthropic) were prompted to identify peer-reviewed studies utilizing machine learning and deep learning to predict IVH outcomes in preterm neonates. Data extraction was prompted to follow TRIPOD artificial intelligence (AI) guidelines, capturing study design, population characteristics, predictor variables, and outcome measures. Semi-automated RAG extraction was performed with manual validation to mitigate hallucination risk.

RESULTS

LLMs initially identified 39 studies, with 28 meeting some or all the validation criteria after excluding references that were hallucinated. From these, 14 distinct prognostic predictors were extracted across four outcome domains: mortality, progression, complications, and resolution. Universal high-impact predictors included gestational age (13 mentions; 41%), birth weight (8 mentions, 25%), and Apgar scores (11 mentions, 34%). Variables were categorized into three clinical tiers based on frequency, outcome breadth, and modifiability. A preliminary risk stratification model demonstrated high-risk neonates (<28 weeks, <1,000g, Apgar <3) with estimated progression risk >70%, and mortality >50%, while low-risk neonates (>32 weeks, >1,500 g, Apgar > 5) showed favorable trajectories.

CONCLUSION

This study demonstrates that LLMs can synthesize medical literature and extract clinically relevant prognostic variables for neonatal IVH outcomes. However, LLM outputs were susceptible to hallucinations and incomplete data synthesis, underscoring the need for rigorous clinical oversight and human validation to ensure reliability. The identified universal predictors provide a foundation for developing AI-assisted clinical decision support tools. Notable research gaps include the complete absence of resolution prediction studies and limited investigation of complication predictors, highlighting opportunities for future investigation in precision neonatology.

KEY POINTS: · LLMs can synthesize medical literature. · LLMs can assist in creating a prognostic ontology. · Human oversight is critical when using LLMs for healthcare.