Benchmarking Gemini 1.5 Pro in Pediatric Emergency Radiology: Sensitivity Gap and Physiological Bias in the Immature Skeleton.

Background/

Objectives

To assess the diagnostic performance of Gemini 1.5 Pro, a general-purpose Large Multimodal Model (LMM), as a zero-shot, smartphone-based decision support tool to distinguish pediatric fractures from healthy controls.

Methods

A retrospective diagnostic accuracy study was conducted on 182 pediatric radiographs (112 confirmed fractures, 70 healthy controls) using the Gemini 1.5 Pro mobile interface with a 4-question prompt protocol (Detection, Characterization, Skeletal Maturity, Management). Diagnostic metrics were calculated with 95% Confidence Intervals (95% CI). Ground truth was established by a senior musculoskeletal radiologist.

Results

The model showed an overall sensitivity of 90.2% (95% CI: 83.1-95.0%) and specificity of 87.1% (95% CI: 77.0-93.9%). A significant sensitivity gap occurred between displaced (100.0%; 95% CI: 91.4-100.0%) and non-displaced fractures (84.5%; 95% CI: 74.0-92.0%; p = 0.007). Qualitative taxonomy revealed a primary physiological bias leading to a 12.8% false-positive rate due to growth plate misinterpretation, alongside minor characterization and localization errors (3.6% of correctly identified fractures). When fractures were correctly identified, proposed management strategies aligned with international standards in 95.0% of cases. Inter-observer reliability was excellent (weighted k = 0.96).

Conclusions

Gemini 1.5 Pro displays robust clinical reasoning for obvious trauma. However, its zero-shot performance is limited by subtle morphological disruptions and developmental anatomical nuances such as open growth plates. It shows promise as a mobile clinical decision support system for non-specialists, but expert human supervision remains necessary to ensure safety in the complex radiographic landscape of the immature skeleton.