Generative Artificial Intelligence Chatbots for Parental Guidance in Pediatric Orthopaedics: A Pilot Feasibility Study.

BACKGROUND

Parents of children with pediatric orthopaedic conditions frequently have questions after their clinic visit, and a reliable postconsultation resource to address these enquiries is lacking. Online resources are variable in quality, and general purpose artificial intelligence (AI) models have demonstrated safety limitations, including unprompted treatment recommendations and citation inaccuracies. A purpose-built, supervised generative AI solution may offer a safer and more reliable alternative.

METHODS

A generative AI chatbot was developed using retrieval-augmented generation and trained on OrthoKids educational materials covering 5 common pediatric orthopaedic conditions: in-toeing, flatfoot, scoliosis, bow legs, and knock knees. After iterative refinement by 2 fellowship-trained pediatric orthopaedic surgeons, the chatbot was evaluated by 18 multidisciplinary clinicians using a structured 12-item 5-point Likert scale survey assessing accuracy, safety, usability, and clinical acceptability.

RESULTS

Mean scores exceeded 4.0 of 5.0 on 11 of 12 domains. Clarity of explanation achieved unanimous positive endorsement (100%), while protocol alignment received the highest mean score (4.50 ± 0.62). Citation accuracy was the lowest-rated domain (4.00 ± 0.77). No hallucinations were identified, and 88.9% of evaluators confirmed the absence of unprompted treatment recommendations. The reverse-scored safety item (Q8) demonstrated the greatest inter-rater variability (mean 3.11 ± 1.57), with orthopaedic specialists rating perceived harm substantially lower (mean 1.33) than nurses (mean 4.40).

CONCLUSION

A purpose-built gen AI chatbot demonstrated high clinical accuracy and acceptability among multidisciplinary evaluators, with effective avoidance of unprompted treatment recommendations. These pilot findings support feasibility for use as a parental education adjunct in pediatric orthopaedic practice.

LEVEL OF EVIDENCE: Level III. See Instructions for Authors for a complete description of levels of evidence.