MicrotiaGPT: Appraisal of Microtia Care Recommendations From an Artificially Intelligent Chatbot.

OBJECTIVE

Evaluate the accuracy, comprehensiveness, and similarity to provider response of ChatGPT-4o in providing patient education regarding microtia and aural atresia management.

METHODS

Ten standardized inquiries were created across three domains (General Information, Hearing/Anatomy, Treatment Decision-Making) and entered into ChatGPT-4o. Responses were evaluated by an international multidisciplinary cohort of 23 experts, including pediatric otolaryngologists and audiologists. Using 5-point Likert scales, graders assessed factual accuracy, comprehensiveness, similarity to physician responses, and overall quality. Readability was analyzed using Flesch-Kincaid metrics.

RESULTS

ChatGPT-generated responses demonstrated high accuracy (mean = 4.03 [SD = 0.66]), comprehensiveness (mean = 4.18 [SD = 0.54]), and similarity to the provider's response (4.02 [0.70]). Audiologists assigned significantly lower ratings than pediatric otolaryngologists regarding accuracy (mean = 3.76 vs. 4.13; p < 0.001) and comprehensiveness (3.94 vs. 4.30; p = 0.046). Qualitative analysis included concerns regarding advanced terminology, omission of specific risks, and a lack of empathetic framing regarding parental guilt. Flesch-Kincaid Reading Grade Level often exceeded the 5th-grade average level for patient education materials (mean = 9.64 [SD = 1.21]).

CONCLUSION

ChatGPT-4o provides reliable introductory information but lacks the technical nuance and multidisciplinary coordination essential for complex microtia care. While a useful adjunct for early information-seeking, ChatGPT-4o lacks the multidisciplinary framing and depth required for ideal counseling.