Comparison Between Oculoplastic Surgeons and Two Large Language Models in the Nonsurgical Management of Thyroid Eye Disease: A Questionnaire-Based Cross-Sectional Study.

BACKGROUND

Thyroid eye disease (TED) presents heterogeneous clinical manifestations requiring nuanced, individualized management. Large language models (LLMs) have shown potential in clinical reasoning, but their consistency with physician decision-making in nonsurgical TED management remains unclear.

METHODS

A structured 19-item questionnaire covering medical, injection-based, and radiotherapeutic treatments for TED was distributed to 17 oculoplastic surgeons across the Asia-Pacific region. Responses were binarized (1 = selected, 0 = not selected) and compared with standardized outputs from GPT and Gemini. Jaccard and cosine similarity indices were calculated to quantify the alignment between physicians and LLMs.

RESULTS

Overall, GPT and Gemini demonstrated comparable alignment with clinicians' treatment preferences. Both models closely matched physician consensus regarding glucocorticoid use and radiotherapy indications but diverged in immunosuppressant selection and injection-based strategies. Similarity was generally consistent across physician experience and patient-volume subgroups.

CONCLUSIONS

GPT and Gemini demonstrated moderate agreement with participating oculoplastic surgeons in structured nonsurgical TED management scenarios. These findings support further evaluation of LLM-assisted applications within controlled clinical validation frameworks, and additional validation using guideline-based benchmarks and real-world clinical scenarios is required.