Evaluation of ChatGPT-5-generated clinical management recommendations for HPV infection and cervical premalignant lesions: a scenario-based expert assessment study.

BACKGROUND

Large language models are increasingly explored as clinical decision-support tools; however, their reliability in gynecologic practice remains insufficiently characterized. This study aimed to evaluate the quality of clinical management recommendations generated by ChatGPT-5 for human papillomavirus (HPV) infection and cervical premalignant lesions using a scenario-based expert assessment approach.

METHODS

Fifteen clinically relevant scenarios reflecting common presentations of HPV infection and cervical premalignant lesions were developed in alignment with contemporary guideline frameworks. Each scenario was submitted to ChatGPT-5, and the resulting management and treatment recommendations were collected. The responses were independently evaluated by 50 experts, including 25 obstetrics and gynecology specialists and 25 gynecologic oncology specialists. Four predefined domains—guideline adherence, coverage, applicability, and overall satisfaction—were assessed using a 5-point Likert scale. Between-group comparisons, effect size analyses, and inter-rater reliability metrics were applied to explore differences in expert evaluations.

RESULTS

The assessments revealed domain-specific variations between expert groups. Gynecologic oncologists assigned higher scores for guideline adherence across multiple scenarios, whereas obstetrics and gynecology specialists reported higher scores for coverage and overall satisfaction. Applicability ratings were consistently high in both groups, with minimal between-group differences. Inter-rater reliability varied across domains, suggesting differing levels of agreement depending on the evaluated aspect of the responses.

CONCLUSIONS

ChatGPT-5 demonstrated the capacity to generate clinically relevant recommendations for HPV-related cervical premalignant conditions. Nonetheless, variability in expert evaluations—particularly in relation to guideline adherence—highlights the need for specialist oversight and cautious integration of large language models into clinical gynecologic decision-making.