Health science reports · 2026
Examining the Performance of ChatGPT in Comprehensive Pre-Internship Exam: The Potential of Artificial Intelligence in Medical Education.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Motaghi Niko Michaeel, Karbasi Zahra, Kazemi Maryam, Zahmatkeshan Maryam
Аннотация
BACKGROUND AND AIMS: ChatGPT is a popular large language model with potential educational applications in medicine. However, its performance in standardized, multi-disciplinary medical exams has not been comprehensively assessed. This study evaluates ChatGPT's accuracy and quality in Iran's national medical pre-internship exam.
METHODS: We tested ChatGPT (GPT-3.5, May 3rd version) on 195 multiple-choice questions from the March 2022 Iranian pre-internship exam, covering 23 medical specialties. Questions with visual content were excluded. Each question was asked in a new chat to avoid memory bias. Responses were evaluated by 55 experts using a 5-point Likert scale and compared against the official answer key. Data were analyzed descriptively using SPSS.
RESULTS: ChatGPT answered 68.6% of questions correctly. Expert ratings averaged 4.23/5 (SD = 1.21), indicating good to excellent quality. Best-performing specialties included pharmacology (85.7%), otorhinolaryngology (83.3%), and dermatology (83.3%). Lower performance was observed in pulmonology (42.9%) and epidemiology (50%).
CONCLUSION: ChatGPT shows promise as a supplemental educational tool in medical education, but its accuracy varies by specialty. Faculty guidance is essential to ensure responsible integration until further improvements and validations are made.
METHODS: We tested ChatGPT (GPT-3.5, May 3rd version) on 195 multiple-choice questions from the March 2022 Iranian pre-internship exam, covering 23 medical specialties. Questions with visual content were excluded. Each question was asked in a new chat to avoid memory bias. Responses were evaluated by 55 experts using a 5-point Likert scale and compared against the official answer key. Data were analyzed descriptively using SPSS.
RESULTS: ChatGPT answered 68.6% of questions correctly. Expert ratings averaged 4.23/5 (SD = 1.21), indicating good to excellent quality. Best-performing specialties included pharmacology (85.7%), otorhinolaryngology (83.3%), and dermatology (83.3%). Lower performance was observed in pulmonology (42.9%) and epidemiology (50%).
CONCLUSION: ChatGPT shows promise as a supplemental educational tool in medical education, but its accuracy varies by specialty. Faculty guidance is essential to ensure responsible integration until further improvements and validations are made.