MedElevator Войти в исследования
← Поиск публикаций

DOI 10.1590/1413-785220263404e300345

Медицинская библиотека · Acta ortopedica brasileira · 2026

USE OF ARTIFICIAL INTELLIGENCE FOR THE EVALUATION OF THE ENTRANCE EXAMINATION OF THE BRAZILIAN SOCIETY OF SHOULDER AND ELBOW SURGERY.

Kenchian Caio Henrique, Ferreira Karolina Stephany Pereira, Sarmento Lucas Pereira, Teixeira Marcelo Silveira, Zan Renato Aroca, Tamaoki Marcel Jun Sugawara

СтатьяПеревод в очереди

Аннотация

INTRODUCTION

AI is increasingly used for medical education and assessment, yet effectiveness on specialized exams remains underexplored. We aim to compare multiple AI models with national human averages on SBCOC entrance exams (2021-2023), evaluate answer accuracy and citation reliability, and contrast models.

METHODS

Five models-ChatGPT-4 (standard and literature-trained), ChatGPT-o1-pro, Gemini, and Meta Llama 3.1-answered official SBCOC exams (50 items/year) individually using a unified prompt and identical images. Each response included one choice (A-D) and a cited source. Outcomes were accuracy and source reliability (peer-reviewed articles/textbooks versus websites). Statistics used chi-square tests and one-way ANOVA with Tukey post-hoc (p<0.05).

RESULTS

Across 150 questions, ChatGPT-o1-pro led (66%, 62%, 68%) and exceeded human national averages each year (p<0.05). Gemini (40%, 28%, 38%) and Meta Llama 3.1 (32%, 44%, 50%) underperformed relative to humans, while both ChatGPT-4 versions hovered near the ≥50% pass threshold without significant differences. Regarding sources, ChatGPT-o1-pro and Llama predominantly cited articles or books, ChatGPT-4 alternated between literature and websites, and Gemini did not specify references.

CONCLUSIONS

ChatGPT-o1-pro outperformed the national human average and mostly used credible sources; ChatGPT-4 matched humans, while Gemini and Meta Llama 3.1 lagged. Level of evidence IV; Case series.

Для цитирования: Kenchian Caio Henrique, Ferreira Karolina Stephany Pereira, Sarmento Lucas Pereira, Teixeira Marcelo Silveira, Zan Renato Aroca, Tamaoki Marcel Jun Sugawara. USE OF ARTIFICIAL INTELLIGENCE FOR THE EVALUATION OF THE ENTRANCE EXAMINATION OF THE BRAZILIAN SOCIETY OF SHOULDER AND ELBOW SURGERY.. Acta ortopedica brasileira. 2026. DOI: 10.1590/1413-785220263404e300345.
Подготовка и перевод публикации
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…

История · было → стало

Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.

Тип публикации указан по данным источника. Уровень доказательности автоматически не присваивается.