Clinical imaging · 2025
Limited performance of ChatGPT-4v and ChatGPT-4o in image-based core radiology cases.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Noy Achiron Romi, Kagasov Shmuel, Neeman Rina, Peri Tamar, Fenton Chifra
Аннотация
OBJECTIVE: Large language models such as ChatGPT have shown potential in clinical reasoning and radiologic interpretation. Recent versions with image-analysing capabilities allow for combined visual and textual processing. This study aims to assess the performance and limitations of ChatGPT-4v and ChatGPT-4o in interpreting image-based multiple-choice questions from official national radiology board examinations, which are designed to reflect core radiologic scenarios.
METHODS: This prospective study used 222 image-based multiple-choice official questions from a national radiology board examinations administered between 2020 and 2024. Questions were entered into ChatGPT-4v and ChatGPT-4o; Generated answers were compared to the official answer key. Accuracy was further analysed by radiologic subspecialty and the presence or absence of clinical information.
RESULTS: ChatGPT-4o achieved a 59 % (130/222) success rate, while ChatGPT-4v achieved a 54 % (119/222), with both models underperforming relative to the board exam passing standard. No significant difference was found between the two versions (two-tailed P-value = 0.339). Analysis by subspecialty revealed that ChatGPT-4v had a similar success rate across all fields (p = 0.330), whereas the success rate of ChatGPT-4o varied significantly (p = 0.0009). Both models achieved significantly higher success rates on questions that included clinical information. ChatGPT-4v: 63.8 % (60/94) vs. 46.1 % (59/128), p = 0.0099; ChatGPT-4o: 67.0 % (63/94) vs. 52.3 % (67/128), p = 0.0384.
CONCLUSION: ChatGPT shows potential as a supportive diagnostic tool, but its accuracy remains below the standard required for board-level image interpretation. The variability across subspecialties highlights current limitations and underscores the need for further research before safe clinical integration.
METHODS: This prospective study used 222 image-based multiple-choice official questions from a national radiology board examinations administered between 2020 and 2024. Questions were entered into ChatGPT-4v and ChatGPT-4o; Generated answers were compared to the official answer key. Accuracy was further analysed by radiologic subspecialty and the presence or absence of clinical information.
RESULTS: ChatGPT-4o achieved a 59 % (130/222) success rate, while ChatGPT-4v achieved a 54 % (119/222), with both models underperforming relative to the board exam passing standard. No significant difference was found between the two versions (two-tailed P-value = 0.339). Analysis by subspecialty revealed that ChatGPT-4v had a similar success rate across all fields (p = 0.330), whereas the success rate of ChatGPT-4o varied significantly (p = 0.0009). Both models achieved significantly higher success rates on questions that included clinical information. ChatGPT-4v: 63.8 % (60/94) vs. 46.1 % (59/128), p = 0.0099; ChatGPT-4o: 67.0 % (63/94) vs. 52.3 % (67/128), p = 0.0384.
CONCLUSION: ChatGPT shows potential as a supportive diagnostic tool, but its accuracy remains below the standard required for board-level image interpretation. The variability across subspecialties highlights current limitations and underscores the need for further research before safe clinical integration.