ChatGPT's Limitations in Athlete ECG Interpretation: Evidence from a Multicenter Diagnostic Study.
Обработка статьи · этапы, ошибки и управление
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Palermi Stefano, Vecchiato Marco, Iacovone Tommaso Remo, Anselmino Matteo, Adorisio Rachele, Biffi Alessandro, Borrelli Francesco, Brugin Erica, Cantarutti Nicoletta, Cavarretta Elena, Cominacini Mattia, Corsi Marco, D'Ascenzi Flavio, De Feo Vittorio, Di Gioia Giuseppe, Dorelli Gianluigi, Foccardi Giulia, Gallina Sabina, Giangrandi Silvia, Graziano Francesca, Lodi Elisa, Livio Alberto, Maestrini Viviana, Manfredi Guglielmo Leonardo, Mansour Davide, Modena Maria Grazia, Neunhaeuserer Daniel, Nigro Antonia, Palermi Andrea, Pellegrino Alessio, Pelliccia Antonio, Quattrini Filippo Maria, Ricci Fabrizio, Scarzella Fiammetta, Squeo Maria Rosaria, Tonelli Riccardo, Zanardo Emanuele, Zorzi Alessandro, D'Ascenzo Fabrizio, De Ferrari Gaetano Maria, Saglietto Andrea
Аннотация
METHODS: In this multicentre diagnostic accuracy study, we evaluated a commercially available LLM (ChatGPT, version 5) in 2950 competitive athletes undergoing PPS. All athletes underwent resting 12-lead ECG, with second- and third-line investigations performed when clinically indicated. The reference outcome was confirmed cardiovascular disease after full diagnostic work-up (n = 450, 15.3%). For each ECG, the LLM generated a numeric score (0-100) representing the inferred likelihood of underlying disease using a standardized prompt and without task-specific fine-tuning. Discriminative performance was assessed using receiver operating characteristic (ROC) analysis. Misclassification patterns were analysed according to International ECG Criteria.
RESULTS: GPT-derived scores demonstrated a marked floor effect, with a median value of 0 (IQR 0-2) in both diseased and non-diseased athletes and substantial overlap between groups. The area under the ROC curve was 0.52 (95% CI 0.49-0.55), indicating performance close to random classification. At the Youden-derived threshold, 79% of athletes with confirmed disease were incorrectly classified as negative. False-negative cases were predominantly characterized by borderline ECG patterns (82%), and a substantial number of red-flag ECG abnormalities were also missed.
CONCLUSIONS: In this PPS cohort, a general-purpose LLM used in a naïve configuration showed no clinically meaningful ability to discriminate between cardiovascular disease and athlete ECGs. Without task-specific training or domain adaptation, such models should not be used for diagnostic triage in athlete screening.