Annals of vascular surgery · 2026
The Illusion of Precision: Artificial Intelligence Cannot yet Reliably Predict Individual Outcomes in Infrapopliteal Chronic Limb-Threatening Ischemia.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Gómez Gutiérrez Oliver Antonio, Imbrognio Franco, De la Torre Oscar A, Herrera Vegas Diego, Fabiani Mario Alejandro
Аннотация
BACKGROUND: The integration of large language models (LLMs) into vascular surgery promises scalable clinical decision support, yet their prognostic reliability remains unproven. This study evaluates the accuracy, calibration, and stability of generative artificial intelligence (AI) in predicting outcomes for chronic limb-threatening ischemia (CLTI).
METHODS: In a retrospective single-center study, we evaluated 49 patients undergoing revascularization for CLTI. Two LLM architectures (GPT-5.5 and DeepSeek-V4 [DS]) were tested using "naive" (unstructured) and "structured" (tabular) zero-shot prompting strategies to predict major adverse events (MAEs) at 6-month, 1-year, and 5-year horizons. Performance was assessed using Brier scores for calibration, sensitivity for discrimination, and Wasserstein distance for stability.
RESULTS: A critical dissociation between calibration and discrimination was observed. DS naive achieved a superior Brier score of 0.068 at 6 months, suggesting high probabilistic accuracy. However, this mathematical precision masked a failure in clinical utility: all models exhibited a sensitivity of 0.00 at 6-month and 1-year horizons. The models adopted an "actuarial hedging" strategy, clustering predictions around the population mean to minimize mathematical error rather than identifying specific high-risk patients. While internal reproducibility (intraclass correlation coefficient [ICCs] >0.500) and distributional stability across prompts were surprisingly high, this consistency proved deceptive, based entirely on nondiscriminatory safety biases.
CONCLUSION: While LLMs demonstrate semantic competence in medical discourse, they currently lack the pragmatic utility required for high-stakes prognostication. The observed "hallucinated calibration" and deceptive stability creates a dangerous illusion of reliability. Until issues of discriminatory failure are resolved, LLMs should remain restricted to administrative rather than predictive roles in patient decision-making.
METHODS: In a retrospective single-center study, we evaluated 49 patients undergoing revascularization for CLTI. Two LLM architectures (GPT-5.5 and DeepSeek-V4 [DS]) were tested using "naive" (unstructured) and "structured" (tabular) zero-shot prompting strategies to predict major adverse events (MAEs) at 6-month, 1-year, and 5-year horizons. Performance was assessed using Brier scores for calibration, sensitivity for discrimination, and Wasserstein distance for stability.
RESULTS: A critical dissociation between calibration and discrimination was observed. DS naive achieved a superior Brier score of 0.068 at 6 months, suggesting high probabilistic accuracy. However, this mathematical precision masked a failure in clinical utility: all models exhibited a sensitivity of 0.00 at 6-month and 1-year horizons. The models adopted an "actuarial hedging" strategy, clustering predictions around the population mean to minimize mathematical error rather than identifying specific high-risk patients. While internal reproducibility (intraclass correlation coefficient [ICCs] >0.500) and distributional stability across prompts were surprisingly high, this consistency proved deceptive, based entirely on nondiscriminatory safety biases.
CONCLUSION: While LLMs demonstrate semantic competence in medical discourse, they currently lack the pragmatic utility required for high-stakes prognostication. The observed "hallucinated calibration" and deceptive stability creates a dangerous illusion of reliability. Until issues of discriminatory failure are resolved, LLMs should remain restricted to administrative rather than predictive roles in patient decision-making.
Предметные рубрики MeSH
HumansRetrospective StudiesReproducibility of ResultsPredictive Value of TestsTreatment OutcomeIschemiaTime FactorsDecision Support TechniquesLarge Language ModelsPeripheral Arterial DiseaseGenerative Artificial IntelligenceRisk AssessmentChronic DiseaseRisk FactorsFemaleAgedMaleVascular Surgical ProceduresClinical Decision-MakingAged, 80 and overMiddle AgedPredictive Learning Models