BMJ health & care informatics · 2026
Performance of large language models in complex coronary artery disease decision-making and potential improvement strategy in a real-world cohort.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Zhang Zhixiang, Chen Runze, Lin Shen, Zheng Zhe
Аннотация
OBJECTIVE: Previous studies have demonstrated that large language models (LLMs) can achieve performance comparable to multidisciplinary heart teams (HTs) in coronary revascularisation decision-making but limited by population bias. We aimed to assess LLMs’ performance in complex coronary artery disease (CAD) decision-making using a real-world cohort and to explore the potential improvement strategy.
METHODS: Patients with stable CAD and angiographically confirmed three-vessel or left main disease randomly selected from a consecutive angiography registry study were included in this cross-sectional study. ChatGPT-4.1 was used to provide five-category treatment decisions based on inputted patient information and a prompt most commonly used by real-world patients. The primary outcome was the agreement rate between HT and LLM recommendations. Three predefined prompting strategies were further evaluated for their potential to enhance agreement rates.
RESULTS: A total of 564 patients were included. The agreement rate between HT and LLM recommendations was 47.0% based on initial prompt. LLM performed poorly in further testing (sensitivity=0.0%) decision. LLM performed well in patients with Synergy between Percutaneous Coronary Intervention with Taxus and Cardiac Surgery score ≥33 (94.1%) but poorly in 23–32 (61.7%) and 0–22 (20.9%). After using prompting strategies, the agreement rate increased significantly (79.3%, p<0.001).
DISCUSSION: Simple patient-style queries cannot independently generate reliable decisions for real-world complex revascularisation scenarios. While prompt engineering and subgroup analysis can partially improve LLM’s performance, the potential safety risk remains emphasised.
CONCLUSION: Among a real-world cohort of patients with complex CAD, the agreement rate of decision-making between LLM and HTs was only half, but could be improved with appropriate prompts.
METHODS: Patients with stable CAD and angiographically confirmed three-vessel or left main disease randomly selected from a consecutive angiography registry study were included in this cross-sectional study. ChatGPT-4.1 was used to provide five-category treatment decisions based on inputted patient information and a prompt most commonly used by real-world patients. The primary outcome was the agreement rate between HT and LLM recommendations. Three predefined prompting strategies were further evaluated for their potential to enhance agreement rates.
RESULTS: A total of 564 patients were included. The agreement rate between HT and LLM recommendations was 47.0% based on initial prompt. LLM performed poorly in further testing (sensitivity=0.0%) decision. LLM performed well in patients with Synergy between Percutaneous Coronary Intervention with Taxus and Cardiac Surgery score ≥33 (94.1%) but poorly in 23–32 (61.7%) and 0–22 (20.9%). After using prompting strategies, the agreement rate increased significantly (79.3%, p<0.001).
DISCUSSION: Simple patient-style queries cannot independently generate reliable decisions for real-world complex revascularisation scenarios. While prompt engineering and subgroup analysis can partially improve LLM’s performance, the potential safety risk remains emphasised.
CONCLUSION: Among a real-world cohort of patients with complex CAD, the agreement rate of decision-making between LLM and HTs was only half, but could be improved with appropriate prompts.