Insights into imaging · 2026
Large language model assistance for report-based carotid plaque AHA classification: a multi-reader study.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Wang Mengfei, Tang Yixiao, Yang Yulu, Tu Yifan, Cao Zhongzheng, Ling Runjianya, Sun Kaiyue, Ma Jian, Zhang Zhicheng, Zhang Ruipeng, Wei Xiaoer, Wang Dan, Li Yuehua
Аннотация
OBJECTIVES: To evaluate whether large language model assistance improves radiologists' agreement with a text-based reference standard in inferring the modified American Heart Association carotid plaque classification from free-text MRI reports, and whether a lightweight model can perform well under criteria-guided prompting.
MATERIALS AND METHODS: This retrospective study included 433 patients from three institutions. In a report-based reader study, four radiologists classified MRI reports with and without large language model assistance. In standalone evaluation, a rule-based baseline and 10 models were compared under naive and criteria-guided prompting. The reference standard was expert consensus based on the report text.
RESULTS: Among 433 patients (mean age, 65.9 years ± 13.4; 248 males), AI assistance improved six-class agreement with the text-based reference standard. At the vessel level, junior radiologists improved from 61.75% to 83.12% (Qwen3-8B) and 87.75% (DeepSeek-R1); senior radiologists from 77.00% to 87.50% and 92.25%, respectively (all Holm-corrected p < 0.001). Patient-level agreement increased from 41.0% to 78.8% (junior) and from 59.8% to 87.8% (senior) with DeepSeek-R1. Agreement increased (quadratic-weighted κ: 0.231 without AI vs 0.775 with DeepSeek-R1). In standalone evaluation, criteria-guided prompting improved Qwen3-8B agreement from 38.91% to 81.52% (95% CI: 78.87-84.06%), remaining 5.89 percentage points below DeepSeek-R1 (87.41%; 95% CI: 85.10-89.61%).
CONCLUSION: In this report-based classification setting, large language model assistance improved radiologists' agreement with the text-based reference standard, with larger gains in junior radiologists. When prompts explicitly encoded AHA criteria, an 8B local candidate model performed well, supporting its potential use as a standardization aid for text-based workflows.
KEY POINTS: Question Report-based modified AHA classification of carotid plaques is experience-dependent and difficult to standardize, as mapping multi-sequence imaging findings to classification conclusions imposes substantial cognitive load. Findings Large language model assistance improved agreement with a text-based reference standard and raised inter-reader consistency (weighted κ 0.231-0.775), narrowing the junior-senior gap. Critical relevance statement Embedding explicit AHA criteria in the prompt let an 8B model approach 671B-model agreement (38.91-81.52%), supporting standardized text-based workflows without fine-tuning.
MATERIALS AND METHODS: This retrospective study included 433 patients from three institutions. In a report-based reader study, four radiologists classified MRI reports with and without large language model assistance. In standalone evaluation, a rule-based baseline and 10 models were compared under naive and criteria-guided prompting. The reference standard was expert consensus based on the report text.
RESULTS: Among 433 patients (mean age, 65.9 years ± 13.4; 248 males), AI assistance improved six-class agreement with the text-based reference standard. At the vessel level, junior radiologists improved from 61.75% to 83.12% (Qwen3-8B) and 87.75% (DeepSeek-R1); senior radiologists from 77.00% to 87.50% and 92.25%, respectively (all Holm-corrected p < 0.001). Patient-level agreement increased from 41.0% to 78.8% (junior) and from 59.8% to 87.8% (senior) with DeepSeek-R1. Agreement increased (quadratic-weighted κ: 0.231 without AI vs 0.775 with DeepSeek-R1). In standalone evaluation, criteria-guided prompting improved Qwen3-8B agreement from 38.91% to 81.52% (95% CI: 78.87-84.06%), remaining 5.89 percentage points below DeepSeek-R1 (87.41%; 95% CI: 85.10-89.61%).
CONCLUSION: In this report-based classification setting, large language model assistance improved radiologists' agreement with the text-based reference standard, with larger gains in junior radiologists. When prompts explicitly encoded AHA criteria, an 8B local candidate model performed well, supporting its potential use as a standardization aid for text-based workflows.
KEY POINTS: Question Report-based modified AHA classification of carotid plaques is experience-dependent and difficult to standardize, as mapping multi-sequence imaging findings to classification conclusions imposes substantial cognitive load. Findings Large language model assistance improved agreement with a text-based reference standard and raised inter-reader consistency (weighted κ 0.231-0.775), narrowing the junior-senior gap. Critical relevance statement Embedding explicit AHA criteria in the prompt let an 8B model approach 671B-model agreement (38.91-81.52%), supporting standardized text-based workflows without fine-tuning.