Academic radiology · 2026
A Hybrid Human-AI Workflow for Ultrasound Report Quality Control: A Multicenter Performance and Efficiency Evaluation.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Liang Xuan, He Hui, Liu Chunzheng, Sang Rui, Zhu Jialin, Wang Xiaoqing, Hou Wenjing, Li Deyi, Wang Junguang, Li Yuanyuan, Su Jiayu, Wang Xiaoyi, Wei Xi
Аннотация
RATIONALE AND OBJECTIVES: This study evaluated a hybrid human-AI workflow for ultrasound report quality assurance using multicenter retrospective data.
MATERIALS AND METHODS: 786 anonymized ultrasound reports covering 22 examination categories were collected from four hospitals. Reports containing naturally occurring or implanted errors were matched across arms. GPT-4o and DeepSeek-v3 performed initial screening before review by six human reviewers (HRs). HR with blinded expert over-reading (HR-ex) served as control. Accuracy, false-positive (FP) and false-negative (FN) rates, and review time were compared. Supplementary validation compared HR and DeepSeek-assisted review using 1680 consecutive reports without implanted errors.
RESULTS: Hybrid workflow (HYB) accuracy was comparable to HR (4o-HYB 80.2%, v3-HYB 82.5%, HR 82.3%) but lower than HR-ex (87.3%). Both HYBs showed higher FN rates than HR-ex (4o-HYB 34.2% and v3-HYB 27.3% vs HR-ex 16.9%, both p < 0.001). Severity assessment did not indicate preferential omission of clinically significant errors. Human verification reduced LLM-generated FNs and FPs. Median review time decreased substantially from 51.0 s (HR-ex) to 11.0 s (4o-HYB) and 12.0 s (v3-HYB). In supplementary validation (error prevalence, 9.6%), v3.2-HYB and HR showed FN rates of 22.6% and 24.3%, FP rates of 7.7% and 10.0%, and median review times of 15.4 and 24.3 s, respectively.
CONCLUSION: The LLM-assisted HYB achieved accuracy comparable to single-physician review while substantially reducing audit time. These findings require prospective real-world validation before routine clinical implementation.
MATERIALS AND METHODS: 786 anonymized ultrasound reports covering 22 examination categories were collected from four hospitals. Reports containing naturally occurring or implanted errors were matched across arms. GPT-4o and DeepSeek-v3 performed initial screening before review by six human reviewers (HRs). HR with blinded expert over-reading (HR-ex) served as control. Accuracy, false-positive (FP) and false-negative (FN) rates, and review time were compared. Supplementary validation compared HR and DeepSeek-assisted review using 1680 consecutive reports without implanted errors.
RESULTS: Hybrid workflow (HYB) accuracy was comparable to HR (4o-HYB 80.2%, v3-HYB 82.5%, HR 82.3%) but lower than HR-ex (87.3%). Both HYBs showed higher FN rates than HR-ex (4o-HYB 34.2% and v3-HYB 27.3% vs HR-ex 16.9%, both p < 0.001). Severity assessment did not indicate preferential omission of clinically significant errors. Human verification reduced LLM-generated FNs and FPs. Median review time decreased substantially from 51.0 s (HR-ex) to 11.0 s (4o-HYB) and 12.0 s (v3-HYB). In supplementary validation (error prevalence, 9.6%), v3.2-HYB and HR showed FN rates of 22.6% and 24.3%, FP rates of 7.7% and 10.0%, and median review times of 15.4 and 24.3 s, respectively.
CONCLUSION: The LLM-assisted HYB achieved accuracy comparable to single-physician review while substantially reducing audit time. These findings require prospective real-world validation before routine clinical implementation.