Journal of the American College of Surgeons · 2026
Evaluation of Vision-Language Models for Surgical Narration and Critical View of Safety Assessment in Laparoscopic Cholecystectomy.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Glazer Rita, Seger Philip, Fox Brendan, Skinner Garrett, Li Jiajie, Tung Kaity, Chang Andrew, Walsh Jeremy, Handmacher Matthew, Ham Ben, Nordin Andrew, Shah Shinil, Yang Gene, Schwaitzberg Steven, Kim Peter Cw
Аннотация
BACKGROUND: Vision-language models (VLMs) may support surgical documentation through operative video narration, but fluent narration may not capture procedural details or safety-critical anatomy. We compared VLM-generated laparoscopic cholecystectomy narrations with surgeon narration and evaluated Critical View of Safety (CVoS) assessment.
STUDY DESIGN: Forty-eight 10-minute segments from 13 Cholec80 videos were evaluated using Gemini 2.5 Flash (Untrained VLM) and Qwen3-VL-CoA (Surgically Trained VLM) under Basic and procedure-specific prompting. VLM and surgeon narrations were compared for structure, semantic similarity, checklist-based procedural content, and CVoS recognition.
RESULTS: Attending narrations averaged 490.6 words versus 243.9-252.5 for VLMs. Specific prompting increased cosine similarity for both VLMs (0.507 to 0.593 and 0.543 to 0.584; both adjusted p<0.001), but checklist F1 improved only for the Untrained VLM (0.405 to 0.501; adjusted p=0.006), not the Surgically Trained VLM (0.337 to 0.380; adjusted p=0.233). Among 6 CVoS-positive and 42 CVoS-negative segments, Untrained Specific had sensitivity 0.33 (95% CI 0.10-0.70) and specificity 0.95 (95% CI 0.84-0.99); Surgically Trained Specific had sensitivity 0.83 (95% CI 0.44-0.97) and specificity 0.19 (95% CI 0.10-0.33).
CONCLUSIONS: Procedure-specific prompting increased similarity to attending narration without consistently improving procedural content. Divergent CVoS errors demonstrate that fluent surgical narration does not establish reliable safety assessment. VLM narration may support surgeon-reviewed documentation, but safety-critical judgments require separate task-specific validation.
STUDY DESIGN: Forty-eight 10-minute segments from 13 Cholec80 videos were evaluated using Gemini 2.5 Flash (Untrained VLM) and Qwen3-VL-CoA (Surgically Trained VLM) under Basic and procedure-specific prompting. VLM and surgeon narrations were compared for structure, semantic similarity, checklist-based procedural content, and CVoS recognition.
RESULTS: Attending narrations averaged 490.6 words versus 243.9-252.5 for VLMs. Specific prompting increased cosine similarity for both VLMs (0.507 to 0.593 and 0.543 to 0.584; both adjusted p<0.001), but checklist F1 improved only for the Untrained VLM (0.405 to 0.501; adjusted p=0.006), not the Surgically Trained VLM (0.337 to 0.380; adjusted p=0.233). Among 6 CVoS-positive and 42 CVoS-negative segments, Untrained Specific had sensitivity 0.33 (95% CI 0.10-0.70) and specificity 0.95 (95% CI 0.84-0.99); Surgically Trained Specific had sensitivity 0.83 (95% CI 0.44-0.97) and specificity 0.19 (95% CI 0.10-0.33).
CONCLUSIONS: Procedure-specific prompting increased similarity to attending narration without consistently improving procedural content. Divergent CVoS errors demonstrate that fluent surgical narration does not establish reliable safety assessment. VLM narration may support surgeon-reviewed documentation, but safety-critical judgments require separate task-specific validation.