Automated extraction of colonoscopy quality indicators from free-text reports using a prompt-based large language model pipeline.

BACKGROUND AND AIMS: Artificial intelligence has increasingly enabled large-scale analysis of clinical documentation, offering new opportunities to improve quality monitoring in endoscopy. In colonoscopy, key quality indicators - such as cecal intubation, bowel preparation, and polyp detection rate - are routinely recorded in free-text reports, which limits automated extraction and large-scale auditing. We aimed to develop and validate a prompt-based large language model pipeline capable of automatically extracting core colonoscopy quality indicators from routine free-text reports.

METHODS

A total of 22,577 consecutive free-text colonoscopy reports (2009-2024) from a tertiary care center were used for model development. Performance was evaluated on an independent test set of 537 reports from 2025, manually annotated by gastrointestinal endoscopists. A prompt-based large language model pipeline was developed to identify maximum colonic extent, bowel preparation quality, and polyp detection; colonoscopy completeness was derived from maximum extent. Performance was assessed using accuracy, precision, recall, and F1-score. Quality indicators derived from model predictions were compared with reference annotations.

RESULTS

The model accurately classified colonoscopy completeness (accuracy 95%, F1-score 0.95) and polyp detection (accuracy 96%, F1-score 0.96). For bowel preparation quality, overall accuracy was 93%, with F1-scores of 0.99 (adequate), 0.80 (inadequate), and 0.88 (not reported). Multiclass polyp quantification (0, 1, 2, or ≥3 polyps) achieved weighted F1-score of 0.93.

CONCLUSIONS

A prompt-based large language model pipeline may facilitate accurate extraction of key quality indicators from free-text documentation, potentially supporting automated and scalable quality monitoring in endoscopy practice. However, performance was lower for inadequate bowel preparation and for some less frequent maximum colonic extent categories, indicating that these clinically relevant scenarios may require further refinement.