Evaluation of Perceptual Realism and Clinical Plausibility of AI-Generated Colon Polyp Images.

Background

Synthetic and pseudosynthetic images can be used to extend colonoscopy datasets, which, in turn, are used to train AI-detection models, yet their clinical acceptability depends on whether medical professionals can still recognize non-real content. Aim: To quantify the ability of practicing gastroenterologists to discriminate real, pseudosynthetic, and synthetic polyp images and to determine how training level and synthesis method impact detection. Materials and

Methods

A total of 32 Romanian gastroenterologists (18 residents and 14 seniors) reviewed 24 images (8 real, 8 augmented, 4 CycleGAN, and 4 diffusion) via an online form. Classification accuracy, 95% confidence intervals (CI), class sensitivity and precision, 3 × 3 confusion matrices, and Fleiss' κ were calculated. Resident vs. senior differences were tested with Pearson χ2; CycleGAN versus diffusion detectability was analyzed with the Wilcoxon signed-rank test (α = 0.05).

Results

Overall accuracy was 61.2% (95% CI 57.7-64.6). Residents and seniors performed similarly (62.3% vs. 59.8%; χ21 = 0.38, p = 0.54). Sensitivity/precision were 70.7%/62.2% for real, 51.6%/58.9% for augmented, and 61.3%/62.1% for synthetic images. Collapsing to "real vs. non-real" yielded 70.7% sensitivity and 78.5% specificity for real images. CycleGAN images were always recognized as synthetic (128/128; 97.1-100% CI), whereas diffusion images were correctly classified only 22.7% of the time (16.3-30.6%; Wilcoxon p < 0.001). The training level did not impact detection performance (χ22 < 1.2, p > 0.5). Inter-rater agreement was fair (κ = 0.30, 95% CI 0.15-0.43).

Conclusions

Clinicians detect non-real colonoscopy images only slightly above chance, irrespective of experience. The diffusion synthesis method creates images that escape human scrutiny, suggesting the need for automated authenticity safeguards before synthetic datasets are applied in clinical or AI-validation contexts.