An Artificial Intelligence Application for Standardized Expert-Level Interpretation of Vascular Clinical Practice Guidelines.

BACKGROUND

Clinical practice guidelines (CPGs) are often complex and subject to the reader's interpretation. The aim was to develop and validate an artificial intelligence (AI)-driven application for standardized interpretation of CPGs. The application was named "VascLink-AI," reflecting the clinical focus and traceable nature of the tool.

METHODS

A comparative study to benchmark AI performance against established clinical standards. The 2017 European Society of Cardiology/European Society for Vascular Surgery Guidelines on the Diagnosis and Treatment of Peripheral Arterial Diseases were vectorised into a knowledge graph. A large language model was locked to this graph, returning structured, citation-anchored answers. Performance was tested on 41 committee-answered vignettes. The app's answers and the committee's answers were scored for accuracy, completeness, clarity, relevance, adaptability, and evidence justification by an automated evaluator (GPT-4o) and 2 independent human experts. Inter-rater reliability and consensus scores were compared. A noninferiority analysis was performed.

RESULTS

The application achieved high alignment with the expert intent, with composite scores favoring the app for both raters: automated 90.5% versus 77.8%, P < 0.001; human consensus 94.2% versus 86.2%, P < 0.001. Reliability analysis revealed an "agreement gap"; human raters' agreement on committee answers was negligible (Kappa = -0.018), but reached a moderate level on AI answers (Kappa = 0.450). The application was found to be noninferior to the experts across all 12 evaluated metrics (all P < 0.001).

CONCLUSION

This study demonstrates how AI can grow from a general research aid into a traceable, guideline-restricted framework capable of delivering advice that matches human expert interpretation of societal guidelines.