External evaluation of a commercial artificial intelligence-augmented digital auscultation platform in valvular heart disease detection using echocardiography as reference standard.

OBJECTIVE

There are few studies evaluating the accuracy of commercially available AI-powered digital auscultation platforms in detecting valvular heart disease (VHD). Therefore, the utility of these systems for diagnosing clinically significant VHD remains unclear. We conducted a comprehensive external evaluation of the Eko murmur analysis software (EMAS) and report its accuracy in detecting murmurs associated with VHD using echocardiography (ECHO) as the reference standard.

METHODS

We analyzed phonocardiogram (PCG) and ECHO data from 1,029 individuals (461 females, mean (SD) age: 61 (29) years, BMI: 29 (9)) at a single academic medical center. PCGs were recorded using the EkoDUO and EkoCORE stethoscopes from the four standard auscultation positions immediately before transthoracic ECHO (TTE) testing. TTE diagnostics were used as reference to calculate the EMAS sensitivity and specificity in detecting murmurs associated with VHD. The 95% confidence intervals are reported.

RESULTS

Of the 4,081 PCGs, 79% were of sufficient quality for murmur analysis. The sensitivity and specificity of the EMAS in detecting VHD were 39.3% (95% CI: 37.2-41.3) and 82.3% (95% CI: 80.0-84.5), respectively. EMAS sensitivity in detecting murmurs associated with common VHD types was 62.5%, 75.0%, 88.9%, and 63.3% for moderate-severe and severe cases of mitral stenosis, aortic regurgitation, aortic stenosis, and mitral regurgitation, respectively.

CONCLUSION

The EMAS algorithm exhibits limited overall sensitivity in detecting VHD. The sensitivity of the algorithm varies across VHD types. These findings suggest that EMAS can be used for diagnosis of specific lesions, but not all VHD types, which limits its clinical applicability as a screening tool.