Clinical validation of autoSCORE: A human-in-the-loop AI tool for interpretation of long-term EEG recordings.

OBJECTIVE

To validate autoSCORE, a convolutional neural network (SCORE-AI) implemented into a commercially available EEG reader, for analysis of long-term EEG (LT-EEG) recordings, and to evaluate it within the human-in-the-loop approach.

METHODS

autoSCORE automatically divides LT-EEG into 2-hour segments, labeling each as normal or as containing one or more of four abnormality categories: focal epileptiform, generalized epileptiform, focal non-epileptiform, or diffuse non-epileptiform. In abnormal segments, it also inserts markers defined by start/stop times and tagged by abnormality type. Within each category, markers can be ranked by decreasing probability. Eighty 2-hour LT-EEG segments (40 in-hospital, 40 ambulatory) from patients aged 0.9-82.3 years were analyzed by autoSCORE. The diagnostic reference standard was derived from independent conventional visual assessment by three human experts.

RESULTS

Accuracy computed across expert-expert and AI-expert pairings, evaluated at the level of the 2-hour segments, were comparable, except for detection of focal non-epilepiform abnormality, where AI outperformed experts for identification accuracy (p = 0.019). The positive predictive value of the automated abnormality markers was 0.446-0.823. In the human-in-the-loop workflow, inspecting 10 automated markers with the highest probability in each category enabled correct identification of all abnormal segments, corresponding to 1.5%-5.5% of the full 2-hour segment.

CONCLUSIONS

autoSCORE accurately screens LT-EEG recordings and may serve as an adjunct to human EEG interpretation to reduce the workload of human experts.

SIGNIFICANCE: autoSCORE is a validated tool that can increase the productivity and efficacy of human experts in assessment of LT-EEG to improve patient diagnosis and management strategies.