Epilepsia · 2026
An adaptive, variable-length test for certifying electroencephalographic spike-detection skill: A retrospective feasibility study.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Zhang Tianyu, Nascimento Fábio A, Karakis Ioannis, Keldsen Elijah W, Sun ChenXi, Jing Jin, Westover M Brandon
Аннотация
OBJECTIVE: Determining whether an individual reader has sufficient electroencephalographic (EEG) spike-detection skill is challenging due to limited expert agreement. We evaluated whether modeling skill in interpretable signal-detection terms-discrimination ability separate from response tendency-could power an adaptive, variable-length test that can reliably and efficiently assess reader competence.
METHODS: We analyzed a multirater dataset containing 20 521 EEG segments and 2574 rater sessions. Case signal was estimated from multirater labels, and each reader was modeled using signal-detection parameters: discrimination skill σ and response bias t . Among 368 calibration-eligible raters across five skill tiers, we used a stratified 50/50 train-validation split to select a certification threshold σ ⋆ in training and to apply it in validation. We replayed an adaptive test on each rater's historical responses, using sequential stopping to issue pass, fail, or retest decisions.
RESULTS: The training-derived threshold was σ ⋆ = .815 (95% bootstrap confidence interval = .663 - 1.018 , Youden J = .828 ). In validation, pass rates preserved the expected tier ordering: experts 100% ( 14 / 14 ), experienced 100% ( 4 / 4 ), novices 14% ( 1 / 7 ), residents 0% ( 0 / 11 ), and crowd raters 7% ( 11 / 150 ). In the 81-rater adaptive replay, clear failures stopped near 20 trials, and passing experts required a median of 95 trials. Split-half replay showed intraclass correlation coefficient 3 , 1 = .87 for σ ̂ at stopping; there were no hard pass-fail reversals among raters whose two halves both reached decisive verdicts.
SIGNIFICANCE: A signal-detection framework quantified spike-identification skill in EEG readers, and adaptive testing estimated this skill reliably with fewer than 100 short EEG segments. This approach could provide a practical, scalable method for assessing EEG reading competence.
METHODS: We analyzed a multirater dataset containing 20 521 EEG segments and 2574 rater sessions. Case signal was estimated from multirater labels, and each reader was modeled using signal-detection parameters: discrimination skill σ and response bias t . Among 368 calibration-eligible raters across five skill tiers, we used a stratified 50/50 train-validation split to select a certification threshold σ ⋆ in training and to apply it in validation. We replayed an adaptive test on each rater's historical responses, using sequential stopping to issue pass, fail, or retest decisions.
RESULTS: The training-derived threshold was σ ⋆ = .815 (95% bootstrap confidence interval = .663 - 1.018 , Youden J = .828 ). In validation, pass rates preserved the expected tier ordering: experts 100% ( 14 / 14 ), experienced 100% ( 4 / 4 ), novices 14% ( 1 / 7 ), residents 0% ( 0 / 11 ), and crowd raters 7% ( 11 / 150 ). In the 81-rater adaptive replay, clear failures stopped near 20 trials, and passing experts required a median of 95 trials. Split-half replay showed intraclass correlation coefficient 3 , 1 = .87 for σ ̂ at stopping; there were no hard pass-fail reversals among raters whose two halves both reached decisive verdicts.
SIGNIFICANCE: A signal-detection framework quantified spike-identification skill in EEG readers, and adaptive testing estimated this skill reliably with fewer than 100 short EEG segments. This approach could provide a practical, scalable method for assessing EEG reading competence.