Journal of biomedical informatics · 2026
Selective classification under imbalance in multiclass settings: A novel metric for bias-aware risk-coverage evaluation.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Sağlam Fatih, Özgen Ünsal, Uygun Alper, Dinçer Oğuz Salih, Albayrak Canan
Аннотация
OBJECTIVE: Selective classification improves reliability by allowing models to abstain on uncertain inputs, which is critical in safety-sensitive domains such as healthcare. However, commonly used evaluation metrics obscure fairness issues under class imbalance, leading to disproportionately high rejection of minority classes and misleadingly favorable assessments of selective strategies.
METHODS: We introduce two imbalance-aware evaluation metrics, Class-Averaged AURC (CA-AURC) and Class-Averaged AUGRC (CA-AUGRC), which integrate risk against class-specific coverage and then average across classes, alongside the Area under the IAM-coverage curve (AUIC) as a complementary imbalance-aware performance metric. In addition, we propose a class-conditional coverage-matching selection strategy that enforces balanced rejection across diagnostic categories. The proposed framework is evaluated on a clinical complete blood count dataset comprising 3316 patient records, 11 laboratory features, and 9 diagnostic classes, and validated on three additional publicly available benchmark datasets covering different domains, class structures, and imbalance levels.
RESULTS: While conventional metrics such as AURC and AUGRC favor classical selective strategies, the proposed class-averaged metrics reveal substantial disparities in rejection behavior under class imbalance. Using CA-AUGRC and AUIC, the class-conditional strategy consistently outperforms both the classical approach and LABEL, a theoretically principled set-valued baseline, across all four datasets and five uncertainty measures. Rank-based statistical comparisons confirm significant advantages of the class-conditional strategy on CA-AUGRC (p=0.009, r=0.638) and AUIC (p<0.001, r=0.957).
CONCLUSION: The results demonstrate that both evaluation and selection in selective classification must be class-aware to ensure fairness and clinical usefulness. Class-averaged metrics and class-conditional selection provide a more reliable basis for assessing selective classifiers in imbalanced medical data, with consistent generalizability across diverse datasets and uncertainty measures.
METHODS: We introduce two imbalance-aware evaluation metrics, Class-Averaged AURC (CA-AURC) and Class-Averaged AUGRC (CA-AUGRC), which integrate risk against class-specific coverage and then average across classes, alongside the Area under the IAM-coverage curve (AUIC) as a complementary imbalance-aware performance metric. In addition, we propose a class-conditional coverage-matching selection strategy that enforces balanced rejection across diagnostic categories. The proposed framework is evaluated on a clinical complete blood count dataset comprising 3316 patient records, 11 laboratory features, and 9 diagnostic classes, and validated on three additional publicly available benchmark datasets covering different domains, class structures, and imbalance levels.
RESULTS: While conventional metrics such as AURC and AUGRC favor classical selective strategies, the proposed class-averaged metrics reveal substantial disparities in rejection behavior under class imbalance. Using CA-AUGRC and AUIC, the class-conditional strategy consistently outperforms both the classical approach and LABEL, a theoretically principled set-valued baseline, across all four datasets and five uncertainty measures. Rank-based statistical comparisons confirm significant advantages of the class-conditional strategy on CA-AUGRC (p=0.009, r=0.638) and AUIC (p<0.001, r=0.957).
CONCLUSION: The results demonstrate that both evaluation and selection in selective classification must be class-aware to ensure fairness and clinical usefulness. Class-averaged metrics and class-conditional selection provide a more reliable basis for assessing selective classifiers in imbalanced medical data, with consistent generalizability across diverse datasets and uncertainty measures.