BMC research notes · 2026
Benchmarking knowledge distillation for lightweight pneumonia detection: a multi-seed calibration study on PneumoniaMNIST.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Neo Ryan Yi Sheng
Аннотация
OBJECTIVE: To benchmark a very small convolutional neural network trained with and without knowledge distillation for pneumonia detection on PneumoniaMNIST, with emphasis on discrimination, calibration, efficiency, and external generalization.
RESULTS: Using the predefined MedMNIST v2 splits and 5 random seeds, a ResNet-18 teacher achieved an area under the receiver operating characteristic curve of 0.9735 (95% confidence interval 0.9687-0.9782). A 60,642-parameter TinyCNN retained about 90% of teacher discrimination (0.8818, 0.8634-0.9001) while reducing parameters by 184-fold, multiply-accumulate operations by 10.9-fold, and batch-1 CPU inference time by 3.7-fold. Knowledge distillation did not provide consistent discrimination gains over vanilla training on this benchmark. The teacher was less well calibrated before temperature scaling, which improved its negative log-likelihood, Brier score, and expected calibration error. External zero-shot evaluation on Kermany and a balanced RSNA subset showed domain-shift degradation for all models, with partial recovery after brief fine-tuning but persistent teacher-student gaps.
RESULTS: Using the predefined MedMNIST v2 splits and 5 random seeds, a ResNet-18 teacher achieved an area under the receiver operating characteristic curve of 0.9735 (95% confidence interval 0.9687-0.9782). A 60,642-parameter TinyCNN retained about 90% of teacher discrimination (0.8818, 0.8634-0.9001) while reducing parameters by 184-fold, multiply-accumulate operations by 10.9-fold, and batch-1 CPU inference time by 3.7-fold. Knowledge distillation did not provide consistent discrimination gains over vanilla training on this benchmark. The teacher was less well calibrated before temperature scaling, which improved its negative log-likelihood, Brier score, and expected calibration error. External zero-shot evaluation on Kermany and a balanced RSNA subset showed domain-shift degradation for all models, with partial recovery after brief fine-tuning but persistent teacher-student gaps.