Journal of biomedical informatics · 2026
TriFuse: A multi-signal three-stage data distillation framework for efficient medical model training.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Xu Ruilong, Fang Huan
Аннотация
OBJECTIVE: Large-scale medical instruction datasets often contain substantial noise and redundancy, leading to high training costs without significant performance gains. Existing data selection and distillation methods rely on single static criteria, failing to capture the diversity and dynamic nature of medical samples during learning. This study proposes TriFuse, a three-stage data distillation framework that integrates multiple complementary learning signals for efficient training of medical question-answering models.
METHODS: The framework comprises three stages: (1) Proxy-student filtering: utilizing masked language modeling loss to remove information-deficient samples; (2) Gradient-median filtering: under the guidance of a weak teacher model, retaining samples with stable and moderate gradient contributions to suppress noise and outliers; (3) Forgettable-count filtering: tracking sample stability across training epochs to identify and remove samples exhibiting unstable learning behaviors. The framework was applied to the large-scale medical instruction dataset MedS-Ins to construct a refined subset (MedCore) comprising less than 30% of the original data.
RESULTS: Models trained on MedCore (28% of the original data) achieved comparable or superior performance across multiple tasks while substantially reducing training time and computational cost compared with full-data training. MedCore-Llama outperformed GPT-4 on French medical QA (65.6%) and HeadQA (63.2%), and achieved state-of-the-art results in Participant Extraction (82.61%) and medical concept explanation. Ablation studies confirmed the necessity of all three filtering stages.
CONCLUSION: TriFuse effectively distills high-value medical training data through multi-signal progressive filtering, significantly reducing computational overhead while maintaining model performance. This framework provides a scalable, low-cost solution for efficient medical language model training in resource-constrained environments.
METHODS: The framework comprises three stages: (1) Proxy-student filtering: utilizing masked language modeling loss to remove information-deficient samples; (2) Gradient-median filtering: under the guidance of a weak teacher model, retaining samples with stable and moderate gradient contributions to suppress noise and outliers; (3) Forgettable-count filtering: tracking sample stability across training epochs to identify and remove samples exhibiting unstable learning behaviors. The framework was applied to the large-scale medical instruction dataset MedS-Ins to construct a refined subset (MedCore) comprising less than 30% of the original data.
RESULTS: Models trained on MedCore (28% of the original data) achieved comparable or superior performance across multiple tasks while substantially reducing training time and computational cost compared with full-data training. MedCore-Llama outperformed GPT-4 on French medical QA (65.6%) and HeadQA (63.2%), and achieved state-of-the-art results in Participant Extraction (82.61%) and medical concept explanation. Ablation studies confirmed the necessity of all three filtering stages.
CONCLUSION: TriFuse effectively distills high-value medical training data through multi-signal progressive filtering, significantly reducing computational overhead while maintaining model performance. This framework provides a scalable, low-cost solution for efficient medical language model training in resource-constrained environments.