Journal of the American Medical Informatics Association : JAMIA · 2026
Small language models in clinical medicine: a systematic review of performance, safety, and deployment feasibility.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Gorenshtein Alon, Omar Mahmud, Barash Yiftach, Kruskal Jonathan B, Ahmed Muneeb, Brook Olga R, Illigens Ben, Nadkarni Girish N, Klang Eyal
Аннотация
OBJECTIVES: To review the clinical evidence for small language models (SLMs), 4 billion parameters or fewer, for performance, safety, and deployment feasibility.
MATERIALS AND METHODS: We searched 5 databases through February 10, 2026 for English-language reports of an SLM on a clinical task, following PRISMA 2020 under a PROSPERO-registered protocol. Two reviewers independently assessed eligibility (Cohen κ = 0.93) and rated methodological quality and transparency. We calculated a relative task score (RTS): the primary metric of each study's best-performing qualifying SLM, divided by a study-specific reference comparator selected by a uniform hierarchy, calculated by the review authors.
RESULTS: Eleven studies (7 peer-reviewed, 4 preprint or technical-report) were eligible. Across the 9 studies with a ratio-scale reference comparator, RTS ranged from 0.30 to 2.36; 1 was a raw difference on a non-ratio scale and 1 had no comparator. We did not pool estimates, given heterogeneity. Hallucination was evaluated in 6 of 11 studies, a scalar calibration-error metric or epistemic uncertainty in 0 of 11 (1 reported a calibration curve only), and on-hardware inference timing in 2 of 11. Our memory model estimated that a 4-billion-parameter model requires about 9.3 GB at a 2048-token context, indicating single-GPU memory feasibility rather than demonstrated deployment.
DISCUSSION: Domain-adapted SLMs are memory-feasible, but the evidence base is too small and heterogeneous for inferential comparison with larger models, and safety properties (including adversarial robustness) go unmeasured.
CONCLUSION: Routine reporting of calibration and uncertainty and measurement of end-to-end clinical performance are prerequisites for responsible SLM deployment.
PROSPERO REGISTRATION: CRD420261331444.
MATERIALS AND METHODS: We searched 5 databases through February 10, 2026 for English-language reports of an SLM on a clinical task, following PRISMA 2020 under a PROSPERO-registered protocol. Two reviewers independently assessed eligibility (Cohen κ = 0.93) and rated methodological quality and transparency. We calculated a relative task score (RTS): the primary metric of each study's best-performing qualifying SLM, divided by a study-specific reference comparator selected by a uniform hierarchy, calculated by the review authors.
RESULTS: Eleven studies (7 peer-reviewed, 4 preprint or technical-report) were eligible. Across the 9 studies with a ratio-scale reference comparator, RTS ranged from 0.30 to 2.36; 1 was a raw difference on a non-ratio scale and 1 had no comparator. We did not pool estimates, given heterogeneity. Hallucination was evaluated in 6 of 11 studies, a scalar calibration-error metric or epistemic uncertainty in 0 of 11 (1 reported a calibration curve only), and on-hardware inference timing in 2 of 11. Our memory model estimated that a 4-billion-parameter model requires about 9.3 GB at a 2048-token context, indicating single-GPU memory feasibility rather than demonstrated deployment.
DISCUSSION: Domain-adapted SLMs are memory-feasible, but the evidence base is too small and heterogeneous for inferential comparison with larger models, and safety properties (including adversarial robustness) go unmeasured.
CONCLUSION: Routine reporting of calibration and uncertainty and measurement of end-to-end clinical performance are prerequisites for responsible SLM deployment.
PROSPERO REGISTRATION: CRD420261331444.