BMC oral health · 2026
Evaluating open-source LLMs for dental EMR generation.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Wang Hao, Du Wen, Yang Bo, Liu Mengyu, Xu Chunwei, Zhang Wei, Xu Chenfan, He Leyang, Zhang Wenbo, Yu Yao, Lin Jianan, Peng Xin
Аннотация
BACKGROUND: Electronic medical records impose significant documentation burdens. While proprietary Large Language Models (LLMs) offer automation, they raise data sovereignty and privacy concerns. This study evaluated the feasibility of using locally deployable open-source LLMs (specifically DeepSeek-V3) versus proprietary models for dental EMR generation.
METHODS: A dataset of 190 de-identified dental EMRs was converted into synthetic Mandarin doctor-patient dialogues, serving as a high-context linguistic stress test. Six LLMs-two open-source (DeepSeek-V3, LLaMA-3.1-405B) and four proprietary (GPT-4, GPT-4o, Claude-3.7-Sonnet, Grok-3)-were evaluated using the structured "CRISPE Framework" and a baseline "Basic Prompt". Assessment utilized a dual-track framework: automated metrics (n = 190) and blinded expert clinical evaluation using a 7-Dimensional Index (7DI) on a stratified subset (n = 50).
RESULTS: The CRISPE strategy significantly outperformed the basic prompt across all models (Mean 7DI: 28.86 ± 2.12 vs. 26.91 ± 2.36; p<.001). Under optimized prompting, DeepSeek-V3 achieved a mean 7DI score of 28.76 ± 2.04, and we did not detect statistically significant differences versus GPT-4o (29.22 ± 2.25; p>.05) in this expert-rated sample. One-way ANOVA similarly did not detect overall between-model differences (F = 0.54, p=.744); however, non-significance should not be interpreted as equivalence, and larger prospective evaluations are warranted.
CONCLUSION: Structured prompt engineering is critical for enhancing AI-generated documentation. Open-source models demonstrated clinical performance comparable to proprietary leaders when optimally prompted, though larger validation studies are needed. These preliminary findings suggest the feasibility of privacy-preserving local deployment strategies, offering a potential pathway to democratize AI support in dental institutions without compromising data sovereignty.
METHODS: A dataset of 190 de-identified dental EMRs was converted into synthetic Mandarin doctor-patient dialogues, serving as a high-context linguistic stress test. Six LLMs-two open-source (DeepSeek-V3, LLaMA-3.1-405B) and four proprietary (GPT-4, GPT-4o, Claude-3.7-Sonnet, Grok-3)-were evaluated using the structured "CRISPE Framework" and a baseline "Basic Prompt". Assessment utilized a dual-track framework: automated metrics (n = 190) and blinded expert clinical evaluation using a 7-Dimensional Index (7DI) on a stratified subset (n = 50).
RESULTS: The CRISPE strategy significantly outperformed the basic prompt across all models (Mean 7DI: 28.86 ± 2.12 vs. 26.91 ± 2.36; p<.001). Under optimized prompting, DeepSeek-V3 achieved a mean 7DI score of 28.76 ± 2.04, and we did not detect statistically significant differences versus GPT-4o (29.22 ± 2.25; p>.05) in this expert-rated sample. One-way ANOVA similarly did not detect overall between-model differences (F = 0.54, p=.744); however, non-significance should not be interpreted as equivalence, and larger prospective evaluations are warranted.
CONCLUSION: Structured prompt engineering is critical for enhancing AI-generated documentation. Open-source models demonstrated clinical performance comparable to proprietary leaders when optimally prompted, though larger validation studies are needed. These preliminary findings suggest the feasibility of privacy-preserving local deployment strategies, offering a potential pathway to democratize AI support in dental institutions without compromising data sovereignty.