Journal of dentistry · 2026
Large language models and retrieval-augmented platforms for dental implant complications: A blinded, expert-rated benchmarking study.
СтатьяСравнительное исследованиеНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Mayer Yaniv, Canullo Luigi, Kivovics Marton, Giladi Hadar Zigdon, Gabay Eran, Ginesin Ofir
Аннотация
OBJECTIVES: To compare expert-rated diagnostic accuracy, patient safety, completeness of management, and freedom from detectable hallucinations across eleven conversational artificial-intelligence (AI) platforms (general-purpose large language models [LLMs], a general-purpose retrieval-augmented [RAG] platform, and a medical-domain RAG platform) using synthetic vignettes of dental implant complications.
METHODS: Thirty vignettes (10 surgical, 10 biological, 10 mechanical/prosthetic) were submitted as independent prompts to each platform. Four blinded specialists (periodontists or oral and maxillofacial surgeons, >10 years' implant experience) scored each response on a 5-point Likert scale across four parameters. Friedman tests (Kendall's W effect size) assessed ordinal scores, and Cochran's Q tested two binary outcomes: safe (Safety ≥ 4) and hallucination-free (Hallucinations ≥ 4).
RESULTS: Across 330 responses (1320 ratings), OpenEvidence achieved 100% safe and 100% hallucination-free responses. Among general-purpose platforms, safe rates ranged from 56.7% (Claude 4.5 Haiku) to 86.7% (Perplexity) and hallucination-free rates from 36.7% (Claude 4.5 Haiku) to 93.3% (GPT-5.4 Thinking; Qwen 3.5-Plus), both significant (Cochran's Q, p < 0.01). Friedman tests were significant for every parameter (lowest p = 1.7 × 10⁻¹⁵; Kendall's W 0.126-0.293). OpenEvidence had the highest composite score (4.90 ± 0.18), with the largest effects in the mechanical/prosthetic subcategory (W = 0.548, large).
CONCLUSIONS: Within a vignette-based design, the medical-domain RAG platform (OpenEvidence) achieved the highest ratings across all domains, with Perplexity second; among general-purpose LLMs, GPT-5.4 Thinking, Claude Opus, and Mistral led. These findings reflect performance under tested conditions, not real-world efficacy. All platforms should serve as adjuncts to specialist judgement, and prospective validation is required before clinical deployment.
CLINICAL SIGNIFICANCE: Retrieval-augmented platforms grounded in peer-reviewed literature earned higher expert ratings than general-purpose LLMs, with the medical-domain platform OpenEvidence rated 100% safe and hallucination-free. Inter-model gaps were largest for mechanical/prosthetic cases; specialist verification remains essential before clinical use.
METHODS: Thirty vignettes (10 surgical, 10 biological, 10 mechanical/prosthetic) were submitted as independent prompts to each platform. Four blinded specialists (periodontists or oral and maxillofacial surgeons, >10 years' implant experience) scored each response on a 5-point Likert scale across four parameters. Friedman tests (Kendall's W effect size) assessed ordinal scores, and Cochran's Q tested two binary outcomes: safe (Safety ≥ 4) and hallucination-free (Hallucinations ≥ 4).
RESULTS: Across 330 responses (1320 ratings), OpenEvidence achieved 100% safe and 100% hallucination-free responses. Among general-purpose platforms, safe rates ranged from 56.7% (Claude 4.5 Haiku) to 86.7% (Perplexity) and hallucination-free rates from 36.7% (Claude 4.5 Haiku) to 93.3% (GPT-5.4 Thinking; Qwen 3.5-Plus), both significant (Cochran's Q, p < 0.01). Friedman tests were significant for every parameter (lowest p = 1.7 × 10⁻¹⁵; Kendall's W 0.126-0.293). OpenEvidence had the highest composite score (4.90 ± 0.18), with the largest effects in the mechanical/prosthetic subcategory (W = 0.548, large).
CONCLUSIONS: Within a vignette-based design, the medical-domain RAG platform (OpenEvidence) achieved the highest ratings across all domains, with Perplexity second; among general-purpose LLMs, GPT-5.4 Thinking, Claude Opus, and Mistral led. These findings reflect performance under tested conditions, not real-world efficacy. All platforms should serve as adjuncts to specialist judgement, and prospective validation is required before clinical deployment.
CLINICAL SIGNIFICANCE: Retrieval-augmented platforms grounded in peer-reviewed literature earned higher expert ratings than general-purpose LLMs, with the medical-domain platform OpenEvidence rated 100% safe and hallucination-free. Inter-model gaps were largest for mechanical/prosthetic cases; specialist verification remains essential before clinical use.