Hospital pediatrics · 2026
Evaluating OpenEvidence Across Pediatric Subspecialties: A Guideline-Anchored Expert Review.
СтатьяНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Hunter R Brandon, Hess Lauren M, Wells Douglas, Dinh Duy M, Abou Hatab Hani, Guzman-Karlsson Mikael C, Khatua Sutapa, Niles Denver, Nguyen Bradford, Qasim Shawki, Sammer Marla Bk, Singh Ila, Mehta Sanjiv, Tsai Andy, Venkatramani Rajkumar, Benjamin Jennifer
Аннотация
OBJECTIVE: To evaluate the quality, safety, and citation characteristics of OpenEvidence across pediatric subspecialties.
METHODS: Guideline-anchored clinical questions spanning high, moderate, and low evidence certainty across 6 pediatric subspecialties were submitted to OpenEvidence. Expert reviewer pairs independently scored responses using the validated Quality Analysis of Medical Artificial Intelligence (QAMAI) instrument (6-30 range, higher scores indicate better quality) and a modified Agency for Healthcare Research and Quality harm assessment (0-7 range, higher scores indicate more harm), followed by consensus reconciliation. Citation analysis examined overlap with guideline references, claim substantiation, and pediatric specificity. Response consistency was assessed across 5 repeated queries.
RESULTS: Seventeen questions were evaluated. The median QAMAI score was 27/30 (IQR, 24-29; "very good" quality). Potential harm was low (median 0/7, range 0-3), though 6 of 17 responses carried nonzero harm potential. OpenEvidence cited the primary guideline in 15 of 17 questions, yet only 17.6% of responses showed strong citation overlap (≥75% shared references) with the corresponding guidelines. OpenEvidence's citations reliably supported the claims they accompanied (88.2% with high substantiation), and 70.8% of reviewer assessments rated the evidence base as comparable in quality to that of the corresponding guidelines. In total, 35.3% of responses cited studies lacking pediatric specificity. Responses were clinically consistent across repeated queries (77.9% had no meaningful or only minor variation); no contradictory recommendations were observed.
CONCLUSIONS: OpenEvidence produced high-quality responses with low expert-rated potential for harm across multiple pediatric subspecialties. The pediatric specificity of cited evidence warrants attention, reflecting the broader challenge of applying artificial intelligence tools that are trained on adult-predominant literature to pediatric populations.
METHODS: Guideline-anchored clinical questions spanning high, moderate, and low evidence certainty across 6 pediatric subspecialties were submitted to OpenEvidence. Expert reviewer pairs independently scored responses using the validated Quality Analysis of Medical Artificial Intelligence (QAMAI) instrument (6-30 range, higher scores indicate better quality) and a modified Agency for Healthcare Research and Quality harm assessment (0-7 range, higher scores indicate more harm), followed by consensus reconciliation. Citation analysis examined overlap with guideline references, claim substantiation, and pediatric specificity. Response consistency was assessed across 5 repeated queries.
RESULTS: Seventeen questions were evaluated. The median QAMAI score was 27/30 (IQR, 24-29; "very good" quality). Potential harm was low (median 0/7, range 0-3), though 6 of 17 responses carried nonzero harm potential. OpenEvidence cited the primary guideline in 15 of 17 questions, yet only 17.6% of responses showed strong citation overlap (≥75% shared references) with the corresponding guidelines. OpenEvidence's citations reliably supported the claims they accompanied (88.2% with high substantiation), and 70.8% of reviewer assessments rated the evidence base as comparable in quality to that of the corresponding guidelines. In total, 35.3% of responses cited studies lacking pediatric specificity. Responses were clinically consistent across repeated queries (77.9% had no meaningful or only minor variation); no contradictory recommendations were observed.
CONCLUSIONS: OpenEvidence produced high-quality responses with low expert-rated potential for harm across multiple pediatric subspecialties. The pediatric specificity of cited evidence warrants attention, reflecting the broader challenge of applying artificial intelligence tools that are trained on adult-predominant literature to pediatric populations.