Acta neurochirurgica · 2025
Specialized AI and neurosurgeons in niche expertise: a proof-of-concept in neuromodulation with vagus nerve stimulation.
СтатьяНазвание и аннотация: Перевод в очереди
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Barrit Sami, Ranuzzi Giovanni, Fetzer Steffen, Al Barajraji Mejdeddine, Hadwe Salim El, Zanello Marc, Ortler Martin, O'Flaherty Julieta, Massager Nicolas, Madsen Joseph R, Dibué Maxine, Carron Romain
Аннотация
OBJECTIVE: Applying large language models (LLM) in specialized medical disciplines presents unique challenges requiring precision, reliability, and domain-specific relevance. We evaluated a specialized LLM-driven system against neurosurgeons in vagus nerve stimulation (VNS) for drug-resistant epilepsy knowledge assessment-a complex neuromodulation therapy requiring transdisciplinary expertise in neural anatomy, epileptic disorders, and device technology.
MATERIALS AND METHODS: Thirty-six European neurosurgeons who completed a 2-day VNS masterclass were assessed using a multiple-choice questionnaire comprising 14 items with 67 binary propositions. We deployed open-source models-LLaMa 2 70B and MXBAI embedding model-using Neura, an AI infrastructure enabling transparent grounding through advanced retrieval augmented generation. The knowledge base consisted of 125 VNS-related publications curated by multidisciplinary faculty. Scoring ranged from -1 to + 1 per question. Performance was analyzed using Wilcoxon signed-rank tests, confusion matrices, and metrics including accuracy, precision, recall, and specificity.
RESULTS: The AI achieved a score of 0.75, exceeding the highest individual clinician score (0.68; median: 0.50), with statistical significance (p < 0.001). AI performed better in questions involving anatomical and technical information, while clinicians excelled in scenarios requiring practical judgment. Confusion matrices revealed higher true correct and true incorrect rates for AI, demonstrating perfect precision and specificity scores with no hallucinations detected.
CONCLUSIONS: Specialized LLM performance in this VNS knowledge assessment, coupled with its verifiability, points to promising applications across neurosurgical subspecialties for clinical decision support and education. The complementary strengths observed suggest that valuable implementations will emerge from synergistic approaches combining human experiential knowledge with AI's information processing capabilities across the broader field of neurosurgery.
MATERIALS AND METHODS: Thirty-six European neurosurgeons who completed a 2-day VNS masterclass were assessed using a multiple-choice questionnaire comprising 14 items with 67 binary propositions. We deployed open-source models-LLaMa 2 70B and MXBAI embedding model-using Neura, an AI infrastructure enabling transparent grounding through advanced retrieval augmented generation. The knowledge base consisted of 125 VNS-related publications curated by multidisciplinary faculty. Scoring ranged from -1 to + 1 per question. Performance was analyzed using Wilcoxon signed-rank tests, confusion matrices, and metrics including accuracy, precision, recall, and specificity.
RESULTS: The AI achieved a score of 0.75, exceeding the highest individual clinician score (0.68; median: 0.50), with statistical significance (p < 0.001). AI performed better in questions involving anatomical and technical information, while clinicians excelled in scenarios requiring practical judgment. Confusion matrices revealed higher true correct and true incorrect rates for AI, demonstrating perfect precision and specificity scores with no hallucinations detected.
CONCLUSIONS: Specialized LLM performance in this VNS knowledge assessment, coupled with its verifiability, points to promising applications across neurosurgical subspecialties for clinical decision support and education. The complementary strengths observed suggest that valuable implementations will emerge from synergistic approaches combining human experiential knowledge with AI's information processing capabilities across the broader field of neurosurgery.