Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research · 2026
Development and Validation of an Artificial Intelligence Tool for Automated Population, Intervention, Comparator, Outcome Scoping for the European Joint Clinical Assessment: A Proof of Concept.
СтатьяВалидационное исследованиеНазвание и аннотация: Перевод пока не подготовлен
Обработка статьи · этапы, ошибки и управление
Загрузка состояния…
Ошибки и ограничения
История · было → стало
Состояние обновляется каждые 5 секунд, пока блок открыт. Процент показывается только там, где известен объём работы. Пауза останавливает продолжение этапа; текущий запрос к источнику может завершиться.
Boverhof Bart-Jan, Takatzoglou Nikos, Redekop Ken, Groot Carin Uyl-de, Visser Jacob Jan, Mölken Maureen Rutten-van
Аннотация
OBJECTIVES: The European Union Health Technology Assessment regulation (EU-HTAR) requires laborious population, intervention, comparator, outcome (PICO) scoping and consolidation across member states during joint clinical assessments. We developed a tool to automate PICO extraction from health technology assessment (HTA) documents and clinical guidelines.
METHODS: We developed a proof-of-concept artificial intelligence (AI) pipeline using retrieval-augmented generation and large language models to process documents, translate to English, extract PICOs, and consolidate findings. The system was validated using 2 oncology cases from nonsmall cell lung cancer (NSCLC) and hepatocellular carcinoma (HCC). We curated data sets for both cases, comprising 49 reports totaling 5463 pages from 16 unique countries. Performance was assessed using recall (sensitivity) and precision (positive predictive value) and compared with human data extraction.
RESULTS: The AI system demonstrated strong recall performance, achieving 0.807 for NSCLC and 0.837 for HCC. The AI outperformed human extraction in the NSCLC case (0.81 vs 0.71 recall) while matching human performance in HCC (0.84 vs 0.81). Extraction of the comparator element showed high recall (0.911 NSCLC; 0.967 HCC). The AI pipeline required 96 minutes of computational time beyond translation, saving approximately 25 hours of human screening work.
CONCLUSIONS: We showed that AI-based PICO extraction was feasible and valuable in 2 use-cases. The system showed good recall and had the potential to reduce workload in the joint clinical assessment. It may be useful for the assessors and coassessors in the EU-HTAR process, the industry in gaining insight into PICO expectations, and national HTA bodies in understanding the expected PICO wishes of other nations.
METHODS: We developed a proof-of-concept artificial intelligence (AI) pipeline using retrieval-augmented generation and large language models to process documents, translate to English, extract PICOs, and consolidate findings. The system was validated using 2 oncology cases from nonsmall cell lung cancer (NSCLC) and hepatocellular carcinoma (HCC). We curated data sets for both cases, comprising 49 reports totaling 5463 pages from 16 unique countries. Performance was assessed using recall (sensitivity) and precision (positive predictive value) and compared with human data extraction.
RESULTS: The AI system demonstrated strong recall performance, achieving 0.807 for NSCLC and 0.837 for HCC. The AI outperformed human extraction in the NSCLC case (0.81 vs 0.71 recall) while matching human performance in HCC (0.84 vs 0.81). Extraction of the comparator element showed high recall (0.911 NSCLC; 0.967 HCC). The AI pipeline required 96 minutes of computational time beyond translation, saving approximately 25 hours of human screening work.
CONCLUSIONS: We showed that AI-based PICO extraction was feasible and valuable in 2 use-cases. The system showed good recall and had the potential to reduce workload in the joint clinical assessment. It may be useful for the assessors and coassessors in the EU-HTAR process, the industry in gaining insight into PICO expectations, and national HTA bodies in understanding the expected PICO wishes of other nations.