AI-assisted thematic synthesis of existing neurological core outcome sets: A descriptive reference framework (COS-Neuro).

BACKGROUND

Neurological disorders affect approximately 3 billion people globally, yet clinical trial success is often hindered by poorly selected outcome measures, impacting trial design, compliance, and interpretation. Over the past 25 years, Core Outcome Sets (COS) have emerged as standardized tools to enhance outcome selection, ensuring comparability across studies and reflecting the priorities of both researchers and patients. Despite the success of COS initiatives in other fields, their development in neurology remains limited, leaving many trialists without disease-specific guidance.

OBJECTIVES

This study aimed to conduct an AI-assisted thematic synthesis of outcome domains from existing COS, with the goal of identifying structural patterns common to these sets and generating descriptive reference framework to inform future COS development.

METHODS

COS-Neuro was developed using AI-assisted thematic framework analysis, complemented by expert review. A modified five-step thematic analysis was conducted without pre-determined codes: 1. Dataset Gathering - Data was collected from the COMET database, and COS domains for neurological disorders were coded. 2. Prompt Design & Testing - Large language models (LLMs), including ChatGPT 3.5, Google Gemini 1.5 Flash and Meta Llama-2-70b, were trialled, and prompts refined based on their outputs. 3. Thematic Analysis - LLMs categorised domains into core areas. 4. Human Refinement - Experts reviewed LLM-generated core areas and selected those most appropriate for further interpretation. 5. Clinical Validation - Experts validated the domains, core areas, and concepts. This approach integrated AI with expert oversight to develop an AI-descriptive thematic map of existing neurological COS.

RESULTS

Utilising LLMs, particularly ChatGPT, an AI-assisted conceptual framework synthesising existing neurological COS was developed based on the analysis of 112 published COS. Through adaptation of the OMERACT model, the final framework comprised four overarching concepts, 13 core areas, and 75 domains identified through expert consensus.

CONCLUSION

COS-Neuro provides a preliminary AI-assisted descriptive synthesis of existing neurological COS, organised using the OMERACT Filter 2.1 as a structural reference. This hypothesis-generating framework may serve as a foundational resource for future COS research and trial design, particularly in areas where no disease-specific COS exists. However, it requires prospective validation through disease-specific, multi-stakeholder consensus processes before clinical application. COS-Neuro also demonstrates the feasibility of AI-assisted thematic synthesis in this context and warrants evaluation in other specialties.