Clinical similarity in cost-comparison evaluations: a systematic review of current methods in NICE appraisals and the development of a framework for the formal assessment of clinical similarity.

OBJECTIVES

To review how statistically non-significant indirect treatment comparison (ITC) results are interpreted within National Institute for Health and Care Excellence (NICE) cost-comparison evaluations (CCEs) and develop a framework to support interpretations of these results from Bayesian network meta-analyses (NMAs).

DESIGN

A systematic review of CCEs between 2017 (first introduced) and April 2025. A framework (point-and-density plots) was developed to better interpret statistically non-significant NMA results for CCEs.

DATA SOURCES: CCEs were identified through NICE website searches, references of similar reviews and communications with NICE.

ELIGIBILITY CRITERIA: NICE technology appraisals (from 2017) that followed a CCE approach ab initio, had final guidance available and used non-statistically significant ITC results were included.

DATA EXTRACTION AND SYNTHESIS: A single reviewer performed screening and data extraction with validation by a second reviewer. Narrative syntheses were performed separately for company, External Assessment Group (EAG) and committee perspectives. Point-and-density plots combine elements of forest plots and density plots alongside reporting the probability that a treatment is non-inferior relative to a comparator. These were applied to a recent CCE (TA1019) for crovalimab for patients with paroxysmal nocturnal haemoglobinuria.

RESULTS

Among 41 CCEs, EAGs raised concerns about statistically non-significant ITC results while companies relied heavily on them. Only ∼32% of CCEs applied formal methods to explore ITC result uncertainty.For the example framework analysis, comparisons of crovalimab to eculizumab (mean difference (MD): 0.018; 95% CIs -0.22 to 0.25) and ravulizumab (MD: 0.079; 95% CIs -0.25 to 0.41) were statistically non-significant, with non-inferiority not demonstrated. However, point-and-density plots indicated a 95.9% and 86.3% probability of non-inferiority of crovalimab versus eculizumab and ravulizumab.

CONCLUSIONS

Interpretations of statistically non-significant ITC results are inconsistent within individual CCEs and across appraisals. Implementation of the presented recommendations and framework would improve the consistency and robustness of CCEs.

PROSPERO REGISTRATION NUMBER: CRD420251034143.