Assessing Representativeness of Randomised Controlled Trials using Serious Adverse Events.

Context: The applicability of randomised controlled trials of pharmacological agents to older people with frailty/multimorbidity is often uncertain, due to concerns that trials are not representative. However, assessing trial representativeness is challenging and complex.

Objectives

We explore an approach assessing trial representativeness by comparing rates of trial Serious Adverse Events (SAEs: most of which reflect hospitalisations/deaths) to rates of hospitalisation/death in routine care (which, in a trial setting, would be SAEs be definition). Study design: Secondary analysis of trial and routine healthcare data. Dataset and population: 483 trials (n=636,267) from clinicaltrials.gov across 21 index conditions. A routine care comparison was identified from SAIL databank (n=2.3M). Instrument: SAIL data were used to derive the expected rate of hospitalisations/deaths by age, sex and index condition. Outcomes: We calculated the expected number of SAEs for each trial compared to the observed number of SAEs (observed/expected SAE ratio). We then re-calculated the observed/expected SAE ratio additionally accounting for comorbidity count in 125 trials for which we accessed individual participant data.

Results

For 12/21 index conditions the observed/expected SAE ratio was <1, indicating fewer SAEs in trials than expected given community rates of hospitalisations and deaths. A further 6/21 had point estimates <1 but the 95% CI included the null. The median observed/expected SAE ratio was 0.60 (95% CI 0.56-0.65; COPD) and the interquartile range was 0.44 (0.34-0.55; Parkinson's disease) to 0.88 (0.59-1.33; IBD). Higher comorbidity count was associated with SAEs/hospitalisations and deaths across index conditions. For most trials, the observed/expected ratio was attenuated but remained <1 after additionally accounting for comorbidity count.

Conclusion

Trial participants experience fewer SAEs than expected based on age/sex/condition hospitalisation and death rates in routine care, confirming the predicted lack of representativeness. This difference is only partially explained by differences in multimorbidity. Assessing observed/expected SAE may help assess applicability of trial findings to older populations in whom multimorbidity and frailty are common.