Variation in bleeding risk estimates among online calculators: Cross-sectional study of apps used by and for patients with atrial fibrillation.

OBJECTIVE

To assess the variation in bleeding risk estimates and risk stratification among Web and mobile applications for patients with atrial fibrillation.

DESIGN

Cross-sectional study.

SETTING

Simulated patient population.

PARTICIPANTS

Hypothetical patient cohorts that encompassed all possible binary risk factor combinations for each clinical prediction model.

INTERVENTIONS: Twenty-five bleeding risk calculators (18 Web and 7 mobile apps), each of which used 1 of 4 clinical prediction models to predict an individual's 12-month bleed risk: ATRIA (Anticoagulation and Risk Factors in Atrial Fibrillation), HAS-BLED (hypertension [systolic blood pressure >160 mm Hg], abnormal renal or liver function, stroke [caused by bleeding], bleeding, labile international normalized ratio, elderly [age >65 years], drugs [acetylsalicylic acid or nonsteroidal anti-inflammatory drugs] or alcohol [≥8 drinks per week]), HEMORR2HAGES (hepatic or renal disease, ethanol abuse, malignancy, older [age >75 years], reduced platelet count or function, rebleeding risk [history of past bleeding], hypertension [uncontrolled], anemia, genetic factors, excessive fall risk, and stroke), and mOBRI (modified Outpatient Bleeding Risk Index).

MAIN OUTCOME MEASURES: Four simulated cohorts were constructed. The coefficient of variation, relative difference (RD), and 95% CI for annual bleeding risk estimates were calculated for all hypothetical patient cohorts. Additionally, pairwise agreement between calculators across low- (<10%), moderate- (10% to 20%), and high-risk (>20%) categories of patients was determined.

RESULTS

The risk estimates the calculators generated were imprecise, with coefficients of variation ranging from 14% for HEMORR2HAGES to 64% for mOBRI. Wide variation was observed in annual risk estimates for calculators using the mOBRI (maximum RD=4.3) and HAS-BLED (maximum RD=3.1) models. The 95% CI of mean annual bleeding risk varied among models; 1 calculator using the HAS-BLED model had a 95% CI of mean annual risk estimates of 5.4% to 6.2%, while another HAS-BLED calculator reported a 95% CI of 17.7% to 18.5%. Concordance for risk category stratification among calculators was high for those based on mOBRI and ATRIA (=1 for both). Poor agreement was observed in 1 calculator using HEMORR2HAGES (=0.54) and another using HAS-BLED ( range=-0.11 to 0.35).

CONCLUSION

Inconsistencies and a lack of precision were observed in annual risk estimates and risk stratification produced by Web and mobile bleeding risk calculators for patients with atrial fibrillation. Clinicians should refer to annual bleeding risks observed in major randomized controlled trials to inform risk estimates communicated to patients.