Defining evidential support for clinical effects in occupational therapy through Bayesian calibration of the paired samples t-test.

PURPOSE: Heuristic effect size benchmarks for Cohen's d, Hedges' g, and p-values lack statistical and clinical grounding, producing misleading interpretations of within-subject occupational therapy (OT) research outcomes. This study aimed to recalibrate d, g, and p-value thresholds for paired samples t-tests using Bayesian posterior probabilities representing evidential support for a clinical effect.

MATERIALS AND

METHODS

Using JASP's Bayesian Paired Samples t-test module, Bayes Factors (BF10) were iteratively derived across 67 sample sizes (N = 5-500) and converted to d/g thresholds corresponding to posterior probabilities of 50%, 75%, 90%, 95%, and 99%. Sensitivity analyses evaluated threshold stability across Cauchy prior scales. Two generalized additive models compared BF10 and p-values for predicting effect sizes using 100 published OT studies from the American Journal of Occupational Therapy.

RESULTS

Recalibrated thresholds corrected small- and large-sample bias. BF10 predicted effect sizes more accurately than p-values (R2 = 0.992 vs. 0.966; ΔAIC = 154.81). Thresholds were robust to prior specification. p-values near .05 consistently failed to achieve posterior probabilities of 75% or greater.

CONCLUSIONS

These recalibrated thresholds provide occupational therapists a principled, sample size-sensitive tool for interpreting within-subject outcomes probabilistically, offering a practical alternative to conventional heuristic benchmarks.