Sensitivity, specificity, reliability, and clinical validity of provider-reported symptoms: a comparison with self-reported symptoms. Outcomes Committee of the AIDS Clinical Trials Group.

BACKGROUND

If symptoms are to be recognized and effectively addressed in clinical research, they must be collected using sensitive, specific, reliable, and clinically meaningful methods.

OBJECTIVE

To perform a comparison of self-administered symptom survey data with data from conventional provider-reports.

DESIGN/

METHODS

Secondary data analysis of AIDS Clinical Trials Group Study 081 (ACTG 081), a randomized trial taking place in 33 sites comparing three approaches to prophylaxis for Pneumocystis carinii-related pneumonia that found no difference among treatment arms. The study was performed on 842 subjects with advanced HIV infection. No intervention was undertaken as a result of this study. ACTG 081 included data on functional status, global quality of life and survival, and two methods of symptom measurement: an open-ended, provider-reported symptom assessment (provider-report) and a self-administered symptom survey (self-report). Agreement was measured using kappa scores. Sensitivity and specificity were calculated using self-report as the standard. Reliability was measured by intersite variation and test-retest reliability (8 weeks later). Clinical validity was evaluated by testing expected associations with functional status, global quality of life, and survival.

RESULTS

Symptom data were available for 808 patients (96%). Patient and provider agreement was poor (mean kappa, 0.14; range, 0.07-0.25). Compared with self-report, providers underreported the presence and severity of symptoms (mean symptom count, 5.2 versus 1.3; mean severity score, 1.3 versus 0.74). provider-report demonstrated greater variability by site (R2 associated with site, 0.02 versus 0.16) and poorer test-retest reliability (mean kappa, 0.34 versus 0.25). Provider-report severity scores were less strongly associated than were self-report with functional status (chi2, 252 versus 80), global quality of life (R2 for model, 0.57 versus 0.15), and survival (chi2, 38 versus 24). Self-reported symptom severity was strongly correlated to patient-reported global quality of life (p, 0.75; p < .0001).

CONCLUSIONS

Provider-reported symptoms as currently collected within the ACTG are less sensitive and reproducible than a self-administered symptom survey. Provider-reported severity scores are also more weakly associated with functional status, global quality of life, and survival. A self-reported symptom survey may provide a better method of symptom measurement for HIV research.