Rotator cuff injuries are the most common type of shoulder disability, 1 leading to approximately 4.5 million physician visits each year. 2 While rotator cuff injuries can be linked to trauma, they also can be considered a part of normal degenerative changes associated with aging. 3 , 4 It is estimated that approximately 25% of adults in their 60s and 50% of adults in their 80s have a full thickness rotator cuff tear. 3 , 4 Symptoms of rotator cuff injuries often include pain, decreased strength, and range of motion, which can significantly limit the ability to complete activities of daily living. 2 In addition to direct cost to individuals for treatment, the loss of function can lead to the inability to work, loss of income, and depression. 5 , 6 Furthermore, injury recurrence rates after surgical repair have been estimated at 57.8%, 7 and several studies have failed to determine a significant difference in efficacy between surgical and nonsurgical management. 8 , 9 , 10 Taken together, the prevalence, symptomatology, and rate of reinjury highlight the need to consider patient perspectives when evaluating management strategies for rotator cuff injuries.
An increased focus on patient-centered care has led to the desire to gather patient perspectives on treatment effects, which can be assessed through patient-reported outcomes (PROs). 11 The Cochrane Collaboration defines PROs as a report directly from the patient pertaining to their treatment, health, and functional status without the interpretation by health care professionals. 12 PROs play a vital role in diseases, such as cancer, 13 and have been deemed useful in identifying a patient’s quality of life as they experience health deterioration, emotional stress, and the economic burden of treatment. 14 In a study assessing physiotherapy for rotator cuff tears, the authors used the Oxford Shoulder Disability Questionnaire and showed that while patients’ pain and overall function improved, their outlook on general health had declined. 15 Owing to the influence that a shoulder injury has on a person’s overall health, it is important that randomized controlled trials (RCTs)—which evaluate the efficacy of treatments—include PROs to assess quality of life following treatment. 16
The use of PROs in orthopaedics has been increasing in recent years 17 ; within shoulder literature, Mosher et al. 18 found that the overall use of PRO measures had increased by 18% from 2007 to 2017. Considering this increase in patient-reported measures, the American Academy of Orthopaedic Surgeons (AAOS) has endorsed the Patient-Reported Outcome Measure Information System (PROMIS) to improve the convenience, validity, and high-quality reporting of PROs. 19 To further enhance the reporting of PROs, the Consolidated Standards of Reporting Trials Patient-Reported Outcome (CONSORT-PRO) provides additional guidance for the promotion of high-quality reporting. 20 The purpose of this study was to evaluate the completeness of PROs reporting using CONSORT-PRO in RCTs involving rotator cuff injuries. We hypothesized that RCTs with PROs as a primary outcome would more completely report PRO measures compared with studies with PROs as a secondary outcome.
Methods
Study Design
We conducted a meta-epidemiologic study of published RCTs regarding the management of rotator cuff injuries. Data were collected by extracting information from published RCTs. This study did not involve humans and, thus, did not meet the regulatory definition for human subjects research per the U.S. Code of Federal Regulations and was not subject to institutional review board oversight. Our study adhered to the reporting guidelines for meta-epidemiological studies. 21
Search Strategy
One researcher (R.O.), with the consultation of a medical research librarian, used Ovid interface to search MEDLINE, Embase, and Cochrane Central Register of Controlled Trials for published RCTs focused on the management of rotator cuff injuries. To identify RCTs, we used the Cochrane search strategy, a validated filter for OVID interfaces. 22 The search string was uploaded to Open Science Framework. 23
Eligibility
RCTs that focused on the management of rotator cuff injuries with a PRO as a primary or secondary outcome were included. Studies published between 2006 and 2020 and written in English were included. We excluded the following: clinical trial protocols, clinical efficacy trials that did not have a PRO measure, observational studies, letters to the editor, animal studies, cost-effectiveness studies, secondary analyses, case reports, meta-analyses, systematic reviews, and other reviews.
Selection Process
The returns from our literature search were then combined and uploaded to a systematic review screening platform, Rayyan ( https://rayyan.qcri.org/ ). Duplicates were removed and 2 authors (L.B and B.H.) screened the title and abstract in a masked, duplicate fashion. A third author (S.S.) was available to resolve disagreements but was not needed.
Data-Collection Process
Two authors (B.H. and T.M.) received training on how to appraise reporting completeness by applying CONSORT-PRO used in previous methodology to evaluate CONSORT-PRO adherence. 20 , 24 Following training, 3 RCTs not included in our sample were extracted in a masked, duplicate fashion until consensus was achieved in an effort to reduce discrepancies. Authors (L.B. and D.N.) were trained for Risk of Bias (RoB) appraisal using Cochrane training modules. 25 Both CONSORT-PRO and RoB were performed in a masked, duplicate fashion using pilot-tested Google forms. Upon completion of data extraction and the RoB assessment, the authors resolved all differences; a third author (S.S.) was available to resolve disagreements, but was not needed.
Data Items
We reported our primary objective as the mean percent completion (see scoring of CONSORT-PRO). 24 Our secondary objective assessed relationships between the mean completeness of PRO reporting and trial characteristics. The characteristics analyzed were (1) year of publication (before or after 2014, a year following the publication of CONSORT-PRO); (2) intervention of RCT (e.g., drug or surgical technique); (3) conflict of interest statement; (4) journal endorsement of CONSORT-PRO; (5) citation of the CONSORT guideline within the publication; (6) whether an RCT used a PRO as a primary or secondary outcome; (7) RoB assessed by the Cochrane RoB 2.0 tool (see RoB assessment); (8) the length of PRO follow-up time; (9) sample size of the trial; and (10) whether the trial cited the use of PROMIS measures.
The publishing journal’s endorsement of CONSORT guidelines were coded as follows: not mentioned, recommended, or required. This data item was evaluated by screening the instructions to authors' pages for mention of EQUATOR, CONSORT, or CONSORT-PRO guidelines.
The following bias domains were evaluated: (1) bias arising from the randomization process, (2) bias due to deviations from intended interventions, (3) bias due to missing endpoint data, (4) bias in measurement of the endpoint, (5) bias in selection of the reported result, and (6) overall RoB assessment.
Scoring CONSORT-PRO
Our scoring framework follows published methodology from Mercieca-Bebber et al. 2017. 24 Of note, items not part of the PRO extension or elaborations were not scored to better evaluate PRO-related reporting. Item 4a of CONSORT-PRO (the use of PROs in eligibility or stratification) was removed from scoring in our study as there is difficulty in the verification of this criteria. Instead, we recorded whether a study describes adherence to this item as a “yes” or “no.” We allocated a maximum value of 0.5 or 1 when information for an item was present. Items that received the maximum value (1 or 0.5 if the item is double-barreled) were considered “complete,” while items that failed to reach maximum value are reported as “not complete.” Item P1b was scored partially complete if an RCT reported the PRO measure used in the study, but failed to identify whether the PRO was a primary or secondary outcome. Therefore, Item P1b could be scored as 0, 0.5, or 1 based on the information available. Item 7a was dependent on the PRO measure being reported as a primary outcome. Because of this dependency, RCTs with a primary PRO had a maximum score of 15 versus RCTs with a secondary outcome had a maximum score of 14. A percent completeness of the checklist per RCT was calculated by adding the items and dividing by the total of possible items.
RoB Assessment
We used a decision algorithm developed by the Cochrane Collaboration to score RoB. If investigators had partially divergent assessments on bias domains (e.g., one investigator answered “yes” and another investigator answered “partial yes”), the overall RoB judgment for the trial outcome was not altered. The overall risk of bias domain was determined per the Excel tool provided by Cochrane as “high,” “some concerns,” or “low’ risk.” 26 , 27 , 28
Data Analysis
First, we report our search return followed by the frequencies and percentages of trial characteristics among the RCTs in our sample. Additionally, we report the frequency of RCTs that mention PROMIS measures. We then report the mean completion percentage of CONSORT-PRO across all RCTs in our sample and by outcome category (i.e., RCTs with PROs as either primary or secondary outcomes). Next, we reported on the frequency and percentage of individual items on CONSORT-PRO for all RCTs and by outcome category. Finally, to address our secondary outcome, we used bivariate regression models to determine the association between mean completion percentage of CONSORT-PRO and the trial characteristics listed in Data Items. All analyses were performed using Stata 16.1 (StataCorp, LLC, College Station, TX).
Reproducibility
The study protocol, data sheets, analysis scripts, a data dictionary, and extraction forms were uploaded on Open Science Framework, to ensure the transparency and reproducibility of our study. This investigation was conducted in tandem with other studies addressing completeness of reporting in other fields of medicine using similar methodology.
Results
General Characteristics
Our literature search returned 467 records. After deduplication, 312 records remained and then were screened by title and abstract. After title and abstract screening, 86 RCTs were included for full-text screening and 33 published RCTs were included in our final sample. Rationales for exclusion can be found in Figure 1 . Fig 1 Rationale for exclusion.
The majority of RCTs were published after 2014 (24/33, 72.73%). The most common of the interventions studied were therapies as defined by our data dictionary (11/33, 33.33%). Fifteen (of 33, 45.45%) RCTs were published in journals that required CONSORT reporting. Among RCTs that included a conflict-of-interest statement, 18 (of 30, 60%) reported no conflicts. Thirty-one (of 33, 93.94%) RCTs did not mention the CONSORT guideline in the manuscript. When evaluating RCTs for risk of bias, we found that 1 RCT was rated as “high” (of 33, 3.03%), 29 RCTs were rated as “some concerns” (of 33, 87.88%), and 3 were rated as “low” (of 33, 9.09%). Twelve (of 33, 36.36%) RCTs had a follow-up time of 3 months or less. Of the 33 RCTs in our sample, no RCTs reported using PROMIS. Additional characteristics of RCTs are found in Table 1 . Table 1 Characteristics of Randomized Controlled Trails and Bivariate Associations With CONSORT-PRO Completion Characteristic Total 33 (100) Coef. (SE) t P Year of publication, No. (%) <2014 9 (27.27) 1 (Ref) – – ≥2014 24 (72.73) –3.97, (6.08) –0.65 .519 Intervention of RCT, No. (%) Combination 4 (12.12) 1 (Ref) – – Device 1 (3.03) –10.42, (17.38) –0.6 .554 Drug 6 (18.18) –18.47, (10.03) –1.84 .077 Other 2 (6.06) –13.75, (13.46) –1.02 .316 Surgical 1 (3.03) –33.75, (17.38) –1.94 .063 Surgical technique 8 (24.24) –15.83, (9.52) –1.66 .108 Therapy 11 (33.33) –14.03, (9.08) –1.55 .134 Includes COI statement, No. (%) No statement 3 (9.09) 1 (Ref) – – Reports COI 12 (36.36) –1.09, (10.14) –0.11 .915 Reports No COI 18 (54.55) –5.93, (9.8) –0.6 .55 Journal requirement of reporting guidelines, No. (%) Not mentioned 8 (24.24) 1 (Ref) – – Recommended 10 (30.3) 0.14, (7.53) 0.02 .986 Required 15 (45.45) 2.86, (6.95) 0.41 .684 Mention of CONSORT or CONSORT-PRO within RCT, No. (%) No 31 (93.94) 1 (Ref) – – Yes 2 (6.06) 9.19, (11.32) 0.81 .423 PRO as a primary or secondary outcome, No. (%) Primary 30 (90.91) 1 (Ref) – – Secondary 3 (9.09) 1.63, (9.49) 0.17 .864 Overall ROB, No. (%) High 1 (3.03) 1 (Ref) – – Some concern 29 (87.88) –11.14, (16) –0.7 .492 Low 3 (9.09) –5.56, (18.16) –0.31 .762 Length of PRO follow-up 3 months or less 12 (36.36) 1 (Ref) – – 3+ to 6 months 2 (6.06) –21.94, (11.41) –1.92 .064 6+ months to 1 year 8 (24.24) 4.31, (6.82) 0.63 .533 1 years + 11 (33.33) 1.64, (6.24) 0.26 .795 Sample size Mean (SD) 85.27 (56.98) 0.1, (0.04) 2.3 .028 Reported PROMIS measures Not reported 33 (100.0) – – – Reported 0 (0.0) – – – COI, conflict of interest; CONSORT, Consolidated Standards of Reporting Trials; CONSORT-PROs; PROMIS, Patient-Reported Outcomes Measurement Information System; RCT, randomized controlled trial; SD, standard deviation. A P-value less than or equal to 0.05 was considered statistically significant indicated in bold.
Table 1
Characteristics of Randomized Controlled Trails and Bivariate Associations With CONSORT-PRO Completion
COI, conflict of interest; CONSORT, Consolidated Standards of Reporting Trials; CONSORT-PROs; PROMIS, Patient-Reported Outcomes Measurement Information System; RCT, randomized controlled trial; SD, standard deviation. A P-value less than or equal to 0.05 was considered statistically significant indicated in bold.
| Characteristic | Total 33 (100) | Coef. (SE) | t | P |
| Year of publication, No. (%) | ||||
| <2014 | 9 (27.27) | 1 (Ref) | – | – |
| ≥2014 | 24 (72.73) | –3.97, (6.08) | –0.65 | .519 |
| Intervention of RCT, No. (%) | ||||
| Combination | 4 (12.12) | 1 (Ref) | – | – |
| Device | 1 (3.03) | –10.42, (17.38) | –0.6 | .554 |
| Drug | 6 (18.18) | –18.47, (10.03) | –1.84 | .077 |
| Other | 2 (6.06) | –13.75, (13.46) | –1.02 | .316 |
| Surgical | 1 (3.03) | –33.75, (17.38) | –1.94 | .063 |
| Surgical technique | 8 (24.24) | –15.83, (9.52) | –1.66 | .108 |
| Therapy | 11 (33.33) | –14.03, (9.08) | –1.55 | .134 |
| Includes COI statement, No. (%) | ||||
| No statement | 3 (9.09) | 1 (Ref) | – | – |
| Reports COI | 12 (36.36) | –1.09, (10.14) | –0.11 | .915 |
| Reports No COI | 18 (54.55) | –5.93, (9.8) | –0.6 | .55 |
| Journal requirement of reporting guidelines, No. (%) | ||||
| Not mentioned | 8 (24.24) | 1 (Ref) | – | – |
| Recommended | 10 (30.3) | 0.14, (7.53) | 0.02 | .986 |
| Required | 15 (45.45) | 2.86, (6.95) | 0.41 | .684 |
| Mention of CONSORT or CONSORT-PRO within RCT, No. (%) | ||||
| No | 31 (93.94) | 1 (Ref) | – | – |
| Yes | 2 (6.06) | 9.19, (11.32) | 0.81 | .423 |
| PRO as a primary or secondary outcome, No. (%) | ||||
| Primary | 30 (90.91) | 1 (Ref) | – | – |
| Secondary | 3 (9.09) | 1.63, (9.49) | 0.17 | .864 |
| Overall ROB, No. (%) | ||||
| High | 1 (3.03) | 1 (Ref) | – | – |
| Some concern | 29 (87.88) | –11.14, (16) | –0.7 | .492 |
| Low | 3 (9.09) | –5.56, (18.16) | –0.31 | .762 |
| Length of PRO follow-up | ||||
| 3 months or less | 12 (36.36) | 1 (Ref) | – | – |
| 3+ to 6 months | 2 (6.06) | –21.94, (11.41) | –1.92 | .064 |
| 6+ months to 1 year | 8 (24.24) | 4.31, (6.82) | 0.63 | .533 |
| 1 years + | 11 (33.33) | 1.64, (6.24) | 0.26 | .795 |
| Sample size | ||||
| Mean (SD) | 85.27 (56.98) | 0.1, (0.04) | 2.3 | .028 |
| Reported PROMIS measures | ||||
| Not reported | 33 (100.0) | – | – | – |
| Reported | 0 (0.0) | – | – | – |
Completeness of Reporting According to CONSORT-PRO
The mean CONSORT-PRO complete percentage of included RCTs was 49.70% (standard deviation [SD] 15.42). RCTs with a PRO as a primary outcome had a mean completion percentage of 49.55% (SD 14.79) and RCTs with a PRO as a secondary outcome had a mean completion percentage of 51.19% (SD 25.08).
Throughout all the RCTs in our sample, we found that item 17aii— results include an estimate of precision —was the most completely reported item (31/33, 93.94%). Additionally, no RCTs completely reported Item 2a— rationale for PRO endpoint . Regarding RCTs with a primary PRO, the most completely reported items included: PRO identified as RCT endpoint in abstract (P1b), number of questionnaires at baseline (13ai), and results include an estimate of precision (17aii) (28/33, 93.33%). Item 2a— rationale for PRO endpoint— was not reported by any RCT with a primary PRO. All RCTs with a secondary PRO (3/3, 100%) completely reported the following items: number of questionnaires at baseline and at time of analysis (13ai and 13aii), results include an estimate of precision (17aii), and PRO study limitations (P20). No RCTs with a secondary PRO completely reported the following items: rationale for PRO endpoint (2a), PRO domains specified in the hypothesis (P2bi and P2bii), and completion of PRO questionnaire (P6aiii). We did not assess Item 7a— sample size determination —for RCTs with secondary PRO. Additional information related to completion of CONSORT PRO may be found in Table 2 . Table 2 Completion of CONSORT-PRO by Primary and Secondary Objective Designation Primary 30 (90.91) Secondary Endpoint 3 (9.09) Total 33 (100) Complete Not Complete Complete Not Complete Complete Not Complete CONSORT-PRO item n (%) n (%) n (%) n (%) n (%) n (%) Introduction P1b. Abstract—PRO as primary/secondary endpoint 28 (93.33) 2 (6.67) 2 (66.67) 1 (33.33) 30 (90.91) 3 (9.09) 2a. Rationale for including PRO endpoint 0 (0) 30 (100) 0 (0) 3 (100) 0 (0) 33 (100) P2bi. PRO hypothesis present 6 (20) 24 (80) 0 (0) 3 (100) 6 (18.18) 27 (81.82) P2bii. PRO domains in hypothesis 1 (3.33) 29 (96.67) 0 (0) 3 (100) 1 (3.03) 32 (96.97) Methods P6ai. Evidence of PRO instrument validity 26 (86.67) 4 (13.33) 2 (66.67) 1 (33.33) 28 (84.85) 5 (15.15) P6aii. Statement of the person completing the questionnaire 15 (50) 15 (50) 2 (66.67) 1 (33.33) 17 (51.52) 16 (48.48) P6aiii. Mode of administration (paper, e-PRO) 3 (10) 27 (90) 0 (0) 3 (100) 3 (9.09) 30 (90.91) P7a. How sample size was determined (not required unless PRO is a primary endpoint) ∗ 25 (83.33) 5 (16.67) – – 25 (83.33) 5 (16.67) P12a. Statistical approach for dealing with missing data (imputation, exclusion, other) 6 (20) 24 (80) 2 (66.67) 1 (33.33) 8 (24.24) 25 (75.76) Results 13ai. Report no. questionnaires submitted/available for analysis at baseline 28 (93.33) 2 (6.67) 3 (100) 0 (0) 31 (93.94) 2 (6.06) 13aii. Report no. questionnaires submitted/available for analysis principal time point for analysis 27 (90) 3 (10) 3 (100) 0 (0) 30 (90.91) 3 (9.09) 15. Demographics table includes baseline PRO 12 (40) 18 (60) 1 (33.33) 2 (66.67) 13 (39.39) 20 (60.61) 16. Number of pts (denominator) included in each PRO analysis 14 (46.67) 16 (53.33) 2 (66.67) 1 (33.33) 16 (48.48) 17 (51.52) 17ai. PRO results reported for the hypothesized domains and time point specified in the hypothesis—OR—reported for each domain of the PRO questionnaire if no PRO hypothesis provided 6 (20) 24 (80) 2 (66.67) 1 (33.33) 8 (24.24) 25 (75.76) 17aii. Results include confidence interval, effect size or some other estimate of precision 28 (93.33) 2 (6.67) 3 (100) 0 (0) 31 (93.94) 2 (6.06) 18. Results of any subgroup/adjusted/exploratory analyses 10 (33.33) 20 (66.67) 1 (33.33) 2 (66.67) 11 (33.33) 22 (66.67) Discussion P20. PRO study limitations 27 (90) 3 (10) 3 (100) 0 (0) 30 (90.91) 3 (9.09) P21. Implications of PRO results for generalizability, clinical practice 6 (20) 24 (80) 1 (33.33) 2 (66.67) 7 (21.21) 26 (78.79) 22. PROs interpreted in relation to clinical outcomes 12 (40) 18 (60) 1 (33.33) 2 (66.67) 13 (39.39) 20 (60.61) CONSORT-PRO, Consolidated Standards of Reporting Trials Patient-Reported Outcomes. ∗ Item P7a only applies to PROs identified as primary outcomes.
Table 2
Completion of CONSORT-PRO by Primary and Secondary Objective Designation
CONSORT-PRO, Consolidated Standards of Reporting Trials Patient-Reported Outcomes.
∗ Item P7a only applies to PROs identified as primary outcomes.
| Primary 30 (90.91) | Secondary Endpoint 3 (9.09) | Total 33 (100) | ||||
| Complete | Not Complete | Complete | Not Complete | Complete | Not Complete | |
| CONSORT-PRO item | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) |
| Introduction | ||||||
| P1b. Abstract—PRO as primary/secondary endpoint | 28 (93.33) | 2 (6.67) | 2 (66.67) | 1 (33.33) | 30 (90.91) | 3 (9.09) |
| 2a. Rationale for including PRO endpoint | 0 (0) | 30 (100) | 0 (0) | 3 (100) | 0 (0) | 33 (100) |
| P2bi. PRO hypothesis present | 6 (20) | 24 (80) | 0 (0) | 3 (100) | 6 (18.18) | 27 (81.82) |
| P2bii. PRO domains in hypothesis | 1 (3.33) | 29 (96.67) | 0 (0) | 3 (100) | 1 (3.03) | 32 (96.97) |
| Methods | ||||||
| P6ai. Evidence of PRO instrument validity | 26 (86.67) | 4 (13.33) | 2 (66.67) | 1 (33.33) | 28 (84.85) | 5 (15.15) |
| P6aii. Statement of the person completing the questionnaire | 15 (50) | 15 (50) | 2 (66.67) | 1 (33.33) | 17 (51.52) | 16 (48.48) |
| P6aiii. Mode of administration (paper, e-PRO) | 3 (10) | 27 (90) | 0 (0) | 3 (100) | 3 (9.09) | 30 (90.91) |
| P7a. How sample size was determined (not required unless PRO is a primary endpoint) ∗ | 25 (83.33) | 5 (16.67) | – | – | 25 (83.33) | 5 (16.67) |
| P12a. Statistical approach for dealing with missing data (imputation, exclusion, other) | 6 (20) | 24 (80) | 2 (66.67) | 1 (33.33) | 8 (24.24) | 25 (75.76) |
| Results | ||||||
| 13ai. Report no. questionnaires submitted/available for analysis at baseline | 28 (93.33) | 2 (6.67) | 3 (100) | 0 (0) | 31 (93.94) | 2 (6.06) |
| 13aii. Report no. questionnaires submitted/available for analysis principal time point for analysis | 27 (90) | 3 (10) | 3 (100) | 0 (0) | 30 (90.91) | 3 (9.09) |
| 15. Demographics table includes baseline PRO | 12 (40) | 18 (60) | 1 (33.33) | 2 (66.67) | 13 (39.39) | 20 (60.61) |
| 16. Number of pts (denominator) included in each PRO analysis | 14 (46.67) | 16 (53.33) | 2 (66.67) | 1 (33.33) | 16 (48.48) | 17 (51.52) |
| 17ai. PRO results reported for the hypothesized domains and time point specified in the hypothesis—OR—reported for each domain of the PRO questionnaire if no PRO hypothesis provided | 6 (20) | 24 (80) | 2 (66.67) | 1 (33.33) | 8 (24.24) | 25 (75.76) |
| 17aii. Results include confidence interval, effect size or some other estimate of precision | 28 (93.33) | 2 (6.67) | 3 (100) | 0 (0) | 31 (93.94) | 2 (6.06) |
| 18. Results of any subgroup/adjusted/exploratory analyses | 10 (33.33) | 20 (66.67) | 1 (33.33) | 2 (66.67) | 11 (33.33) | 22 (66.67) |
| Discussion | ||||||
| P20. PRO study limitations | 27 (90) | 3 (10) | 3 (100) | 0 (0) | 30 (90.91) | 3 (9.09) |
| P21. Implications of PRO results for generalizability, clinical practice | 6 (20) | 24 (80) | 1 (33.33) | 2 (66.67) | 7 (21.21) | 26 (78.79) |
| 22. PROs interpreted in relation to clinical outcomes | 12 (40) | 18 (60) | 1 (33.33) | 2 (66.67) | 13 (39.39) | 20 (60.61) |
Associations Between Trial Characteristics and Completeness of Reporting
Our bivariate regression analyses revealed that an increase in trial sample size was significantly associated with increased completeness of reporting (t 1 = 2.3; P = .028). We found no other significant associations between trial characteristics and completeness of reporting. Relationships are demonstrated in Table 1 .
Discussion
Our study identified that PRO reporting was incomplete within RCTs focused on rotator cuff injuries, with an overall mean completeness of less than 50%. Studies with PROs as secondary outcomes were found to be associated with a greater level of completeness; this finding is ambiguous due to the contrast in number of studies with PROs as primary outcomes versus secondary. One commonly underreported item in CONSORT-PRO checklist was the description of statistical methods for handling missing data. Conversely, a well reported item within RCTs was the documentation of instrument validity. Additionally, our analysis revealed that none of the included RCTs mentioned the use of PROMIS. In the following discussion, we will elaborate on our findings within the context of existing literature, address the implications of the aforementioned checklist items, and recommend the implementation of PROMIS to improve PRO reporting within orthopaedics.
In accordance with our findings, studies have found that CONSORT-PRO completeness was suboptimal. 29 , 30 Additionally, a systematic review assessing PRO reporting in multiple myeloma found similar findings with commonly omitted checklist items including the statistical plan for handling missing data (23.0% reported) and implications for generalizability and clinical practice (26.7% reported). 31 In a study looking at PROs in 23 publications related to osteoporosis, the authors found that the mean CONSORT-PRO completeness was approximately two-thirds. 32 Taken together, these findings suggest reporting deficits, which can be addressed through the application of the CONSORT-PRO checklist.
An inconsistently reported item on the CONSORT-PRO checklist was a lack of reporting of an approach to handling missing data. Nearly three-fourths of the RCTs in our sample failed to report this item. This finding is not unique to our study. An investigation of the Journal of the American Medical Association , The New England Journal of Medicine , the Lancet , and The BMJ found that 77% of RCTs published between 2013 and 2014 failed to report on appropriate handling of missing data within their publication. 33 Missing data can introduce bias and decrease the power of the study when researchers fail to report their statistical approach. 34 These deficits in reporting could impair interpretation of the quality of data collection and analysis. 35
While deficiencies in reporting were common in our sample, information regarding PRO instrument validity was well documented at 84.85% across rotator cuff injury RCTs. This finding contrasts with previous literature, suggesting that medical research may not report necessary information regarding instrument selection. In our previous example by LeBlanc et al., 31 only 50% of studies in their systematic review of multiple myeloma literature cited validation studies for the included PROs. Additionally, a study analyzing PRO quality across head and neck cancer RCTs found that 34% of these RCTs reported evidence of PRO instrument validity. 36 Without the use of a validated PRO instrument, clinicians are unable to assess the relevance of RCTs to patients and whether or not the PROs are appropriate for the trial design. 36
Despite the strengths in reporting that were found in orthopaedic literature, opportunities to further improve PRO reporting exist. The endorsement of PROMIS by the AAOS offers the opportunity to overcome historical barriers associated with PRO reporting within clinical trials. 37 In a study assessing the use of PROMIS in upper extremity injuries—including rotator cuff injuries—PROMIS physical function computerized adaptive testing was found to be more efficient and have a strong correlation with American Shoulder and Elbow Surgeons score and the Simple Shoulder Test scores. 38 Several elements which make PROMIS superior to legacy systems are: (1) computerized adaptive testing is a form of computer based questioning which is built on item-response theory to adapt to the ability level of the patient; (2) the ability to report baseline functional T-score using a standard set of outcomes across differing subspecialties in orthopaedics 39 ; (3) the multidomain capability which PROMIS offers with its physical function, pain, emotional distress, and fatigue domains would normally require multiple scoring systems to be used. 40 Eighteen RCTs in our sample were published after the AAOS endorsement of PROMIS in 2016, 19 and no trials in our sample incorporated PROMIS measures. Our findings are supported by a cross-sectional analysis among orthopaedic trauma literature from 2014 to 2018 looking at 319 RCTs. 41 They found only 7 trials used PROMIS measures. 41 Given these findings, there is still opportunity for the field of orthopaedics to consistently adopt PROMIS and become a leader in reporting patient outcomes.
Recommendations
To increase high-quality reporting in orthopaedics, we agree with Hussain et al. 42 in their recommendation of emphasizing the use of the CONSORT checklist. To further emphasize the importance of RCTs abiding by CONSORT-PRO, Mercieca-Bebber et al. 19 found that greater rates of complete reporting correlated with RCTs that cited CONSORT-PRO in their study. 43 Despite the adoption of PROMIS by the AAOS in 2016, https://paperpile.com/c/WAvgO0/oThM use of this tool is lacking in recent studies. Following the guidance of the AAOS, we recommend the use of PROMIS to increase the quality in which PRO data is captured. 19 Additionally, reporting standardization of PROs will allow for incorporation of convenient, value-based assessments in clinical practice. 44 Finally, further investigation of the relationship between sample size and completeness of reporting is encouraged. Factors influencing the sample size of RCT s are numerous and may have influenced our results. These factors may include funding source, amount of funding, recruitment methods, and study design.
Limitations
This study is not without limitations. First, three RCTs in our sample used a PRO as a secondary outcome. This limited the assessment of our hypothesis. Second, although MEDLINE, Embase, and Cochrane Central Register of Controlled Trials database searches were performed, it is possible that all relevant RCTs were not included. Lastly, the results of our cross-sectional study should not be generalized to other qualifying studies, other journals, or studies published from other years.
Conclusions
Randomized controlled trials involving rotator cuff injuries frequently use PRO measures as primary outcomes. Reporting of these PRO measures is suboptimal and may benefit from rigorous standardization.
References
1 Weber S. Chahal J. Management of rotator cuff injuries J Am Acad Orthop Surg 28 2020 e193 e201 31599763 10.5435/JAAOS-D-19-00463
2 Oh L.S. Wolf B.R. Hall M.P. Levy B.A. Marx R.G. Indications for rotator cuff repair: A systematic review Clin Orthop Relat Res 455 2007 52 63 17179786 10.1097/BLO.0b013e31802fc175
3 Tashjian R.Z. Epidemiology, natural history, and indications for treatment of rotator cuff tears Clin Sports Med 31 2012 589 604 23040548 10.1016/j.csm.2012.07.001
4 Sher J.S. Uribe J.W. Posada A. Murphy B.J. Zlatkin M.B. Abnormal findings on magnetic resonance images of asymptomatic shoulders J Bone Joint Surg Am 77 1995 10 15 7822341 10.2106/00004623-199501000-00002
5 Mather R.C. 3rd Koenig L. Acevedo D. The societal and economic value of rotator cuff repair J Bone Joint Surg Am 95 2013 1993 2000 24257656 10.2106/JBJS.L.01495 PMC3821158
6 Cho C.-H. Seo H.-J. Bae K.-C. Lee K.-J. Hwang I. Warner J.J.P. The impact of depression and anxiety on self-assessed pain, disability, and quality of life in patients scheduled for rotator cuff repair J Shoulder Elbow Surg 22 2013 1160 1166 23594716 10.1016/j.jse.2013.02.006
7 Shin Y.K. Ryu K.N. Park J.S. Jin W. Park S.Y. Yoon Y.C. Predictive factors of retear in patients with repaired rotator cuff tear on shoulder MRI AJR Am J Roentgenol 210 2018 134 141 29045184 10.2214/AJR.17.17915
8 Ryösä A. Laimi K. Äärimaa V. Lehtimäki K. Kukkonen J. Saltychev M. Surgery or conservative treatment for rotator cuff tear: a meta-analysis Disabil Rehabil 39 2017 1357 1363 27385156 10.1080/09638288.2016.1198431
9 Kukkonen J. Joukainen A. Lehtinen J. Treatment of nontraumatic rotator cuff tears: A randomized controlled trial with two years of clinical and imaging follow-up J Bone Joint Surg Am 97 2015 1729 1737 26537160 10.2106/JBJS.N.01051
10 Longo U.G. Risi Ambrogioni L. Candela V. Conservative versus surgical management for patients with rotator cuff tears: a systematic review and META-analysis BMC Musculoskelet Disord 22 2021 50 33419401 10.1186/s12891-020-03872-4 PMC7796609
11 Snyder C.F. Jensen R.E. Segal J.B. Wu A.W. Patient-reported outcomes (PROs): Putting the patient perspective in patient-centered outcomes research Med Care 51 8 suppl 3 2013 S73 S79 23774513 10.1097/MLR.0b013e31829b1d84 PMC3771694
12 HIGGINS J.P. Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. www.cochrane-handbook.org. Published online 2011. https://ci.nii.ac.jp/naid/20000796633/
13 Chen J. Ou L. Hollis S.J. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting BMC Health Services Res 13 1 2013 10.1186/1472-6963-13-211 PMC3700832 23758898
14 Deshpande P. Sudeepthi B. Rajan S. Nazir C.P.A. Patient-reported outcomes: A new era in clinical research Perspect Clin Res 2 2011 137 22145124 10.4103/2229-3485.86879 PMC3227331
15 Ainsworth R. Physiotherapy rehabilitation in patients with massive, irreparable rotator cuff tears Musculoskelet Care 4 2006 140 151 10.1002/msc.85 17042025
16 Osoba D. Translating the science of patient-reported outcomes assessment into clinical practice J Natl Cancer Inst Monogr 2007 5 11 17951224 10.1093/jncimonographs/lgm002
17 Gagnier J.J. Patient reported outcomes in orthopaedics J Orthop Res 35 2017 2098 2108 28513993 10.1002/jor.23604
18 Mosher Z.A. Ewing M.A. Collins C.S. Usage trends of patient-reported outcome measures in shoulder literature J Am Acad Orthop Surg 28 2020 e774 e781 31663915 10.5435/JAAOS-D-19-00455
19 Patel A.A. Patient-reported outcome measures: The promise of PROMIS J Am Acad Orthop Surg 24 2016 743 27755260 10.5435/JAAOS-D-16-00436
20 Calvert M. Blazeby J. Altman D.G. Reporting of patient-reported outcomes in randomized trials: The CONSORT PRO extension JAMA 309 2013 814 822 23443445 10.1001/jama.2013.879
21 Murad M.H. Wang Z. Guidelines for reporting meta-epidemiological methodology research Evid Based Med 22 2017 139 142 28701372 10.1136/ebmed-2017-110713 PMC5537553
22 Lefebvre C Manheimer. E Glanville. Searching for studies Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.1.0 2011 The Cochrane Collaboration www.handbook.cochrane.org Accessed June 30, 2021
23 Shepard S. Wise A. Heigle B. Rotator cuff injuries (July 2021). Open Science Framework Published online June 15, 2021. Accessed July 21, 2021. https://osf.io/scy9j/
24 Mercieca-Bebber R. Rouette J. Calvert M. Preliminary evidence on the uptake, use and benefits of the CONSORT-PRO extension Qual Life Res 26 2017 1427 1437 28168602 10.1007/s11136-017-1508-6
25 RoB 2.0: A revised tool to assess risk of bias in randomized trials [webinar] Accessed July 21, 2021. https://training.cochrane.org/resource/rob-20-webinar
26 Risk of bias tools—Current version of RoB 2 https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool/current-version-of-rob-2?authuser=0
27 Risk of bias tools—RoB 2 for crossover trials https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool/rob-2-for-crossover-trials?authuser=0
28 Risk of bias tools—RoB 2 for cluster-randomized trials https://sites.google.com/site/riskofbiastool/welcome/rob-2-0-tool/rob-2-for-cluster-randomized-trials?authuser=0
29 Bylicki O. Gan H.K. Joly F. Maillet D. You B. Péron J. Poor patient-reported outcomes reporting according to CONSORT guidelines in randomized clinical trials evaluating systemic cancer therapy Ann Oncol 26 2015 231 237 25355720 10.1093/annonc/mdu489
30 Coleman R.L. Beck J.T. Baranda J.C. The use of patient-reported outcome measures in phase I oncology clinical trials Oncology 99 2021 444 453 33823518 10.1159/000514874
31 LeBlanc M.R. Hirschey R. Leak Bryant A. LeBlanc T.W. Smith S.K. How are patient-reported outcomes and symptoms being measured in adults with relapsed/refractory multiple myeloma? A systematic review Qual Life Res 29 2020 1419 1431 31848847 10.1007/s11136-019-02392-6 PMC7428159
32 Mack D.E. Wilson P.M. Santos E. Brooks K. Standards of reporting: the use of CONSORT PRO and CERT in individuals living with osteoporosis Osteoporos Int 29 2018 305 313 28971256 10.1007/s00198-017-4249-z
33 Fielding S. Ogbuagu A. Sivasubramaniam S. MacLennan G. Ramsay C.R. Reporting and dealing with missing quality of life data in RCTs: Has the picture changed in the last decade? Qual Life Res 25 2016 2977 2983 27650288 10.1007/s11136-016-1411-6 PMC5102945
34 Little R.J. D’Agostino R. Cohen M.L. The prevention and treatment of missing data in clinical trials N Engl J Med 367 2012 1355 1360 23034025 10.1056/NEJMsr1203730 PMC3771340
35 Devereaux P.J. Manns B.J. Ghali W.A. Quan H. Guyatt G.H. The reporting of methodological factors in randomized controlled trials and the association with a journal policy to promote adherence to the Consolidated Standards of Reporting Trials (CONSORT) checklist Control Clin Trials 23 2002 380 388 12161081 10.1016/s0197-2456(02)00214-3
36 Mercieca-Bebber R.L. Perreca A. King M. Patient-reported outcomes in head and neck and thyroid cancer randomised controlled trials: A systematic review of completeness of reporting and impact on interpretation Eur J Cancer 56 2016 144 161 26851406 10.1016/j.ejca.2015.12.025
37 Brodke D.J. Saltzman C.L. Brodke D.S. PROMIS for Orthopaedic Outcomes Measurement J Am Acad Orthop Surg 24 2016 744 749 27661391 10.5435/JAAOS-D-15-00404
38 Cheung E.C. Moore L.K. Flores S.E. Lansdown D.A. Feeley B.T. Zhang A.L. Correlation of PROMIS with Orthopaedic Patient-Reported Outcome Measures JBJS Rev 7 2019 e9 10.2106/JBJS.RVW.18.00190 31460991
39 Horn M.E. Reinke E.K. Yan X. Use of Patient-Reported Outcomes Measurement Information System (PROMIS) measures to characterise health status for patients seeking care from an orthopaedic provider: a retrospective cohort study BMJ Open 11 2021 e047156 10.1136/bmjopen-2020-047156 PMC8413970 34475157
40 Wong L.H. Meeker J.E. The promise of computer adaptive testing in collection of orthopaedic outcomes: an evaluation of PROMIS utilization J Patient Rep Outcomes 6 2022 2 34982280 10.1186/s41687-021-00407-w PMC8727662
41 O’Hara N.N. Richards J.T. Overmann A. Slobogean G.P. Klazinga N.S. Is PROMIS the new standard for patient-reported outcomes measures in orthopaedic trauma research? Injury 51 suppl 2 2020 S43 S50 31708090 10.1016/j.injury.2019.10.076
42 Hussain J.A. Bland M. Langan D. Johnson M.J. Currow D.C. White I.R. Quality of missing data reporting and handling in palliative care trials demonstrates that further development of the CONSORT statement is required: A systematic review J Clin Epidemiol 88 2017 81 91 28532739 10.1016/j.jclinepi.2017.05.009 PMC5590708
43 Mercieca-Bebber R. King M.T. Calvert M.J. Stockler M.R. Friedlander M. The importance of patient-reported outcomes in clinical trials and strategies for future optimization Patient Relat Outcome Meas 9 2018 353 367 30464666 10.2147/PROM.S156279 PMC6219423
44 Makhni E.C. Hamamoto J.T. Higgins J.D. How comprehensive and efficient are patient-reported outcomes for rotator cuff tears? Orthop J Sports Med 5 2017 2325967117693223 10.1177/2325967117693223 PMC5400217 28451596
.Supplementary Material
ICMJE author disclosure forms
Acknowledgments
We are grateful to April Schweikhard, who assisted in the development of our search strategy and to the OSU medical library for their procurement of relevant literature.
The authors report the following potential conflicts of interest or sources of funding M.H. reports receiving funding from the 10.13039/100005289 National Institute of Justice for work unrelated to the current subject. M.V. reports receipt of funding from the 10.13039/100000026 National Institute on Drug Abuse , the 10.13039/100000027 National Institute on Alcohol Abuse and Alcoholism , the U.S. Office of Research Integrity, Oklahoma Center for Advancement of Science and Technology, and internal grants from 10.13039/100007069 Oklahoma State University Center for Health Sciences, all outside of the present work. Full ICMJE author disclosure forms are available for this article online, as supplementary material .