The Musculoskeletal Tumor Society Scoring system is a valid subjective and objective tool to evaluate outcomes of surgical treatment of patients affected by upper and lower extremity tumors.

Introduction

Treatments of oncological diseases of musculoskeletal system mostly aim to guarantee function and quality of life (QoL) at their best, especially when eradication of the neoplasia is not achievable. Although in the past most outcome studies focused on survival rate and local recurrences as primary outcomes, in the last decades more emphasis has been placed on patient’s perspective. It has been shown that taking into account patient perception promotes communication, improves decision-making process and increases patient satisfaction. Objective measures integrated with patient perception could provide better medical care [ 1 – 3 ].

The Musculoskeletal Tumor Society (MSTS) score was developed in 1993 as an objective tool to measure functional outcome in patients affected by neoplasms [ 4 ]. Even if the MSTS score has been never properly validated in its original version [ 4 ], it is widely used in clinical practice. As a matter of fact, the original version underwent cross-cultural adaptation and validation in several languages, such as Greek [ 5 ], Danish [ 6 ], Brazilian [ 7 ], Chinese [ 8 ], Japanese [ 9 ], French [ 10 ] and Turkish [ 11 ].

The score is available for upper and lower limb [ 12 ]. Main strengths of the score mainly rely on ease of use and briefness [ 13 ]. Main concern is that the MSTS score was formulated as an objective tool (hetero-administered), but it is currently worldwide used as a patient-reported outcome measure (PROM) (self-administered).

The main purpose of the present study is to evaluate if there is a difference between objective or subjective administration of the MSTS score in a cohort of patients affected by musculoskeletal oncological diseases. The hypothesis of the study is that there are no differences between patient- and clinician-reported outcomes using the MSTS score for both lower and upper limb.

Materials and methods

Study design

An observational study was conducted, after approval of the study protocol by the local ethic committee (NP 4912 Spedali Civili, Brescia).

Patients

All patients who underwent surgery for bone or soft tissue localization of neoplastic disease in lower or upper limb from June 2015 to June 2020 at Spedali Civili in Brescia, Italy, were considered eligible for the study. Patients were included regardless of previous treatment and disease stage. Onco-emathologic diseases were also included. Inclusion criteria also included: Italian as mother language, age of 18 years or above, minimum twelve-month follow-up from surgery, willingness to enter the study and ability to provide informed consent. Patients with a Karnofsky’s score lower than 30% [ 14 ], those who did not undergo surgical treatment and those who had diagnosis of dementia (any type) or were in a state of altered metal status were excluded.

Intervention

In order to administer the score as a PROM, the first part of the study consisted of translation and cross-cultural adaptation of an Italian version of the questionnaire according to well-established guidelines [ 15 , 16 ]. Questionnaires were then administered during the postoperative follow-up visits in an outpatient setting. Thirty to sixty minutes after completion of self-administered questionnaires, patients underwent an interview by an orthopedic surgeon, based on the MSTS questionnaire. The examiner was blinded to the patients’ answers at the self-administration of the questionnaire. Retest was conducted after a period of two weeks after first administration in order to avoid any recall bias.

Outcome measures

Besides the Italian version of MSTS questionnaire, all patients filled out the national validated version of the Toronto Extremity Savage Score (TESS) [ 17 ]. Each patient completed the upper or the lower limb version of both MSTS and TESS score, depending on the localization of the disease. The national validated version of the SF-36 [ 18 ] was used as general health measurement.

The MSTS questionnaire [ 4 ] consists of six domains, each scored on a scale from 0 to 5, with a higher score indicating better function. The total score, ranging from 0 (maximum disability) to 30 (no impairment), can be transformed to a point scale of 0 to 100.

The TESS score assesses functional outcome in musculoskeletal tumor patients aged 12–85 years [ 19 ]. It consists of 29 items for upper extremity and 30 items for lower extremity. The degree of disability is rated from 0 (complete disability) to 5 (no functional impairment) in each item. Similar to MSTS, the final TESS score can be converted to a score ranging from 0 to 100 points.

The SF-36 is a non-pathology-related questionnaire aiming to test both physical and mental components of patient perception of QoL. It is composed of 36 questions divided into eight different domains. Each of these domains can be rated from 0 (worst) to 100 (best).

Data analysis

Sample size was estimated according to established guidelines for questionnaire validation [ 16 , 17 ]. The Italian version of MSTS questionnaire was first test retested on at least 30 patients per group (upper and lower limb). Psychometric properties of the questionnaire were then assessed on a sample of 50 patients per group.

All the data were analyzed by SPSS 25 (IBM Statistics, Harmonk, NY, USA). Descriptive statistics were used to report scores and answers distribution for each question. Data normality was ascertained by Shapiro–Wilk test. Discrete data were expressed as mean ± standard deviation in case of normal data distribution, otherwise as median and interquartile range (IQR). Categories were expressed as frequencies and percentages.

Ceiling and floor effect were considered significant if more than 15% of patient reached the lowest or the highest possible score, respectively.

Content validity could not be tested through the multi-trait analysis because each question corresponds to a domain. The structure of the questionnaire was determined by the factor analysis. Factor’s number was calculated using Kaiser criteria (eigenvalue > 1) and a scree plot.

Construct validity was calculated through Pearson’s coefficient correlation. The Italian version of MSTS score was compared to TESS and SF-36 score. Correlation between objective and self-administered MSTS score was assessed through Pearson’s coefficient. Correlation was deemed as very weak when ranging 0 to 0.19, weak if between 0.20 and 0.39, mild if between 0.40 and 0.69, strong if between 0.70 and 0.89 and very strong if between 0.90 and 1 [ 15 ].

Reliability was assessed by internal consistency and test–retest reliability. Cronbach’s alpha coefficient measured internal consistency for every domain. Internal consistency higher than 0.70 indicates good reproducibility [ 20 ]. Intraclass correlation coefficient (ICC) measured test–retest reliability. ICC values ranged between 0 (absolute disagreement) and 1 (maximum disagreement). Values were interpreted as follows: poor reliability when less than 0.50, moderate when between 0.50 and 0.75, good when between 0.75 and 0.90 and excellent when greater than 0.90 [ 21 ].

Significance at probability tests was estimated for p value < 0.05.

Results

No major issues were encountered during translation from the original version. No major difficulties in comprehension were revealed during testing the pre-final version. Patients took about 5–10 min to complete the questionnaire (see Appendix 1).

The psychometric properties were tested on a finale sample of 110 patients: 59 affected by lower extremity involvement and 51 affected by upper extremity involvement. Patients’ characteristics are shown in Tables 1 and 2 , for upper and lower limb, respectively.

Psycometric properties of MSTS score for upper extremity

The descriptive statistics data are shown in Tables 3 (self-administered) and 4 (hetero-administered). No missing data were reported, thus confirming that the translated questionnaire was well understood by the patients. A ceiling effect > 15% was observed for all items in both administration modalities.

Factor analysis, as indicated in the scree plots (Fig. 1 ), showed that the appropriate number of factors was 1. This was visible in both the self-administered and hetero-administered modalities.

The assessment of construct validity (Appendix 2) showed that both administration modalities had overall good correlation with TESS ( r = 0.78 and r = 0.80 for the self-administered version and for the hetero-administered version, respectively). On the opposite, both administrations showed poor correlation with SF-36: ( r = 0.19 and and r = 0.1 for the self-administered version and for the hetero-administered version, respectively).

Internal consistency was as high as Cronbach’s alpha = 0.84 for self-admnistered MSTS and 0.77 for hetero-administered MST versions (Appendix 3). Test–retest reliability was good in both versions, with an overall ICC of 0.84 for self-administered version and 0.78 for hetero-administered version (Appendix 4).

Correlation between self-administered and hetero-administered version of the questionnaire was high ( r = 0.96) (Table 5 ).

Psycometric properties of MSTS score for lower extremity

Descriptive statistics are shown in Tables 6 (self-administered) and 7 (hetero-administered). No missing data were reported. A ceiling effect > 15% was observed for all items in both administration modalities.

Factor analysis, as indicated in the scree plots (Fig. 2 ) , showed that the appropriate number of factors is 1. This was confirmed for both the self-administered and the hetero-administered modalities.

Assessment of construct validity showed that both administration modalities had overall strong correlation with TESS, as high as r = 0.80 for both versions, and mild correlation with SF-36, equal to r = 0.5 for both versions (Appendix 5).

Internal consistency was very high for both self- and hetero-administered MSTS versions. Self-administered version showed an overall Cronbach’s alpha = 0.96, while the hetero-administered version reached an overall Cronbach’s alpha = 0.98 (Appendix 6). Test–retest reliability was excellent in both versions: ICC = 0.96 for the self-administered version and ICC = 0.98 for the hetero-administered version (Appendix 7).

The correlation between self-administered and hetero-administered version of the questionnaire was as high as r = 0.97 (Table 8 ).

Discussion

Main finding of the present study was that MSTS score can be interchangeably used as a PROM or as an objective tool because both administration modalities showed to be valid and the correlation between the two was very high. At the same time, it must be highlighted that the Italian version of the MSTS score showed good psychometric properties for both lower and upper extremity.

MSTS score has been originally developed as a clinician-administered questionnaire and it is now widely used to evaluate residual function in patients with extremity tumors [ 4 ]. However, even if it has been developed to be completed by an examiner, MSTS score is often reported in the literature as a self-administered tool [ 22 ] or even as a mixed version with some questions completed by the patient and some others completed by the clinician.

Marchese et al. [ 23 ] and Ginsberg et al. [ 24 ] assessed functional outcomes in patients affected by lower extremity sarcomas. Both studies used the MSTS score by asking patients to complete pain, emotional acceptance and supports, while physical therapists rated gait and walking abilities.

Janssen et al. [ 25 ] first compared self- and hetero-administered modality of the original version, in a study about functional outcome after surgery in patients affected by lower and upper extremity bone metastasis. According to the authors, clinician reports overestimate function as compared to the patient perceived score. This assumption strongly differs from the outcome of the present study. The reason probably relies in the study design. As recognized by the authors [ 25 ], they collected the clinician reports by resuming information from previously noted medical records, but the score was developed to be completed at the time of consultation. As a matter of fact, the discrepancy was the largest for the common overall function and emotional acceptance domains. In the present study, patients were directly visited and interviewed by the clinician, who filled out the form during the clinical examination, thus reducing the risk of possible misinterpretations.

Looking at the results of the psychometric properties, some issues deserve further explanation.

The original version of MSTS score was never properly validated, therefore results of the present study can be only compared to other cross-cultural adaptations [ 5 – 11 , 26 ]. We observed that ceiling effect was high for all questions both for upper and lower extremity forms, which it means that the questionnaire cannot distinguish higher functioning patients. This finding is in agreement with previous studies [ 6 , 7 , 26 , 27 ]. At least two possible explanations can be attempted. First, MSTS score was developed in a time when limb savage surgery and reconstructive options were less common, and expectations on functional results were quite low. Secondly, a sensitivity analysis aiming to distinguish between hystotypes or at least between aggressive, intermediate and benign tumors could have probably lowered or partially better explained the effect. Saebye et al. [ 6 ], in their study on cross-cultural adaptation and validation of Danish version of MSTS score, found no ceiling effect among patients with lower extremity bone sarcomas or high aggressive tumors after stratification. Rebolledo et al. [ 7 ] provided cross-cultural adaptation and validation of the Brazilian Portuguese MSTS score. They only included patients who underwent limb salvage surgery or amputation for primary sarcoma of the lower limb, and no ceiling effect was reported. Unfortunately, sample size of the present study did not allow stratification for hystotype, albeit a strong and valid outcome measurement tool should be as universal as possible. Inclusion criteria of the present study were kept wide on purpose. In fact, MSTS score is widely used for any kind of tumor, and therefore, it was deemed important to be as inclusive as possible.

In agreement with previous translations of MSTS questionnaire [ 5 , 7 – 9 , 26 ], we observed that the instrument composed by all six items is able to evaluate one latent factor (e.g., lower or upper limb function).

TESS and SF-36 were chosen to test the construct validity to be consistent with the previously translated versions [ 5 , 7 – 9 , 11 , 26 ]. MSTS and TESS reported moderate to strong correlation in all studies. Results differed and became controversial when it comes to SF-36. While some studies [ 9 , 11 , 26 , 27 ] showed better correlation with the physical component of SF-36, some others did not [ 5 ]. However, the SF-36 is a tool designed and widely used as a general health and health-related QoL assessment measurement, thus this controversial correlation can be easily understood.

In terms of reliability, we found a Cronbach’s alpha coefficient > 0.95 for both upper and lower extremity in each administration modality. A general accepted rule is that Cronbach’s alpha of 0.6–0.7 indicates an acceptable level of reliability, and 0.8 or greater a very good level. However, values higher than 0.95 are not necessarily good, since they might be an indication of redundance [ 28 ]. Overall, previous cross-cultural adaptations of MSTS score showed a Cronbach’s alpha between 0.70 and 0.90. Once again, the different values of Cronbach’s alpha in the present study are possibly due to indirect influence from external factors such as heterogeneity of study population [ 20 ].

The present study has some limitations. First, as already mentioned, a sensitivity analysis could have been clarified some controversial psychometric properties. However, it must be highlighted that results are comparable to other cross-cultural adaptations of the questionnaire. Therefore, the major flaw is probably that the original version has never been properly tested. At the same time, as the MSTS score is the most popular and widely used, it was mandatory to provide an Italian version. The added value of the study relies on its main purpose: the comparison between self- and hetero-administration of the score. Second, only patients attending the outpatient clinic were asked to participate in the study. It must be considered that somehow patients with progressive disease or unsatisfied patients are less likely to be available for testing in the same setting.

In conclusion, the Italian version of the MSTS is a valid tool to evaluate outcomes of surgical treatment of patients affected by extremities tumors and it can be used as a subjective tool for both lower and upper extremity.

Appendix 1

See appendix Tables 9 and 10 .

Appendix 2

See appendix Tables 11 and 12 .

Appendix 3

See appendix Tables 13 and 14 .

Appendix 4

See appendix Tables 15 and 16 .

Appendix 5

See appendix Tables 17 and 18 .

Appendix 6

See appendix Tables 19 and 20 .

Appendix 7

See appendix Tables 21 and 22 .

Authors contribution

All authors contributed to the study conception and design. Prof. Giuseppe Milano, Dr.ssa Maristella F. Saccomanno, Dr Marco Paderno and Dr.ssa Francesca Milano were involved in the conceptualization, Prof. Giuseppe Milano and Dr.ssa Maristella F. Saccomanno contributed to the methodology, Dr. Marco Paderno and Dr.ssa Arianna Rizzo assisted in the formal analysis and investigation, Dr.ssa Arianna Rizzo contributed to the writing—original draft preparation, Prof. Giuseppe Milano and Dr.ssa Maristella F. Saccomanno were involved in the writing—review and editing, Prof. Giuseppe Milano assisted in the supervision. All authors read and approved the final manuscript.

Funding

Open access funding provided by Università degli Studi di Brescia within the CRUI-CARE Agreement.

Declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved by the Bioethics Committee of the Medical University of Brescia (NP 4912).

Informed consent

Informed consent was obtained from all individual participants included in the study.

Location where the work was performed

Department of Bone and Joint Surgery, Spedali Civili, Brescia, Italy.

Footnotes

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

1. Valderas JM, Kotzeva A, Espallargues M, et al. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res. 2008;17:179–193. doi: 10.1007/s11136-007-9295-0.

2. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13:211. doi: 10.1186/1472-6963-13-211.

3. Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S (2015) Patient reported outcome measures in practice

4. Enneking W (1993) A system for the functional evaluation of reconstructive procedures after surgical treatment of tumors of the musculoskeletal system. 286:241–246

5. Bolia IK, Savvidou OD, Kang HP, et al. Cross-cultural adaptation and validation of the Musculoskeletal Tumor Society (MSTS) scoring system and Toronto Extremity Salvage Score (TESS) for musculoskeletal sarcoma patients in Greece. Eur J Orthop Surg Traumatol. 2021 doi: 10.1007/s00590-021-02921-5.

6. Saebye CKP, Keller J, Baad-Hansen T. Validation of the Danish version of the musculoskeletal tumour society score questionnaire. World J Orthop. 2019;10:23–32. doi: 10.5312/wjo.v10.i1.23.

7. Rebolledo DCS, Vissoci JRN, Pietrobon R, et al. Validation of the Brazilian version of the Musculoskeletal Tumor Society rating scale for lower extremity bone sarcoma. Clin Orthop. 2013;471:4020–4026. doi: 10.1007/s11999-013-3211-4.

8. Xu L, Li X, Wang Z, et al. Functional evaluation for patients with lower extremity sarcoma: application of the Chinese version of Musculoskeletal Tumor Society scoring system. Health Qual Life Outcomes. 2017;15:107. doi: 10.1186/s12955-017-0685-x.

9. Uehara K, Ogura K, Akiyama T, et al. Reliability and validity of the Musculoskeletal Tumor Society Scoring system for the upper extremity in Japanese patients. Clin Orthop. 2017;475:2253–2259. doi: 10.1007/s11999-017-5390-x.

10. Mallet J, El Kinani M, Crenn V, et al. French translation and validation of the cross-cultural adaptation of the MSTS functional assessment questionnaire completed after tumor surgery. Orthop Traumatol Surg Res. 2023;109:103574. doi: 10.1016/j.otsr.2023.103574.

11. Ocaktan B, Deveci MA, et al. Cross-cultural adaptation and validation of the Turkish version of the Musculoskeletal Tumor Society scoring system in patients with musculoskeletal tumors. Acta Orthop Traumatol Turc. 2021;55:141–146. doi: 10.5152/j.aott.2021.20195.

12. Enneking WF, et al. A system for the surgical staging of musculoskeletal sarcoma. Clin Orthop Relat Res. 1980;153:106–120. doi: 10.1097/00003086-198011000-00013.

13. Enneking W (1987) Modification of the system for functional evaluation of surgical Management of Musculoskeletal Tumors. N. Y. NY Churchill-Livingstone

14. Karnofsky DA, Burchenal JH (1949) Evaluation of chemotherapeutic agents. Clin Eval Chemother Agents Cancer 191–205

15. Beaton DE, Bombardier C, Guillemin F, Ferraz MB. Guidelines for the process of cross-cultural adaptation of self-report measures. Spine. 2000;25:3186–3191. doi: 10.1097/00007632-200012150-00014.

16. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46:1417–1432. doi: 10.1016/0895-4356(93)90142-N.

17. Rossi L, Boffano M, Comandone A, et al. Validation process of Toronto Exremity Salvage Score in Italian: a quality of life measure for patients with extremity bone and soft tissue tumors. J Surg Oncol. 2020;121:630–637. doi: 10.1002/jso.25849.

18. Apolone G, Mosconi P. The Italian SF-36 health survey: translation, validation and norming. 12

19. Davis AM, Wright JG, Williams JI, et al. Development of a measure of physical function for patients with bone and soft tissue sarcoma. Qual Life Res. 1996;5:508–516. doi: 10.1007/BF00540024.

20. Ursachi G, Horodnic IA, Zait A. How reliable are measurement scales? External factors with indirect influence on reliability estimators. Procedia Econ Finance. 2015;20:679–686. doi: 10.1016/S2212-5671(15)00123-9.

21. Artino AR, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach. 2014;36:463–474. doi: 10.3109/0142159X.2014.889814.

22. Downie S, Stillie A, Moran M, et al. Patient-reported assessment of outcome after surgery for bone metastases. Orthop Rev. 2021 doi: 10.4081/or.2021.9062.

23. Marchese VG, Rai SN, Carlson CA, et al. Assessing functional mobility in survivors of lower-extremity sarcoma: reliability and validity of a new assessment tool. Pediatr Blood Cancer. 2007;49:183–189. doi: 10.1002/pbc.20932.

24. Ginsberg JP, Rai SN, Carlson CA, et al. A comparative analysis of functional outcomes in adolescents and young adults with lower-extremity bone sarcoma. Pediatr Blood Cancer. 2007;49:964–969. doi: 10.1002/pbc.21018.

25. Janssen SJ, van Rein EAJ, Paulino Pereira NR, et al. The discrepancy between patient and clinician reported function in extremity bone metastases. Sarcoma. 2016;2016:1–6. doi: 10.1155/2016/1014248.

26. Iwata S, Uehara K, Ogura K, et al. Reliability and validity of a Japanese-language and culturally adapted version of the musculoskeletal tumor society scoring system for the lower extremity. Clin Orthop Relat Res. 2016;474:2044–2052. doi: 10.1007/s11999-016-4880-6.

27. Wada T, Kawai A, Ihara K, et al. Construct validity of the Enneking score for measuring function in patients with malignant or aggressive benign tumours of the upper limb. J Bone Jt Surg. 2007;89:5. doi: 10.1302/0301-620X.89B5.18498.

28. Hulin C, Netemeyer R, Cudeck R (2001) Can a reliability coefficient be too high? 10(1):55–58