Challenges in the cultural adaptation of the German Myeloma Patient Outcome Scale (MyPOS): an outcome measure to support routine symptom assessment in myeloma care.

Background

With the ageing of society and an increasing incidence [ 1 – 3 ], cancer is a major public health concern. Haematological cancers and multiple myeloma (MM) in particular exemplify this changing face of cancer with conditions whose management resembles that of a chronic illness, with recurrent treatment patterns followed by maintenance therapy [ 4 , 5 ]. MM is an incurable cancer of the bone marrow characterised by bone destruction, bone marrow failure with anaemia, immune deficiency, and renal insufficiency [ 6 ]. It belongs to the heterogeneous group of plasma cell dyscrasias, which vary from asymptomatic forms to malignant disease with severe end-organ damage and high patient morbidity [ 7 ]. Front-line treatment with high-dose chemotherapy and hematopoietic stem cell transplant (HSCT) has improved the median survival for those under the age of 65 to 5 years or longer after a myeloma diagnosis [ 8 ]. However, due to its incurability, patients must cope with the incrementally progressive nature of the disease, interspersed with intervals of stable disease and maintenance treatment, while also experiencing long-lasting effects of previous treatments [ 9 , 10 ]. Moreover, the median age at diagnosis is 69 years, with more than 60% of multiple myeloma patients aged 65 years or older at diagnosis. Due to the aging baby boomer generation, an increase in the proportion of the general population older than 65 years of age is expected between 2020 and 2030. Therefore, in tandem with new and better treatment options, the incidence and prevalence, and subsequently also deaths due to multiple myeloma, will inevitably increase [ 3 ].

Therefore, MM patients represent a group of older, haematological cancer patients faced with complex treatment pathways, a long duration of a chronic, yet life-threatening disease with repeated relapses and high levels of uncertainty around disease and treatment progression [ 11 – 13 ]. This results in a high prevalence of physical and psychosocial symptoms and problems throughout the disease trajectory. MM patients may suffer more symptoms and problems than other patients with haematological malignancies. On average, several cross-sectional surveys have shown a mean of 5.6 symptoms, of which 2.3 were rated as severe [ 9 ]. A recent meta-analysis showed high prevalence rates for pain and fatigue as well as limitations in aspects of quality of life (QoL) such as physical, role, and social functions [ 14 ].

Haematological and psychosocial care of MM patients could be improved by incorporating a longitudinal assessment of symptoms and QoL into routine clinical practice. The routine use of QoL measures allows monitoring of symptoms/QoL, thus leading to better symptom control, improved communication, and higher patient satisfaction [ 15 , 16 ]. However, few measures have been designed for monitoring QoL in the routine clinical setting; a systematic review of 13 generic and disease-specific health-related QoL measures for multiple myeloma found no single tool developed or validated specifically for routine clinical care [ 17 ]. The most common measures, the Functional Assessment of Cancer Therapy-Multiple Myeloma questionnaire (FACT-MM) [ 18 ], the EORTC QLQ-MY20 [ 19 ], and the M. D. Anderson Cancer Centre Multiple Myeloma measure (MDASI-MM) [ 20 ] are disease-specific measures that have been developed for use in clinical trials. Currently, only the EORTC QLQ-MY20 is available in German [ 21 ].

The Myeloma Patient Outcome Scale (MyPOS) is a questionnaire developed and validated specifically to measure disease-specific QoL in patients with MM in a routine clinical setting and for monitoring purposes. The initial validation comprised 380 patients at different disease stages in the UK [ 22 ]. The MyPOS is a module of the Integrated Palliative/Patient care Outcome Scale (IPOS) [ 23 – 25 ], a short, multidimensional questionnaire to assess palliative care concerns in patients with advanced disease. The MyPOS takes the core items of the POS and extends them with myeloma-specific concerns. The MyPOS comprises a list of 13 symptoms and 20 QoL items which are scored on a 5-point Likert scale and summed into a total score and three subscale scores. Content and construct validity as well as reliability of the MyPOS have been established in clinically representative samples of all disease stages [ 22 , 26 , 27 ]. Its reliability and responsiveness in longitudinal monitoring of QoL have been shown to be satisfactory to good [ 27 ].

However, the conceptualization of QoL for patients with MM used in the original MyPOS is UK-focused and must be checked before a German translation of the instrument can be used in practice. Differences in care processes, treatment pathways and cultural background may affect its cross-cultural equivalence [ 28 , 29 ]. We therefore aimed to explore the cultural, linguistic, and contextual issues during the adaptation of the MyPOS to the German context. Our aim was to translate and cross-culturally adapt the MyPOS and establish its content and face validity for monitoring QoL in MM patients.

Methods

This study used a multi-step, explorative and sequential mixed-methods design [ 30 , 31 ]. The procedure for the translation and cultural adaptation of the MyPOS followed the guidelines reported in the manual for cross-cultural adaptation and validation of the parent’s measure POS development group [ 32 ], based on commonly accepted standards for cross-cultural adaptation and psychometric testing by the European Organization for Research and Treatment of Cancer (EORTC) and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) [ 33 , 34 ]. Because there are already German translation versions of the POS and the IPOS and therefore challenges with adapting items to the German language and healthcare context are already known, we tailored the proposed, six-step process towards supporting a wider exploration of issues of cultural equivalence instead of focusing on psychometric evaluation (see Fig. 1 ). The cultural adaptation of MyPOS was further informed by the Cultural Equivalency Model for Translating and Adapting Instruments [ 35 ], focusing on conceptual (Does the content relate to constructs in the culture?), content (Is the content of each item relevant?), semantic (Is the meaning of the item’s wording the same?), and technical equivalence (Do layout, format, and answer options work the same way?). Fig. 1 Six-step process of translation and cross-cultural adaptation of the MyPOS based on EORTC and ISPOR standards [33, 34]. (EORTC: European Organization for Research and Treatment of Cancer; HCP: healthcare professional; ISPOR: International Society for Pharmacoeconomics and Outcomes Research)

Initial phases: translation, conceptual equivalence, focus groups, and expert review

Conceptual definitions and equivalence of key concepts can be identified by an array of methods. Given the nature of haematological care within Germany, we opted to hold focus groups with the target population, consisting of patients, family members, and healthcare professionals (HCP) to discuss cultural equivalence of key concepts. The combined use of cognitive interviews and focus groups is the recommended best practice for establishing content validity in both new and existing patient-reported outcome measures [ 36 ]. The combined use of these methods helps confirm the validity of results through triangulation [ 37 ].

Two focus groups, one with patients and their family members ( n = 5) and one with HCPs ( n = 6), were held. Recruitment for the patient and family member focus group followed same eligibility criteria and procedures as specified below for cognitive interviews (phase 5). All groups were chaired by CG and were audio recorded. The topic guide was partly based on the topic guides used in the development of IPOS [ 25 ] and MyPOS [ 38 ] and included discussions about what constitutes QoL in multiple myeloma, the dimensions of QoL, feedback on MyPOS as a measure, aspects of layout, and feedback on MyPOS single items (see Additional file 2 ). The clinical value of QoL questionnaires was also discussed as well as possible ways in which these measures could be implemented into routine myeloma care [ 39 ].

The discussion of cultural equivalence was preceded by the translation of MyPOS to provide a base for feedback on MyPOS. Forward translation (phase 1) of the MyPOS was performed by two independent, native German speakers from different backgrounds (a medical student proficient and fluent in English and a palliative care clinician). Both translations were reviewed for discrepancies. Consensus was reached by discussion with a third, independent researcher not involved in the initial forward translation. MyPOS as a myeloma-specific module includes the original IPOS items that have already been validated in German [ 25 , 40 ]. The translated items were used but checked by both translators. The backward translation (phase 2) was carried out by two native English speakers who were blinded to the original English version. They independently back-translated the questionnaire from German to English. One back translator had a nursing background and patient experience, and the other translator had an epidemiology background. Again, discrepancies were reviewed and resolved by consensus discussions with a third, independent researcher. A record of all items and aspects challenging conceptual, semantic, content, and cultural equivalence was kept and discussed in an expert review (phase 3). The multidisciplinary group included researchers with knowledge in haemato-oncology and palliative care, all the translators, and a statistician/psychometrician (CR). Based on the discrepancies noted during the translation process, a consensus on content, instructions, wording of items, and response options was reached. We used criteria specified by Koller et al. [ 41 ] to characterise changes made during this process of reconciliation. The consolidated version was then pre-tested in cognitive interviews (phase 5).

Phase 5: cognitive interviews

Cognitive interviews ( n = 9) were performed in a separate sample of myeloma patients. As recommended by guidelines [ 42 , 43 ], these semi-structured interviews served the purpose of further testing the translated and culturally adapted version by using verbal probes and recording cognitive processes (think-aloud) during completion of the questionnaire. Cognitive interviews were supported by a topic guide (see Additional file 3 ) and complemented the focus group discussions (phase 4) in terms of content. They were based on Tourangeau’s four-stage question response model [ 44 ], as adapted for cognitive interviewing by Willis [ 30 , 42 ]: comprehension, retrieval, judgement, and response formulation. The interviews addressed the patient’s comprehension of the instructions, items and response options, clarity and layout of MyPOS, its length, and difficulties in understanding and answering the questions. Participants were asked to elaborate on reasons why items were perceived as difficult, and to make suggestions as to how instructions, items, and response options could be rewritten to achieve better clarity. Verbal probes concerned missing aspects of QoL in the MyPOS, its acceptability and clinical utility, and the burden associated with its completion.

Patients were recruited from the haematology department of the university hospital, local private practices, and the regional self-help group (LHRM, Leukämiehilfe Rhein-Main). Patients were approached individually, by their treating physician, and via an information leaflet in outpatient clinics. Inclusion criteria were selected in order to approach patients in need of myeloma treatment: a confirmed diagnosis of multiple myeloma (MM) Salmon & Durie stage III, high-risk smoldering MM with positive CRAB-criteria (hyperCalcemia, Renal failure, Anemia, Bone lesions) [ 6 ], and any other MM stage triggering supportive therapy. Patients had to be age 18 years or older, sufficiently fluent in written and spoken German, and had to have the capacity to give written informed consent. Exclusion criteria were those too unwell or distressed to participate as judged by their clinical team; any cognitive or communication impairment; and being close to death within the next couple of days. Participants were purposively sampled to achieve maximum variation across the key characteristics: gender, age (</≥ 65 years), stage of disease, and Australia-modified Karnofsky Performance Status (AKPS) [ 45 ] with low (≤ 60%), medium (70 and 80%) and high (≥90%) performance status.

Eligible patients were provided with written information about the study and, if willing to participate, gave written informed consent. Interviews were audio-recorded and transcribed verbatim completely for focus groups; cognitive interview quotes were analysed from the interview records.

Data analysis

Interviews were analysed using thematic content analysis for the development and testing of survey questionnaires [ 37 , 46 ]. For the focus groups, transcripts were analysed based on an initial coding frame developed by CG. Themes regarding missing QoL aspects in the MyPOS, acceptability, clinical utility, and burden were developed inductively from the material. The analysis aimed towards saturation with no new themes emerging, evaluated through a saturation grid [ 47 ].

Feedback on instructions, individual items, and answer options of the MyPOS were recorded in a standardised spreadsheet in Excel, with participant numbers across the top and questionnaire items/attributes listed down the left-hand column. The deductive analytical approach followed Tourangeau’s question response model (comprehension, retrieval, judgment, response formulation). The results from focus groups and cognitive interviews were aggregated separately for each item. The analysis was performed by one researcher (CG), discussing discrepancies with a second researcher (CR) to reach consensus.

Written summaries of the results were used for a final expert review as guidance for making changes to the questionnaire. Proposals for changes were reported to and approved by the POS development team ( phase 6 ).

Ethical issues

All participants provided written informed consent. The study protocol was approved by the Ethics Committee of the Medical Council of the federal state of Rhineland-Palatine (837.109.17(10943) on 24 Mar 2017).

Results

Demographics

A total of 20 participants, n = 11 in two focus groups and n = 9 in cognitive interviews, were recruited into the study from April to October 2017. Of the eligible patient participants approached, three declined to participate because of a scheduled transplantation ( n = 1), sepsis ( n = 1), and time constraints ( n = 1). Two patients wanted to include their closest relative, resulting in two family members taking part in the first focus group, consisting of 3 women, and 2 men. Six HCPs, a nurse from the haematology department, a hospice nurse working in the community, a physician specialized in haemato-oncology, and three psychologists in psycho-oncology, participated in the second focus group. Another physician and the social worker were scheduled to take part but cancelled at short notice because of patient care obligations. In addition, two psychologists volunteered to take part, resulting in a disproportionate distribution of professions in the HCP focus group.

The demographic background of those patients taking part in focus groups and cognitive interviews represented the typical patient population in terms of age, marital status, and education. However, no patient with a migration background could be recruited. The HCP focus group included one participant with a background other than Christian. Sociodemographic data are presented in Table 1 . Table 1 Baseline characteristics of participants for all phases Focus group: patients & relatives Focus group: Healthcare professionals Cognitive interviews: patients ( n = 5) ( n = 6) ( n = 9) Sex (Women: men) 3: 2 3: 3 4: 5 Age Median: 63 Range: 55–79 30–39 1 40–49 0 2 50–59 2 3 60–69 1 1 70–79 1 Religious affiliation 5 Christian 2 Christian; 1 Muslim; 2 none; 1 n.d. 7 Christian; 2 n.d. Marital status Married 4 4 6 Widowed – – 1 Divorced – – 1 Missing 1 2 2 Education Secondary school graduate 3 – 2 Technical qualification – 2 4 University degree 2 2 1 Ph.D. and higher degree – 2 – Missing – – 2 Professional experience (years) 16–33 Time since diagnosis (years) – – Median: 1 Range: 0.5–10 First line treatment – – 4 Plateau phase 2 after HSCT – 3 Relapsed after transplantation 1 – 1 Progressive disease – – 1 Performance status (AKPS) 60% – – 2 70% – – 4 80% – – 1 90% – – 2 AKPS Australia modified Karnofsky Performance Status, HSCT Hematopoietic stem cell transplantation, Ph.D.: Philosophical Doctorate, n: sample size; n.d.: no data

The duration of the focus groups was 60–150 min. Time to complete the MyPOS was 10–15 min on average. The mean duration of the cognitive interviews was 54 min (minimum 24 min, maximum 100 min), and took place in a meeting room of the palliative care department ( n = 3), at a private practice ( n = 2), on the transplantation ward ( n = 2), and at the patient’s home ( n = 2).

The presentation of results follows the cultural equivalency model with the dimensions conceptual, content, semantic, and technical equivalence. The original items with results pooled from focus groups, expert review, and interviews and potential revision of an item or aspect of the questionnaire are shown in Table 2 . Table 2 Issues regarding MyPOS items identified in focus groups ( n = 11) and cognitive interviews ( n = 9) # Original MyPOS item Subscale/ original POS item? Issues Item revised? 1 What have been your main problems and concerns at the moment? −/ POS The patients wished for a free text field instead of the original enumerated list, and the question line was shifted into the free-text field in order to avoid patients overlooking the question. Yes 2 Below is a list of symptoms, which you may or may not have experienced. For each symptom please tick one box that best describes how it has affected you over the past week. Patients differentiated between problems associated with illness and with therapy with a tendency to overlook therapy side effects. In order to clarify, we added “Below is a list of symptoms and side effects”. Yes The time frame of 1 week was perceived to be too short by patients and HCP. Patients reported about serious issues that happened sometime in the past - severity of the issue was the pivotal factor rather than point of time. Experts discussed accepting that patients may also report issues from the past still relevant to their current situation to inform the consultation. No Pain Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Shortness of breath Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Weakness or lack of energy Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Nausea (feeling like you are going to be sick) Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Vomiting (being sick) Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Appetite loss Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Constipation Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Sore or dry mouth Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Drowsiness Symptoms/ POS One patient wanted to add sleeping problems, which she discovered to do in the list thereafter. No Poor mobility Symptoms/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No Diarrhoea Symptoms Good comprehension, retrieval, judgement, and response choice by all patients. No Tingling in hands/feet Symptoms Good comprehension, retrieval, judgement, and response choice by all patients. No Problems remembering Symptoms One patient differentiated between problems remembering things and concentration problems. This was not addressed by other focus group members. No. Over the past week… 3 Have you been feeling anxious or worried about your illness and treatment? Emotions/ POS Expert discussion whether to delete “illness” or both “illness and treatment” because of content overload and Q17 also addressing illness worries. Some patients found that worries about treatment differed from illness. The parent instrument developer team confirmed deletion of “illness” from the German Q3. Yes 4 Have any of your family or friends been anxious or worried about you? Emotions/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No 5 Have you been feeling depressed? Emotions/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No 6 Have you felt at peace? Emotions/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No 7 Have you been able to share how you are feeling with your family or friends as much as you wanted? Emotions/ POS Good comprehension, retrieval, judgement, and response choice by all patients. No 8 Have you had as much information as you wanted? Emotions/ POS Patients from the German population and the generation/ age of typical myeloma manifestation may prefer to receive no or less information in general or currently. Further testing involved showing patients the former POS answer options for that item. All but one patient preferred the latter. Yes 9 Have any practical problems resulting from your illness been addressed? (such as financial or personal) Emotions/ POS This item proved to be the most challenging in the translation. Although most patients understood it well and reported no problems during cognitive interviews, there was a high potential for misunderstanding with this item. Participants proposed asking this question in a simple and more direct way. The consented adaptation now reads: ‘Do you wish you had support for practical problems that have arisen from your disease (e.g. financial or personal issues)?’ Yes 10 Have you been able to carry out your usual activities without help from others? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 11 Have you been able to pursue your hobbies and leisure activities? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 12 Have you been able to spend quality time with family and friends? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No We would like you to answer this question whether or not you are sexually active. If you prefer not to answer then please tick here: Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 13 Have you been worrying about your sex life? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 14 Have you been worrying about infections? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 15 Have you been worrying about your physical appearance? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 16 Have you been worrying about your financial situation? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 17 Have you been worrying that your illness will get worse? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 18 Have you felt able to cope with your illness and treatment? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No 19 Are you able to contact your doctors or nurses for advice if needed? Support Interviewed patients, family, and staff argued for further differentiating these three questions. It was felt that their content does not directly address the patients’ QoL. Patients found doctors and nurses should be asked about separately. However, they were concerned about inadvertently blaming members of the health care profession: patients meet a high number of HCP and the questions ask to make a judgment for this whole group, contrary to the real-world situation in which patients might only have had one bad encounter. Overall, these concerns could very well lead to diminished acceptance of the MyPOS with patients and their families as well as HCP and lead to non-responses. Therefore, these items were removed. Yes 20 Do your doctors and nurses show a good standard of knowledge and skill when treating you? Support Yes 21 Do your doctors and nurses show care and respect when treating you? Support Yes 22 Do you have enough information about what might happen to you in the future? Emotions Good comprehension, retrieval, judgement, and response choice by all patients. No End of question-naire Some patients wished for a free-text field at the end of the questionnaire due to this being the place to expect them. A note referring to the free-text option at the beginning of MyPOS would help preserve the open-ended focus on the patient’s main problems at the beginning of MyPOS and facilitates the use of the free-text field for issues important to the patient but not addressed in the questionnaire. Yes

Conceptual equivalence

Overall, patients in cognitive interviews and focus groups, their family members, and HCPs all confirmed that the MyPOS is a comprehensive measure and includes all items relevant to capture disease-specific QoL in multiple myeloma. No patient or family member felt that important aspects of the QoL experience were missing from the questionnaire. Regarding redundant or unnecessary items, items Q19 to Q21, advice, knowledge, care and respect from doctors and nurses , were identified by patients and HCPs as not belonging to the QoL construct, but rather measuring patient satisfaction. This was interpreted to be a separate issue. These questions were also felt to hamper return of the MyPOS due to these questions asking the respondent to make a global judgment of the whole HCP group. Patients feared that such feedback could be perceived as a criticism of their HCPs and could make them seem ungrateful for their care. This led to concerns regarding the confidentiality of the questionnaire and its use in daily clinical practice. It was also advised to enable separate judgments regarding doctors and nurses. After pooling results from all phases, it was decided to remove Q19 to Q21 from the German translation in concordance with the POS development team.

Content and semantic equivalence

In the expert review and cognitive interviews, the instructions for Question 2 were criticized. These instructions specify that the following list of symptoms should be scored according to how much the patient had been affected by each symptom in the past week. Patients were found to be at risk for underreporting subsequent symptoms as they distinguished between symptoms arising from their illness and symptoms arising from their treatment. It was decided that the instructions needed to be more precise, and the phrase “symptoms and side effects” was added to the instructions.

Questions 8 and 22, having as much information as wanted and having enough information about what might happen in the future , posed problems in the translation process because of a lack of answer options for patients preferring less information. HCPs in the focus group further elaborated on patient-centred care also entailing honouring patients’ and families’ wishes for less information: “No, but it’s about killing people with information, and I think that’s a legitimate question, because people hear what kind of disease they have, and then they’re fed tens of thousands of pieces of information that go right in and out anyway, right? ” (HCP 2, nurse). Following the discussion around answer options and information needs, patients in the focus groups also suggested adding to the instructions for completion of the MyPOS and making it clearer that patients were allowed to not answer questions if they felt concerned or unable to deal with a problem. Although the MyPOS questionnaire ends with instructions for the patient to speak to a member of a clinical team if concerns about any of the issues raised persist, it was felt that these instructions had to be presented earlier in the questionnaire. Patients also proposed to open up the item information needs so that it also includes their informal caregivers: ‘Have your family or confidential persons had as much information as they wanted?’

Question 5, feeling depressed, challenged semantic equivalence in the sense that the term depression was perceived by patients in the focus groups and cognitive interviews as referring to psychiatric disease. This item had already been translated into German during the cultural adaptation of the parent measure (IPOS). However, the existing translation was deemed inappropriate and containing ambiguous wording. An adapted translation was tested in the cognitive interviews and the item now translates as: ‘ Have you been feeling sad or depressed/gloomy? ’.

Questions (Q) addressing practical issues and financial worries (Q9 and Q16) were found to be challenging in terms of understanding and were perceived to be intrusive. One patient in particular highlighted this issue during the cognitive interview. A more direct and non-disturbing rephrasing was discussed in the final expert review: ‘Do you wish you had support for practical problems that have arisen from your disease (e.g. financial or personal issues)? ’.

Among the QoL items of the MyPOS, item Q12, being able to spend quality time with family and friends , could not be translated verbatim into German as no such concept as quality time exists in the German language. The verbatim translation has a connotation of duration rather than quality of the time spent together. A translation of “spending time together” was perceived as not mirroring the aspect of enjoyment. The proposed revised translation which was cognitively tested therefore specified whether the patient was able to cherish those moments s/he spent with family and friends.

The MyPOS contains an item on sexual well-being (Q13). Patients in the focus groups and cognitive interviews welcomed this question and did not feel uncomfortable or embarrassed answering this. In contrast sexuality was controversial when discussed in the HCP focus group. While the importance of this aspect was not denied, it was also perceived to be an embarrassing issue that could hamper completion of the questionnaire.

“I don’t think patients dare to address it with the doctors. Maybe they also have feelings of shame, which is easier with the nurses. So, you are somehow without fear or favour. It’s a bit more distant with the doctors, I think, there are a lot of questions coming up towards us. Can I have sex with my partner at all? What do I have to pay attention to? Can I hurt her or him? And I think these are very, very big points. Something that also moves the patients, because there are also many younger ones. They’re even afraid to cuddle up with each other, because it’s always said, watch out because of the risk of infection and I think that’s really bad.” (HCP 2, nurse).

Technical equivalence

The layout of the original questionnaire only needed to be improved for Q1, main problems and concerns . Patients advised using a free-text field instead of three separate lines. A box was added to help patients not overlook this item. Response options were perceived to work well, except for items Q8 and Q22, information needs , and item Q9, practical matters . Following the revision of these items, the answer options were adapted accordingly (see Table 2 ). Q19 to Q21 in particular generated a discussion on gender issues. German is a language with gender-specific nouns, thus having female and male versions of the English ‘doctor’ and ‘nurse’. Appropriating doctor with ‘he’ and nurse with ‘she’ conveys an outdated gender stereotype. However, mentioning both the female and male version can make the text cluttered. It was therefore opted to include the general term for healthcare staff.

The original time frame of the MyPOS is asking about the past week. Different alternative time frames were discussed in the patient focus group and the cognitive interviews. It was noted that while a time frame of 1 week might work well with physical symptoms, emotional issues might need a longer time frame. Patients also reported that severe symptoms that happened to be beyond the time frame of 1 week would be reported nonetheless. After much discussion, the one-week time frame was concluded to be the most acceptable.

Due to polyneuropathy, patients preferred a paper/pencil version of the MyPOS presented on a clipboard rather than a tablet computer version of the questionnaire. They felt that completion before a clinical visit would help them identify the critical issues they would like to discuss with their doctor. The tool was perceived as an aid but not as a substitute for the consultation. Patients in both focus groups and cognitive interviews advocated for developing a caregiver-reported version of the MyPOS. The MyPOS is presented in Additional file 1 . The original English and the revised German version of the MyPOS are available for download at https://pos-pal.org/ .

Discussion

In this study, we translated and culturally adapted the Myeloma Patient Outcome Scale, a patient-reported symptom and QoL tool to support monitoring of patient-centred issues in routine clinical practice. The MyPOS is the first disease-specific myeloma questionnaire available in German that supports both clinical assessment/monitoring as well as use within research studies. The only other available translation of a health-related QoL questionnaire, the EORTC QLQ-MY20 only offers utility within research [ 21 ]. The German translation of the MyPOS has been shown to possess content/face validity and acceptability for patients and staff. However, certain adaptations honouring patient-centred wishes for more or less information, separating patient satisfaction from QoL, and involving family members in the assessment process were needed to achieve cultural equivalence within a German healthcare context. The questionnaire was perceived as clinically important in preparing and guiding individual clinical encounters with HCPs and helping patients raise embarrassing or challenging issues with their HCPs. MyPOS may help to direct and focus conversations when employed longitudinally. The sequential use of focus groups to establish conceptual equivalence and cognitive interviews to cognitively test individual items benefitted the content validity by stressing separate issues regarding the comprehension and utility of the questionnaire.

Several of the issues regarding comprehension and equivalence of individual items observed in this study have also been reported in recent cultural adaptation studies of the parent measure, the IPOS. Our study also found more problems with the adaptation of items from the parent measure than from the myeloma-part of the measure. Comprehension issues consistently reported in the literature concern item Q5, feeling depressed . The association of the term ‘depressed’ with a medical diagnosis was also noted in a study testing a Swedish version of the IPOS [ 48 ]. Similar to our finding, the Swedish research group opted for a colloquial term. They found that this translation was better fitted towards eliciting the patient’s mood. Some authors argue for directly asking patients whether they think they are depressed rather than opting for a wording of feeling depressed . However, results are inconsistent [ 49 , 50 ]. Similar findings have been reported in cognitive interviews during the German translation of the IPOS [ 25 ] and its Italian translation [ 51 ]. The issue around culturally adapting Q9 as well as Q19, both referring to problems of a financial or personal nature being addressed, was also reported in the French and Italian translation of Q9 in the IPOS [ 51 , 52 ]. Additional problems concerning the response options have been described.

Items in the healthcare support subscale caused the most concerns from a content and face validity point of view. Patients identified issues regarding these questions measuring a construct other than QoL and issues regarding wording, adaptation and the global nature of these questions. Both the cross-sectional and longitudinal validity studies of the English MyPOS [ 22 , 26 ] reported poor psychometric criteria for these items, with pronounced ceiling effects in the scale and poor reliability. Similar issues were found for a set of healthcare satisfaction items that were part of the original 24-item version of the EORTC QLQ-MY20 [ 53 ]. Due to the same reasons, these items were removed from subsequent versions of the questionnaire. Despite patient satisfaction being assigned a central role in patient outcomes [ 54 ], it is regarded as a patient experience measure rather than an outcome measure and thus perceived as distinct from QoL as a construct [ 55 ]. However, healthcare satisfaction and QoL do seem to overlap. The addition of these items to the original MyPOS was based on findings from the qualitative interviews to develop the measure [ 38 ]. In stem cell transplant populations, it has also been reported that patients with higher levels of satisfaction with medical care reported higher levels of QoL, despite ongoing physical and psychosocial morbidity after transplant [ 56 ]. The main critique in this German sample centred on worries of negative satisfaction statements sending the wrong message to HCPs and jeopardizing the long-lasting relationships with doctors and nurses who care for these patients throughout their illness trajectory. Therefore, the negative connotation of these items might well point towards different care models and patient experiences with care in the German context.

This less fragmented care model that was reported by patients in our study also serves the clinical utility of the MyPOS. Unlike the MDASI-MM and the EORTC-QLQ-MY20, the MyPOS contains more items regarding worry about the future, information needs, and coping processes as well as adaptation processes, all relevant to and reflecting the prolonged disease trajectory. These issues have also been highlighted as important in recent qualitative studies focusing on the advanced myeloma population [ 11 – 13 , 57 , 58 ]. The MyPOS was developed as a tool to go beyond the simple assessment of symptom status and asking for an evaluation of physical, emotional, spiritual, and QoL morbidity. Outcome measures to be used in routine clinical care have been shown to be powerful instruments to improve wellbeing and outcomes of cancer patients as well as improve communication with HCPs [ 15 ]. These tools cannot replace the interventions required to meet patients’ and their family caregiver’s needs [ 59 , 60 ], but they can help to identify and monitor high levels of unmet supportive care needs and help integrate those services into care [ 61 , 62 ]. The interface of supportive care and PC with haematology differs from the one with oncology because MM patients usually have a long and close relationship with the haematology team. LeBlanc & El-Jawahri even proposed unifying PC with haematology [ 63 ] by having haemato-oncological staff with a qualification in PC follow a primary PC approach supported by close collaboration with PC specialists when required, as indicated e.g. by patient-reported outcomes and joint case reviews. Porta-Sales et al. demonstrated the benefit of a cooperative PC-haematology approach for outpatients guided by impaired QoL in a retrospective analysis of their Multiple Myeloma Palliative Care Clinic [ 64 ]. The MyPOS could support such a care model.

Methodological limitations

The sample size for this study can be considered a limitation. Both the focus group study as well as the cognitive interviews would have benefitted from a larger number of patients taking part and being interviewed. However, we followed a purposive sampling approach using a sampling matrix to ensure maximum variation. We also aimed to represent the diversity of care providers in the HCP sample, which was challenged due to the high workloads, resulting in a disproportionate representation of psychologists in the HCP sample. Although we managed to sample a group of patients from different age groups, myeloma patients unfit to receive active therapy or at the end of life represented by lower AKPS levels are under-represented in this study. It would also be worthwhile to sample extreme cases to explore aberrant views on the utility of outcome measures in clinical practice. However, we included multiple settings in a hospital, private practice, outpatients and inpatients, including those undergoing haematopoietic stem cell transplantation. HSCT in particular is the first haematology setting where PC was integrated successfully into the care of patients challenged by an uncertain prognosis and high, multidimensional symptom burden [ 65 , 66 ].

Conclusion

In this study, we translated and culturally adapted the Myeloma Patient Outcome Scale, a QoL questionnaire to support the assessment and monitoring of symptoms and wellbeing in routine clinical care. After adaptations to the German care context, the MyPOS demonstrated good face and content validity. MyPOS German is now ready to support successful clinical approaches to better meet myeloma patients’ needs for palliation of symptoms as well as psychosocial concerns which impact on QoL. Next steps in its German validation include formal psychometric and clinical utility testing. Further, the demand for a proxy version should be appreciated and addressed, including assessment of informal caregiver burden.

Supplementary information

Additional file 1: Myeloma Patient Outcome Scale (MyPOS) English Version. Additional file 2: Topic guide for focus groups with patients, family, and healthcare professionals about conceptual equivalence, and feedback on MyPOS as a measure, aspects of layout, and feedback on MyPOS single items. Additional file 3: Topic guide for cognitive interviews with patients addressing the patient’s comprehension and layout of MyPOS, suggestions for improvement, its acceptability and clinical utility, and the burden associated with its completion.

Abbreviations

Australia-modified Karnofsky Performance Status

Hypercalcemia, Renal failure, Anemia, Bone lesions

European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Myeloma 20

Functional Assessment of Cancer Therapy-Multiple Myeloma questionnaire

Healthcare professional

Haematopoietic stem cell transplantation

Integrated Palliative/Patient care Outcome Scale

International Society for Pharmacoeconomics and Outcomes Research

M. D. Anderson Cancer Centre Multiple Myeloma measure

Multiple Myeloma

Myeloma Patient Outcome Scale

Palliative care

United Kingdom

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information accompanies this paper at 10.1186/s12885-020-06730-7.

We thank Dr. Eva Schildmann for methodological advice, and Lara Kuerzer, Angelika Stippler, and Anita Waldmann from Leukemia Aid Rhine-Main, Patricia Meinhard, Leonie Schorrlepp, Dr. Christoph Schulz, patients and families for their time and energy to support the project.

Authors’ contributions

CG led the application for funding in collaboration with MW. CG designed the overall study. CR, MF, KT and MM contributed to the conduct of the study with CG acting as senior researcher overseeing the project. CG and CR drafted the manuscript. All other authors provided comments and critical revisions. The final manuscript was approved by all authors prior to submission.

Funding

This work was supported by intramural research funding of the University Medical Center, Johannes Gutenberg University Mainz, Germany. The funder of the study had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The corresponding author had full access to all the data and had responsibility for the decision to submit for publication.

Availability of data and materials

The datasets generated and/or analysed during the current study are not publicly available due to the large size of the transcripts from each interview and sensitivity and specificity of some of the information from the interviews, but the demographic information and some of the de-identified descriptions can be made available upon reasonable request to the corresponding author.

The MyPOS questionnaire is available for download at https://pos-pal.org/ .

Ethics approval and consent to participate

All procedures performed involving human participants were in accordance with the 1964 Helsinki declaration and its later amendments and the ethical standards of the institutional research ethics committee of the Medical Council of the federal state of Rhineland-Palatine. The study was approved by the research ethics committee of the Medical Council of the state of Rhineland-Palatine on 24 March 2017 (REC No. 837.109.17(10943). Written informed consent to participate was obtained from all individual participants included in the study.

Consent for publication

Not applicable - only de-identified data was used in the manuscript.

Competing interests

The authors declare that they have no competing interests.

References

1. Siegel RL Miller KD Jemal A Cancer statistics, 2019 CA Cancer J Clin 2019 69 7 34 30620402 10.3322/caac.21551

2. Böckmann L, Jedinger S, Kirschey T. Vierte regionalisierte Bevölkerungs-vorausberechnung [4th regionalised population forecast]: Office of Statistics Rhineland-Palatine; 2015. https://www.statistik.rlp.de/dokumente/stat_analysen/rp2060_bj2013 . Accessed 17 Oct 2019.

3. Robert-Koch-Institut. Krebs in Deutschland 2011/2012 [Cancer in Germany 2011/2012]: Robert-Koch-Institut; 2015. https://www.krebsdaten.de/Content/Krebs_in_Deutschland/kid_2015 . Accessed 17 Oct 2019.

4. Sirohi B Powles R Epidemiology and outcomes research for MGUS, myeloma and amyloidosis Eur J Cancer 2006 42 1671 1683 10.1016/j.ejca.2006.01.065 16870424

5. Rajkumar SV Multiple myeloma: 2013 update on diagnosis, risk-stratification, and management Am J Hematol 2013 88 226 235 10.1002/ajh.23390 23440663

6. Mikhael J Ismaila N Cheung MC Costello C Dhodapkar MV Kumar S Lacy M Lipe B Little RF Nikonova A Omel J Peswani N Prica A Raje N Seth R Vesole DH Walker I Whitley A Wildes TM Wong SW Martin T Treatment of multiple myeloma: ASCO and CCO joint clinical practice guideline J Clin Oncol 2019 37 1228 1263 10.1200/JCO.18.02096 30932732

7. Palumbo A Anderson KC Multiple myeloma N Engl J Med 2011 364 1046 1060 10.1056/NEJMra1011442 21410373

8. Phekoo KJ Schey SA Richards MA Bevan DH Bell S Gillett D Møller H Consultant Haematologists, South Thames Haematology Specialist Committee A population study to define the incidence and survival of multiple myeloma in a National Health Service Region in UK Br J Haematol 2004 127 299 304 10.1111/j.1365-2141.2004.05207.x 15491289

9. Johnsen AT Tholstrup D Petersen MA Pedersen L Groenvold M Health related quality of life in a nationally representative sample of haematological patients Eur J Haematol 2009 83 139 148 10.1111/j.1600-0609.2009.01250.x 19284418 PMC2730555

10. Sonneveld P Verelst SG Lewis P Gray-Schopfer V Hutchings A Nixon A Petrucci MT Review of health-related quality of life data in multiple myeloma patients treated with novel agents Leukemia. 2013 27 1959 1969 10.1038/leu.2013.185 23783393 PMC3806249

11. Molassiotis A Wilson B Blair S Howe T Cavet J Unmet supportive care needs, psychological well-being and quality of life in patients living with multiple myeloma and their partners Psychooncology. 2011 20 88 97 10.1002/pon.1710 20187072

12. Molassiotis A Wilson B Blair S Howe T Cavet J Living with multiple myeloma: experiences of patients and their informal caregivers Support Care Cancer 2011 19 101 111 10.1007/s00520-009-0793-1 19998045

13. Maher K de Vries K An exploration of the lived experiences of individuals with relapsed multiple myeloma Eur J Cancer Care (Engl) 2011 20 267 275 10.1111/j.1365-2354.2010.01234.x 20950368

14. Ramsenthaler C Kane P Gao W Siegert RJ Edmonds PM Schey SA Higginson IJ Prevalence of symptoms in patients with multiple myeloma: a systematic review and meta-analysis Eur J Haematol 2016 97 416 429 10.1111/ejh.12790 27528496

15. Basch E Deal AM Kris MG Scher HI Hudis CA Sabbatini P Rogak L Bennett AV Dueck AC Atkinson TM Chou JF Dulko D Sit L Barz A Novotny P Fruscione M Sloan JA Schrag D Symptom monitoring with patient-reported outcomes during routine Cancer treatment: a randomized controlled trial J Clin Oncol 2016 34 557 565 10.1200/JCO.2015.63.0830 26644527 PMC4872028

16. Etkind SN Daveson BA Kwok W Witt J Bausewein C Higginson IJ Murtagh FE Capture, transfer, and feedback of patient-centered outcomes data in palliative care populations: does it make a difference? A systematic review J Pain Symptom Manag 2015 49 611 624 10.1016/j.jpainsymman.2014.07.010 25135657

17. Osborne TR Ramsenthaler C Siegert RJ Edmonds PM Schey SA Higginson IJ What issues matter most to people with multiple myeloma and how well are we measuring them? A systematic review of quality of life tools Eur J Haematol 2012 89 437 457 10.1111/ejh.12012 22985406

18. Wagner LI Robinson D Weiss M Katz M Greipp PR Fonseca R Cella D Content development for the functional assessment of Cancer therapy-multiple myeloma (FACT-MM): use of qualitative and quantitative methods for scale construction J Pain Symptom Manag 2012 43 1094 1104 10.1016/j.jpainsymman.2011.06.019 PMC3419469 22575718

19. Cocks K Cohen D Wisløff F Sezer O Lee S Hippe E Gimsing P Turesson I Hajek R Smith A Graham L Phillips A Stead M Velikova G Brown J EORTC Quality of Life Group. An international field study of the reliability and validity of a disease-specific questionnaire module (the QLQ-MY20) in assessing the quality of life of patients with multiple myeloma Eur J Cancer 2007 43 1670 1678 10.1016/j.ejca.2007.04.022 17574838

20. Jones D Vichaya EG Wang XS Williams LA Shah ND Thomas SK Johnson VE Champlin RE Cleeland CS Mendoza TR Validation of the M. D. Anderson Symptom Inventory multiple myeloma module J Hematol Oncol 2013 6 13 10.1186/1756-8722-6-13 23384030 PMC3598689

21. Proskorovsky I Lewis P Williams CD Jordan K Kyriakou C Ishak J Davies FE Mapping EORTC QLQ-C30 and QLQ-MY20 to EQ-5D in patients with multiple myeloma Health Qual Life Outcomes 2014 12 35 10.1186/1477-7525-12-35 24618388 PMC4007827

22. Osborne TR Ramsenthaler C Schey SA Siegert RJ Edmonds PM Higginson IJ Improving the assessment of quality of life in the clinical care of myeloma patients: the development and validation of the myeloma patient outcome scale (MyPOS) BMC Cancer 2015 15 280 10.1186/s12885-015-1261-6 25884627 PMC4404617

23. Hearn J Higginson IJ Development and validation of a core outcome measure for palliative care: the palliative care outcome scale. Palliative care Core audit project advisory group Qual Health Care 1998 8 219 227 10.1136/qshc.8.4.219 PMC2483665 10847883

24. Murtagh FE Ramsenthaler C Firth A Groeneveld EI Lovell N Simon ST Denzel J Guo P Bernhardt F Schildmann E van Oorschot B Hodiamont F Streitwieser S Higginson IJ Bausewein C A brief, patient- and proxy-reported outcome measure in advanced illness: validity, reliability and responsiveness of the integrated palliative care outcome scale (IPOS) Palliat Med 2019 33 1045 1057 10.1177/0269216319854264 31185804 PMC6691591

25. Schildmann EK Groeneveld EI Denzel J Brown A Bernhardt F Bailey K Guo P Ramsenthaler C Lovell N Higginson IJ Bausewein C Murtagh FE Discovering the hidden benefits of cognitive interviewing in two languages: the first phase of a validation study of the integrated palliative care outcome scale Palliat Med 2016 30 599 610 10.1177/0269216315608348 26415736 PMC4873725

26. Ramsenthaler C Gao W Siegert RJ Schey SA Edmonds PM Higginson IJ Longitudinal validity and reliability of the myeloma patient outcome scale (MyPOS) was established using traditional, generalizability and Rasch psychometric methods Qual Life Res 2017 26 2931 2947 10.1007/s11136-017-1660-z 28752440 PMC5655545

27. Davies JM Osborne TR Edmonds PM Schey SA Devereux S Higginson IJ Ramsenthaler C The myeloma patient outcome scale is the first quality of life tool developed for clinical use and validated in patients with follicular lymphoma Eur J Haematol 2017 98 508 516 10.1111/ejh.12864 28160316 PMC5413861

28. Warnecke RB Johnson TP Chavez N Sudman S O'Rourke DP Lacey L Horm J Improving question wording in surveys of culturally diverse populations Ann Epidemiol 1997 7 334 342 10.1016/S1047-2797(97)00030-6 9250628

29. Sidani S Guruge S Miranda J Ford-Gilboe M Varcoe C Cultural adaptation and translation of measures: an integrated method Res Nurs Health 2010 33 133 143 20069612 10.1002/nur.20364

30. Willis GB Cognitive interviewing: a tool for improving questionnaire design 2005 Thousand Oaks Sage Publications

31. Creswell JW Clark Vicki LP Designing and conducting mixed methods research 2017 Newbury Park Sage

32. Antunes B, Daveson B, Ramsenthaler C, Benalia H, Lopes Ferreira P, Higginson I. The palliative care outcome scale manual for cross-cultural adaptation and psychometric validation: Cicely Saunders Institute, King’s College London; 2012. http://pos-pal.org/Resources.php . Accessed 17 Oct 2019.

33. Kulis D, Bottomley A, Velikova G, Greimel E, Koller M. EORTC Quality of Life Group translation procedure manual. European Organization for Research and Treatment of Cancer. 2017. http://www.eortc.org/app/uploads/sites/2/2018/02/translation_ manual_2017.pdf . Accessed 17 Oct 2019.

34. Wild D Grove A Martin M Eremenco S McElroy S Verjee-Lorenz A Erikson P Translation ITFf, Cultural A Principles of good practice for the translation and cultural adaptation process for Patient-Reported Outcomes (PRO) measures: report of the ISPOR Task Force for Translation and Cultural Adaptation Value Health 2005 8 94 104 10.1111/j.1524-4733.2005.04054.x 15804318

35. Chavez L, Canino G. Toolkit on translating and adapting instruments. Cambridge: Human Services Research Institute; 2005.

36. Brod M Tesler LE Christensen TL Qualitative research and content validity: developing best practices based on science and experience Qual Life Res 2009 18 1263 1278 10.1007/s11136-009-9540-9 19784865

37. Green J Thorogood N Qualitative methods for Health Research 2009 2 London Sage Publications

38. Osborne TR Ramsenthaler C de Wolf-Linder S Schey SA Siegert RJ Edmonds PM Higginson IJ Understanding what matters most to people with multiple myeloma: a qualitative study of views on quality of life BMC Cancer 2014 14 496 10.1186/1471-2407-14-496 25005145 PMC4227056

39. Velikova G Awad N Coles-Gale R Wright EP Brown JM Selby PJ The clinical value of quality of life assessment in oncology practice-a qualitative study of patient and physician views Psychooncology 2008 12 690 698 10.1002/pon.1295 18033733

40. Bausewein C Fegg M Radbruch L Nauck F von Mackensen S Borasio GD Higginson IJ Validation and clinical application of the German version of the palliative care outcome scale J Pain Symptom Manag 2005 30 51 62 10.1016/j.jpainsymman.2005.01.017 16043007

41. Koller M Kantzer V Mear I Zarzar K Martin M Greimel E Bottomley A Arnott M Kuliś D ISOQOL TCA-SIG The process of reconciliation: evaluation of guidelines for translating quality-of-life questionnaires Expert Rev Pharmacoecon Outcomes Res 2012 12 189 197 10.1586/erp.11.102 22458620

42. Willis GB Royston P Bercini D The use of verbal report methods in the development and testing of survey questionnaires Appl Cogn Psychol 1991 5 251 267 10.1002/acp.2350050307

43. Lee J Conducting cognitive interviews in cross-National Settings Assessment. 2014 21 227 240 10.1177/1073191112436671 22327207

44. Tourangeau R Jabine TB Cognitive sciences and survey methods Cognitive aspects of survey methodology: building a bridge between disciplines 1984 Washington, DC National Academy Press 73 101

45. Abernethy AP Shelby-James T Fazekas BS Woods D Currow DC The Australia-modified Karnofsky performance status (AKPS) scale: a revised scale for contemporary palliative care clinical practice [ISRCTN81117481] BMC Palliat Care. 2005 4 7 10.1186/1472-684X-4-7 16283937 PMC1308820

46. Braun V Clarke V Using thematic analysis in psychology Qual Res Psychol 2006 3 77 101 10.1191/1478088706qp063oa

47. Kerr C Nixon A Wild D Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research Expert Rev Pharmacoecon Outcomes Res. 2010 10 269 281 10.1586/erp.10.30 20545592

48. Beck I Olsson Möller U Malmström M Klarare A Samuelsson H Lundh Hagelin C Rasmussen B Fürst CJ Translation and cultural adaptation of the integrated palliative care outcome scale including cognitive interviewing with patients and staff BMC Palliat Care 2017 16 49 10.1186/s12904-017-0232-x 28893215 PMC5594532

49. Blozik E Scherer M Lacruz ME Ladwig KH KORA study group. Diagnostic utility of a one-item question to screen for depressive disorders: results from the KORA F3 study BMC Fam Pract 2013 14 198 10.1186/1471-2296-14-198 24359193 PMC3906912

50. Vahter L Kreegipuu M Talvik T Gross-Paju K One question as a screening instrument for depression in people with multiple sclerosis Clin Rehabil 2007 21 460 464 10.1177/0269215507074056 17613567

51. Veronese S Rabitti E Costantini M Valle A Higginson I Translation and cognitive testing of the Italian integrated palliative outcome scale (IPOS) among patients and healthcare professionals PLoS One 2019 14 e0208536 10.1371/journal.pone.0208536 30601810 PMC6314603

52. Sterie AC Bernard M Challenges in a six-phase process of questionnaire adaptation: findings from the French translation of the integrated palliative care outcome scale BMC Palliat Care. 2019 18 38 10.1186/s12904-019-0422-9 30999910 PMC6474036

53. Stead ML Brown JM Velikova G Kaasa S Wisløff F Child JA Hippe E Hjorth M Sezer O Selby P Development of an EORTC questionnaire module to be used in health-related quality-of-life assessment for patients with multiple myeloma. European Organization for Research and Treatment of Cancer study group on quality of life Br J Haematol 1999 104 605 611 10.1046/j.1365-2141.1999.01206.x 10086801

54. Heinemann AW Fisher WP Gershon R Improving health care quality with outcomes management J Prosthet Orthot 2006 18 46 50 10.1097/00008526-200601001-00005

55. Kaplan SH Kravitz RL Greenfield S A critique of current uses of health status for the assessment of treatment effectiveness and quality of care Med Care 2000 38 184 191 10.1097/00005650-200009002-00029 10982106

56. Ferrell B Grant M Schmidt GM Rhiner M Whitehead C Fonbuena P Forman SJ The meaning of quality of life for bone marrow transplant survivors. Part 1. The impact of bone marrow transplant on quality of life Cancer Nurs 1992 15 153 160 1611601

57. Dahan JF Auerbach CF A qualitative study of the trauma and posttraumatic growth of multiple myeloma patients treated with peripheral blood stem cell transplant Palliat Support Care 2006 4 365 387 10.1017/S1478951506060470 17133896

58. Parsons JA Greenspan NR Baker NA McKillop C Hicks LK Chan O Treatment preferences of patients with relapsed and refractory multiple myeloma: a qualitative study BMC Cancer 2019 19 1 264 10.1186/s12885-019-5467-x 30909874 PMC6434792

59. Carlson LE Screening alone is not enough: the importance of appropriate triage, referral, and evidence-based treatment of distress and common problems J Clin Oncol 2013 31 3616 3617 10.1200/JCO.2013.51.4315 24002494

60. Hollingworth W Metcalfe C Mancero S Harris S Campbell R Biddle L McKell-Redwood D Brennan J Are needs assessments cost effective in reducing distress among patients with cancer? A randomized controlled trial using the distress thermometer and problem list J Clin Oncol 2013 31 3631 3638 10.1200/JCO.2012.48.3040 24002506

61. Dionne-Odom JN Azuero A Lyons KD Hull JG Tosteson T Li Z Frost J Dragnev KH Akyar I Hegel MT Bakitas MA Benefits of early versus delayed palliative care to informal family caregivers of patients with advanced Cancer: outcomes from the ENABLE III randomized controlled trial J Clin Oncol 2015 33 1446 1452 10.1200/JCO.2014.58.7824 25800762 PMC4404423

62. McCaughan D Roman E Smith AG Garry AC Johnson MJ Patmore RD Howard MR Howell DA Perspectives of bereaved relatives of patients with haematological malignancies concerning preferred place of care and death: a qualitative study Palliat Med 2019 33 518 530 10.1177/0269216318824525 30696347 PMC6507303

63. LeBlanc TW El-Jawahri A Hemato-oncology and palliative care teams: is it time for an integrated approach to patient care? Curr Opin Support Palliat Care 2018 12 530 553 10.1097/SPC.0000000000000385 30379755

64. Porta-Sales J Guerrero-Torrelles M Moreno-Alonso D Sarrà-Escarré J Clapés-Puig V Trelis-Navarro J Sureda-Balarí A Fernández De Sevilla-Ribosa a. is early palliative care feasible in patients with multiple myeloma? J Pain Symptom Manag 2017 54 692 700 10.1016/j.jpainsymman.2017.04.012 28807703

65. El-Jawahri A Traeger L Greer JA VanDusen H Fishman SR LeBlanc TW Pirl WF Jackson VA Telles J Rhodes A Li Z Spitzer TR McAfee S Chen YA Temel JS Effect of inpatient palliative care during hematopoietic stem-cell transplant on psychological distress 6 months after transplant: results of a randomized clinical trial J Clin Oncol 2017 35 3714 3721 10.1200/JCO.2017.73.2800 28926288 PMC5675739

66. El-Jawahri A LeBlanc T VanDusen H Traeger L Greer JA Pirl WF Jackson VA Telles J Rhodes A Spitzer TR McAfee S Chen YA Lee SS Temel JS Effect of inpatient palliative care on quality of life 2 weeks after hematopoietic stem cell transplantation: a randomized clinical trial JAMA. 2016 316 2094 2103 10.1001/jama.2016.16786 27893130 PMC5421101