Adaptation of digital integration of PROMs and PREMs in oncology during implementation: a scoping review.

Relevance Statement

i) What is already known about the topic? To date, there has been no agreed system to capture adaptations of ePROMs in oncology, and currently, there is no system or framework to assess ePROMs’ efficacy. This review offers a complete summary description of the reasons and types of adaptations made to oncology ePROMs during an implementation process.

ii) What this paper adds We created a complete overview of reasons and types of adaptations made to ePROMs which can be used in future development and implementation of ePROMs in oncology evidence.

iii) Implications for practice, theory or policy Our review showed the complete absence of studies that include adaptations to oncology ePREMs, potentially indicating a gap in reporting on patients’ experiences, while much focus has been on patient outcomes (ePROMs). Our review highlighted the low availability of data for in-patient groups and these patient groups could benefit from being included more frequently in the literature. Our review also found no relationship between the type of an implementation framework and adaptation made to ePROMs

Introduction

In recent years, health information technology (HIT) has become an integral part of oncology care to facilitate the electronic assessment of patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs) [ 1 , 2 ]. Electronic PROMs (ePROMs) assess patients’ views on their health, well-being, or symptoms related to their care (e.g. nausea and pain). ePREMs capture patients’ perspectives on the overall quality of and satisfaction with received care, which is essential for personalised oncology treatment and holistic care [ 3 ]. Assessing ePROMs and ePREMs can help to identify unmet patient needs, improve patient-clinician communication, patient satisfaction, increase supportive care measures, alleviate symptom burden, and improve patient-centred care [ 4 – 7 ]. Real-world implementation of ePROMs, and in particular ePREMs, outside the context of feasibility or research intervention in cancer care remains sparse, and adoption/implementation attempts rates are low [ 2 , 8 , 9 ].

Up to now, the scientific literature shows that ePROMs and ePREMs are prone to different barriers at distinct stages of the implementation process in healthcare [ 10 – 13 ]. These include the following: lack of clinicians’ knowledge; disbelief in their value; lack of digital literacy of both patients and clinicians; technology confidence; inadequate infrastructures; and challenges with integrating ePROMs and ePREMs into the electronic medical health record [ 2 , 10 ]. Adaptations are an integral part of the implementation process of ePROMs and ePREMs. Adaptations refer to a process of purposeful or unplanned modifications to system design or implementation to fit the local context better, to retain fidelity (maintaining the integrity of the intervention and its implementation), or to facilitate acceptability [ 14 – 17 ]. Many early-stage implementation reports provide preliminary observations rather than established findings. We aimed to base the review on evidence derived from fully executed interventions to avoid drawing conclusions based on incomplete or tentative data. In the ePROMs/ePREMs literature, most studies do not track nor report on the adaptations performed. Adaptations to ePROMs/ePREMs in oncology should be a topic of research because they directly impact data quality, clinical decision-making, and patient engagement. ePROMs/PREMs must be tailored to the specific context to be successfully integrated, as there is no one-size-fits-all solution. Consequently, ongoing formative evaluation and adaptation are essential strategies. Optimised electronic measures can play an important role in early detection of complications and tailoring of patients’ disease management, potentially improving quality of life or survival [ 5 ].

In this scoping review, therefore, we try to map those studies that describe ePROMs and ePREMs adaptations and map how and what types of adaptations are reported to try to improve fit with the local context during the implementation study. Learning about adaptations made to ePROMs/ePREMs in oncology could facilitate informing the development of other systems or outcome measures applied in oncology.

Methods

Design

The Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist and explanation [ 18 ] was used to draft the protocol which was subsequently registered with the Open Science Framework 2022 https://osf.io/z3wc7/overview . A systematic scoping review methodology was chosen as it was most appropriate to the study aim of mapping the existing literature to provide a more exploratory overview of a topic that has not been previously characterised in this manner.

Eligibility criteria

To be included in this review, articles needed to (i) focus on routine cancer care, (ii) report on the implementation of an ePROMs and/or ePREMs intervention, and (iii) (retrospectively) describe an iterative process of adapting the system and/or methods of implementation. As adaptations often take place after the initial stages of the ePROMs/ePREMs implementation, it is therefore important that studies describing complete implementations were identified. RCTs are not designed to assess the sustainability of implementing ePROMs in daily practice and often do not allow for the adaptations needed for real-world use—both important elements of our scoping review. Articles were excluded if they did not/were (i) not oncology focussed; (ii) implemented paper-based PROMs and PREMs; (iii) exclusively focused on the impact, effectiveness, measurement properties, and/or language validity of ePROMs and ePREMs; (iv) used ePROMs and ePREMs as a measure of therapy effect of drugs, such as in clinical trials; and (v) described the implementation of ePROMs and ePREMs in research contexts (e.g. clinical trials) or were prospective because these were usually still ongoing. Furthermore, the search was limited to primary studies written in English. Duplicates, protocols, non-empirical publications, editorials, letters to editors, conference abstracts, and commentaries were also excluded. Reviews were not included in the synthesis, but reference lists of relevant reviews were screened for primary studies. Full criteria are available in the Supplementary Material 01 .

Literature search

We systematically searched the following academic databases: PubMed, Embase, PsychINFO, and CINAHL (from inception to May 5, 2023). Through multidisciplinary team discussion, a search strategy was developed combining indexing terms, keywords, and synonyms for cancer (e.g. neoplasm), implementation and adaptation (e.g. adoption), and PROMs/PREMs (e.g. patient-reported outcomes). The full search strategy was adapted for each electronic database (see Supplementary Material 01 ). All records were imported using the reference manager EndNote (Clarivate), and the screening process was facilitated by the web-based tool Rayyan (Rayyan Systems Inc.). EndNote allowed us to import the references more quickly than downloading each reference individually and importing it into Rayyan. After removing duplicates, records were screened for eligibility based on title and abstract by E.R. (see Acknowledgements). In this first round of screening (i.e. based on title and abstract), a second reviewer (N.F.) independently screened at least 10% ( n = 494) to check inter-rater reliability. This was followed by a screening on full-text level, in which all records were screened by the two reviewers (E.R. and N.F.) blinded to each other. Disagreements and conflicts were resolved through discussion. The electronic database search was supplemented by screening reference lists of relevant reviews and included papers, by exploring grey literature (the last search was on July 24, 2023), all in the English language. The latter included exploring Dutch literature (in the English language) in Medline, MedRxiv (i.e. preprints), the International Network of Agencies for Health Technology Assessment (HTA) database, and Open Access Theses and Dissertations. We used this because a significant proportion of the authors on our paper are based in the Dutch-speaking part of Belgium.

Data extraction and synthesis

This review was guided by the Population, Concept, and Context (PCC) framework [ 14 ], which helped us construct clear and meaningful objectives and categories for data extraction in our scoping review. The data from the included studies were extracted and summarised in a table in Excel (Microsoft Inc) by one researcher (E.R.), and uncertainties were discussed with a second (N.F.) and third reviewer (A.-L.S.). The extracted data included PCC dimensions: context, content, evaluation, and training—categories that shaped our data organisation and discussion.

The extracted data also included (i) background information: first author, year, and country where the implementation took place; (ii) details on the population and setting: number of sites where the implementation took place, in- and/or outpatient setting, and target group; (iii) details on the ePROMs and/or ePREMs intervention: measurement instruments, timing, and core features of the digital system (e.g. alert system and integration in the electronic medical health record); (iv) information on the process and reporting of adaptation: theoretical frameworks, process for making adaptations, clinical champions supporting the adaptation process, qualitative and/or quantitative methods supporting the adaptation process (methodology described for the adaptation), and presentation of adapted elements in the manuscript (i.e. first aim); and (v) reasons for and types of adaptations (i.e., second aim). Stirman and associates’ typology of modifications was used to categorise adaptations [ 14 ]. Contextual adaptations refer to changes to the delivery of the ePROMs/ePREMs in terms of format, setting, personnel, and the target population, while content adaptations refer to changes to the procedures, materials, or delivery. Adapting training and evaluation processes refer to training and evaluation adaptations, respectively [ 14 ]. Numerical analyses using descriptive methods and a narrative synthesis were conducted to summarise results.

Results

Study selection

As shown in the PRISMA-ScR flow diagram [ 18 ] (Fig. 1 ), the systematic search in academic databases resulted in n = 5597 publications. The titles and abstracts of 3700 records were screened for eligibility after the removal of duplicates. From these, 244 were retrieved for the full-text screening round. The full-text screening round identified 15 studies eligible for inclusion. The main reasons for exclusion were the wrong publication type (i.e. conference abstracts and literature reviews) and the lack of an iterative process of adapting the system and/or implementation methods. Per cent agreement between the first and second reviewers was 92.5% and 94.7% in the first (i.e. title and abstract) and second (i.e. full text) screening rounds, respectively. Scanning reference lists of relevant reviews resulted in five additional studies being identified. We did not find grey literature publications eligible for inclusion. In total, n = 20 articles were included in this scoping review, in which two publications reported on the same ePROM intervention and the accompanying process of adapting the digital tool [ 19 , 20 ]. Fig. 1 PRISMA-ScR flow diagram

General study characteristics

Among the included studies, the oldest one dated from 2010 [ 21 ], and n = 17 (85%) of the studies were published between 2019 and 2022, with eight studies alone in 2022 [ 16 , 19 , 20 , 22 – 25 ]. Studies reported locations in which they were conducted as follows: USA ( n = 7) [ 16 , 22 , 25 – 29 ], Europe ( n = 6; i.e. Austria ( n = 3), Germany, Belgium, and the Netherlands) [ 21 , 23 , 24 , 30 – 32 ], Australia ( n = 3) [ 19 , 20 , 33 , 34 ], and the UK ( n = 2) [ 35 , 36 ]. In one study, the implementation took place in different countries [ 37 ]. Concerning the target group, n = 15 (70%) targeted outpatients [ 19 – 21 , 23 , 26 – 31 , 33 – 37 ] and n = 12 (60%) included patients with various cancer diagnoses [ 16 , 21 , 24 – 27 , 29 , 32 , 33 , 37 ] (Table 1 ). Implementation of ePROMs assessments was guided by various groups such as study authors, clinicians, administration staff, champions, nurses, etc. (see under “ Process of integration and adaptations ”). Two studies included children with cancer [ 21 , 26 ]. Table 1 Study characteristics First author, year Country Number of centres b In- and/or outpatients Types of cancer(s) d Bamgboje‐Ayodele, 2022 [ 20 ] a Girgis, 2022 [ 19 ] a Australia 3 Outpatients Lung cancer Engelen, 2010 [ 21 ] Netherlands 4 Outpatients Children Fernández-Méndez, 2018 [ 35 ] UK 2 Outpatients Brain tumour or metastasis Furlong, 2019 [ 37 ] Norway; Austria; Greece; Ireland; UK 13 Outpatients Breast cancer, colorectal cancer, Hodgkin’s disease, or non-Hodgkin’s lymphoma Hassett, 2022 [ 22 ] USA 6 In- and outpatients Gynecologic, thoracic, and GI malignancies McClaery, 2022 [ 26 ] USA 1 Outpatients Children and adults diagnosed with cancer or blood disorders Misplon, 2022 [ 23 ] Belgium 1 Outpatients Lung cancer Neal, 2021 [ 27 ] USA 2 Outpatients Breast and gastrointestinal (GI) Nordhausen, 2022 [ 24 ] Germany 1 Inpatients Radiation oncology patients Palos, 2021 [ 28 ] USA 4 Outpatients All cancer types Roberts, 2020 [ 33 ] Australia 2 Outpatients A wide range of cancer types, care needs, and level of complexity in children/pediatric oncology Rocque, 2022 [ 16 ] USA 2 In- and outpatients All cancer types Schuler, 2021 [ 34 ] Australia 1 Outpatients Lung, prostate, colorectal, breast and other Strachna, 2022 [ 25 ] USA 20 c In- and outpatients All cancer types Sztankay, 2019 [ 30 ] Austria 1 Outpatients Multiple myeloma Van Egdom, 2019 [ 31 ] Netherlands 1 Outpatients Breast cancer Warrington, 2019 [ 36 ] UK 1 Outpatients Early breast cancer Wintner, 2020 [ 32 ] Austria 1 Inpatients Various diagnoses Wu, 2016 [ 29 ] USA 1 Outpatients Breast and prostate cancer a Reporting on the same ePROM system b Centres refer to any hospitals, clinics, or care wards c Approximately 20 sites d The cancer types are reported in line with the wording in the original papers

Intervention characteristics

All the included studies focused on ePROMs, and none reported on ePREMs. A variety of measurement instruments were used to assess PROMs, including European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Core (EORTC QLQ) and its specific modules ( n = 7) [ 23 , 24 , 29 – 32 , 35 ], Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCATE) ( n = 5) [ 16 , 22 , 23 , 33 , 36 ], EQ-5D (health-related quality of life) and its three-level version ( n = 3) [ 30 – 32 ], and Patient-Reported Outcome Measurement Information System (PROMIS) ( n = 2) [ 27 , 29 ] (Table 2 ). The frequency and timing points of the ePROMs assessment varied ranging from daily ( n = 2) [ 24 , 37 ], weekly ( n = 3) [ 16 , 23 , 36 ], monthly ( n = 3) [ 19 , 20 , 27 ] to yearly ( n = 1) [ 26 ]. More than half of the studies reported that the frequency of ePROM assessment was either varied or very specific to their study (e.g., at a new cancer diagnosis, prior treatment, depending on treatment, and before a consultation) ( n = 11) [ 21 , 22 , 28 , 29 , 31 , 32 , 35 ]. In terms of the access location, the sites offered either remote, in situ, or a combination of both as follows: in situ ( n = 5) [ 21 , 24 , 30 , 33 , 35 ], remote ( n = 7) [ 26 , 31 , 36 , 38 – 41 ], and both (in situ and remote) ( n = 7) [ 19 , 20 , 22 , 26 , 27 , 29 , 32 , 34 ]. In addition, 11 ePROM systems offered alerts [ 16 , 19 , 20 , 22 , 23 , 25 , 28 , 29 , 33 , 36 , 37 ] and n = 14 (64%) studies [ 16 , 19 , 20 , 22 , 25 , 29 , 31 , 33 ] reported integration of patients’ ePROMs into the electronic health record (EHR). Table 2 Intervention characteristics First author, year PROMs Measurement instrument and additional questions Frequency or timing points Access location Alert system Integration into EHR Bamgboje‐Ayodele, 2022 [ 20 ]; Girgis, 2022 [ 19 ] ESAS; DT; Problem checklist Monthly In situ and remote Yes Yes Engelen, 2010 [ 21 ] PedsQL; TAPQOL Three consecutive consultations In situ Unknown Unknown Fernández-Méndez, 2018 [ 35 ] EORTC QLQ-C30; EORTC QLQ-BN20 Prior to consultation In situ Unknown Unknown Furlong, 2019 [ 37 ] Chemotherapy Toxicity Self-Assessment Questionnaire Daily Remote Yes unknown Hassett, 2022 [ 22 ] PRO-CTCAE; pictogram questions for overall well-being and functional status Depending on treatment In situ and remote Yes Yes McClaery, 2022 [ 26 ] e-NPIQ Yearly or with new cancer diagnosis In situ and remote Unknown Yes Misplon, 2022 [ 23 ] PRO-CTCAE; questions on psychological, spiritual, palliative, social, family-related and financial needs; EORTC QLQ-C30; EORTC QLQ-LC13 Weekly Remote Yes Yes Neal, 2021 [ 27 ] PROMIS; question to request additional help; Faith and Belief, Importance, Community, Address in Care spiritual health questions; open-ended question to add symptoms Monthly or less frequently (three days in advance of a scheduled visit) In situ and remote Unknown Yes Nordhausen, 2022 [ 24 ] EORTC QLQ-C30; QSC-R10; three-eight questions for specific symptoms for lung, head-neck, colorectal, brain, breast, and prostate cancer Daily In situ Unknown Yes Palos, 2021 [ 28 ] MDASI Depending on site Remote Yes Yes Roberts, 2020 [ 33 ] PRO-CTCAE Prior to consultation In situ Yes Yes Rocque, 2022 [ 16 ] PRO-CTCAE Weekly Remote Yes Yes Schuler, 2021 [ 34 ] NPRS At baseline and four weeks post therapy In situ and remote Unknown Unknown Strachna, 2022 [ 25 ] Acute Side Effects From Breast Radiation; COVID-19 Symptom Questionnaire; Daily Symptom Assessment; Recovery Tracker Varying Remote Yes Yes Sztankay, 2019 [ 30 ] EORTC QLQ-C30; EORTC QLQ‐MY20; EQ‐5D‐5L At any given post‐diagnostic time point (according to treatment schedule) In situ Unknown Unknown Van Egdom, 2019 [ 31 ] EORTC QLQ-C30; EORTC QLQ‐BR23; BREAST-Q; EQ‐5D‐5L; DT; RCS-NL; CarerQoL-7D Three weeks before the scheduled consultation, following the last course of therapy, six months after surgery and annually thereafter Remote Unknown Yes Warrington, 2019 [ 36 ] PRO-CTCAE; option to add symptoms; free text option Weekly or more frequently Remote Yes Yes Wintner, 2020 [ 32 ] EORTC QLQ-C30; HADS-D; SSS-PSD; EQ-5D-3L; four questions assessing patients’ wish for a psycho-oncology treatment, previous psychiatric disorder and/or psychotherapeutic treatment Before admission and up to three days before discharge In situ and remote Unknown Yes Wu, 2016 [ 29 ] EORTC QLQ-C30; Supportive Care Needs Survey-short Form; PROMIS-short form Prior to consultation In situ and remote Yes Yes PRO-CTCAE Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events, EHR electronic health record, EORTC QLQ-C30 European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Core 30, HADS-D Hospital Anxiety and Depression Scale, SSS-PS Short Screening Scale for DSM-IV Post-Traumatic Stress Disorder, EQ-5D-3L EuroQOL-5-Dimensions, QSC-R10 Questionnaire on Stress in Cancer Patients, ESAS Edmonton Symptom Assessment Scale, DT distress thermometer, e-NPIQ Electronic New Patient Intake Questionnaire, PROMIS Patient-Reported Outcomes Measurement Information System questionnaire, MDASI MD Anderson Symptom Inventory, EORTC QLQ-BN20 European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Brain Tumor Module, EORTC QLQ-LC13 European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Lung Cancer Module, EORTC QLQ‐MY20 European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Myeloma Module, EORTC QLQ‐BR23 European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Lung Cancer Module, EORTC QLQ‐MY20 European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Breast Cancer Module, RCS-NL Reproductive Concerns Scale, CarerQoL-7D Care-related Quality of Life instrument, PedsQL Pediatric Quality of Life Inventory, TAPQOL TNO-AZL Preschool Children Quality of Life, NPRS Numerical Pain Rating Scale

Types of adaptations

As shown in Table 3 , studies included different types of adaptations, which were categorised into four dimensions. For complete details and descriptions of each type of adaptation, please refer to the table. Types of adaptations included context adaptations (e.g. increasing visibility of the tool in services, changing complex onboarding) ( n = 12) [ 16 , 19 , 20 , 27 , 30 , 32 , 33 , 35 ] and content adaptations (e.g. changing visibility, readability, and feedback within the ePROMs system) [ 16 , 19 – 37 ] which were included in all 20 studies. Other adaptations were made to the evaluation (e.g. mostly configuration of alerts) ( n = 3) [ 22 , 33 , 35 ] and training (e.g., group training, training new staff) ( n = 10) [ 19 – 22 , 26 , 28 , 33 – 37 ] (see also Supplementary Material 02 for an overview). Table 3 Adaptation types First author, year Adaptation type 1. Context example 2. Content example 3. Evaluation example 4. Training example Bamgboje‐Ayodele, 2022 [ 20 ]; Girgis, 2022 [ 19 ] Difficulty in identifying whether patients were missed in the onboarding process, which resulted in the refinement of the criteria for patient eligibility Data required to measure implementation is unavailable, which resulted in integrating reporting measures as part of the implementation workflow to measure implementation Limited insights into the changes in readiness for, and views on, implementing PROMs as routine care, which resulted in surveys to assess organisational readiness for change and health care professionals’ views on the conducted PROMs implementation New staff joining the cancer service with no knowledge of PROMs, which resulted in orientations to train new staff Engelen, 2010 [ 21 ] Feelings of triumph or disappointment when children’s score was over or under the red dotted line, which resulted in parents and children receiving PROMs without line graphs The theoretical part was not as useful as the practical component of the training, which resulted in the group training being shortened to one hour, and the theoretical part was limited Fernández-Méndez, 2018 [ 35 ] Progressive departure of the support team from the neuro-oncology clinic, which resulted in rearranging the roles in the implementation Lack of clarity of PROMs answers, which resulted in including graph bars to summarise patient answers Insufficient time for members of the care team to prepare PROMs during/before clinic, which resulted in weekly monitoring of PROMs preparation Members of the care team sometimes forget how to implement the PROMs, which resulted in guidelines with detailed descriptions of how to access and administer the PROMs during clinic Furlong, 2019 [ 37 ] Inefficient charging (one handset), which resulted in two active handsets being necessary at each cancer centre, with one in use and the second on charge A four-hour response timeframe was not feasible in busy cancer centres, which resulted in the timeframe for handling mild to moderate patient symptoms being changed from four to eight hours Technical issues in relation to using the clinician website, which resulted in the technology partner providing additional training on using the system Hassett, 2022 [ 22 ] Limited access to internet-enabled devices and technology confidence, which resulted in the implementation of alternative PROMs collection methods, including tablets in the clinic and proxy reporting at home The report displayed all reported symptoms rather than summarising just the severe ones, which resulted in severe symptoms being highlighted in yellow to make them stand out No encouragement from the care team to submit symptom reports, which resulted in additional training materials outlining the importance of PROMs collection McClaery, 2022 [ 26 ] Low enrolment rates, which resulted in staff-assisted patient enrolment support Issues with visibility and readability, which resulted in the question flow was rearranged to be more logically ordered Low enrolment rates, which resulted in staff education Misplon, 2022 [ 23 ] COVID-19, which resulted in the introduction of tele-consultations A questionnaire based on their own experience with lung cancer care, which resulted in replacing the self-developed questionnaire with an internationally validated instrument Neal, 2021 [ 27 ] Supportive services at the cancer centre were not coordinated, which resulted in a coordinated and centralised umbrella organisation being developed, which allowed for increased visibility of available resources to both clinical teams and patients The frequency of questionnaires was too frequent, which resulted in frequency was changed to every 60 days Nordhausen, 2022 [ 24 ] Not all patients appeared in the software program, which resulted in the optimisation of the functionality Palos, 2021 [ 28 ] Patients’ lack of knowledge or understanding of the role of PROMs, which resulted in tip sheet was designed to explain the role of PROMs in patient care Providers lack knowledge of how to assess, interpret, and score the results, which resulted in professional education and training sessions to explain how to use the PROMs and interpret scores Roberts, 2020 [ 33 ] Patients were concerned that they may miss their doctor’s appointment by stopping to complete the PROMs, which resulted in the touchscreen computer station being located where patients could hear their name called Difficulties with integrating the report into workflows, which resulted in the alert system being abandoned No clear process for responding to PROMs symptom information, which resulted in the evaluation of the different pathways for responding to symptom information The purpose of PROMs not defined and is perceived as just another task added to workloads, which resulted in education about purpose and the evidence for potential uses Rocque, 2022 [ 16 ] High workload for clinic nurses, which resulted in formalised assignment of survey initiation and compliance to navigators; alert response to clinic nurses Alert fatigue, which resulted in added a snooze option by setting the expected alert threshold for up to 4 weeks Schuler, 2021 [ 34 ] Marked deterioration of PROMs completion, which resulted in scheduling a shared patient-specific PROMs collection task for radiation therapists Marked deterioration of PROMs completion, which resulted in staff education Strachna, 2022 [ 25 ] Cognitive load associated with reading patient messages without colours, which resulted in adding colour labels that indicate the level of severity Sztankay, 2019 [ 30 ] Lack of privacy (no designated room) in situ, which resulted in completing the PROMs in a separate room To increase adherence rates, which resulted in adapting assessment frequency to patient preference Van Egdom, 2019 [ 31 ] Low compliance rates, which resulted in a brochure to explain the initiative and the different time points of the survey-assessments were created Warrington, 2019 [ 36 ] Severe symptom notifications were being triggered for patients reporting retrospective problems, which resulted in a branching question being added to ask patients ‘Is this a current problem?’ Staff found the symptom reports less useful when patients were not completing regularly, which resulted in future training staff being asked to encourage patients to complete regularly and explicitly refer to and use the results in consultations Wintner, 2020 [ 32 ] Staff did not feel well prepared for PROMs-related critical inquiries of patients, which resulted in phone support for front desk staff Low response rates, which resulted in refining the cover letter to patients presenting PROMs as a substantial part of the routine practice Wu, 2016 [ 29 ] Need more than 1 reminder, which resulted in increasing the frequency of email notifications sent to the patient

Different dimensions of adaptation for ePROMs and ePREMs—such as context, content, evaluation, and training—address unique barriers and needs; for example, context adaptations involve refining eligibility criteria or clinical workflow, content adaptations change the format, presentation, or symptom highlights, evaluation focuses on readiness surveys and feedback loops, and training covers orientation, role changes, and the practical/technical skill-building required for effective use.

Common reasons for adaptations

Common reasons for adaptations included a combination of those reported by patients and some reported by clinicians: (i) low response rates and non-completion of ePROMs by patients ( n = 6) [ 16 , 26 , 31 , 32 , 34 , 36 ], (ii) high workload of clinicians ( n = 5) [ 16 , 27 , 29 , 33 , 35 ], (iii) time constraints of clinicians ( n = 3) [ 29 , 33 , 35 ], (iv) alert fatigue and unwarranted alerts for both ( n = 5) [ 16 , 25 , 29 , 36 , 37 ], (v) technical issues for both ( n = 5) [ 29 , 33 , 35 – 37 ], (vi) staff’s lack of knowledge about how to assess, interpret, and use the ePROMs system ( n = 5) [ 16 , 20 , 28 , 33 , 35 ], and (vii) a high frequency of positive responses that overburdened staff because it resulted in increased assessment (follow-up) for the clinical team ( n = 1) [ 27 ]. Low response rates and non-completion of ePROMs by patients were, for instance, attributed to a combination of infrastructural and human factors, such as a lack of computer access [ 26 ], forgetting [ 36 ], and patients being unaware of needing to complete ePROMs [34, 35].

To address low response and engagement, possible adaptations included using automated patient reminders [ 16 , 36 ], refining cover letters or patient training to emphasise the importance of PROMs and the time points of survey assessment [ 31 , 32 , 36 ], providing patients with a tablet device in situ on which to complete ePROMs [ 26 ], and staff education (orientations to train new staff and existing staff) [ 20 , 26 , 34 ]. Although some studies provided automated alerts, adaptations included managing those alerts better and making workflow adaptations to address irrelevant and excessive notifications involved. For example, better management of alerts for outpatient staff meant that the staff believed they were not the appropriate person to receive inpatient information when not managing inpatient care [ 16 , 35 ]. Additionally, making workflow adaptations meant abandoning the alert system in favour of better integration of reports into workflows [ 33 ] to address irrelevant and excessive notifications such as revising alert thresholds [ 20 , 36 ], silencing alerts [ 16 , 25 , 37 ], allowing patients to opt out of receiving calls [ 16 ], and adding branching questions to filter current problems [ 36 ]. Technical issues, such as connectivity issues [ 29 , 33 , 35 , 37 ] and software problems [ 35 ], were addressed by adapting the ePROMs system to monitor for and make clinicians aware of connectivity issues [ 37 ] and software upgrades [ 35 ]. Finally, the staff’s lack of knowledge led to different adaptations, including organising (additional) professional education and training sessions [ 20 , 28 ], as well as developing detailed guidelines [ 35 ].

Process of integration and adaptations

Various procedures for the integration and adaptation process were reported. Across studies, multiple champions were involved in the adaptation process, including study staff, medical and allied healthcare staff, management and administrative personnel, and information technology experts (Table 4 ). Twelve (55%) studies used a combination of qualitative (e.g. observation, interviews, focus groups, and written feedback) and quantitative methods (e.g. surveys and outcomes extracted from the EHR) to support and inform the adaptation process, with the majority receiving feedback from the users of ePROMs [ 16 , 19 , 20 , 26 , 30 , 32 , 33 , 35 , 36 ]. The adaptations were collected and presented in different formats: written text ( n = 11) [ 21 , 24 – 26 , 30 – 32 , 34 , 37 ], a combination of a table and written text ( n = 6) [ 16 , 22 , 28 , 29 , 33 , 36 ], and supplementary materials (e.g. healthcare provider survey) ( n = 3) [ 19 , 20 , 35 ]. Table 4 Adaptation process Approach/framework and referenced studies Description of the integration and adaptation process Stakeholders involved Methods and data sources CFIR (Consolidated Framework for Implementation Research) Bamgboje-Ayodele, 2022 [ 20 ]; Girgis, 2022 [ 19 ]; McClaery, 2022 [ 26 ] Monthly evaluation and engagement meetings; three pilot studies optimising accessibility Oncologists, allied professionals, consumer reps, IT/content experts, council, study staff Quantitative: implementation status, surveys; qualitative: meetings, interviews, focus groups RE-AIM Bamgboje-Ayodele, 2022 [ 20 ]; Girgis, 2022 [ 19 ]; Neal, 2021 [ 27 ] Monthly and iterative feedback in three phases Multidisciplinary implementation and IT teams, study staff Quantitative: abstracted outcomes, reports; qualitative: meeting notes, feedback FRAME (Framework for Modifications and Adaptations) Rocque, 2022 [ 16 ] Formative evaluation, six steps for adapting implementation in pilot studies Physicians, nursing, admin, tech vendors, support, study staff Quantitative: abstracted outcomes; qualitative: verbal feedback, interviews REP (Replicating Effective Programs) Sztankay, 2019 [ 30 ]; Wintner, 2020 [ 32 ] Regular evaluation meetings, continuous feedback and improvement for technical/logistic/workflow issues Multidisciplinary implementation team, institution reps, medical/therapy directors Quantitative: surveys, outcomes; qualitative: observations, notes FITT (Fit between Individuals, Task, and Technology) Strachna, 2022 [ 25 ]; Sztankay, 2019 [ 30 ] Regular meetings for feedback collection and process improvement End users, program managers, clinicians, system developers, work groups Quantitative: abstracted outcomes, surveys; qualitative: observational notes iPARIHS (Promoting Action Research in Health Services) Roberts, 2020 [ 33 ] Five PDSA cycles at prespecified protocol time points Staff, researchers, patients, study team Quantitative: abstracted outcomes; qualitative: observation PDSA (Plan-Do-Study-Act cycles) Roberts, 2020 [ 33 ]; Wu, 2016 [ 29 ] Five and two cycles, respectively, informing adaptations Staff, researchers, clinicians, patients, team Quantitative: abstracted outcomes; qualitative: interviews, observation MRC (Medical Research Council Framework) Nordhausen, 2022 [ 24 ] Three phases with continuous on-site monitoring and team meetings for emerging issues Study staff Quantitative: outcomes; qualitative: notes, observation Usability and Agile Principles Warrington, 2019 [ 36 ] Iterative testing, feedback collection and integration through regular review Oncologists, nurses, informatics experts, patient reps, researchers Quantitative: outcomes, surveys; qualitative: interviews, feedback, observations User-Centered/Design Principles Hassett, 2022 [ 22 ] Weekly/monthly meetings to refine design and troubleshoot Build team, technical leads, site investigators, patients, content experts Qualitative: user feedback Action Research/Principles Fernández-Méndez, 2018 [ 35 ] Iterative cycles of planning, action and monitoring, with regular practical issue discussions Clinical/admin teams, patients, software developers Quantitative: outcomes, survey; qualitative: logs Framework for Implementation Outcomes Misplon, 2022 [ 23 ] Feedback and workload/validation data collection Study/clinician team Quantitative: validation questionnaire, workload, outcomes; qualitative: interviews No Specific Framework or Approach Palos, 2021 [ 28 ]; Schuler, 2021 [ 34 ]; Van Egdom, 2019 [ 31 ] Provider-reported challenges, group solutions, continuous improvement or phased evaluation Multidisciplinary teams: clinical/administrative, cancer survivors, study authors Quantitative: outcomes, survey; qualitative: feedback, informal observation Greenhalgh Integration Feedback Engelen, 2010 (21 ) Feedback during implementation Study authors, pediatric oncologists Qualitative: unstructured feedback No Specific Approach Furlong, 2019 [ 37 ] Monthly evaluation meetings in two phases Multinational representatives, clinicians, tech partner, researchers Quantitative: outcomes, feasibility evaluation forms FRAME Framework for Modifications and Adaptations, REP replicating effective programmes, MRC Medical Research Council framework, CFIR Consolidating Framework for Implementation Research, RE-AIM Reach, Effectiveness, Adoption, Implementation, and Maintenance framework, iPARIHS Promoting Action Research in Health Services framework, PDSA Plan-Do-Study-Act cycles, FITT Fit between Individuals, Task and Technology framework

Theoretical frameworks for implementation processes

In n = 15 (68%) studies, one (or more) framework(s) or principles were used to guide the development, implementation, adaptation, and/or evaluation [ 16 , 19 – 27 , 29 , 30 , 33 , 35 , 36 ] such as Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM), Consolidated Framework for Implementation Research (CIFR), and Fit between Individuals, Task and Technology (FITT) (see all listed in Table 4 ). Some frameworks or principles were specifically used to guide the adaptation process. The frameworks or principles guided the researchers to make adaptations at varying time points in the implementation process, which varied from 3 to 52 months (mean duration = 24.5 months SD 3.2 months). The majority of the studies made adaptations at pre-specified time points ( n = 11/19), some at random time points ( n = 7/11), and one included a combination of both ( n = 1/19). The reason we report out of 19 and not 20 studies is that Bamgboje‐Ayodele [ 20 ] and Girgis [ 19 ] used the same dataset and were part of the same study. The frameworks were applied in ways that either provided step-wise guidance on how to perform the adaptation process [ 19 , 20 , 29 , 33 , 35 ] or by guiding the reporting on and categorisation of adaptations [ 26 ] (e.g. creating three pilot versions; prioritising adaptations in order of importance). Only one study exclusively reported the use of user-centred design principles [ 22 ]. To provide an example of how a framework guided the implementation process, Furlong et al. [ 37 ] used RE-AIM framework, which has five domains: (i) reach of the target population, (ii) effect on key outcomes, (iii) adoption by people responsible for its delivery, (iv) success of its implementation, and (v) potential for it to be maintained. Each of these five domains includes specific questions that researchers or implementers can ask or address, using data sources (e.g., clinical audit logs completed over 3 months, EHR data, and ePROM survey) and at different time points. This is applied arbitrarily or as decided by the particular team. There was no consistency or pattern in terms of the framework used and phases or frequency of data collection, stakeholders involved, access location, ePROMs measurement use, or frequency or types of adaptations made.

Discussion

Principal findings

This review summarises the type of adaptations made to ePROMs during implementation using data from n = 20 studies. Various data collection methods and stakeholders were utilised to make adaptations, guided by one or more implementation frameworks (80% of studies). Common types of adaptations included changing context (e.g. complex onboarding), content (e.g. readability) (all studies), evaluation (e.g. alerts), and training of patients and clinicians (e.g. training). No evidence was found for the type of implementation framework or the number of types of adaptations made. Overall, the existing evidence of ePROMs adaptations made during implementation in clinical care in the published literature is rather limited, and although the studies contain varying degrees of details of the adaptations made, there is no standardised method for reporting these adaptations or assessing their efficacy.

At the time of writing this review, this was the first review to date on the topic of how studies undertake and report on adaptations of ePROMs/ePREMs systems for integration into oncology care. In addition, there are no reports on the reasons and types of adaptations made to system design and/or implementation strategy to promote implementation. Similar reviews exist on the implementation of ePROMs systems in general [ 2 , 8 , 39 , 42 ], but not explicitly on the adaptations made during process implementation, which is a unique contribution of this review to the literature gap. One of the recent reviews also shows similar findings related to the adaptations [ 39 ]. For example, the review demonstrated that adaptations require personalisation and are often not aligned with clinical reality, calling for the development of ePROMs that would fit better with the existing workflows and programmes [ 39 ]. We found no evidence that an application of a particular implementation framework results in capturing or applying a different number or types of adaptations to any of the dimensions: content, context, evaluation, or training. The data also show no consistency in terms of the implementation framework that guided the implementation process and all other factors such as phases or frequency of data collection, stakeholders involved, access location, choice of ePROMs, or frequency or types of adaptations.

What this study adds and implications for practice

We did not aim to report on the successes or failures of the implementations and adaptations of ePROMs in clinical practice. Instead, we attempted to map which types of adaptations are made within a local context, and how they are reported and captured during the implementation of each study. Each study had made sets of adaptations specific to their local context, but we could not identify any patterns apart from that they have all ended up making changes to the content (e.g. changing visibility or presentation, readability, and feedback for users within the ePROMs system) of ePROMs during implementation, and that all included various stakeholders in the adaptation process. The majority of the changes were therefore to the presentation, but also to alerts, response options, and frequency of administration of ePROMs. We did not collect data on whether the changes improved ePROMs response rates because this was not the objective of the review. It is possible that the complexity of changes was likely individual to every context, cancer group, ePROM, and individual study, which would warrant further review in future. The studies included a large variety of stakeholders in the adaptation process, ranging from oncologists, nursing and allied staff, clinical and administrative teams, technology partners, study staff, study authors, health informatics experts, researchers, advisory workgroups, etc. Although this was not our aim, we became aware that the studies do not describe which method or group of stakeholders was more or less effective in making adaptations and when, because this was entirely individual to each study, their target clinical and patient population, their chosen ePROMs system, the chosen implementation framework(s) and their local contexts [ 11 ].

The studies included in this review have captured the adaptations in various ways, using a structured and unstructured collection of feedback; through mostly qualitative but also quantitative feedback, such as (semi-structured) interviews, focus groups, reports, technological feasibility evaluation forms, observation notes, meeting logs, evaluation of transcripts at engagement meetings and detailed notes, and abstracted outcomes. Mixed methods are a preferred method to capture data in this context; however, it was not an objective of this review to evaluate the levels of organisational readiness for implementing change, nor the effect the clinicians had on the success or use of ePROMs [ 19 , 43 ].

We could not assess the clinical impact of the adaptations made as has been done in some reviews [ 44 ] because this was not the objective of this review but also because many studies did not report the outcomes of the adaptations but just the adaptations made (usually as a list or within the text). However, we succeeded in offering a comprehensive, detailed (up to May 5, 2023, for databases and July 24, 2023, for grey literature) description of types and reasons for adaptations of oncology ePROMs during implementation—which should be of great value to any implementation scientists wishing to develop, evaluate, or implement PROMs into clinical practice across different clinical specialties.

There were no studies found that detailed the adaptation of ePREMs in routine cancer care. The reasons for this are unknown; however, it could possibly be due to differences in how ePROMs versus ePREMs are used and/or studied. Future research should focus on the use of ePREMs as well as ePROMs more, particularly how they are adapted for local contexts. Also, this review identified the limited data on the use of ePROMs in in-patients, and these groups could benefit from being included more frequently in this literature. This could point to a broader problem of not (yet) including people’s experience measures (ePREMs) in care as frequently as their medical outcomes (ePROMs) [ 45 ].

Strengths and limitations of the study

The strengths of this review include compliance with PRISMA-ScR guidelines, the development of a comprehensive search strategy, and the review of results applying the PCC framework [ 14 ]. The results of the review help further the conversation on ePROMs implementation needs in oncological clinical practice and shed light on the interaction between people and technology in a real-world clinical setting. A closer inspection is required to elucidate the interplay between patient management based on ePROMs results and the success of care. The findings of this review are likely transferable and generalisable to other ePROMs in medical fields, which include other patient populations and countries. This is because digital ePROMs are digital tools, and digital tools of a certain nature often share similar characteristics. For example, ePROMs are designed with a specific purpose of gathering patient information for more streamlined and patient-centred care. The issue with patient-centred data is that it is not always collected efficiently, in a cost-effective manner or frequently enough. Introducing ways to improve this process, i.e. by adapting the implementation and use of ePROMs, may yield increased efficiency, better response rates, and data quality, which in turn enhances clinical decision-making or enables better identification of patients’ health changes or health needs. Since all ePROMs are designed with this purpose and implemented into various clinical fields, they likely share similar challenges.

The review also has some limitations, including the relatively small number of databases and papers included in the review. The grey literature search resulted in adding one paper which was published in the English language. Grey literature from other languages could potentially result in capturing more studies and more insights or details of adaptations made during the implementation of ePROMs as part of the routine oncology care. Another limitation of this review is that it was not possible to capture or describe each of the implementation frameworks used in each study due to the sheer size and aspects of this information. A separate review could be carried out in describing just the implementation of different frameworks used in ePROMs in oncology.

Clearly, we could only examine the material available to us. We did not investigate the effectiveness or outcomes of the adaptations, but rather focused on identifying and reporting them. Our aim was to understand the nature of the adaptations, not their impact (e.g. in practice or for oncology populations), as this was beyond the scope of our study. The general trend we observed was that adaptations made to ePROMs during the implementation are highly contextualised to the local setting in which they are implemented. This observation aligns with findings from other studies [ 2 ].

Studies have also pointed to the importance of clinicians in implementing systematic ePROMs/ePREMs, with increased engagement from clinicians leading to higher success rates in patient enrolment and responses [ 40 ]. Our review did not assess the clinicians’ involvement, be it to elicit patients’ feedback or as part of the adaptation process. Not describing clinicians’ involvement with ePROMs/ePREMs as part of adaptations more broadly can mean that we have not considered adaptations that happen on a more macro level [ 41 , 46 ].

Lastly, this review had several methodological limitations. We could not capture the function of PROMs at different levels (e.g. PROMs as a screening tool, a management tool to identify and prioritise issues, or a tool to improve patient-physician information [ 41 , 47 ]). Additionally, our search of literature could have been widened, and findings of ePROMs adaptations could have been compared based on the different conditions, such as cancer and non-cancer diseases, for example. We could not assess if specific feedback and later types of adaptations were linked to different functions of PROMs, nor if different implementation frameworks yielded different adaptation types. Also, our exclusion criteria included studies that exclusively focus on looking at the effectiveness of ePROMs/PREMs, but details of adaptations may have been included in such studies. We also recognise that the review was conducted in a specific timeframe and data extracted by specific researchers and as such, it is apparent that any potentially relevant new literature published afterwards will not have been included. This time lag may limit the comprehensiveness of the most-up-to-date evidence.

Conclusion

To our knowledge, this was the first scoping review to explicitly describe and categorise adaptations made to ePROMs in oncology during digital implementation. Previous and existing reviews briefly mention adaptations, but do not describe them in detail. Although an increasing amount of literature continues to be published on the implementation of ePROMs systems in oncology care worldwide, several “grey areas” remain. These include systematically reporting and managing feedback from patients and clinicians, as well as measuring the effectiveness of adaptations. This review may broadly suggest that the adaptations are dependent on local needs and structures, and what works for whom and when. The findings of this review demonstrate that adaptations during an implementation process for ePROMs in oncology happen by using phased approaches, holding regular meetings with stakeholders, continuous testing and monitoring of the tool and feedback, using multiple clinical champions, and not relying solely on the quantitative data but instead relying more heavily on qualitative reports of immediate users. Future research could focus on describing factors for the overall success of an adaptation (i.e. better outcomes).

Supplementary Information

Below is the link to the electronic supplementary material. ESM 1 (DOCX 17.6 KB)

Marie Fallon and Luc Deliens shared co-last authorship.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Acknowledgements

The authors would like to thank Horizon 2020 for sponsoring the wider project this study is part of and all authors of the studies who have responded to our emails, providing additional data. We would also like to thank all consortium members.

**The MyPath consortium:

Kate Absolom 11,12 , Morten Andresen 9 , Marek Atter 10 , Dag Ausen 9 , Sara Bea 10 , Kim Beernaert 3 , Augusto Caraceni 13,14 , Andres Cervantes 15,16 , Kathrin Cresswell 4 , Olav Dajani 6 , Judith de Vos-Geelen 8 , Luc Deliens 3 , Felicity Evans 17 , Marie Fallon 10 , Victòria Freitas-Durks 18 , Viviana Fusetti 13 , Inez Gonzalez-Barrallo 18 , Peter Hall 10 , Marianne Jensen Hjermstad 6 , Marisol Huerta 18 , Kristin Solheim Hustad 6 , An Jacobs 3,5 , Stein Kaasa 6 , Lisa Heide Koteng 6 , Geana Paula Kurita 19,20 , Henrik Larsen 21 , Ulrik Lassen 28,29 , Nicola Jane Latino 23 , Tonje Lundeby 6 , Elias David Lundereng 6 , Camilla Charlotte Lykke 21,23 , Giacomo Massa 13,14 , Ulla Mathiesen 21 , Nicoleta Mitrea 24,25 , Daniela Mosoiu 25,26 , Steven Olde Damink 27,28,29 , Helle Pappot 20,21,22 , Koen Pardon 3 , Cathy Payne 30 , Oana Predoiu 25 , Anne-Lore Scherrens 3 , Morena Shkodra 13 , Per Sjøgren 21 , Eivind Storaas 6 , Amaia Urrizola 6 , Peder Heyderdahl Utne 6 , Femke Van Landschoot 3 , Galina Velikova 11,12 , Lorraine Warrington 12 , Naomi White 10 , Robin Williams 7

11. Leeds Cancer Centre, St James’s University Hospital, Leeds, UK

12. Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK

13. Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy

14. Dipartimento di Scienze Cliniche e di Comunità – Dipartimento di eccellenza 2023–2027 Università degli studi di Milano, Milan, Italy

15. Department of Medical Oncology INCLIVA, Biomedical Research Institute, University of Valencia, Valencia, Spain

16. CIBERONC, Instituto Salud Carlos III, Madrid, Spain

17. Department of Scientific & Medical Affairs, European Society for Medical Oncology (ESMO), Lugano, Switzerland

18. Department of Medical Oncology INCLIVA, Biomedical Research Institute, Valencia, Spain

19. Rigshospitalet Copenhagen University Hospital, Department of Oncology and Department of Anaesthesiology, Pain and Respiratory Support, Copenhaguen, Denmark

20. University of Copenhagen, Department of Clinical Medicine, Copenhaguen, Denmark

21. Rigshospitalet Copenhagen University Hospital, Department of Oncology, Section of Palliative Medicine, Copenhaguen, Denmark

22. Rigshospitalet Copenhagen University Hospital, Department of Oncology, Copenhaguen, Denmark

23. Department of Oncology and Palliative Care, North Zealand Hospital, Denmark

24. Department of Fundamental Disciplines and Clinical Prevention, Faculty of Medicine, University of Transilvania from Brasov, Romania

25. Department of Education and National Development, HOSPICE Casa Sperantei, Brasov, Romania

26. Department of Medical and Surgical Specialties, Faculty of Medicine, University of Transilvania from Brasov, Romania

27. Department of Surgery, Maastricht University Medical Centre, Maastricht, The Netherlands

28. NUTRIM Institute of Nutrition and Translational Research in Metabolism, Maastricht University, Maastricht, The Netherlands

29. Department of General, Visceral, Vascular and Transplant Surgery, University Hospital Essen, Essen, Germany

30. European Association for Palliative Care, Belgium.

Author contributions

Conceptualization: LD, KB, AJ, ALS, ER, MF, NF. Methodology: KB, ALS, ER, NF; Formal analysis and investigation: ER, NF, ALS; Writing - original draft preparation: NF, ER; Writing - review and editing: NF, ALS, ER, KB, KC, AJ, TL, KP, RW, FVL, JVG, DA, AU, MJH, MF, LD Resources: LD, MF; Supervision: LD, KB, ALS, MF, SK.

Funding

This research is part of the multicentre European oncology study including 9 cancer centres and 14 partners. MyPath study is funded by the European Union (grant no. 101057514) and supported by Innovate UK and the Swiss State Secretariat for Education, Research, and Innovation (SERI). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Health and Digital Executive Agency (HADEA). Neither the European Union nor the granting authority can be held responsible for them.

Data availability

Further information about the search strategies and data extraction are available from the corresponding author on reasonable request.

Declarations

Ethics approval

As this study represents a scoping literature review, no formal ethics approval was required.

Consent for publication

N/A.

Competing interests

The authors declare no competing interests.

References

1. Bennett AV Jensen RE Basch E Electronic patient-reported outcome systems in oncology clinical practice CA Cancer J Clin 2012 62 5 336 347 10.3322/caac.21150 22811342 Bennett AV, Jensen RE, Basch E (2012) Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 62(5):336–347 10.3322/caac.21150 22811342

2. Glenwright BG Simmich J Cottrell M O’Leary SP Sullivan C Pole JD Facilitators and barriers to implementing electronic patient-reported outcome and experience measures in a health care setting: a systematic review J Patient Rep Outcomes 2023 7 1 13 36786914 10.1186/s41687-023-00554-2 PMC9928985 Glenwright BG, Simmich J, Cottrell M, O’Leary SP, Sullivan C, Pole JD et al (2023) Facilitators and barriers to implementing electronic patient-reported outcome and experience measures in a health care setting: a systematic review. J Patient Rep Outcomes 7(1):13 36786914 10.1186/s41687-023-00554-2 PMC9928985

3. Kingsley C Patel S Patient-reported outcome measures and patient-reported experience measures Br J Anaesth 2017 17 4 137 144 Kingsley C, Patel S (2017) Patient-reported outcome measures and patient-reported experience measures. Br J Anaesth 17(4):137–144

4. Chen J Ou L Hollis SJ A systematic review of the impact of routine collection of patient-reported outcome measures on patients, providers, and health organisations in an oncologic setting BMC Health Serv Res 2013 13 1 211 23758898 10.1186/1472-6963-13-211 PMC3700832 Chen J, Ou L, Hollis SJ (2013) A systematic review of the impact of routine collection of patient-reported outcome measures on patients, providers, and health organisations in an oncologic setting. BMC Health Serv Res 13(1):211 23758898 10.1186/1472-6963-13-211 PMC3700832

5. Kotronoulas G Kearney N Maguire R Harrow A Di Domenico D Croy S What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials J Clin Oncol 2014 32 14 1480 1501 24711559 10.1200/JCO.2013.53.5948 Kotronoulas G, Kearney N, Maguire R, Harrow A, Di Domenico D, Croy S et al (2014) What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 32(14):1480–1501 24711559 10.1200/JCO.2013.53.5948

6. Yang LY Manhas DS Howard AF Olson RA Patient-reported outcome use in oncology: a systematic review of the impact on patient-clinician communication Support Care Cancer 2018 26 1 41 60 28849277 10.1007/s00520-017-3865-7 Yang LY, Manhas DS, Howard AF, Olson RA (2018) Patient-reported outcome use in oncology: a systematic review of the impact on patient-clinician communication. Support Care Cancer 26(1):41–60 28849277 10.1007/s00520-017-3865-7

7. Howell D Molloy S Wilkinson K Green E Orchard K Wang K Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors Ann Oncol 2015 26 9 1846 1858 25888610 10.1093/annonc/mdv181 Howell D, Molloy S, Wilkinson K, Green E, Orchard K, Wang K et al (2015) Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol 26(9):1846–1858 25888610 10.1093/annonc/mdv181

8. Anatchkova M Donelson SM Skalicky AM McHorney CA Jagun D Whiteley J Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer-reviewed literature J Patient Rep Outcomes 2018 2 1 64 30588562 10.1186/s41687-018-0091-0 PMC6306371 Anatchkova M, Donelson SM, Skalicky AM, McHorney CA, Jagun D, Whiteley J (2018) Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer-reviewed literature. J Patient Rep Outcomes 2(1):64 30588562 10.1186/s41687-018-0091-0 PMC6306371

9. Miranda R Raemdonck E Deliens L Kaasa S Zimmermann C Rodin G Do cancer centres and palliative care wards routinely measure patients’ quality of life? An international cross-sectional survey study Support Care Cancer 2023 31 8 499 37523097 10.1007/s00520-023-07964-4 Miranda R, Raemdonck E, Deliens L, Kaasa S, Zimmermann C, Rodin G et al (2023) Do cancer centres and palliative care wards routinely measure patients’ quality of life? An international cross-sectional survey study. Support Care Cancer 31(8):499 37523097 10.1007/s00520-023-07964-4

10. Foster A Croot L Brazier J Harris J O’Cathain A The facilitators and barriers to implementing patient-reported outcome measures in organisations delivering health-related services: a systematic review of reviews J Patient Rep Outcomes 2018 2 1 46 30363333 10.1186/s41687-018-0072-3 PMC6170512 Foster A, Croot L, Brazier J, Harris J, O’Cathain A (2018) The facilitators and barriers to implementing patient-reported outcome measures in organisations delivering health-related services: a systematic review of reviews. J Patient Rep Outcomes 2(1):46 30363333 10.1186/s41687-018-0072-3 PMC6170512

11. Stover AM Haverman L Van Oers HA Greenhalgh J Potter CM Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings Qual Life Res 2021 30 11 3015 3033 32651805 10.1007/s11136-020-02564-9 PMC8528754 Stover AM, Haverman L, Van Oers HA, Greenhalgh J, Potter CM (2021) Using an implementation science approach to implement and evaluate patient-reported outcome measures (PROM) initiatives in routine care settings. Qual Life Res 30(11):3015–3033 32651805 10.1007/s11136-020-02564-9 PMC8528754

12. Nguyen H Butow P Dhillon H Sundaresan P A review of the barriers to using patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs) in routine cancer care J Med Radiat Sci 2021 68 2 186 195 32815314 10.1002/jmrs.421 PMC8168064 Nguyen H, Butow P, Dhillon H, Sundaresan P (2021) A review of the barriers to using patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs) in routine cancer care. J Med Radiat Sci 68(2):186–195 32815314 10.1002/jmrs.421 PMC8168064

13. Black N Patient-reported outcome measures could help transform healthcare BMJ 2013 346 f167 23358487 10.1136/bmj.f167 Black N (2013) Patient-reported outcome measures could help transform healthcare. BMJ 346:f167 23358487 10.1136/bmj.f167

14. Stirman SW Miller CJ Toder K Calloway A Development of a framework and coding system for modifications and adaptations of evidence-based interventions Implement Sci 2013 8 1 65 23758995 10.1186/1748-5908-8-65 PMC3686699 Stirman SW, Miller CJ, Toder K, Calloway A (2013) Development of a framework and coding system for modifications and adaptations of evidence-based interventions. Implement Sci 8(1):65 23758995 10.1186/1748-5908-8-65 PMC3686699

15. Wiltsey Stirman S Baumann AA Miller CJ The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions Implement Sci 2019 14 1 1 10 31171014 10.1186/s13012-019-0898-y PMC6554895 Wiltsey Stirman S, Baumann AA, Miller CJ (2019) The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci 14(1):1–10 31171014 10.1186/s13012-019-0898-y PMC6554895

16. Rocque GB Dent DN Ingram SA Caston NE Thigpen HB Lalor FR Adaptation of remote symptom monitoring using electronic patient-reported outcomes for implementation in real-world settings JCO Oncol Pract 2022 18 11 e683 e691 10.1200/OP.22.00360 PMC9750550 36306496 Rocque GB, Dent DN, Ingram SA, Caston NE, Thigpen HB, Lalor FR et al (2022) Adaptation of remote symptom monitoring using electronic patient-reported outcomes for implementation in real-world settings. JCO Oncol Pract 18(11):e683–e691 10.1200/OP.22.00360 PMC9750550 36306496

17. Escoffery C Lebow-Skelley E Haardoerfer R Boing E Udelson H Wood R A systematic review of adaptations of evidence-based public health interventions globally Implement Sci 2018 13 1 125 30257683 10.1186/s13012-018-0815-9 PMC6158804 Escoffery C, Lebow-Skelley E, Haardoerfer R, Boing E, Udelson H, Wood R et al (2018) A systematic review of adaptations of evidence-based public health interventions globally. Implement Sci 13(1):125 30257683 10.1186/s13012-018-0815-9 PMC6158804

18. (2018) PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. https://www.acpjournals.org/doi/full/10.7326/M18-0850 . Accessed 02 Nov 2024 10.7326/M18-0850 30178033

19. Girgis A Bamgboje-Ayodele A Rincones O Vinod SK Avery S Descallar J Stepping into the real world: a mixed-methods evaluation of the implementation of electronic patient-reported outcomes in routine lung cancer care J Patient-Rep Outcomes 2022 6 1 70 35723827 10.1186/s41687-022-00475-6 PMC9207870 Girgis A, Bamgboje-Ayodele A, Rincones O, Vinod SK, Avery S, Descallar J et al (2022) Stepping into the real world: a mixed-methods evaluation of the implementation of electronic patient-reported outcomes in routine lung cancer care. J Patient-Rep Outcomes 6(1):70 35723827 10.1186/s41687-022-00475-6 PMC9207870

20. Bamgboje-Ayodele A Avery S Pearson J Mak M Smith K Rincones O Adapting an integrated care pathway for implementing electronic patient-reported outcomes assessment in routine oncology care: lessons learned from a case study J Eval Clin Pract 2022 28 6 1072 1083 35470525 10.1111/jep.13688 Bamgboje-Ayodele A, Avery S, Pearson J, Mak M, Smith K, Rincones O et al (2022) Adapting an integrated care pathway for implementing electronic patient-reported outcomes assessment in routine oncology care: lessons learned from a case study. J Eval Clin Pract 28(6):1072–1083 35470525 10.1111/jep.13688

21. Engelen V, Haverman L, Koopman H, Schouten - Van Meeteren N, Meijer - Van Den Bergh E, Vrijmoet-Wiersma J et al (2010) Development and implementation of a patient-reported outcome intervention (QLIC-ON PROfile) in clinical pediatric oncology practice. Patient Educ Couns 81(2):235–244 10.1016/j.pec.2010.02.003 20189747

22. Hassett MJ Cronin C Tsou TC Wedge J Bian J Dizon DS eSyM: an electronic health record-integrated patient-reported outcomes-based cancer symptom management program used by six diverse health systems JCO Clin Cancer Inform 2022 6 e2100137 34985914 10.1200/CCI.21.00137 PMC9848544 Hassett MJ, Cronin C, Tsou TC, Wedge J, Bian J, Dizon DS et al (2022) eSyM: an electronic health record-integrated patient-reported outcomes-based cancer symptom management program used by six diverse health systems. JCO Clin Cancer Inform 6:e2100137 34985914 10.1200/CCI.21.00137 PMC9848544

23. Misplon S, Marneffe W, Himpe U, Hellings J, Demedts I (2022) Evaluation of the implementation of value-based healthcare with a weekly digital follow-up of lung cancer patients in clinical practice. Eur J Cancer Care 31(6). 10.1111/ecc.13653 10.1111/ecc.13653 35819130

24. Nordhausen T Lampe K Vordermark D Holzner B Al-Ali HK Meyer G An implementation study of electronic assessment of patient-reported outcomes in inpatient radiation oncology J Patient Rep Outcomes 2022 6 1 77 35852715 10.1186/s41687-022-00478-3 PMC9296709 Nordhausen T, Lampe K, Vordermark D, Holzner B, Al-Ali HK, Meyer G et al (2022) An implementation study of electronic assessment of patient-reported outcomes in inpatient radiation oncology. J Patient Rep Outcomes 6(1):77 35852715 10.1186/s41687-022-00478-3 PMC9296709

25. Strachna O Asan O Stetson PD Managing critical patient-reported outcome measures in oncology settings: system development and retrospective study JMIR Med Inform 2022 10 11 e38483 36326801 10.2196/38483 PMC9672998 Strachna O, Asan O, Stetson PD (2022) Managing critical patient-reported outcome measures in oncology settings: system development and retrospective study. JMIR Med Inform 10(11):e38483 36326801 10.2196/38483 PMC9672998

26. McCleary NJ Haakenstad EK Cleveland JLF Manni M Hassett MJ Schrag D Framework for integrating electronic patient-reported data in routine cancer care: an oncology intake questionnaire JAMIA Open 2022 5 3 ooac064 35898610 10.1093/jamiaopen/ooac064 PMC9315161 McCleary NJ, Haakenstad EK, Cleveland JLF, Manni M, Hassett MJ, Schrag D (2022) Framework for integrating electronic patient-reported data in routine cancer care: an oncology intake questionnaire. JAMIA Open 5(3):ooac064 35898610 10.1093/jamiaopen/ooac064 PMC9315161

27. Neal JW Roy M Bugos K Sharp C Galatin PS Falconer P Distress screening through patient-reported outcomes measurement information system (PROMIS) at an academic cancer center and network site: implementation of a hybrid model JCO Oncol Pract 2021 17 11 e1688 e1697 33830852 10.1200/OP.20.00473 Neal JW, Roy M, Bugos K, Sharp C, Galatin PS, Falconer P et al (2021) Distress screening through patient-reported outcomes measurement information system (PROMIS) at an academic cancer center and network site: implementation of a hybrid model. JCO Oncol Pract 17(11):e1688–e1697 33830852 10.1200/OP.20.00473

28. Palos GR Suarez-Almazor ME Launching an electronic patient-reported outcomes initiative in real-time clinical practice JNCI Monogr 2021 2021 57 23 30 10.1093/jncimonographs/lgab005 34478509 Palos GR, Suarez-Almazor ME (2021) Launching an electronic patient-reported outcomes initiative in real-time clinical practice. JNCI Monogr 2021(57):23–30 10.1093/jncimonographs/lgab005 34478509

29. Wu AW White SM Blackford AL Wolff AC Carducci MA Herman JM Improving an electronic system for measuring PROs in routine oncology practice J Cancer Surviv 2016 10 3 573 582 26644190 10.1007/s11764-015-0503-6 PMC4864116 Wu AW, White SM, Blackford AL, Wolff AC, Carducci MA, Herman JM et al (2016) Improving an electronic system for measuring PROs in routine oncology practice. J Cancer Surviv 10(3):573–582 26644190 10.1007/s11764-015-0503-6 PMC4864116

30. Sztankay M, Neppl L, Wintner LM, Loth FL, Willenbacher W, Weger R et al (2019) Complementing clinical cancer registry data with patient-reported outcomes: a feasibility study on routine electronic patient-reported outcome assessment for the Austrian Myeloma Registry. Eur J Cancer Care 28(6). 10.1111/ecc.13154 10.1111/ecc.13154 PMC6900154 31465136

31. Van Egdom LSE Lagendijk M Van Der Kemp MH Van Dam JH Mureau MAM Hazelzet JA Implementation of value-based breast cancer care Eur J Surg Oncol 2019 45 7 1163 1170 30638807 10.1016/j.ejso.2019.01.007 Van Egdom LSE, Lagendijk M, Van Der Kemp MH, Van Dam JH, Mureau MAM, Hazelzet JA et al (2019) Implementation of value-based breast cancer care. Eur J Surg Oncol 45(7):1163–1170 30638807 10.1016/j.ejso.2019.01.007

32. Wintner LM, Sztankay M, Riedl D, Rumpold G, Nickels A, Licht T et al (2021) How to implement routine electronic patient-reported outcome monitoring in oncology rehabilitation. Int J Clin Pract 75(4) 10.1111/ijcp.13694 PMC8047907 32885565

33. Roberts NA Alexander K Wyld D Janda M Statistical process control assessed implementation fidelity of patient-reported outcome measures (PROMs) in routine care J Clin Epidemiol 2020 127 76 86 32569618 10.1016/j.jclinepi.2020.06.022 Roberts NA, Alexander K, Wyld D, Janda M (2020) Statistical process control assessed implementation fidelity of patient-reported outcome measures (PROMs) in routine care. J Clin Epidemiol 127:76–86 32569618 10.1016/j.jclinepi.2020.06.022

34. Schuler T Back M Hruby G Carroll S Jayamanne D Kneebone A Introducing computed tomography simulation-free and electronic patient-reported outcomes-monitored palliative radiation therapy into routine care: clinical outcomes and implementation experience Adv Radiat Oncol 2021 6 2 100632 33851063 10.1016/j.adro.2020.100632 PMC8039552 Schuler T, Back M, Hruby G, Carroll S, Jayamanne D, Kneebone A et al (2021) Introducing computed tomography simulation-free and electronic patient-reported outcomes-monitored palliative radiation therapy into routine care: clinical outcomes and implementation experience. Adv Radiat Oncol 6(2):100632 33851063 10.1016/j.adro.2020.100632 PMC8039552

35. Fernández-Méndez R Rastall RJ Sage WA Oberg I Bullen G Charge AL Quality improvement of neuro-oncology services: integrating the routine collection of patient-reported health-related quality-of-life measures Neuro-Oncol Pract 2019 6 3 226 236 10.1093/nop/npy040 PMC6656295 31385996 Fernández-Méndez R, Rastall RJ, Sage WA, Oberg I, Bullen G, Charge AL et al (2019) Quality improvement of neuro-oncology services: integrating the routine collection of patient-reported health-related quality-of-life measures. Neuro-Oncol Pract 6(3):226–236 10.1093/nop/npy040 PMC6656295 31385996

36. Warrington L Absolom K Holch P Gibson A Clayton B Velikova G Online tool for monitoring adverse events in patients with cancer during treatment (eRAPID): field testing in a clinical setting BMJ Open 2019 9 1 e025185 30782751 10.1136/bmjopen-2018-025185 PMC6361332 Warrington L, Absolom K, Holch P, Gibson A, Clayton B, Velikova G (2019) Online tool for monitoring adverse events in patients with cancer during treatment (eRAPID): field testing in a clinical setting. BMJ Open 9(1):e025185 30782751 10.1136/bmjopen-2018-025185 PMC6361332

37. Furlong E Darley A Fox P Buick A Kotronoulas G Miller M Adaptation and implementation of a mobile phone-based remote symptom monitoring system for people with cancer in Europe JMIR Cancer 2019 5 1 e10813 30869641 10.2196/10813 PMC6437605 Furlong E, Darley A, Fox P, Buick A, Kotronoulas G, Miller M et al (2019) Adaptation and implementation of a mobile phone-based remote symptom monitoring system for people with cancer in Europe. JMIR Cancer 5(1):e10813 30869641 10.2196/10813 PMC6437605

38. Greenhalgh J Long AF Flynn R The use of patient-reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med 2005 60 4 833 843 15571900 10.1016/j.socscimed.2004.06.022 Greenhalgh J, Long AF, Flynn R (2005) The use of patient-reported outcome measures in routine clinical practice: lack of impact or lack of theory? Soc Sci Med 60(4):833–843 15571900 10.1016/j.socscimed.2004.06.022

39. Sasseville M Supper W Gartner JB Layani G Amil S Sheffield P Gagnon MP Hudon C Lambert S Attisso E Ouellet S Breton M Poitras ME Roux-Lévy PH Plaisimond J Bergeron F Ashcroft R Wong ST Groulx A Paquette JS D'Anjou N Langlois S LeBlanc A Electronic implementation of patient-reported outcome measures in primary health care: mixed methods systematic review J Med Internet Res 2025 27 e63639 40324173 10.2196/63639 PMC12089857 Sasseville M, Supper W, Gartner JB, Layani G, Amil S, Sheffield P, Gagnon MP, Hudon C, Lambert S, Attisso E, Ouellet S, Breton M, Poitras ME, Roux-Lévy PH, Plaisimond J, Bergeron F, Ashcroft R, Wong ST, Groulx A, Paquette JS, D’Anjou N, Langlois S, LeBlanc A (2025) Electronic implementation of patient-reported outcome measures in primary health care: mixed methods systematic review. J Med Internet Res 27:e63639 40324173 10.2196/63639 PMC12089857

40. Lungu DA Pennucci F De Rosis S Romano G Melfi F Implementing successful systematic Patient Reported Outcome and Experience Measures (PROMs and PREMs) in robotic oncological surgery: the role of physicians Int J Health Plann Manage 2020 35 3 773 787 31793689 10.1002/hpm.2959 Lungu DA, Pennucci F, De Rosis S, Romano G, Melfi F (2020) Implementing successful systematic Patient Reported Outcome and Experience Measures (PROMs and PREMs) in robotic oncological surgery: the role of physicians. Int J Health Plann Manage 35(3):773–787 31793689 10.1002/hpm.2959

41. Deyo RA Carter WB Strategies for improving and expanding the application of health status measures in clinical settings. A researcher-developer viewpoint Med Care 1992 30 5 Suppl MS176 MS186 1583932 10.1097/00005650-199205001-00015 Deyo RA, Carter WB (1992) Strategies for improving and expanding the application of health status measures in clinical settings. A researcher-developer viewpoint. Med Care 30(5 Suppl):MS176–MS186 1583932 10.1097/00005650-199205001-00015

42. Fontaine G Poitras ME Sasseville M Barriers and enablers to the implementation of patient-reported outcome and experience measures (PROMs/PREMs): protocol for an umbrella review Syst Rev 2024 13 96 38532492 10.1186/s13643-024-02512-5 PMC10964633 Fontaine G, Poitras ME, Sasseville M et al (2024) Barriers and enablers to the implementation of patient-reported outcome and experience measures (PROMs/PREMs): protocol for an umbrella review. Syst Rev 13:96 38532492 10.1186/s13643-024-02512-5 PMC10964633

43. Rutherford C Campbell R Tinsley M Speerin R Soars L Butcher A Implementing patient-reported outcome measures into clinical practice across NSW: mixed methods evaluation of the first year App Res Qual Life 2021 16 3 1265 1284 Rutherford C, Campbell R, Tinsley M, Speerin R, Soars L, Butcher A et al (2021) Implementing patient-reported outcome measures into clinical practice across NSW: mixed methods evaluation of the first year. App Res Qual Life 16(3):1265–1284

44. Arumalla N Adas M Chan CKD Gibson M Man YL Norton S Galloway J Garrood T The clinical impact of electronic patient-reported outcome measures in remotely monitoring inflammatory arthritis: a systematic review Rheumatology 2023 75 11 1892 1903 10.1002/art.42559 37204273 Arumalla N, Adas M, Chan CKD, Gibson M, Man YL, Norton S, Galloway J, Garrood T (2023) The clinical impact of electronic patient-reported outcome measures in remotely monitoring inflammatory arthritis: a systematic review. Rheumatology 75(11):1892–1903 10.1002/art.42559 37204273

45. Zhao J, Gao S, Wang J, Liu X, Hao Y (2016) Differentiation between two healthcare concepts: person-centered and patient-centered care. Int J Nurs Sci 3. 10.1016/j.ijnss.2016.08.009

46. Krawczyk M Sawatzky R Schick-Makaroff K Micro-Meso-Macro practice tensions in using patient-reported outcome and experience measures in hospital palliative care Qual Health Res 2019 29 4 510 521 29542400 10.1177/1049732318761366 Krawczyk M, Sawatzky R, Schick-Makaroff K et al (2019) Micro-Meso-Macro practice tensions in using patient-reported outcome and experience measures in hospital palliative care. Qual Health Res 29(4):510–521 29542400 10.1177/1049732318761366

47. Boyce MB Browne JP Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review Qual Life Res 2013 22 2265 2278 23504544 10.1007/s11136-013-0390-0 Boyce MB, Browne JP (2013) Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review. Qual Life Res 22:2265–2278 23504544 10.1007/s11136-013-0390-0