'Lessons learned' from trialists who adapted a complex intervention for remote delivery within a trial as a result of the COVID-19 pandemic: a scoping review.

Introduction

In March 2020, many trialists evaluating complex interventions faced an unprecedented adaptation situation. For a trial to continue during the COVID-19 pandemic restrictions, an intervention had to be rapidly adapted from face-to-face to remote delivery, rather than being designed to be delivered remotely from the outset. Decisions about how best to adapt a complex intervention, however, are not straightforward [ 1 ]. A single adaptation can cause a number of unintended consequences and affect important aspects of trial generalisability and interpretation [ 2 , 3 ]. Without clear signposting and guidance, trialists will have responded to these challenges in the best way they could [ 4 ]. There is much we can learn from this period about what trialists need to consider when adapting a complex intervention for remote delivery in the future.

Complex interventions are typically defined as interventions with multiple different, interacting components [ 5 , 6 ]. One of these interacting components is intervention mode of delivery, which can be defined as the method or methods by which intervention content is brought to its target population [ 7 ], for example, via video call or telephone [ 8 ]. During the COVID-19 pandemic, trialists had little opportunity to consider the appropriateness of intervention mode of delivery with the study design and population. Decisions were often guided by the ability to deliver intervention content remotely within pandemic lockdown restrictions, typically achieved using digital technologies [ 9 ]. However, intervention content does not always translate seamlessly from one mode of delivery to another [ 10 ]. Additional adaptations would likely have been required that could mean the intervention adapted for remote delivery is not the equivalent to that delivered before adaptation [ 2 ]. Furthermore, trialists will have had to consider how participants could engage with the intervention in their own homes, instead of the originally planned setting. This likely required additional planning and logistics, particularly if delivering the intervention via digital technologies [ 11 , 12 ].

While guidance exists to support the adaptation of complex interventions [ 13 – 16 ], considerations for the adaptation of mode of delivery during the evaluation stages are lacking. Furthermore, the adaptation of complex interventions for remote delivery via digital technologies requires specific considerations of the environment in which the intervention is being implemented [ 14 , 17 ]. There is limited evidence on how best to implement interventions remotely using digital technologies within a trial [ 18 ].

A scoping review can provide an overview of a broad research field and identify areas for more in-depth research [ 19 ]. This is particularly valuable where evidence is widely dispersed and extensive [ 20 ], as is the case for research from the COVID-19 period. Gaining a better understanding of trialists’ experiences of adapting interventions for remote delivery during the COVID-19 pandemic will identify where more in-depth investigation and guidance is needed to support remote intervention delivery within a trial. This learning is particularly relevant for the conduct of future trials within health and care services that have changed to more remote delivery models as a result of COVID-19 [ 21 ].

Aims and objectives

This scoping review aims to explore the literature reporting interventions that were adapted to remote delivery within a trial as a result of the COVID-19 pandemic to identify the ‘lessons learned’ related to intervention adaptation and remote delivery . Specifically, it aims to 1. identify the methods trial teams reported to adapt a complex intervention for remote delivery and 2. identify the challenges and facilitators of the process of adaptation of a complex intervention and its remote delivery.

Methods

The review was conducted and is reported according to the JBI methodology for scoping reviews which provides a structured approach to defining objectives, identifying relevant studies, and synthesising evidence [ 20 ]. It is reported in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) guidelines [ 22 ]. The review followed an a priori protocol, which is registered and publicly available on the Open Science Framework [ 23 ].

Search strategy

An initial limited search of PUBMED and SCOPUS was undertaken in May 2024 to identify articles on the review topic and inform the search strategy. In accordance with the JBI manual, a selection of articles was used to identify keywords and phrases related to this study [ 24 – 31 ]. The final search terms incorporated a combination of four key concepts from the research question: 1. adaptation, 2. complex intervention, 3. remote delivery, and 4. randomised controlled trial. An information specialist was consulted to develop the final search strategy. The original search was developed for MEDLINE and adapted for Web of Science, PsycINFO, EMBASE, and COCHRANE.

The term ‘COVID-19’ was not used in the search due to the large number of papers retrieved. Searches were restricted to March 2020–May 2024, with May 2024 reflecting the date the searches were conducted. This timeframe ensured the inclusion of literature published during the COVID-19 period. Although the WHO declared COVID-19 was no longer a global emergency in May 2023, relevant literature continues to be published beyond this point. There was a lack of consistency in how papers related to ‘lessons learned’ had been indexed. Therefore, MESH terms were not used. The full search strategy is presented in Additional file 1.

Eligibility (inclusion/exclusion) criteria

This scoping review considered studies reporting quantitative, qualitative, and mixed methods research. Reviews in any form (literature, scoping, or systematic) that met the criteria were considered. Primary studies included in existing reviews were assessed for eligibility. Conference abstracts and protocols were not considered.

This review used the Participant, Concept, Context (PCC) approach for developing eligibility criteria [ 20 ]. We included research literature that met the following PCC criteria in this scoping review.

Population

The focus of our scoping review is lessons learned from trialists delivering trials of complex interventions conducted on human participants.

Concept

The key concepts are ‘lessons learned’ about adapting a complex intervention for remote delivery within a trial as a result of the COVID-19 pandemic. This is further specified by [ 1 ] methods taken by trial teams to adapt a complex intervention for remote delivery; [ 2 ] challenges and facilitators related to the process of adapting a complex intervention to remote delivery within a trial; and [ 3 ] challenges and facilitators related to the remote delivery of a complex intervention within a trial.

Moore et al. [ 16 ] define adaptation as the ‘intentional modification(s) of an evidence-informed intervention, in order to achieve a better fit between an intervention and a new context’. They distinguish between planned adaptations and responsive adaptations. This scoping review is interested in understanding the challenges and facilitators of the responsive adaptations that trial teams made to a complex intervention as a result of the COVID-19 pandemic.

The NIHR Remote Trial Delivery Working Group defines remote delivery as ‘all activities related to clinical trial delivery undertaken without in-person, face-to-face contact’ [ 4 ]. This review is interested in the challenges and facilitators of remote delivery of the intervention being evaluated (as opposed to other trial processes such as recruitment and collecting consent).

Methods taken to adapt a complex intervention are defined as any process a trial team takes to adapt an intervention from in-person to remote delivery.

Context

Reports were included from randomised controlled trials (pilot/feasibility/phase 3 or 4) conducted in any geographical location. Reports published between March 2020 and May 2024 were included in this review. This review team selected March 2020 as the cut-off as this was when the World Health Organisation declared COVID-19 a pandemic.

Identification of eligible studies

Following the search, all identified citations were uploaded into Rayyan [ 32 ] and duplicates were removed. Titles and abstracts were screened by EH against the eligibility criteria for the review, and a random 10% were checked by LZ. Potentially relevant reports that met the PCC criteria outlined above were retrieved in full and their details imported into Rayyan. The full text of these reports was assessed in detail against the eligibility criteria by EH with a further 20% checked by LZ. Discrepancies were discussed and resolved by EH and LZ. Reasons for exclusion at the full text stage were recorded.

Data extraction

Data were extracted from papers included in the scoping review by EH using a data extraction tool developed for the purpose of this review. The tool was used to extract specific details about the publication as well as data related to the review questions. Extracted data on study characteristics included country of origin, study design, and aims. The data extracted for the review questions included mode of delivery to deliver intervention content remotely, methods reported to adapt the intervention, and challenges and facilitators of the process of adaptation and of remote intervention delivery. We deviated from the protocol by not extracting data on ‘adaptations made to the intervention’ and ‘adaptations made to comparator’ as data was not easily presentable in the studies included. See Additional file 2 for the data extraction tool.

The table was piloted by EH using a sample of three full-text articles. Issues were discussed with AJF, SS, and KG, and the extraction table modified and revised accordingly.

Data analysis and presentation

We used the upper level of the Behaviour Change Intervention Ontology (BCIO) [ 33 ] as a framework to organise the challenges and facilitators related to remote intervention delivery.

An ontology represents knowledge in a domain by defining key entities within it and the relationship between them [ 34 ]. The BCIO upper level provides a high-level classification of the components of a behaviour change intervention and its evaluation [ 35 ]. The BCIO has applicability to complex interventions more generally. Firstly, the majority of complex interventions have a behavioural aspect [ 36 ]. Secondly, the development of the BCIO was informed by the CONSORT guidelines for reporting of clinical trials and the Template for Intervention Description and Replication (TIDieR) [ 37 – 39 ]. These frameworks are used to support the reporting of complex interventions in trials (Fig. 1 ). Fig. 1 Upper level of the Behaviour Change Intervention Ontology adapted from Michie et al. [ 33 ] with example lower classes of mode of delivery

Each challenge or facilitator that related to the remote delivery of the complex intervention was mapped according to the BCIO upper level. Data within each category was grouped and descriptively summarised [ 40 ]. This supported a more granular understanding of the challenges and facilitators that related to remote delivery. For example, data was organised depending on whether a specific challenge related more to the mode in which the intervention was delivered (e.g. via telephone or video call), or the setting in which it was remotely delivered. It also provided an organised and effective way to present the large quantity of qualitative data. A second reviewer (LZ) cross-checked the grouping of the data. Disagreements were resolved through discussion.

Study characteristics were summarised using descriptive statistics. The challenges and facilitators of adaptation and remote intervention delivery were presented in a table and aligned where possible. The challenges and facilitators of adaptation and remote intervention delivery were also summarised narratively. No quality assessment of the included articles was conducted since the aim of the scoping review was to map the breadth of emerging evidence regarding trialists’ experience of adapting a complex intervention for remote delivery, not to critique the quality of included articles.

Results

Identified studies

In the initial database search, 2563 records were identified. Once duplicates were removed, 1522 titles and abstracts were screened for inclusion. Seventy-three full-text studies were assessed for eligibility. Fifteen papers met the eligibility criteria. The search process and results are documented in the PRISMA flow diagram. One paper could not be found online. The authors were contacted but the full text was not retrieved (Fig. 2 ). Fig. 2 PRISMA flow diagram

Characteristics of included studies

Fifteen studies were included, describing insights from 16 randomised controlled trials. Of the 15 reports identified, 13 were from the USA, 1 from Canada, and 1 from the UK. Eleven of the 16 trials were two-arm trials [ 24 , 28 , 41 – 49 ], three were three-arm trials [ 50 – 52 ], and two were four-arm trials [ 31 , 53 ]. Most trials were evaluating an intervention that involved a psychotherapy component, such as motivational interviewing and cognitive behavioural therapy ( n = 8) and/or an exercise component ( n = 6). Three of the trials included were group-based interventions [ 31 , 43 , 48 ]. A range of modes of delivery were used to remotely deliver the intervention. The most common mode of delivery used for remote delivery was video call ( n = 15). As demonstrated in Table 1 , there was a range of participants included in the trials included in this review. Table 1 Study characteristics of included studies Authors and date Country Included study aim Randomised controlled trial design Target population and planned sample size Intervention And Comparator Mode(s) of delivery of originally planned intervention Mode(s) of delivery of intervention adapted for remote delivery Face to face Video call Telephone Paper materials* Text messages Computerised** Not clear Face to face Video call Telephone Paper materials* Text messages Computerised** Not clear Binder 2021 [ 50 ] USA Report of the methods of this study, including adjustments made to the protocol because of the coronavirus-2019 disease (COVID-19) pandemic A three-arm, individually randomised, phase 3 trial of an exercise and testosterone therapy Women > 65 with a recent hip fracture n = 300 Supervised exercise + testosterone gel*** X X X supervised exercise and placebo gel X X X Enhanced usual care (home exercise + health education (Comparator) X X X Everhart 2021 [ 53 ] USA Report to provide updates on modifications made to the RVA Breaths Trial A four-arm, cluster randomised, phase 3 trial of an asthma education intervention (RVA breaths) Children (aged 5–11) in low-income areas n = 300 Asthma education with community health workers X X X X X Home remediation with home assessors, and a school nurse component X X X X Asthma education and home remediation X X X Enhanced standard of care (comparator) X X Fanning 2023 [ 31 ] USA Report to describe process of adapting the IMOVE intervention for delivery via videocall in response to COVID-19 A 2X2 randomised factorial pilot trial of a group-based, improvisational dance intervention Adults with early-stage Alzheimer’s Disease and their caregiver n = 80 (40 dyads) Movement group (movement and social) X X Movement alone X X Social Group X X Usual Group X X Fisher 2021 [ 41 ] USA Report to outline the procedural modifications used to facilitate conversion of an in-person intervention to remote delivery A two-arm individually randomised pilot trial of a cognitive behavioural therapy (CBT) for depression Individuals aged 21–69 with traumatic brain injury n = 40 Cognitive Behavioural Therapy X X X X X Control: 12-week waitlist - - - - - - - - - - - - - - Johnson 2022 [ 42 ] USA Report to describe study challenges and adaptations as a result of COVID-19 research restrictions A two-arm individually randomised phase 3 trial of a healthy relationship tool and safety decision aid to reduce intimate partner violence (IPV) during perinatal period Women aged 18–45 who screen positive for IPV n = 186 Online healthy relationship tool and safety decision aid X X X X Attention-time information matched control condition X X Kashikar-Zuck 2022 [ 51 ] USA Report to describe modifications made to the FIT Teens study protocol in response to the COVID-19 pandemic A three-arm individually randomised phase 3 multi-site trial of a specialised neuromuscular training intervention with Cognitive Behavioural Therapy Adolescents aged 12–17 juvenile fibromyalgia n = 420 FIT teen intervention (combines Cognitive Behavioural Therapy with neuromuscular training) X X X X Cognitive Behavioural Therapy only X X X X X Graded Aerobic Exercise only X X X X Klasko-Foster 2022 [ 29 ] USA Report to provide descriptions of remote adaptations to two RCTs of behavioural interventions for primary and secondary HIV prevention**** 1) A two-arm individually randomised phase 3 trial to establish the efficacy of a behavioural intervention to optimize ART adherence and viral suppression [ 44 ] HIV-infected youth, ages 16 to 29 n = 192 Text messaging reminders to take medication and in-person counselling X X X Standard of care X X 2) A three-arm individually randomised phase 3 trial testing an integrated HIV risk reduction and behavioural activation counselling intervention (IMPACT) for HIV-uninfected, stimulant [ 52 ] Men aged 18 or older n = 286 Behavioural Activation and Cognitive Behavioural Therapy with HIV sexual risk reduction counselling X X A time- and intensity-matched control including relaxation therapy and educational support with counselling X X A standard of care (SOC) comparison condition including counselling only X X Lalande 2021 [ 28 ] USA A tutorial to help clinical researchers transition their in-person research programs to a web-based format A two-arm phase 3 trial, individually randomised at the dyad, of a relationship-education counselling program for couples Couples in which one partner has experienced a cardiac event n = 608 Healing Hearts Together counselling programme X X X Usual care at the hospital X X McCourt 2022 [ 43 ] UK A report to document changes to a previously published protocol, as well as facilitate study design in the immediate post pandemic era A two-arm, individually randomised, pilot trial of an exercise prehabilitation before and during stem cell transplantation Patients with multiple myeloma n = 60–75 Group-based supervised exercise X X X Usual care X X Meline 2021 [ 24 ] USA A report to describe the experience of transitioning to virtual methods to deliver intervention and conduct study visits A two-arm, individually randomised, phase 3 trial of a lung cancer screening decision tool Veterans aged 55–80 with a history of smoking n = 200 A lung Cancer Screening Decision Tool (LCSDecTool) website X X X X X Control intervention website X X X Shear 2022 [ 45 ] USA A report to provide lessons learned throughout the process of adapting to a virtual format A two-arm, individually randomised phase 3 trial examining graded exposure treatment compared to multidisciplinary pain management Youth aged 8–18 with Chronic muscular-skeletal pain n = 74 Graded Exposure Treatment X X X X Multidisciplinary Pain Management (outpatient Cognitive Behavioural Therapy and physical therapy) X X X Sperl-Hillen 2021 [ 46 ] USA Report to describe an RCT to improve chronic kidney disease (CKD) care and the challenges encountered due to COVID-19 A two-arm cluster randomised pragmatic trial to test a chronic kidney disease clinical decision support tool Patients with stage 3 or 4 chronic kidney disease n = 47 primary care clinics Chronic-kidney disease clinical decision support (CKD-CDS) X X X X X Usual care X X Tang 2021 [ 47 ] Canada A report to examine health educators’ experiences and perceptions of transitioning from home visits to remote delivery of motivational interviewing A two-arm cluster randomised, phase 3 trial to test the impact of a home-based obesity prevention intervention Families with children between ages 1.5–5 n = 285 Motivational interviewing X X X X Control X X Vinci 2022 [ 48 ] USA A report to describe the technical and logistical considerations of transitioning from in-person to remote-based delivery for group-based treatment A two-arm group-based individually randomised pilot trial for smoking cessation/alcohol used Adults aged 18 + who are daily cigarette smokers n = 64 Mindfulness-Based Relapse Prevention for Smoking and Alcohol (MBRP-SA) X X X Cognitive Behavioural Therapy X X X Zaccari 2022 [ 49 ] USA A report to present modifications made to pivot a multisite RCT from in-person to virtual study implementation A two-arm group-based individually randomised pilot trial of a yoga and cognitive processing group therapy Women veterans with Post-Traumatic Stress Disorder (PTSD) n = 40 Cognitive Processing Therapy (CPT) X X X X Trauma Centre Trauma Sensitive Yoga (TCTSY) X X X * Printed materials were given as a hard copy to participants in the original intervention. In the adapted intervention, these materials were either posted and/or emailed to participants to print ** Computerised mode(s) of delivery captures intervention content that was viewed/interacted with via a computer (e.g. on a website or video) *** Women were told to continue their prescribed gel (provided to them before the pandemic) unless it was determined it was unsafe **** Trial description has come from the protocol paper for the two papers discussed in this study

Findings

Most papers reported that they had adapted the intervention in order for it to be conducted remotely, but provided little detail about how this was achieved [ 24 , 29 , 41 , 42 , 45 – 51 ]. Five studies briefly reflected on the challenges and facilitators of adaptation [ 24 , 28 , 31 , 45 , 53 ]. The majority of discussion reported focused on the challenges and facilitators of the remote delivery of the complex intervention. Most of these challenges related to the setting in which an intervention was delivered (e.g. challenges around privacy and safety), and the mode of delivery in which an intervention was delivered (e.g. challenges of remote delivery via video call).

Methods that trialists report using to adapt a complex intervention for remote delivery

Four papers highlight that participants were involved in the adaptation process [ 28 , 31 , 43 , 53 ], for example, by consulting them on the proposed changes and gaining their approval [ 43 ]. One study of the adaptation of a group-based movement intervention highlighted how they aimed for fidelity to key aspects that were hypothesised to be important for improving quality of life [ 31 ]. A second study, which was written as a tutorial for researchers adapting to web-based formats, detailed five steps undertaken [ 28 ]. This included consultation and assessments of needs, adaptation of procedures for remote delivery, adaptation of procedures and materials for the research protocol, pilot sessions, and the final launch. Feedback was gathered in the pilot stages from participants who received the in-person intervention. The authors specified how this feedback contributed to treatment fidelity and the acceptability of the web-based format. Lalande et al. [ 28 ] report that the iterative process they took to adapt the intervention was guided by a four-step framework outlined by Hom et al. [ 54 ]. This included consultation, adaptation, pilot testing, and test launch.

Challenges and facilitators of the process of adapting a complex intervention for remote delivery

Five studies briefly reflected on the challenges and facilitators of adaptation [ 24 , 28 , 31 , 45 , 53 ]. Challenges discussed included the time available to iteratively adapt the intervention and specific features of the intervention that made it more difficult to adapt. For example, it was more challenging to adapt the intervention when there were more interactive features [ 24 ]. Key facilitators reported included consulting with others and good communication among members of the team [ 24 , 28 ] (Table 2 ). Table 2 Challenges and facilitators of adaptation Challenges Facilitators • The more complex the intervention, the more refinement it needed [ 31 ] • Good communication among members of the research team facilitated the adaptation process [ 24 , 28 ] • Limited time available to iteratively adapt and get feedback on acceptability and feasibility [ 45 ] • Specific features of an intervention, such as the amount of interactive features [ 24 ] and the degree to which the intervention component was non-verbal [ 31 ] • Having team members with expertise in the programming of the intervention tools [ 24 ] • Consulting with others, such as trial participants and members of the team [ 28 , 53 ]

Challenges and facilitators of remote intervention delivery

All reports discussed challenges and facilitators related to remote intervention delivery. These challenges and facilitators have been organised using the upper level of the Behaviour Change Intervention Ontology [ 33 ]. Specifically, they are organised according to whether the challenges and facilitators are mostly related to the setting in which the intervention was delivered (setting), the individual delivering the intervention (source), the mode in which intervention content was delivered (mode of delivery), the style in which an intervention was communicated (style of delivery), how often the intervention could be delivered (schedule of delivery), the way in which remote delivery of the intervention impacted intervention fidelity (fidelity), and any participant characteristics that made remote intervention delivery more or less challenging (population). Each of these is discussed in detail below. Table 3 details the challenges and facilitators identified within each of these categories. Table 3 Challenges and facilitators of remote intervention delivery BCIO upper level Challenges Potential facilitators Setting The environment in which an intervention is delivered [ 79 ] • Privacy to engage in the intervention in the home environment [ 29 , 48 , 49 , 51 ] • Carrying out an environmental inventory/checklist to identify if participants were in a private space [ 48 ] • Additional precautions implemented (e.g. at the start of group sessions participants are reminded that they must be in a private space or else they must leave the session) [ 51 ] • Availability of space in the home [ 45 ] • Carrying out an environmental inventory/checklist to identify if participant had movement space of a sufficient size [ 31 ] • Equipment availability [ 45 , 50 , 51 ] • Carrying out an environmental inventory/checklist to identify if participants had the equipment needed to carry out the intervention in their home [ 28 , 31 , 45 , 48 , 53 ] • Interruptions and distractions in the home environment [ 45 , 48 , 49 ] • Providing extra information and instruction on how to limit distraction during the session [ 41 ] • Safety concerns related to the risk of falling from completing certain exercises [ 43 , 45 , 50 ] • Safety issues related to who participants live with [ 46 ] • Identifying participants physical location before the start of intervention delivery [ 41 ] • Remove certain balance exercise from intervention [ 50 ] • Updating eligibility criteria for vulnerable participants [ 42 ] and those at risk of falling [ 43 ] • Exercising in non-air-conditioned homes in hot weather [ 45 ] Source The person, population, or organisation that delivers the intervention [ 80 ] • Dealing with distractions and interruptions [ 31 , 48 ] • Difficulties building rapport with participants [ 47 , 48 ] • Providing additional training on how to build rapport online [ 53 ] • Difficulties as a result of not being ‘in the room’ [ 24 , 28 , 45 ] • Extra burden placed on the intervention source [ 24 , 47 , 48 , 51 ] • Emotional fatigue experienced by staff as a result of participants turning to them for support during the COVID-19 pandemic [ 51 ] • Providing extra support for the interventions source [ 29 ] • Questioning importance of role in a virtual setting [ 53 ] • Making deliberate efforts to validate contributions of individuals delivering the intervention [ 53 ] • Providing additional training around using videoconferencing software [ 28 , 48 ] Style of delivery The manner in which intervention content is communicated to intervention participants [ 81 ] • Inability to read subtle non-verbal cues and eye contact [ 31 , 48 ] • Increasing quantity of communication with participants [ 29 , 46 , 53 ] • Adding extra time specifically to build rapport and increase bonding [ 31 , 48 , 53 ] • Reliance on verbal instruction and cues [ 45 ] as therapists were unable to make corrections to form as they might normally do • Describing what should be felt during an exercise [ 45 ] • Using agenda-setting to let participants know what will be happening and using plain language to describe new procedures [ 29 ] • Risk of conversation being dominated by one member of the group [ 31 ] • Active management of group dynamics [ 31 ] • Using open-ended questions to understand participant experiences of COVID-19 [ 45 ] Fidelity The degree to which an intervention is put in practice as intended [ 61 ] • Updating training for the intervention source [ 28 , 51 ], for example, by creating training videos around the updated study protocols [ 51 ] • Challenges with fidelity assessment [ 24 , 46 , 48 , 49 ]. For example, issues related to being unable to directly observe participants and record their progression [ 24 ] and confidentiality issues around recording intervention delivery online [ 48 , 49 ] • Uncertainty of intervention receipt [ 24 , 45 , 46 ]. For example, uncertainty if materials had been printed and/or received in the mail [ 24 , 45 ] • Missed/forgotten sessions online [ 45 ] • Providing text and email reminders for attendance to the virtual settings [ 45 ] • Internet issues accessing an online tool [ 24 ] Mode of delivery The method or methods by which intervention content is brought to its target population [ 7 ] Video call (e.g. Zoom, Microsoft Teams) • Technological issues related to video conferencing software [ 24 , 29 , 31 , 45 , 47 , 49 , 51 ] which one paper reported as being heightened in a group setting [ 49 ] • Providing extra instruction, technological support and assistance for participants [ 28 , 31 , 41 , 48 , 49 ] • Spending time prior to intervention delivery to check equipment was functioning and participants were comfortable with the set up [ 28 , 31 , 41 ] • Having a plan in place for navigating potential difficulties [ 29 , 51 , 53 ] for example, a back-up plan for what participants should do if connection is lost [ 51 ] and agreeing on a cut-off point at which the intervention was delivered via phone instead of via video conferencing • Risk of spyware for a vulnerable participant group [ 42 ] • Difficulties from using devices with different sized screens to engage in the intervention. For example, using a phone or an iPad [ 48 ] • Access to devices in the home if delivering an intervention to more than one person at the same time (e.g. a parent and a child) [ 45 ] • Providing technological equipment for participants [ 41 , 48 ], for example, loaning tablets for those that needed Audio call (telephone) • Participants more distracted on the phone [ 47 ] • Non-verbal cues not possible [ 47 ] • Being ‘more comfortable with the silence’ [ 47 ] • Asking more probing questions [ 47 ] Printed material (posted or emailed to participants) • Difficulties mailing materials to participants with unstable housing [ 29 ] • Having an alternative location available for participants to collect materials, and leveraging community partnerships [ 29 ] • Uncertainty if intervention materials have been received [ 24 ] • Disruptions to mailing service [ 48 ] • Sending out materials was resource consuming [ 48 ] • Variability in who printed the materials that were sent to them [ 45 ] Computerised (website) • User experience impacted by screen size [ 24 ] • Computer programme being used in the intervention was not designed for screen share [ 46 ] • Navigation issues with the intervention might go undetected or misinterpreted [ 24 , 46 ] • Difficulties accessing the study website as a result of internet access [ 24 ] Schedule of delivery The duration, frequency, and timing of intervention delivery • Missed and forgotten virtual visits [ 45 , 46 ] • Providing text and email reminders for attendance to the virtual settings [ 45 ] • Scheduling difficulties related to changes in the home environment as a result of COVID-19 [ 47 , 50 ], for example, having children at home while parents worked full time • Increased opportunity to schedule appointments [ 28 , 46 , 47 ] Population The group of participants involved in the trial • Individuals with photosensitivity could not look at the screen for too long [ 41 ] • Reducing the quantity of time participants needed to look at a screen [ 41 ] • Increased safety risks of engaging with the intervention for vulnerable people who live with abusive partners [ 42 ] • Changing the eligibility criteria of the trial. Specifically, participants were excluded if it was deemed their risk of spyware was too high, or they were at a greater risk for severe injury and/or homicide [ 42 ] • Safety risks for individuals with balance issues [ 43 ] • Including an extra assessment tool to identify participants with balance issues and make necessary adjustments to their participation [ 43 ]

Setting

All the studies in this review reported that they remotely delivered the adapted intervention into participants’ homes. The key challenge trialists report facing in the home setting related to the privacy of engaging in the intervention [ 29 , 48 , 49 , 51 ] and safety concerns related to the risk of falling [ 43 , 45 , 50 ]. Privacy challenges were more commonly associated with interventions that contained a counselling component. Safety issues were more commonly discussed in interventions that contained an exercise component. The most frequently reported facilitator was the use of an environmental inventory/checklist to check the space in which participants were located and the equipment they had available [ 28 , 31 , 45 , 48 , 53 ].

Source

Trialists reported challenges the intervention provider faced as a result of not being ‘in the room’ [ 24 , 28 , 45 ]. For example, not being able to physically correct form during exercise [ 45 ] or directly observe participants’ progression [ 24 ]. Four studies highlighted the extra burden remote delivery placed on the individual delivering the intervention [ 24 , 47 , 48 , 51 ]. For example, by increasing the amount of communication needed with participants and the extra time needed to set up a visit. Key facilitators discussed included providing additional training related to building rapport with participants online [ 53 ] and videoconferencing skills [ 28 , 48 ]. In one study, intervention deliverers were provided with more opportunities for debriefing [ 29 ].

Style of delivery

The inability to read non-verbal cues was reported as a challenge to how intervention content could be communicated [ 31 , 48 ]. Potential solutions included increasing the quantity of communication with participants and adapting communication styles. For example, by using plain language to describe the new procedures [ 29 ].

Fidelity

Remote delivery of the intervention was reported as causing several challenges related to fidelity assessment [ 24 , 46 , 48 , 49 ]. These challenges were related to the mode of delivery in which an intervention was delivered. For example, two papers reported uncertainty of intervention receipt when posting printed materials to participants [ 24 , 45 ]. Recording intervention delivery on a video call to measure fidelity was reported as causing confidentiality issues [ 48 , 49 ]. Few potential solutions to these challenges were discussed by trialists.

Mode of delivery

Video call

Nearly all papers included in this review reported using video call (e.g. Zoom or Microsoft Teams) to deliver the intervention remotely. This mode of delivery was associated with the greatest number of reported challenges. The majority of these challenges related to technological issues [ 24 , 29 , 31 , 45 , 47 , 49 , 51 ]. Potential solutions discussed by trialists mostly related to providing extra instruction, support, and assistance for participants [ 28 , 31 , 41 , 48 , 49 ]. Two papers discussed how certain issues around relational connectivity and rapport building during a video call were heightened in group-based interventions [ 28 , 53 ].

Audio call (telephone)

Several papers provided the option for intervention delivery via the telephone. One study discussed challenges this caused as a result of the lack of non-verbal cues and the potential for participants to get more distracted [ 47 ].

Printed materials (posted or emailed to participants)

Several papers that used printed materials to deliver intervention content reported either posting these materials to participants or emailing them to be printed. However, one study highlighted that when materials were emailed, there was variability in which participants printed the materials [ 45 ]. They highlight that this could have reduced the saliency of the materials, as participants were less able to write and reflect directly on them.

Posting materials to participants was reported as being a resource-consuming activity [ 48 ]. One paper reported challenges if participants had unstable housing [ 29 ]. Having an alternative location available to send materials was discussed as a potential facilitator of this [ 29 ].

Computerised (website)

While the mode of delivery could stay the same for parts of an intervention that were originally delivered via a website, challenges were associated with its remote delivery as a result of not being present with participants while they were set up with the intervention. Other challenges reported included navigation issues going undetected [ 24 , 46 ] and the website not being designed for screen share on a video call [ 46 ]. One study reported that some eligible patients were not able to participate in the intervention because they lacked internet access at home [ 24 ].

Schedule of delivery

Three papers reported the increased opportunity remote delivery provided for scheduling appointments for intervention delivery [ 28 , 46 , 47 ]. For example, it meant there was no need to cancel appointments if a family member was ill or there were parental work commitments [ 46 ]. Two papers reported issues with missed and forgotten visits [ 45 , 46 ]. Providing text reminders was used to address this [ 45 ].

Population

Two papers reported on characteristics of the population for which remote delivery resulted in a few potential safety issues. These included vulnerabilities from a potentially unsafe living situation [ 42 ] and safety issues as a result of balance issues [ 43 ]. One paper reported changing the eligibility criteria to address this [ 42 ]. Specifically, participants were excluded if it was deemed their risk of spyware was too high, or they were at a greater risk for severe injury and/or homicide [ 42 ]. An additional balance tool was introduced to identify potential balance issues and adjust participant participation as a result [ 43 ].

Discussion

This scoping review synthesised insights from trialists about the adaptation of a complex intervention to remote delivery as a result of the COVID-19 pandemic. The review identified a broad range of challenges and facilitators of remote delivery of an intervention within a trial. The majority of these challenges related to the setting in which the intervention was delivered, and the mode of delivery used to deliver the intervention remotely. While this review aimed to identify methods reported to adapt interventions, and challenges and facilitators of the adaptation process, few included studies described this. Several areas for which guidance is needed to support trialists to adapt an intervention for remote delivery have been identified.

Most of the challenges identified in this review related to the home setting in which the intervention was remotely delivered. Some of these challenges, such as interruptions in the environment, were likely heightened during the lockdown period when family members were present in the home environment. However, other challenges, such as those related to the privacy and safety of intervention delivery, have been discussed in the pre-pandemic literature [ 55 – 58 ]. A recent review on privacy and confidentiality in telemedicine also highlighted patients’ concerns about being overheard or seen by others during online consultations [ 59 ]. Within the context of a trial, these issues need to be planned for and anticipated. Not only could they affect intervention fidelity and the validity of trial results [ 60 , 61 ] they raise important questions around the potential risks associated with remote intervention delivery. For example, what happens if participants do not have the space to safely engage in a certain part of the intervention? What should trialists do if participants become distressed? Several papers in this review reported steps taken to manage these issues, for example, carrying out additional privacy and safety checks [ 41 , 48 ]. However, it is unclear how effective these strategies employed by trialists were at mitigating these risks. Future work should identify how trialists can best plan for and mitigate the potential safety and privacy risks of delivering an intervention remotely into a participant’s home. The facilitators identified in this review provide a starting point for future investigation.

To address some of the safety and privacy concerns of remote intervention delivery, some trialists reported making changes to the eligibility criteria for intervention participation. For example, Johnson et al. [ 42 ] excluded people from participating remotely who had a high risk of spyware on their computer. Kashikar-Zuck et al. [ 51 ] describe how unless participants were in a private space, they were not allowed to participate in the remotely delivered group intervention. While these responses might reflect the limited time available to prepare for the safety challenges, it highlights the potential impact of remote intervention delivery on research inclusion for trials. The implications of remotely delivering an intervention using digital technologies on inclusivity have been widely discussed [ 62 – 64 ]. These trials, typically known as decentralised trials (DCTs), could make trials more accessible for individuals who are home bound [ 65 , 66 ]. However, they could also introduce more severe health inequities for socially disadvantaged groups who experience greater challenges accessing digital health technologies [ 67 , 68 ]. This review has highlighted several additional factors in the home setting that could influence inclusion in trials where the intervention is remotely delivered. For example, remote delivery for individuals living in crowded or unstable housing may pose logistical and practical challenges that hinder trial participation [ 69 , 70 ].

This review demonstrated that remote delivery of a complex intervention can be achieved via several different modalities. The most frequently used mode of delivery was video call. Technological and connectivity challenges were commonly reported. To address these challenges, many trialists used additional resources and technological support to ensure participants were comfortable with the new setup. For example, equipment was provided for participants who needed it [ 41 , 48 ] and equipment was checked to ensure functioning prior to intervention delivery [ 28 , 31 , 41 ]. Providing additional support to remotely deliver an intervention has also been identified as an unanticipated consideration of remote delivery in a recent qualitative study by Cooney et al. [ 10 ]. These responses likely incurred additional time and efficiency costs that are important considerations for the long-term viability of a trial intervention and its post-trial implementation [ 71 , 72 ]. To support trialists to adapt an intervention for remote delivery, an understanding of the potential organisation and resource implications is required.

Many of the challenges trialists reported were specific to the modes of delivery used to deliver the intervention remotely. Connectivity issues were a particular challenge for delivery via video call, whereas adapting communication style was more of a consideration for intervention delivery via the telephone. Previous studies have also noted these mode of delivery specific challenges. Irvine et al. [ 73 ] emphasise the need for more explicit verbal cues during telephone interactions to demonstrate ‘active listening’, whereas technological challenges are frequently reported in video-based intervention delivery [ 74 – 76 ]. There is a growing interest in the ability to deliver an intervention remotely in a trial using digital technologies. However, there is limited guidance on how best to do this [ 18 ]. The results from this review demonstrate the value of developing guidance that is tailored to the type of mode of delivery being used. Different modes of delivery will require different practical and communication-related considerations.

Three reports in this review originally used a web-based mode of delivery to deliver all, or part of the intervention [ 24 , 42 , 46 ]. However, this did not make the intervention immediately suitable for remote delivery. Trialists reported challenges as a result of not being present with participants while they were set up with the intervention and engaged with it remotely [ 24 , 46 ]. This finding suggests that the suitability of an intervention for remote delivery is not just related to whether it is delivered via a digital technology but also the degree to which it requires human facilitation and support. Evidence from other web-based interventions similarly indicates that effective participant engagement often depends on human support during onboarding and throughout the intervention [ 77 ]. Cooney et al. [ 10 ] highlight that there is not a dichotomous divide between in-person and digital delivery. They introduce the concept of a continuum of digitalisation to demonstrate the multiple points in between complete in-person delivery and fully automated delivery. Different considerations for adaptation are required depending on the degree of digitalisation and the degree of human support needed. This finding supports the need for clarity in the reporting of digitally delivered complex interventions [ 78 ]. It is important to report not just the mode of delivery used, but the degree of human support and facilitation required.

Strengths and limitations

A key strength of this review is the use of the upper level of the Behaviour Change Intervention Ontology (BCIO) as a framework to organise the challenges and facilitators related to remote intervention delivery. The BCIO supported a more granular examination of where guidance is needed. For example, by distinguishing between different modes of delivery, it was possible to see the challenges that relate more specifically to the use of video call compared to the telephone. Another strength of this review is that the studies included are from a range of trials with different populations and trial designs. This supports the generalisability of results and learnings related to the remote delivery of a complex intervention within a trial.

The review had limitations. Most papers in this review came from trials conducted in the USA, which may limit generalisability of the results. Most studies in this review, however, reported challenges related to remote delivery in the home environment (e.g. interruptions and disruptions) and to intervention mode of delivery (e.g. technological difficulties with video conferencing software). It is reasonable to assume that these challenges have wide relevance and applicability.

The studies included in this review came from trialists who had successfully adapted an intervention for remote delivery. Therefore, insights from trialists who were unable to adapt their intervention were not included, and key, potentially insurmountable, challenges to intervention adaptation and remote delivery might be missed.

While this review has identified several ‘facilitators’ of both adaptation and remote intervention delivery, these are self-reported by trialists. It is unclear how effective they were in managing the challenges that were faced. The facilitators identified in this review can be used as a starting point for more in-depth investigation.

Finally, the literature from the COVID-19 period is still maturing. Therefore, a number of potentially eligible papers and relevant insights may be missed. The purpose of this scoping review, however, was not to definitively identify every lesson learned related to adaptation and remote delivery. Instead, it aimed to identify key characteristics in the data that can be used to inform more in-depth future research.

Conclusion

This systematic scoping review identified a broad range of challenges that trialists faced as a result of remote intervention delivery. Little detail was provided about how trialists adapted their interventions for remote delivery. Key challenges of remote delivery in the home environment included privacy and safety concerns, which in some instances impacted trial participation. Technological and connectivity challenges were commonly reported with the use of video call. Suitability of an intervention for remote delivery depends not only on whether it is originally delivered via digital technology, but the extent to which it requires human facilitation and support. Further research is needed to explore how trialists can effectively prepare for and manage the challenges of remote delivery highlighted in this review. Any guidance that is developed to support the adaptation of an intervention for remote delivery within a trial should be specific to the mode of delivery used.

Supplementary Information

Additional file 1: Final search strategy Additional file 2: Data extraction table. Acronym list: BCIO, Behaviour Change Intervention Ontology; JBI, Joanna Briggs Institute; DCT, decentralised trial

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Acknowledgements

We thank Dierdre Andre, a research support adviser at Leeds University, for her support in developing the search strategy.

Authors’ contributions

Under supervision from AJF, SS, and KG, EH devised and conducted this scoping review and drafted the manuscript. LZ was involved in data analysis and acted as a second reviewer of the data. All authors read and approved the final manuscript.

Funding

This project is supported by funding from MRC DTP (MR/W006049/1). Samuel Smith is funded by a National Institute for Health Research NIHR Advanced Fellowship (NIHR300588). Katie Gillies is funded by a Medical Research Council Senior Non-Clinical Fellowship (MR/X007646/1). The views expressed here are those of the author(s) and not necessarily those of the NHS, the National Institute for Health and Social Care Research, Department of Health and Social Care, or the Medical Research Council. The funders had no role in the design of the study, data collection, analysis, interpretation of data, and in the writing of this manuscript.

Data availability

The protocol is available on the Open Science Framework. The extracted data is available upon request.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Smith declares receiving a personal consultancy payment from Lilly.

References

1. Movsisyan A Arnold L Evans R Hallingberg B Moore G O’Cathain A Adapting evidence-informed complex population health interventions for new contexts: a systematic review of guidance Implement Sci 2019 14 1 105 10.1186/s13012-019-0956-5 31847920 PMC6918624 Movsisyan A, Arnold L, Evans R, Hallingberg B, Moore G, O’Cathain A, et al. Adapting evidence-informed complex population health interventions for new contexts: a systematic review of guidance. Implement Sci. 2019Dec 17;14(1):105. 31847920 10.1186/s13012-019-0956-5 PMC6918624

2. Mara CA Peugh JL Validity of data collected from randomized behavioral clinical trials during the COVID-19 pandemic J Pediatr Psychol 2020 45 9 971 976 10.1093/jpepsy/jsaa078 32968774 PMC7797740 Mara CA, Peugh JL. Validity of data collected from randomized behavioral clinical trials during the COVID-19 pandemic. J Pediatr Psychol. 2020Oct 1;45(9):971–6. 32968774 10.1093/jpepsy/jsaa078 PMC7797740

3. Orkin AM Gill PJ Ghersi D Campbell L Sugarman J Emsley R Guidelines for reporting trial protocols and completed trials modified due to the COVID-19 pandemic and other extenuating circumstances: the CONSERVE 2021 statement JAMA 2021 326 3 257 265 10.1001/jama.2021.9941 34152382 Orkin AM, Gill PJ, Ghersi D, Campbell L, Sugarman J, Emsley R, et al. Guidelines for reporting trial protocols and completed trials modified due to the COVID-19 pandemic and other extenuating circumstances: the CONSERVE 2021 statement. JAMA. 2021Jul 20;326(3):257–65. 34152382 10.1001/jama.2021.9941

4. Masoli JAH Down K Nestor G Hudson S O’Brien JT Williamson JD A report from the NIHR UK working group on remote trial delivery for the COVID-19 pandemic and beyond Trials 2021 22 1 911 10.1186/s13063-021-05880-8 34895305 PMC8665850 Masoli JAH, Down K, Nestor G, Hudson S, O’Brien JT, Williamson JD, et al. A report from the NIHR UK working group on remote trial delivery for the COVID-19 pandemic and beyond. Trials. 2021Dec 11;22(1):911. 34895305 10.1186/s13063-021-05880-8 PMC8665850

5. Craig P Dieppe P Macintyre S Michie S Nazareth I Petticrew M Developing and evaluating complex interventions: the new Medical Research Council guidance BMJ 2008 29 337 a1655 10.1136/bmj.a1655 PMC2769032 18824488 Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ. 2008Sep;29(337):a1655. 10.1136/bmj.a1655 PMC2769032 18824488

6. O’Cathain A Croot L Duncan E Rousseau N Sworn K Turner KM Guidance on how to develop complex interventions to improve health and healthcare BMJ Open 2019 9 8 e029954 10.1136/bmjopen-2019-029954 31420394 PMC6701588 O’Cathain A, Croot L, Duncan E, Rousseau N, Sworn K, Turner KM, et al. Guidance on how to develop complex interventions to improve health and healthcare. BMJ Open. 2019Aug 1;9(8):e029954. 31420394 10.1136/bmjopen-2019-029954 PMC6701588

7. Marques MM Carey RN Norris E Evans F Finnerty AN Hastings J Delivering behaviour change interventions: development of a mode of delivery ontology Wellcome Open Res 2021 26 5 125 10.12688/wellcomeopenres.15906.2 PMC7993627 33824909 Marques MM, Carey RN, Norris E, Evans F, Finnerty AN, Hastings J, et al. Delivering behaviour change interventions: development of a mode of delivery ontology. Wellcome Open Res. 2021Feb;26(5):125. 10.12688/wellcomeopenres.15906.1 PMC7993627 33824909

8. Beall RF Baskerville N Golfam M Saeed S Little J Modes of delivery in preventive intervention studies: a rapid review Eur J Clin Invest 2014 44 7 688 696 10.1111/eci.12279 24828885 Beall RF, Baskerville N, Golfam M, Saeed S, Little J. Modes of delivery in preventive intervention studies: a rapid review. Eur J Clin Invest. 2014;44(7):688–96. 24828885 10.1111/eci.12279

9. Coyle J Rogers A Copland R De Paoli G MacDonald TM Mackenzie IS Learning from remote decentralised clinical trial experiences: a qualitative analysis of interviews with trial personnel, patient representatives and other stakeholders Br J Clin Pharmacol 2022 88 3 1031 42 10.1111/bcp.15003 34296777 PMC9290051 Coyle J, Rogers A, Copland R, De Paoli G, MacDonald TM, Mackenzie IS. Learning from remote decentralised clinical trial experiences: a qualitative analysis of interviews with trial personnel, patient representatives and other stakeholders. Br J Clin Pharmacol. 2022;88(3):1031–42. 34296777 10.1111/bcp.15003 PMC9290051

10. Cooney E Toomey E Ryan K Meade O McSharry J What’s different about digital? A qualitative interview study exploring experiences of adapting in-person behaviour change interventions for digital delivery Br J Health Psychol 2024 29 4 972 99 10.1111/bjhp.12741 39075024 Cooney E, Toomey E, Ryan K, Meade O, McSharry J. What’s different about digital? A qualitative interview study exploring experiences of adapting in-person behaviour change interventions for digital delivery. Br J Health Psychol. 2024;29(4):972–99. 39075024 10.1111/bjhp.12741

11. Chen J Application of mobile learning in higher English education systems using cognitive web services Int J e-Collab 2023 19 2 23 10.4018/IJeC.316654 Chen J. Application of mobile learning in higher English education systems using cognitive web services. Int J e-Collab. 2023;23. 10.4018/IJeC.316654.

12. Hincapié MA Gallego JC Gempeler A Piñeros JA Nasner D Escobar MF Implementation and usefulness of telemedicine during the COVID-19 pandemic: a scoping review J Prim Care Community Health 2020 10 11 2150132720980612 10.1177/2150132720980612 PMC7734546 33300414 Hincapié MA, Gallego JC, Gempeler A, Piñeros JA, Nasner D, Escobar MF. Implementation and usefulness of telemedicine during the COVID-19 pandemic: a scoping review. J Prim Care Community Health. 2020Dec;10(11):2150132720980612. 10.1177/2150132720980612 PMC7734546 33300414

13. Chambers DA, Norton WE. The Adaptome: advancing the science of intervention adaptation. American J Prev Med. 2016;51(4, Supplement 2):S124–31. 10.1016/j.amepre.2016.05.011 PMC5030159 27371105

14. Copeland L Littlecott H Couturiaux D Hoddinott P Segrott J Murphy S The what, why and when of adapting interventions for new contexts: a qualitative study of researchers, funders, journal editors and practitioners’ understandings PLoS ONE 2021 16 7 e0254020 10.1371/journal.pone.0254020 34242280 PMC8270163 Copeland L, Littlecott H, Couturiaux D, Hoddinott P, Segrott J, Murphy S, et al. The what, why and when of adapting interventions for new contexts: a qualitative study of researchers, funders, journal editors and practitioners’ understandings. PLoS ONE. 2021Jul 9;16(7):e0254020. 34242280 10.1371/journal.pone.0254020 PMC8270163

15. Escoffery C Lebow-Skelley E Udelson H Böing EA Wood R Fernandez ME A scoping study of frameworks for adapting public health evidence-based interventions Translational Behavioral Medicine 2019 9 1 1 10 10.1093/tbm/ibx067 29346635 PMC6305563 Escoffery C, Lebow-Skelley E, Udelson H, Böing EA, Wood R, Fernandez ME, et al. A scoping study of frameworks for adapting public health evidence-based interventions. Translational Behavioral Medicine. 2019Jan 1;9(1):1–10. 29346635 10.1093/tbm/ibx067 PMC6305563

16. Moore G Campbell M Copeland L Craig P Movsisyan A Hoddinott P Adapting interventions to new contexts—the ADAPT guidance BMJ 2021 3 n1679 10.1136/bmj.n1679 PMC8329746 34344699 Moore G, Campbell M, Copeland L, Craig P, Movsisyan A, Hoddinott P, et al. Adapting interventions to new contexts—the ADAPT guidance. BMJ. 2021Aug;3:n1679. 10.1136/bmj.n1679 PMC8329746 34344699

17. Goodridge D Marciniuk D Rural and remote care: overcoming the challenges of distance Chron Respir Dis 2016 13 2 192 203 10.1177/1479972316633414 26902542 PMC5734598 Goodridge D, Marciniuk D. Rural and remote care: overcoming the challenges of distance. Chron Respir Dis. 2016May 1;13(2):192–203. 26902542 10.1177/1479972316633414 PMC5734598

18. Rogers A, Mackenzie I, Hawkins K; BEST Work Package 1 Team, Trials@Home Project. D1.1 First set of recommendations for RDCTs Version 1 [Internet]. Dundee: Trials@Home – Centre of Excellence – Remote Decentralised Clinical Trials; 2020. Available from: https://trialsathome.com/ .

19. Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020Oct;18(10):2119–26. 10.11124/JBIES-20-00167 33038124

20. Peters MDJ Marnie C Colquhoun H Garritty CM Hempel S Horsley T Scoping reviews: reinforcing and advancing the methodology and application Syst Rev 2021 10 1 263 10.1186/s13643-021-01821-3 34625095 PMC8499488 Peters MDJ, Marnie C, Colquhoun H, Garritty CM, Hempel S, Horsley T, et al. Scoping reviews: reinforcing and advancing the methodology and application. Syst Rev. 2021Oct 8;10(1):263. 34625095 10.1186/s13643-021-01821-3 PMC8499488

21. De Brouwer W, Patel CJ, Manrai AK, Rodriguez-Chavez IR, Shah NR. Empowering clinical research in a decentralized world. npj Digit Med. 2021;4(1):1–5. 10.1038/s41746-021-00473-w PMC8249659 34211085

22. Tricco AC Lillie E Zarin W O’Brien KK Colquhoun H Levac D PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation Ann Intern Med 2018 169 7 467 473 10.7326/M18-0850 30178033 Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018Oct 2;169(7):467–73. 30178033 10.7326/M18-0850

23. Howes E, Smith S, Gillies K, Farrin A. What are the ‘lessons learned’ from trialists who adapted and remotely delivered a complex intervention within a trial during the COVID-19 pandemic? A scoping review protocol. 2024; Available from: https://osf.io/8ev5f/ . [cited 2024 Oct 17].

24. Meline J Prigge JM Dye D Rieder J Asan O Chhatre S Adapting the design of a web-based decision support clinical trial during the COVID-19 pandemic Trials 2021 22 1 734 10.1186/s13063-021-05700-z 34688297 PMC8541813 Meline J, Prigge JM, Dye D, Rieder J, Asan O, Chhatre S, et al. Adapting the design of a web-based decision support clinical trial during the COVID-19 pandemic. Trials. 2021Oct 23;22(1):734. 34688297 10.1186/s13063-021-05700-z PMC8541813

25. Saadatian-Elahi M Alexander N Möhlmann T Ariffin FD Schmitt F Richardson JH Addressing the COVID-19 pandemic challenges for operational adaptations of a cluster randomized controlled trial on dengue vector control in Malaysia BMC Public Health 2022 22 1 667 10.1186/s12889-022-13026-x 35387611 PMC8985393 Saadatian-Elahi M, Alexander N, Möhlmann T, Ariffin FD, Schmitt F, Richardson JH, et al. Addressing the COVID-19 pandemic challenges for operational adaptations of a cluster randomized controlled trial on dengue vector control in Malaysia. BMC Public Health. 2022Apr 6;22(1):667. 35387611 10.1186/s12889-022-13026-x PMC8985393

26. Farmer C Santer E West A Darby J Rees-Roberts M Doulton T Delivering clinical studies of exercise in the COVID-19 pandemic: challenges and adaptations using a feasibility trial of isometric exercise to treat hypertension as an exemplar BMJ Open 2023 10.1136/bmjopen-2022-068204 36931669 PMC10030278 Farmer C, Santer E, West A, Darby J, Rees-Roberts M, Doulton T, et al. Delivering clinical studies of exercise in the COVID-19 pandemic: challenges and adaptations using a feasibility trial of isometric exercise to treat hypertension as an exemplar. BMJ Open. 2023. 10.1136/bmjopen-2022-068204. 36931669 10.1136/bmjopen-2022-068204 PMC10030278

27. Afari N Yarish NM Wooldridge JS Materna K Hernandez J Blanco BH Lessons learned from transition of an in-person to a virtual randomized controlled trial for weight and fitness concerns in active-duty service members: survey study J Med Internet Res 2022 24 11 e37797 10.2196/37797 36201851 PMC9693705 Afari N, Yarish NM, Wooldridge JS, Materna K, Hernandez J, Blanco BH, et al. Lessons learned from transition of an in-person to a virtual randomized controlled trial for weight and fitness concerns in active-duty service members: survey study. J Med Internet Res. 2022Nov 10;24(11):e37797. 36201851 10.2196/37797 PMC9693705

28. Lalande K Greenman PS Bouchard K Johnson SM Tulloch H The Healing Hearts Together randomized controlled trial and the COVID-19 pandemic: a tutorial for transitioning from an in-person to a web-based intervention J Med Internet Res 2021 23 4 e25502 10.2196/25502 33729984 PMC8025918 Lalande K, Greenman PS, Bouchard K, Johnson SM, Tulloch H. The Healing Hearts Together randomized controlled trial and the COVID-19 pandemic: a tutorial for transitioning from an in-person to a web-based intervention. J Med Internet Res. 2021Apr 6;23(4):e25502. 33729984 10.2196/25502 PMC8025918

29. Klasko-Foster LB Biello KB Lodge W Olson J Mimiaga MJ Transitioning from face to face to the digital space: best practices and lessons learned leveraging technology to conduct HIV-focused interventions Telemed e-Health 2022 28 7 1070 1073 10.1089/tmj.2021.0190 PMC9469739 34995162 Klasko-Foster LB, Biello KB, Lodge W, Olson J, Mimiaga MJ. Transitioning from face to face to the digital space: best practices and lessons learned leveraging technology to conduct HIV-focused interventions. Telemed e-Health. 2022;28(7):1070–3. 10.1089/tmj.2021.0190 PMC9469739 34995162

30. Brody AA Convery KA Kline DM Fink RM Fischer SM Transitioning to remote recruitment and intervention: a tale of two palliative care research studies enrolling underserved populations during COVID-19 J Pain Symptom Manage 2022 63 1 151 159 10.1016/j.jpainsymman.2021.06.017 34161811 PMC8685301 Brody AA, Convery KA, Kline DM, Fink RM, Fischer SM. Transitioning to remote recruitment and intervention: a tale of two palliative care research studies enrolling underserved populations during COVID-19. J Pain Symptom Manage. 2022;63(1):151–9. 34161811 10.1016/j.jpainsymman.2021.06.017 PMC8685301

31. Fanning JT Barnstaple R Babcock P Black A Collier N Linville MC Virtual delivery of improvisational movement and social engagement interventions in the IMOVE trial during the COVID-19 pandemic Contemp Clin Trials Commun 2023 18 33 101102 10.1016/j.conctc.2023.101102 PMC10023198 36969988 Fanning JT, Barnstaple R, Babcock P, Black A, Collier N, Linville MC, et al. Virtual delivery of improvisational movement and social engagement interventions in the IMOVE trial during the COVID-19 pandemic. Contemp Clin Trials Commun. 2023Mar;18(33):101102. 10.1016/j.conctc.2023.101102 PMC10023198 36969988

32. Ouzzani M Hammady H Fedorowicz Z Elmagarmid A Rayyan—a web and mobile app for systematic reviews Syst Rev 2016 5 1 210 10.1186/s13643-016-0384-4 27919275 PMC5139140 Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan—a web and mobile app for systematic reviews. Syst Rev. 2016Dec 5;5(1):210. 27919275 10.1186/s13643-016-0384-4 PMC5139140

33. Michie S West R Finnerty AN Norris E Wright AJ Marques MM Representation of behaviour change interventions and their evaluation: development of the upper level of the behaviour change intervention ontology Wellcome Open Res 2020 5 123 10.12688/wellcomeopenres.15902.1 33614976 PMC7868854 Michie S, West R, Finnerty AN, Norris E, Wright AJ, Marques MM, et al. Representation of behaviour change interventions and their evaluation: development of the upper level of the behaviour change intervention ontology. Wellcome Open Res. 2020;5:123. 33614976 10.12688/wellcomeopenres.15902.1 PMC7868854

34. Arp R, Smith B, Spear AD. Building ontologies with basic formal ontology. The MIT Press; 2015. Available from: https://direct.mit.edu/books/monograph/4044/Building-Ontologies-with-Basic-Formal-Ontology . [cited 2024 May 29].

35. Michie S Thomas J Johnston M Aonghusa PM Shawe-Taylor J Kelly MP The human behaviour-change project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation Implement Sci 2017 12 1 121 10.1186/s13012-017-0641-5 29047393 PMC5648456 Michie S, Thomas J, Johnston M, Aonghusa PM, Shawe-Taylor J, Kelly MP, et al. The human behaviour-change project: harnessing the power of artificial intelligence and machine learning for evidence synthesis and interpretation. Implement Sci. 2017;12(1):121. 29047393 10.1186/s13012-017-0641-5 PMC5648456

36. Wells M Williams B Treweek S Coyle J Taylor J Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions Trials 2012 13 1 95 10.1186/1745-6215-13-95 22742939 PMC3475073 Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012Jun 28;13(1):95. 22742939 10.1186/1745-6215-13-95 PMC3475073

37. Hoffmann TC Glasziou PP Boutron I Milne R Perera R Moher D Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide BMJ 2014 7 348 g1687 10.1136/bmj.g1687 24609605 Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014Mar;7(348):g1687. 10.1136/bmj.g1687 24609605

38. Johnston M Progress in conducting and reporting behaviour change intervention studies: a prospective retrospection Health Psychol Behav Med 2021 9 1 567 581 10.1080/21642850.2021.1939701 34211803 PMC8218683 Johnston M. Progress in conducting and reporting behaviour change intervention studies: a prospective retrospection. Health Psychol Behav Med. 2021Jan 1;9(1):567–81. 34211803 10.1080/21642850.2021.1939701 PMC8218683

39. Montgomery P Grant S Mayo-Wilson E Macdonald G Michie S Hopewell S Reporting randomised trials of social and psychological interventions: the CONSORT-SPI 2018 Extension Trials 2018 19 1 407 10.1186/s13063-018-2733-1 30060754 PMC6066921 Montgomery P, Grant S, Mayo-Wilson E, Macdonald G, Michie S, Hopewell S, et al. Reporting randomised trials of social and psychological interventions: the CONSORT-SPI 2018 Extension. Trials. 2018Jul 31;19(1):407. 30060754 10.1186/s13063-018-2733-1 PMC6066921

40. Sandelowski M Whatever happened to qualitative description? Res Nurs Health 2000 23 4 334 340 10.1002/1098-240X(200008)23:4<334::AID-NUR9>3.0.CO;2-G 10940958 Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23(4):334–40. 10940958 10.1002/1098-240x(200008)23:4<334::aid-nur9>3.0.co;2-g

41. Fisher L, Tuchman S, Curreri A, Markgraf M, Nyer M, Cassano P, et al. Transitioning from in-person to remote clinical research on depression and traumatic brain injury during the COVID-19 pandemic: study modifications and preliminary feasibility from a randomized controlled pilot study. JMIR Form Res. 2021. 10.2196/28734. 10.2196/28734 PMC8638786 34662285

42. Johnson E Jenssen S Wernette GT Tweel T Johnson D Zlotnick C Web-based intervention to reduce intimate partner violence during perinatal period: a modified protocol in response to the COVID-19 pandemic Psychiatry Res 2022 317 114895 10.1016/j.psychres.2022.114895 37406076 PMC9556943 Johnson E, Jenssen S, Wernette GT, Tweel T, Johnson D, Zlotnick C. Web-based intervention to reduce intimate partner violence during perinatal period: a modified protocol in response to the COVID-19 pandemic. Psychiatry Res. 2022;317:114895. 37406076 10.1016/j.psychres.2022.114895 PMC9556943

43. McCourt O Fisher A Ramdharry G Roberts AL Land J Rabin N Adaptation of the PERCEPT myeloma prehabilitation trial to virtual delivery: changes in response to the COVID-19 pandemic BMJ Open 2022 12 4 e059516 10.1136/bmjopen-2021-059516 35396313 PMC8995568 McCourt O, Fisher A, Ramdharry G, Roberts AL, Land J, Rabin N, et al. Adaptation of the PERCEPT myeloma prehabilitation trial to virtual delivery: changes in response to the COVID-19 pandemic. BMJ Open. 2022Apr 1;12(4):e059516. 35396313 10.1136/bmjopen-2021-059516 PMC8995568

44. Mimiaga MJ Kuhns LM Biello KB Olson J Hoehnle S Santostefano CM Positive STEPS - a randomized controlled efficacy trial of an adaptive intervention for strengthening adherence to antiretroviral HIV treatment among youth: study protocol BMC Public Health 2018 18 1 867 10.1186/s12889-018-5815-9 30001703 PMC6043988 Mimiaga MJ, Kuhns LM, Biello KB, Olson J, Hoehnle S, Santostefano CM, et al. Positive STEPS - a randomized controlled efficacy trial of an adaptive intervention for strengthening adherence to antiretroviral HIV treatment among youth: study protocol. BMC Public Health. 2018Jul 13;18(1):867. 30001703 10.1186/s12889-018-5815-9 PMC6043988

45. Shear D Harrison LE O’Brien S Khazendar Z Lyons S Morgan J Rapid transition to virtual assessment and treatment in an interdisciplinary randomized clinical trial for youth with chronic pain: adaptations and implications for future trials Clin J Pain 2022 38 7 459 469 10.1097/AJP.0000000000001040 35686576 PMC9199594 Shear D, Harrison LE, O’Brien S, Khazendar Z, Lyons S, Morgan J, et al. Rapid transition to virtual assessment and treatment in an interdisciplinary randomized clinical trial for youth with chronic pain: adaptations and implications for future trials. Clin J Pain. 2022;38(7):459–69 ( Jul 1; ). 35686576 10.1097/AJP.0000000000001040 PMC9199594

46. Sperl-Hillen J Crain A Chumba L Ekstrom H Appana D Kopski K Pragmatic clinic randomized trial to improve chronic kidney disease care: design and adaptation due to COVID disruptions Contemp Clin Trials 2021 10.1016/j.cct.2021.106501 34271175 PMC8276567 Sperl-Hillen J, Crain A, Chumba L, Ekstrom H, Appana D, Kopski K, et al. Pragmatic clinic randomized trial to improve chronic kidney disease care: design and adaptation due to COVID disruptions. Contemp Clin Trials. 2021. 10.1016/j.cct.2021.106501. 34271175 10.1016/j.cct.2021.106501 PMC8276567

47. Tang L Broad J Lewis R Ma DWL Haines J Guelph Family Health Study Transitioning a home-based, motivational interviewing intervention among families to remote delivery during the COVID-19 pandemic: key lessons learned Patient Educ Couns 2021 104 9 2286 91 10.1016/j.pec.2021.02.043 33674132 PMC7906506 Tang L, Broad J, Lewis R, Ma DWL, Haines J, Guelph Family Health Study. Transitioning a home-based, motivational interviewing intervention among families to remote delivery during the COVID-19 pandemic: key lessons learned. Patient Educ Couns. 2021;104(9):2286–91. 33674132 10.1016/j.pec.2021.02.043 PMC7906506

48. Vinci C Hemenway M Baban SS Yang M-J Brandon KO Witkiewitz K Transition to telehealth: challenges and benefits of conducting group-based smoking and alcohol treatment virtually Contemp Clin Trials 2022 114 106689 10.1016/j.cct.2022.106689 35085833 PMC8785408 Vinci C, Hemenway M, Baban SS, Yang M-J, Brandon KO, Witkiewitz K, et al. Transition to telehealth: challenges and benefits of conducting group-based smoking and alcohol treatment virtually. Contemp Clin Trials. 2022;114:106689. 35085833 10.1016/j.cct.2022.106689 PMC8785408

49. Zaccari B Loftis JM Haywood T Hubbard K Clark J Kelly UA Synchronous telehealth yoga and cognitive processing group therapies for women veterans with posttraumatic stress disorder: a multisite randomized controlled trial adapted for COVID-19 Telemedicine and e-Health 2022 10.1089/tmj.2021.0612 35357957 PMC9519809 Zaccari B, Loftis JM, Haywood T, Hubbard K, Clark J, Kelly UA. Synchronous telehealth yoga and cognitive processing group therapies for women veterans with posttraumatic stress disorder: a multisite randomized controlled trial adapted for COVID-19. Telemedicine and e-Health. 2022. 10.1089/tmj.2021.0612. 35357957 10.1089/tmj.2021.0612 PMC9519809

50. Binder E, Christensen J, Stevens-Lapsley J, Bartley J, Berry S, Dobs A, et al. A multi-center trial of exercise and testosterone therapy in women after hip fracture: design, methods and impact of the COVID-19 pandemic. Contemp Clin Trials. 2021. 10.1016/j.cct.2021.106356. 10.1016/j.cct.2021.106356 PMC9119796 33716173

51. Kashikar-Zuck S Barnett K Williams S Pfeiffer M Thomas S Beasley K FIT teens RCT for juvenile fibromyalgia: protocol adaptations in response to the COVID 19 pandemic Contemp Clin Trials Commun 2022 10.1016/j.conctc.2022.101039 36467389 PMC9707024 Kashikar-Zuck S, Barnett K, Williams S, Pfeiffer M, Thomas S, Beasley K, et al. FIT teens RCT for juvenile fibromyalgia: protocol adaptations in response to the COVID 19 pandemic. Contemp Clin Trials Commun. 2022. 10.1016/j.conctc.2022.101039. 36467389 10.1016/j.conctc.2022.101039 PMC9707024

52. Mimiaga MJ Pantalone DW Biello KB Glynn TR Santostefano CM Olson J A randomized controlled efficacy trial of behavioral activation for concurrent stimulant use and sexual risk for HIV acquisition among MSM: project IMPACT study protocol BMC Public Health 2018 18 1 914 10.1186/s12889-018-5856-0 30045702 PMC6060452 Mimiaga MJ, Pantalone DW, Biello KB, Glynn TR, Santostefano CM, Olson J, et al. A randomized controlled efficacy trial of behavioral activation for concurrent stimulant use and sexual risk for HIV acquisition among MSM: project IMPACT study protocol. BMC Public Health. 2018Jul 25;18(1):914. 30045702 10.1186/s12889-018-5856-0 PMC6060452

53. Everhart R, Dempster K, Withers D, Mazzeo S, Corona R, Thacker L, et al. Adaptations to the RVA breathes clinical trial due to the COVID-19 pandemic. Contemp Clin Trials Commun. 2021. 10.1016/j.conctc.2021.100871. 10.1016/j.conctc.2021.100871 PMC8592848 34805614

54. Hom MA Weiss RB Millman ZB Christensen K Lewis EJ Cho S Development of a virtual partial hospital program for an acute psychiatric population: lessons learned and future directions for telepsychotherapy J Psychother Integr 2020 30 2 366 82 10.1037/int0000212 Hom MA, Weiss RB, Millman ZB, Christensen K, Lewis EJ, Cho S, et al. Development of a virtual partial hospital program for an acute psychiatric population: lessons learned and future directions for telepsychotherapy. J Psychother Integr. 2020;30(2):366–82.

55. O’Neil J Egan M Marshall S Bilodeau M Pelletier L Sveistrup H Remotely supervised home-based intensive exercise intervention to improve balance, functional mobility, and physical activity in survivors of moderate or severe traumatic brain injury: protocol for a mixed methods study JMIR Research Protocols 2019 8 10 e14867 10.2196/14867 31599733 PMC6812480 O’Neil J, Egan M, Marshall S, Bilodeau M, Pelletier L, Sveistrup H. Remotely supervised home-based intensive exercise intervention to improve balance, functional mobility, and physical activity in survivors of moderate or severe traumatic brain injury: protocol for a mixed methods study. JMIR Research Protocols. 2019Oct 9;8(10):e14867. 31599733 10.2196/14867 PMC6812480

56. Wrape ER McGinn MM Clinical and ethical considerations for delivering couple and family therapy via telehealth J Marital Fam Ther 2019 45 2 296 308 10.1111/jmft.12319 29361194 Wrape ER, McGinn MM. Clinical and ethical considerations for delivering couple and family therapy via telehealth. J Marital Fam Ther. 2019;45(2):296–308. 29361194 10.1111/jmft.12319

57. Gros DF Morland LA Greene CJ Acierno R Strachan M Egede LE Delivery of evidence-based psychotherapy via video telehealth J Psychopathol Behav Assess 2013 35 4 506 521 10.1007/s10862-013-9363-4 Gros DF, Morland LA, Greene CJ, Acierno R, Strachan M, Egede LE, et al. Delivery of evidence-based psychotherapy via video telehealth. J Psychopathol Behav Assess. 2013Dec 1;35(4):506–21.

58. Luxton DD Sirotin AP Mishkind MC Safety of telemental healthcare delivered to clinically unsupervised settings: a systematic review Telemed J E-Health 2010 16 6 705 711 10.1089/tmj.2009.0179 20583951 Luxton DD, Sirotin AP, Mishkind MC. Safety of telemental healthcare delivered to clinically unsupervised settings: a systematic review. Telemed J E-Health. 2010;16(6):705–11. 20583951 10.1089/tmj.2009.0179

59. Ritunga I, Harsa C, Santoso J, Mellyanawati, Nurhadi S, Nugraheni E, et al. Privacy and confidentiality in telemedicine: a literature review. Christian J for Global Hlth. 2025;12(1). Available from: 10.15566/cjgh.v12i1.377 . [cited 2025 Aug 13].

60. Roben CKP Kipp E Schein SS Costello AH Dozier M Transitioning to telehealth due to COVID-19: maintaining model fidelity in a home visiting program for parents of vulnerable infants Infant Ment Health J 2022 43 1 173 184 10.1002/imhj.21963 34964152 PMC9015513 Roben CKP, Kipp E, Schein SS, Costello AH, Dozier M. Transitioning to telehealth due to COVID-19: maintaining model fidelity in a home visiting program for parents of vulnerable infants. Infant Ment Health J. 2022;43(1):173–84. 34964152 10.1002/imhj.21963 PMC9015513

61. Toomey E Hardeman W Hankonen N Byrne M McSharry J Matvienko-Sikar K Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions Health Psychol Behav Med 2020 8 1 132 151 10.1080/21642850.2020.1738935 34040865 PMC8114368 Toomey E, Hardeman W, Hankonen N, Byrne M, McSharry J, Matvienko-Sikar K, et al. Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions. Health Psychol Behav Med. 2020Jan 1;8(1):132–51. 34040865 10.1080/21642850.2020.1738935 PMC8114368

62. Sinha SD Chary Sriramadasu S Raphael R Roy S Decentralisation in clinical trials and patient centricity: benefits and challenges Pharm Med 2024 38 2 109 120 10.1007/s40290-024-00518-x 38453755 Sinha SD, Chary Sriramadasu S, Raphael R, Roy S. Decentralisation in clinical trials and patient centricity: benefits and challenges. Pharm Med. 2024Mar 1;38(2):109–20. 10.1007/s40290-024-00518-x 38453755

63. Vayena E Blasimme A Sugarman J Decentralised clinical trials: ethical opportunities and challenges Lancet Digit Health 2023 5 6 e390 e394 10.1016/S2589-7500(23)00052-3 37105800 PMC10129131 Vayena E, Blasimme A, Sugarman J. Decentralised clinical trials: ethical opportunities and challenges. Lancet Digit Health. 2023Jun;5(6):e390–4. 37105800 10.1016/S2589-7500(23)00052-3 PMC10129131

64. Goodson N, Wicks P, Morgan J, Hashem L, Callinan S, Reites J. Opportunities and counterintuitive challenges for decentralized clinical trials to broaden participant inclusion. NPJ Digit Med. 2022;5(1):58. 10.1038/s41746-022-00603-y PMC9072305 35513479

65. Adams DV Long S Fleury ME Association of remote technology use and other decentralization tools with patient likelihood to enroll in cancer clinical trials JAMA Netw Open 2022 5 7 e2220053 10.1001/jamanetworkopen.2022.20053 35788672 PMC9257574 Adams DV, Long S, Fleury ME. Association of remote technology use and other decentralization tools with patient likelihood to enroll in cancer clinical trials. JAMA Netw Open. 2022Jul 5;5(7):e2220053. 35788672 10.1001/jamanetworkopen.2022.20053 PMC9257574

66. Gray SM Franke T Sims-Gould J McKay HA Rapidly adapting an effective health promoting intervention for older adults—choose to move—for virtual delivery during the COVID-19 pandemic BMC Public Health 2022 22 1 1172 10.1186/s12889-022-13547-5 35690744 PMC9188419 Gray SM, Franke T, Sims-Gould J, McKay HA. Rapidly adapting an effective health promoting intervention for older adults—choose to move—for virtual delivery during the COVID-19 pandemic. BMC Public Health. 2022Jun 11;22(1):1172. 35690744 10.1186/s12889-022-13547-5 PMC9188419

67. Guo C Ashrafian H Ghafur S Fontana G Gardner C Prime M Challenges for the evaluation of digital health solutions—a call for innovative evidence generation approaches npj Digit Med 2020 3 1 1 14 10.1038/s41746-020-00314-2 32904379 PMC7453198 Guo C, Ashrafian H, Ghafur S, Fontana G, Gardner C, Prime M. Challenges for the evaluation of digital health solutions—a call for innovative evidence generation approaches. npj Digit Med. 2020;3(1):1–14. 32904379 10.1038/s41746-020-00314-2 PMC7453198

68. Yao R Zhang W Evans R Cao G Rui T Shen L Inequities in health care services caused by the adoption of digital health technologies: scoping review J Med Internet Res 2022 24 3 e34144 10.2196/34144 35311682 PMC8981004 Yao R, Zhang W, Evans R, Cao G, Rui T, Shen L. Inequities in health care services caused by the adoption of digital health technologies: scoping review. J Med Internet Res. 2022Mar 21;24(3):e34144. 35311682 10.2196/34144 PMC8981004

69. Békási S Girasek E Győrffy Z Telemedicine in community shelters: possibilities to improve chronic care among people experiencing homelessness in Hungary Int J Equity Health 2022 21 1 1 17 10.1186/s12939-022-01803-4 36528777 PMC9758475 Békási S, Girasek E, Győrffy Z. Telemedicine in community shelters: possibilities to improve chronic care among people experiencing homelessness in Hungary. Int J Equity Health. 2022Dec;21(1):1–17. 36528777 10.1186/s12939-022-01803-4 PMC9758475

70. Toseef MU Armistead I Bacon E Hawkins E Bender B Podewils LJ Missed appointments during COVID-19: the impact of telehealth for persons experiencing homelessness with substance use disorders Asian J Psychiatr 2022 1 69 102987 10.1016/j.ajp.2021.102987 PMC9761744 34979474 Toseef MU, Armistead I, Bacon E, Hawkins E, Bender B, Podewils LJ, et al. Missed appointments during COVID-19: the impact of telehealth for persons experiencing homelessness with substance use disorders. Asian J Psychiatr. 2022Mar;1(69):102987. 10.1016/j.ajp.2021.102987 PMC9761744 34979474

71. Sasangohar F Bradshaw MR Carlson MM Flack JN Fowler JC Freeland D Adapting an outpatient psychiatric clinic to telehealth during the COVID-19 pandemic: a practice perspective J Med Internet Res 2020 22 10 e22523 10.2196/22523 32936768 PMC7546859 Sasangohar F, Bradshaw MR, Carlson MM, Flack JN, Fowler JC, Freeland D, et al. Adapting an outpatient psychiatric clinic to telehealth during the COVID-19 pandemic: a practice perspective. J Med Internet Res. 2020Oct 1;22(10):e22523. 32936768 10.2196/22523 PMC7546859

72. Loudon K Treweek S Sullivan F Donnan P Thorpe KE Zwarenstein M The precis-2 tool: designing trials that are fit for purpose BMJ 2015 8 350 h2147 10.1136/bmj.h2147 25956159 Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The precis-2 tool: designing trials that are fit for purpose. BMJ. 2015;8(350):h2147. 10.1136/bmj.h2147 25956159

73. Irvine A Drew P Bower P Brooks H Gellatly J Armitage CJ Are there interactional differences between telephone and face-to-face psychological therapy? A systematic review of comparative studies J Affect Disord 2020 265 120 131 10.1016/j.jad.2020.01.057 32090733 PMC7049904 Irvine A, Drew P, Bower P, Brooks H, Gellatly J, Armitage CJ, et al. Are there interactional differences between telephone and face-to-face psychological therapy? A systematic review of comparative studies. J Affect Disord. 2020;265:120–31 ( Mar 15; ). 32090733 10.1016/j.jad.2020.01.057 PMC7049904

74. Buckman JEJ Saunders R Leibowitz J Minton R The barriers, benefits and training needs of clinicians delivering psychological therapy via video Behav Cogn Psychother 2021 49 6 696 720 10.1017/S1352465821000187 33966666 PMC12092168 Buckman JEJ, Saunders R, Leibowitz J, Minton R. The barriers, benefits and training needs of clinicians delivering psychological therapy via video. Behav Cogn Psychother. 2021Nov;49(6):696–720. 33966666 10.1017/S1352465821000187 PMC12092168

75. Singhal A Riley JP Cowie MR Benefits and challenges of telemedicine for heart failure consultations: a qualitative study BMC Health Serv Res 2023 23 1 847 10.1186/s12913-023-09872-z 37563576 PMC10413528 Singhal A, Riley JP, Cowie MR. Benefits and challenges of telemedicine for heart failure consultations: a qualitative study. BMC Health Serv Res. 2023Aug 10;23(1):847. 37563576 10.1186/s12913-023-09872-z PMC10413528

76. Thomas N McDonald C de Boer K Brand R Nedeljkovic M Seabrook L Review of the current empirical literature on using videoconferencing to deliver individual psychotherapies to adults with mental health problems PSYCHOLOGY AND PSYCHOTHERAPY-THEORY RESEARCH AND PRACTICE 2021 94 3 854 883 10.1111/papt.12332 PMC8451850 33620133 Thomas N, McDonald C, de Boer K, Brand R, Nedeljkovic M, Seabrook L. Review of the current empirical literature on using videoconferencing to deliver individual psychotherapies to adults with mental health problems. PSYCHOLOGY AND PSYCHOTHERAPY-THEORY RESEARCH AND PRACTICE. 2021Sep;94(3):854–83. 10.1111/papt.12332 PMC8451850 33620133

77. Eisner E Faulkner S Allan S Ball H Di Basilio D Nicholas J Barriers and facilitators of user engagement with digital mental health interventions for people with psychosis or bipolar disorder: systematic review and best-fit framework synthesis JMIR Ment Health 2025 20 12 e65246 10.2196/65246 PMC11791459 39832352 Eisner E, Faulkner S, Allan S, Ball H, Di Basilio D, Nicholas J, et al. Barriers and facilitators of user engagement with digital mental health interventions for people with psychosis or bipolar disorder: systematic review and best-fit framework synthesis. JMIR Ment Health. 2025Jan;20(12):e65246. 10.2196/65246 PMC11791459 39832352

78. Rhon DI Fritz JM Kerns RD McGeary DD Coleman BC Farrokhi S TIDier-telehealth: precision in reporting of telehealth interventions used in clinical trials - unique considerations for the template for the intervention description and replication (TIDieR) checklist BMC Med Res Methodol 2022 22 161 10.1186/s12874-022-01640-7 35655144 PMC9161193 Rhon DI, Fritz JM, Kerns RD, McGeary DD, Coleman BC, Farrokhi S, et al. TIDier-telehealth: precision in reporting of telehealth interventions used in clinical trials - unique considerations for the template for the intervention description and replication (TIDieR) checklist. BMC Med Res Methodol. 2022;22:161. 35655144 10.1186/s12874-022-01640-7 PMC9161193

79. Norris E Marques MM Finnerty AN Wright AJ West R Hastings J Development of an intervention setting ontology for behaviour change: specifying where interventions take place Wellcome Open Res 2020 10 5 124 10.12688/wellcomeopenres.15904.1 PMC7489274 32964137 Norris E, Marques MM, Finnerty AN, Wright AJ, West R, Hastings J, et al. Development of an intervention setting ontology for behaviour change: specifying where interventions take place. Wellcome Open Res. 2020Jun;10(5):124. 10.12688/wellcomeopenres.15904.1 PMC7489274 32964137

80. Norris E Wright AJ Hastings J West R Boyt N Michie S Specifying who delivers behaviour change interventions: development of an intervention source ontology Wellcome Open Res 2021 8 6 77 10.12688/wellcomeopenres.16682.1 PMC8406443 34497878 Norris E, Wright AJ, Hastings J, West R, Boyt N, Michie S. Specifying who delivers behaviour change interventions: development of an intervention source ontology. Wellcome Open Res. 2021Apr;8(6):77. 10.12688/wellcomeopenres.16682.1 PMC8406443 34497878

81. Wright AJ Zhang L Howes E Specifying how intervention content is communicated: development of a style of delivery ontology Wellcome Open Res 2023 8 456 10.12688/wellcomeopenres.19899.1 39193088 PMC11347912 Wright AJ, Zhang L, Howes E, et al. Specifying how intervention content is communicated: development of a style of delivery ontology. Wellcome Open Res. 2023;8:456. 10.12688/wellcomeopenres.19899.1. 39193088 10.12688/wellcomeopenres.19899.1 PMC11347912