Detection bias in open-label trials of anticancer drugs: a meta-epidemiological study.

What is already known on this topic

The US Food and Drug Administration and European Medicines Agency recommend the use of central reviewers in oncological open-label trials to avoid detection bias; however, this recommendation is not based on evidence. Previous meta-epidemiological studies have compared progression-free survival (PFS) or objective response rate (ORR) adjudicated by central reviewers and local investigators, but have not clearly identified an overestimation by local investigators. However, these studies did not specifically focus on open-label trials, and there is a concern that an adequate number of studies were not included through an appropriate search strategy.

What this study adds

This meta-epidemiological study found that only a small fraction of the studies on anticancer drugs adjudicated PFS or ORR using both central and local investigators, and only half of these studies reported both outcomes. A meta-analysis of these studies showed that PFS may have been slightly overestimated by local investigators, while ORR was not. A sensitivity analysis which used a range of assumptions did not change these results. Our study suggests that the impact of the differences between central and local adjudications is not substantial. However, as this study did not access data from all trials, the results may not fully evaluate detection bias. Among the studies that claimed both assessors were used, half of them only reported the results from one assessor, indicating potential selective outcome reporting bias. Until the findings of this study

are validated by studies that overcome this limitation, it is desirable to establish central reviewers in oncological open-label trials.

How might this affect research, practice or policy

It is expected that the results of this study will lead to a trend towards reporting outcomes adjudicated by both central and local investigators. Additional meta-epidemiological studies would then be conducted to validate this study and accumulate knowledge on detection bias in open-label trials of anticancer drug.

Introduction

Detection bias can systematically distort the results of randomised controlled trials. 1 This bias, also known as observer bias or measurement bias, can occur when the outcomes are subjective and the adjudicators are not blinded (or masked), leading them to evaluate the results optimistically. In general medicine, several meta-epidemiological studies have assessed the magnitude of detection bias by comparing the same outcomes between blinded and non-blinded adjudicators, with conflicting findings and a lack of consensus. 2–5 Recently, the MetaBLIND study found no difference in the estimated treatment effect between trials with and without blinded outcome adjudicators; however, since they compared the outcomes in different trials, the study had a risk of confounding. 6 Furthermore, these meta-epidemiological studies have diverse specialties and outcomes, and whether they can be generalised to specific outcomes in oncology requires validation.

Open-label trials are more common in oncological clinical trials than in non-oncological trials, 7 and there has been a trend towards using progression-free survival (PFS) and objective response rate (ORR) as primary outcomes in oncological trials rather than overall survival (OS). 8 This is due to the abbreviated time required to evaluate efficacy, smaller sample size requirements and lack of subsequent treatments. However, PFS and ORR involve more subjective judgement than OS, particularly in open-label settings. Given recent trends, detection bias is a particular concern in oncological clinical trials.

The US Food and Drug Administration recommends the use of central reviewers blinded to study treatment to verify tumour assessment to minimise bias in oncological clinical trials when the primary endpoint is PFS or ORR. 9 The European Medicines Agency also emphasises the importance of blinded independent central reviewers, especially when the majority of events are based on imaging rather than clinical progression. 10 However, previous oncological meta-epidemiological studies have yielded inconsistent findings and do not necessarily demonstrate overestimation by local investigators. 11–15 Nevertheless, these studies did not focus on open-label trials, and there is a concern that an adequate number of studies were not included through an appropriate search strategy. Therefore, while the importance of central reviewers is widely recognised in oncology, there is limited evidence to support their importance.

This meta-epidemiological study focused on open-label trials of anticancer drugs and aimed to evaluate detection bias by confirming whether local investigators overestimate the PFS and ORR compared with the estimates by central reviewers.

Methods

Study design

This was a meta-epidemiological study of randomised controlled trials registered in the UMIN Clinical Trial Registry ( CTR-UMIN000044623 ). The protocol was published as a preprint in the Open Science Foundation 16 and descriptive summary results of the identified studies have been reported elsewhere. 17 The study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. 18

Eligibility criteria

Types of studies

We included open-label, parallel-group superiority and randomised trials that investigated the efficacy of anticancer drugs. Non-inferiority and equivalence trials were excluded because the null hypothesis was different from that of the superiority trials. We also excluded records that were not in English language. 19

Types of participants

We focused on solid tumours and excluded haematological diseases, including leukaemia, lymphoma and multiple myeloma. Because haematological cancers are evaluated biologically, their evaluations are less likely to be influenced by adjudicators than solid tumours. Therefore, we only included solid cancers of all histological types and stages.

Types of interventions

Eligible interventions included molecularly targeted therapy, immune checkpoint inhibitors, immune therapy, chemotherapy or hormone therapy; and eligible comparisons included standard therapy, supportive care or no treatment. Combination therapies were also included (eg, therapy A plus standard therapy vs standard therapy). Owing to heterogeneity, we excluded neoadjuvant and adjuvant interventions.

Types of outcomes

We included trials that used PFS or ORR as measurements of treatment efficacy.

Information sources

We identified relevant trials from the MEDLINE database.

Search strategy

We searched MEDLINE from 1 January 2010 to 30 June 2021. Online supplemental appendix lists the search terms used on 1 July 2021. Furthermore, we conducted a manual search of the reference lists attached to relevant articles. We restricted the search period to 2010 to allow the possibility of contacting the corresponding authors.

bmjebm-2023-112332supp001.pdf

Selection process

First, seven independent pairs of researchers (SF and TY; SF and YF; SF and SY; SF and NN; YL and YK; TY and MK; TY and MT) screened the titles and abstracts of the records identified by the literature search. Second, different seven pairs of researchers (SF and YL; SF and YK; SF and TY; SF and YF; SF and SY; SF and MK; SF and MT) independently screened the full texts of the records. After full-text screening, we classified the articles by outcome adjudicators into four categories: ‘central and local’, ‘only central’, ‘only local’ and ‘unclear’. Based on this classification, we selected articles in which the outcome was adjudicated by both the central and local investigators. Any disagreement was resolved through discussion or consultation with another pair or TAF if necessary. The PRISMA flow chart ( figure 1 ) summarises the reasons for excluding studies.

Data collection process

Eight pairs of researchers independently collected detailed information from each trial using a prepiloted form (SF and YL; SF and YK; SF and TY; SF and YF; SF and SY; SF and MK; SF and MT; SF and NN). When no details were reported for the data items, the corresponding authors were queried.

Data items

We collected the data items as follows:

General information of the study: publication journal, publication year, authors, trial registry number (National Clinical Trial number, European Union Clinical Trial number and others), trial phase (phase II, phase III and others), cancer type and number of randomised participants.

Intervention and comparison information: drug name, drug classification (targeted therapy, immune checkpoint inhibitor, immune therapy, chemotherapy or hormone therapy), comparison treatment (standard treatment, BSC or no treatment) and treatment line.

Outcome information: primary outcome, response criteria (response evaluation criteria in solid tumours (RECIST) 1.0, 20 RECIST V.1.1, 21 WHO criteria 22 and others), point estimates and 95% CIs of the HR of PFS assessed by central reviewers (HR central ), HR of PFS assessed by local investigators (HR local ), OR of ORR assessed by central reviewers (OR central ) and OR of ORR assessed by local investigators (OR local ). If the study did not report the 95% CIs, we collected the p value or the other CI (eg, the 90% CIs) to obtain the SE. 23

Statistical analysis

The characteristics of the included studies were summarised as numbers and relative frequencies for categorical variables. Our primary outcomes were the ratio of HR (RHR) and the ratio of OR (ROR) between HR central (or OR central ) and HR local (or OR local ). We performed a meta-analysis of RHR using the two-step process proposed by Sterne et al . 24 Within each trial, we divided HR local by HR central to calculate the RHR and divided OR local by OR central to calculate the ROR. An RHR<1 indicates that the local investigators overestimate the effect compared with the findings of central reviewers, which suggests the existence of a detection bias. In order to calculate the SE of RHR in each trial, we transformed RHR to log RHR and calculated the SE of the logRHR (SE(logRHR)) using the following equation:

where ρ is correlation coefficients between HR central and HR local in each trial. Then, we performed a random-effects meta-analysis using the inverse variance method. We were unable to calculate ρ without individual patient data. We, therefore, assumed no dependency between HR central and HR local in each trial (ρ=0) for the main analysis, as it is the most conservative approach (ie, it generates the largest SE estimation and hence widest CIs). 4 To test robustness, we also performed sensitivity analyses assuming ρ=0.25, 0.50, 0.75 and 0.95. We presented a forest plot to visualise the RHR in all included studies. We evaluated the heterogeneity of RHR across different trials using the tau 2 and I 2 statistics. We prespecified and explored whether the direction or magnitude of detection bias would vary in the following subgroups: (1) trial phases, (2) cancer types and (3) drug classifications. We calculated the p values for interactions using a meta-regression model. We performed a meta-analysis of the ROR in the same manner. Note that unlike RHR, an ROR>1 indicates that the local investigators overestimate the effect compared with the findings of central reviewers, indicating detection bias. The ‘metafor’ package (V.3.8–1) 25 in R (V.4.0.3; R Foundation for Statistical Computing, Vienna, Austria) was used for meta-analysis. The collected data and codes used for our analysis are provided in the online supplemental materials .

Patient and public involvement

Patients and members of the public were not involved in this study because it was conducted to answer a methodological question that was not directly dependent on patient priorities, experiences or participant preferences.

Results

Study selection

Figure 1 illustrates the study selection process. We identified 6339 records, 1517 of which were eligible after screening titles and abstracts. From 1517 records, we retrieved 1197 eligible records after full-text screening and assessed the adjudicators of PFS and ORR. A total of 181 records (PFS: 157 records, ORR: 141 records) were adjudicated by both central reviewers and local investigators ( online supplemental table 1 ), and the trend of the prevalence of outcome adjudicators did not change from 2010 to 2021 in both PFS ( online supplemental figure 2 ) and ORR ( online supplemental figure 3 ). Of the 181 records, we excluded 10 due to data duplication and included the remaining records that could extract the outcomes adjudicated by both central reviewers and local investigators. Finally, we included 114 records in this analysis, of which 92 were analysed for PFS and 74 for ORR. In other words, among the records that we judged were adjudicated by both central reviewers and local investigators for PFS, only 61.7% (92/149) reported results from both assessors and 54.4% (74/136) for ORR. Online supplemental materials list the exclusion criteria for records during the full-text screening stage (n=320), data extraction stage (n=57) and duplication records stage (n=10). We sent 121 emails to the corresponding authors to request details of the data (26 October 2022) and received 15 responses, of which only one provided sufficient information. The other 14 corresponding authors stated that they could not access the data.

Study characteristics

Table 1 presents the characteristics of the included records. The analysis included 114 records, 92 for PFS and 74 for ORR. The majority of the trials were phase III (n=74); the most common tumour types were non-small cell lung cancer (n=29), breast cancer (n=22) and renal cell cancer (n=14); and the most common type of drug was targeted therapy (n=70). PFS was the primary outcome in 82 records, whereas ORR was the primary outcome in 13. Most of the reports were from high-impact factor (>10) journals (n=102), with only 5 funded by the public sector.

Primary outcome: RHR and ROR

Table 2 and figure 2 present the RHR for PFS. Under the assumption of no dependency between central reviewers and local investigators in each trial (ρ=0), the RHR for PFS was 0.95 (95% CI 0.91 to 0.98), indicating that local investigators slightly overestimated the HR compared with the findings of central investigators. No heterogeneity was detected in the meta-analysis (tau 2 =0; I 2 =0%; p>0.99). Table 3 and figure 3 present ROR for ORR and, in contrast, the ROR for ORR was 1.00 (95% CI 0.91 to 1.09), and no heterogeneity was observed in meta-analyses (tau 2 =0; I 2 =0%; p>0.99). Sensitivity analysis showed the RHR for PFS and ROR for ORR under the assumption of dependency between the central reviewers and local investigators in each trial ( online supplemental table 2 and online supplemental figures 4a–4d and 5a–5d ). The RHR remained constant at 0.95, and the upper 95% CIs did not exceed 1.00, indicating that local investigators consistently overestimated the HR for PFS compared with the findings of central reviewers under any assumptions of dependency. Conversely, the ROR ranged from 1.00 to 1.03, with lower 95% CIs consistently below 1.00, indicating that local investigators did not overestimate the OR for ORR under any assumption of dependency. Heterogeneity increased for both PFS and ORR from ρ=0 (independent) to ρ=0.95 (nearly completely dependent). Tables 2 and 3 and online supplemental figures 6a,b and 8a,b show the prespecified subgroup analyses of trial phases, cancer types and drug classifications. No interactions were observed between these subgroups for either the PFS or ORR.

Discussion

Principal findings

This meta-epidemiological study found that local investigators tended to slightly overestimate the HR for PFS compared with the findings of central reviewers in oncological open-label trials. In contrast, there was no evidence of overestimation in ORR. These results remained consistent in the sensitivity analysis which accounted for various assumptions, and the subgroup analysis did not identify any factors that might influence the findings. However, it is important to note that these results were based on a small subset of oncological open-label trials, and only approximately half of the reports that claimed the outcomes were adjudicated by both central reviewers and local investigators reported both results.

Strengths and limitations of the study

This study has several strengths. First, because it excluded any terms related to outcome adjudicators in its search strategy, this study represents the most comprehensive meta-analysis to date, including the largest number of studies of any existing reports. Second, we were able to provide a breakdown of adjudicated outcomes in recent oncological open-label trials. This enabled us to detect how many studies adjudicated PFS and ORR by central reviewers and local investigators, as well as the corresponding number of reported outcomes. Third, the results remained unchanged after sensitivity analysis, highlighting their robustness. Fourth, as our study compared the same outcomes between central and local investigators within the same trial, the risk of confounding was low.

Although this study has several strengths, it also has some limitations. First, the records in which both central reviewers and local investigators adjudicated the outcomes were only a small proportion of the total number of oncological open-label trials. Furthermore, only half of all reports were available for analysis, with the rest not reporting results from either assessor. Additionally, our preliminary study showed that outcome adjudicators might differ from those prespecified in the protocol or trial registry, 17 suggesting the possibility of selective outcome reporting bias in our results. In other words, among studies that claimed that both central reviewers and local investigators adjudicated the outcome, we expect studies that reported results by both assessors (ie, studies included in our analysis) to show less significant differences between the two assessors. Moreover, when we contacted the investigators of those trials that reported only central or local adjudicators’ outcomes, very few investigators provided data regarding local adjudicator outcomes. Because trial sponsors are unlikely to publish the results of trials in which the central and local adjudications are inconsistent, the low response rate may underestimate the extent of detection bias by local investigators due to selective outcome reporting bias. However, it should be noted that although the number of studies included in the analysis was limited, they showed a slight overestimation in the HR of PFS by local investigators. Actual detection bias could be greater in real-world settings. Second, we assumed that the dependence (ρ) between central and local adjudicators in all studies was 0 in the primary analysis, and between 0.25 and 0.95 in the sensitivity analysis. However, the presence of no dependence (ρ=0) between central and local adjudications is implausible, and the dependence may vary from trial to trial; therefore, this assumption may not be entirely accurate. A more rigorous synthesis could be achieved by calculating the true dependence in each study using individual patient data. Nevertheless, as our sensitivity analysis indicated that changing the values of ρ from 0 to 0.95 did not affect the final results, we assume that the variation in ρ observed among the studies would not affect the final RHR or ROR. Third, the period of the search was limited to studies published after 2010. Although this meta-analysis includes more studies than previously published reviews, this search strategy was not entirely systematic.

Comparison with other studies

Regarding general medicine, the MetaBLIND study, which evaluated the effect of blinding outcome assessors to the intervention, did not find an apparent overestimation by non-blinded assessments, 6 However, as this meta-epidemiological study compared the outcomes of different trials, there was a risk of confounding. Contrarily, when examining studies that compared blinded and non-blinded outcome adjudicators within the same trials, several previous meta-epidemiological studies found an overestimation of non-blinded adjudication by 27%–36%. 2–4 Nevertheless, as these reviews did not focus on the same medical field and outcomes and included both double-blinded and open-label trials, these findings may not be generalisable to oncology.

Narrowing the focus to oncology and the outcomes to PFS and ORR, several meta-epidemiological studies have been published to date, with inconsistent findings. Two studies that evaluated PFS found no apparent differences between central and local investigators. 12 13 However, these studies did not focus exclusively on open-label trials, and included 36 and 21 open-label trials, respectively. Of the three studies that evaluated ORR, one study found an overestimation by local investigators of 17.5% in 33 trials. However, this was limited to phase II trials and did not distinguish between double-blind and open-label trials. 14 Two studies, which included 22 and 23 open-label trials, found no evidence of overestimation. 11 12 The most recent published review included 38 and 33 open-label trials in the meta-analysis of PFS and ORR, respectively, and found that local investigators overestimated PFS but not ORR, which is consistent with the findings of our study. 15 In comparison to these meta-epidemiological studies, we specifically focused on open-label trials and included a large number of reports using the updated search strategy. Our results show that local investigators slightly overestimated PFS by 5%, but not did not overestimate ORR. While there were some discrepancies in the results between the studies; however, the impact of the discrepancies between central and local adjudications is not substantial.

Mechanisms and implications

There are many possible reasons for discrepancies between central reviewers and local investigators in the adjudication of tumour outcomes. 26 Local investigators often lack formal training in radiology, and adjudication may be influenced by knowledge of a patient’s clinical status. Central reviewers help ensure consistency in data collection and adjudication across sites, reducing the potential for measurement error and bias. In this study, we found detection bias in the estimation of the HR of PFS but not in the estimation of the OR of ORR. This could be because PFS is a more subjective outcome than ORR. ORR may be less prone to bias, as it is usually defined according to imaging evaluation using manuals such as RECIST, making it more objective. Although there was bias in HR of PFS, the bias was not marked and may not have significantly impacted the estimation and interpretation of the effect. This suggests that central reviewers are not necessary in oncological open-label trials.

However, it is important to note that methodological biases may have distorted the true values and affected these results. In other words, the results represent three possibilities: (1) low risk of detection bias, (2) underestimation of detection bias due to selective outcome reporting bias and (3) underestimation of detection bias due to the use of central reviewers. (1) If there is no methodological bias in this study, these results are true values and there is low risk of detection bias in oncological open-label trials. (2) If selective outcome reporting is present in this study (ie, the analysed studies that reported results from both assessors tended to show smaller differences between the two reviewers than studies that reported results from only one assessor), these results might underestimate the detection bias. (3) If the central adjudicators’ judgement was influenced by the local investigators, 27 or the influence of informative censoring is higher than that assumed, 28 the difference between the central and local adjudications would be reduced, resulting in an underestimation of the detection bias. Informative censoring occurs when patients whose progression is judged by local investigators but not by central reviewers are treated as censored. Previous studies, including this study, have only analysed openly available data, so it is difficult to verify these possibilities. To address this challenge, all studies with central adjudication should also report the results of the local investigators, and a meta-analysis of the ratios should be performed. We expect that further epidemiological studies will accumulate over time, enabling our findings to be investigated.

Conclusions

This meta-epidemiological study found that compared with the findings of central reviewers, local investigators may slightly overestimate the PFS, but not the ORR, in oncological open-label oncological trials. These findings suggest that detection bias of local investigators may not be substantial in the field of oncology. However, this analysis did not extract data from all identified trials and thus may not reflect true detection bias in oncological trials. Further studies that overcome this limitation are necessary before conclusions can be drawn.

Acknowledgments

We thank Yasushi Tsujimoto for his valuable comments on this protocol and Makoto Ueno for responding to our query. We acknowledge the National Institute of Public Health for the support of the language editing/ article publishing charge.

Footnotes

Twitter: @funada_satoshi, @yluo06, @YukiKataoka3

Contributors: Conceptualisation, data curation, formal analysis, investigation, methodology, project administration, resources, software, visualisation, writing—original Draft: SF. Conceptualisation, investigation, methodology, resources, writing—review and editing: YL,YK. Investigation, resources, writing—review and editing: TY,YF,SY,MK,MT,NN. Conceptualisation, writing—review and editing, supervision: YN,RU. Conceptualisation, writing—review and editing: KU. Conceptualisation, methodology, writing—review and editing, supervision: TAF. SF is the guarantor and accepts full responsibility for the work and the conduct of the study, had access to the data and controlled the decision to publish.

Funding: This study was funded by the Pfizer Health Research Foundation, and a Kyoto University School of Public Health Crossover Award.

Competing interests: SF received a research grant from Japan Society for the Promotion of Science KAKENHI (grant number JP 20K18964) and the Pfizer Health Research Foundation. YL received a Grant-in-Aid for JSPS Fellows (grant number 21J15050) for research. YK received a research grant from the Systematic Review Workshop Peer Support Group, the Japan Osteoporosis Foundation, and the Yasuda Memorial Medical Foundation for other research purposes. TY received a research grant from JSPS KAKENHI (grant number JP 21K17228) and the National Cancer Center Research Grant (grant number 2022-A-25) outside this work. NN was supported by a grant-in-aid for multicentre clinical research from Japanese Association for Acute Medicine. YN reported research funding from Taiho Pharmaceutical, Chugai Pharmaceutical, Guardant Health, Genomedia, Daiichi Sankyo and Seagen. KU reported personal fees from Sumitomo Pharma for other than the submitted work. RU reported personal fees from Eisai, Sawai Pharmaceutical and EPS Corporation for other than the submitted work, and a research grant from JSPS KAKENHI (grant number JP 20H04147 and 21KK0205). TAF reported grants and personal fees from Mitsubishi-Tanabe, personal fees from MSD, grants and personal fees from Shionogi, outside the submitted work; TAF has a patent pending (2020-548587) concerning a smartphone CBT app, and intellectual properties for Kokoro-app licensed to Mitsubishi-Tanabe.

Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review: Not commissioned; externally peer reviewed.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Data availability statement

Data are available in a public, open access repository. The collected data and codes used for our analysis are provided in online supplemental materials ( https://github.com/SatoshiFunada/2023_detection_bias ).

Ethics statements

Patient consent for publication

Not applicable.

References

1. Higgins JPT, Altman DG, Gøtzsche PC, et al. The Cochrane collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011;343:d5928. 10.1136/bmj.d5928

2. Hróbjartsson A, Thomsen ASS, Emanuelsson F, et al. Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ 2012;344:e1119. 10.1136/bmj.e1119

3. Hróbjartsson A, Thomsen ASS, Emanuelsson F, et al. Observer bias in randomized clinical trials with measurement scale outcomes: a systematic review of trials with both blinded and nonblinded assessors. CMAJ 2013;185:E201–11. 10.1503/cmaj.120744

4. Hróbjartsson A, Thomsen ASS, Emanuelsson F, et al. Observer bias in randomized clinical trials with time-to-event outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. Int J Epidemiol 2014;43:937–48. 10.1093/ije/dyt270

5. Ndounga Diakou LA, Trinquart L, Hróbjartsson A, et al. Comparison of central adjudication of outcomes and onsite outcome assessment on treatment effect estimates. Cochrane Database Syst Rev 2016;3:MR000043. 10.1002/14651858.MR000043.pub2

6. Moustgaard H, Clayton GL, Jones HE, et al. Impact of blinding on estimated treatment effects in randomised clinical trials: meta-epidemiological study. BMJ 2020;368:l6802. 10.1136/bmj.l6802

7. Hirsch BR, Califf RM, Cheng SK, et al. Characteristics of oncology clinical trials: insights from a systematic analysis of Clinicaltrials.Gov. JAMA Intern Med 2013;173:972–9. 10.1001/jamainternmed.2013.627

8. Chen EY, Haslam A, Prasad V. FDA acceptance of surrogate end points for cancer drug approval: 1992-2019. JAMA Intern Med 2020;180:912–4. 10.1001/jamainternmed.2020.1097

9. U.S. Food and Drug Administration . Clinical trial endpoints for the approval of cancer drugs and Biologics. 2018. Available: https://www.fda.gov/regulatory-information/search-fda-guidance-documents/clinical-trial-endpoints-approval-cancer-drugs-and-biologics

10. European Medicines Agency . Appendix 1 to the guideline on the evaluation of anticancer medicinal products in man. Methodological consideration for using progression-free survival (PFS) or disease-free survival (DFS) in Confirmatory trials. 2012. Available: https://www.ema.europa.eu/en/documents/scientific-guideline/appendix-1-guideline-evaluation-anticancer-medicinal-products-man-methodological-consideration-using_en.pdf

11. Zhang J, Zhang Y, Tang S, et al. Evaluation bias in objective response rate and disease control rate between blinded independent central review and local assessment: a study-level pooled analysis of phase III randomized control trials in the past seven years. Ann Transl Med 2017;5:481. 10.21037/atm.2017.11.24

12. Zhang J, Zhang Y, Tang S, et al. Systematic bias between blinded independent central review and local assessment: literature review and analyses of 76 phase III randomised controlled trials in 45 688 patients with advanced solid tumour. BMJ Open 2018;8:e017240. 10.1136/bmjopen-2017-017240

13. Dello Russo C, Cappoli N, Navarra P. A comparison between the assessments of progression-free survival by local investigators versus blinded independent central reviews in phase III oncology trials. Eur J Clin Pharmacol 2020;76:1083–92. 10.1007/s00228-020-02895-z

14. Dello Russo C, Cappoli N, Pilunni D, et al. Local investigators significantly overestimate overall response rates compared to blinded independent central reviews in phase 2 oncology trials. J Clin Pharmacol 2021;61:810–9. 10.1002/jcph.1790

15. Lian Q, Fredrickson J, Boudier K, et al. Meta-analysis of 49 Roche oncology trials comparing blinded independent central review (BICR) and local evaluation to assess the value of BICR. Oncologist 2023:oyad012. 10.1093/oncolo/oyad012

16. Funada S, Luo Y, Kataoka Y, et al. Detection bias in open-label trials of cancer drug: a meta-epidemiological study. 2021. 10.17605/OSF.IO/CSX7H

17. Funada S, Luo Y, Kataoka Y, et al. Inadequate reporting of adjudicators in open-label trials of anticancer drugs between 2017 and 2021: a methodological review. J Clin Epidemiol 2022;150:80–9. 10.1016/j.jclinepi.2022.06.020

18. Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Rev 2021;10:89. 10.1186/s13643-021-01626-4

19. Nussbaumer-Streit B, Klerings I, Dobrescu AI, et al. Excluding non-English publications from evidence-syntheses did not change conclusions: a meta-epidemiological study. J Clin Epidemiol 2020;118:42–54. 10.1016/j.jclinepi.2019.10.011

20. Therasse P, Arbuck SG, Eisenhauer EA, et al. New guidelines to evaluate the response to treatment in solid tumors. J Natl Cancer Inst 2000;92:205–16. 10.1093/jnci/92.3.205

21. Eisenhauer EA, Therasse P, Bogaerts J, et al. New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1). Eur J Cancer 2009;45:228–47. 10.1016/j.ejca.2008.10.026

22. Miller AB, Hoogstraten B, Staquet M, et al. Reporting results of cancer treatment. Cancer 1981;47:207–14. 10.1002/1097-0142(19810101)47:1<207::aid-cncr2820470134>3.0.co;2-6

23. Altman DG, Bland JM. How to obtain the confidence interval from a P value. BMJ 2011;343:d2090. 10.1136/bmj.d2090

24. Sterne JAC, Jüni P, Schulz KF, et al. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-Epidemiological’ research. Stat Med 2002;21:1513–24. 10.1002/sim.1184

25. Viechtbauer W. 'Package Metafor' 2022. Available: https://www.metafor-project.org/doku.php/metafor

26. Tang PA, Pond GR, Chen EX. Influence of an independent review committee on assessment of response rate and progression-free survival in phase III clinical trials. Ann Oncol 2010;21:19–26. 10.1093/annonc/mdp478

27. Raunig D, Goldmacher G, Conklin J. Local evaluation and blinded central review comparison: a victim of meta-analysis shortcomings. Ther Innov Regul Sci 2013;47:1–2. 10.1177/2168479013499572

28. Amit O, Mannino F, Stone AM, et al. Blinded independent central review of progression in cancer clinical trials: results from a meta-analysis. Eur J Cancer 2011;47:1772–8. 10.1016/j.ejca.2011.02.013

Associated Data

Supplementary Materials

bmjebm-2023-112332supp001.pdf

Data Availability Statement

Data are available in a public, open access repository. The collected data and codes used for our analysis are provided in online supplemental materials ( https://github.com/SatoshiFunada/2023_detection_bias ).