The increased availability of real-world evidence (RWE) for many treatment options makes it important for advanced practice providers (APPs) to understand how real-world data (RWD) studies fit into the evidence landscape to optimize clinical care for their patients. Here, we aim to provide a primer for APPs on the interpretation and practical application of RWE for clinical use and shared decision-making by summarizing the key elements of RWD studies, including what comprises a strong, high-quality RWD study. We demonstrate how RWD studies complement randomized controlled trials (RCTs) using recently published studies for palbociclib.
DEFINING RWD AND RWE
Real-world data refers to information relating to patient health status and/or the delivery of health care that is routinely collected outside of the confines of a clinical trial ( FDA, 2018 ). Sources of RWD include electronic health records (EHRs), claims and billing databases, product and disease registries, patient-generated data, and mobile/wearable devices ( FDA, 2018 ). Real-world evidence is derived from the analysis of RWD and consists of “clinical evidence about the usage and potential benefits or risks of a medical product” ( FDA, 2018 ).
WHAT IS THE PURPOSE OF RWE?
Historically, health-care decisions have been driven by evidence derived from RCTs. Randomized controlled trials examine cause and effect relationships between treatments and outcomes and, therefore, are considered the gold standard for the evaluation of a treatment's efficacy and safety ( Blonde et al., 2018 ; Hariton & Locascio, 2018 ; Kurtin & Taher, 2020 ); however, this type of research has challenges and limitations ( Table 1 ). For example, patient populations in RCTs may not accurately reflect the diversity of patients seen in routine clinical practice because of strict inclusion and exclusion criteria, which are necessary to reduce the number of variables that could affect treatment outcomes. Randomized controlled trials may exclude patients with certain comorbidities or restrict the age groups that are included ( Singh et al., 2017 ; Talarico et al., 2004 ; Tang et al., 2023 ). Furthermore, patients in some race/ethnicity groups are underenrolled in clinical trials, limiting their representation, which may impact the generalizability of RCTs ( Singh & Jemal, 2017 ; Turner et al., 2022 ). Additionally, RCTs are typically cost and resource intensive, costing an estimated median of $1.1 billion for a single drug program ( Wouters et al., 2020 ). Owing to cost and resource limitations, RCTs may not have a large enough sample size ( Cummings, 2018 ; Rohrig et al., 2010 ) or not be long enough in duration to generate significant data on rare side effects or long-term efficacy ( Broglio & Berry, 2009 ; Monti et al., 2018 ).
Table 1
Key Strengths and Limitations of Randomized Controlled Trials and Real-World Data Studies
Note. Information from Blonde et al. (2018) ; Hariton & Locascio (2018) ; Kurtin & Taher (2020) ; O'Leary & Cavender (2020) ; Tang et al. (2023) .
| Randomized controlled trials | Real-world data studies | |
| Strengths | Designed to show cause–effect relationship between an intervention and outcome Data are collected prospectively with research intent; therefore, are of high quality and completeness Random assignment of patients to treatment arms increases internal validity Strict inclusion and exclusion criteria can reduce variability | Can include a large and diverse sample size; therefore, are generalizable to more patients Data can be collected over long periods of time and stored; therefore, can detect rare and long-term events Can answer research questions about the long-term use and prescribing patterns of a treatment |
| Limitations | Limited generalizability Cost and resource intensive May be unable to answer all questions about real-world uses of a drug in clinical practice May be unable to answer research questions related to rare disease or rare outcomes May not be long enough to assess long-term events (e.g., overall survival) | Do not demonstrate causality May have missing or erroneous data entry in source data Unobserved variables cannot be controlled Potential for bias (e.g., selection, information, time-related, and prevalent user biases) Variation in assessments of clinical outcome (e.g., physicians may not use blinded, centralized, or standardized assessments) |
There are also some limitations on the types of research questions that can be answered by RCTs. The health-care landscape is constantly evolving in response to new evidence and technologies. This can influence how a drug is used in the real-world setting compared with how it was originally evaluated in RCTs ( Subbiah, 2023 ). For instance, daratumumab is routinely administered over two separate infusions in clinical practice, even though that is not how it was originally evaluated in RCTs ( Rifkin et al., 2019 ). Additionally, there can be ethical concerns about the use of a placebo or continued use of a comparator drug if there is sufficient evidence to support the use of one intervention over another. In other instances, such as in rare diseases, standard-of-care options may be lacking altogether, and no comparator is possible ( Monti et al., 2018 ; Thorlund et al., 2020 ).
Both RCTs and RWD studies can be used to assess the effects of a treatment. To draw a distinction between the two types of analyses, the term “efficacy” is used to describe the treatment effects observed in RCTs, whereas “effectiveness” is used to describe these same findings in real-world settings ( Singal et al., 2014 ). Owing to the nature of RWD sources, RWD studies have a different set of strengths and limitations compared with RCTs ( Table 1 ). Real-world data can provide evidence of treatment effects in patients with broader clinical and demographic characteristics (e.g., comorbidities, advanced age, ethnicity, or socioeconomic diversity), who may not have been eligible for, or had access to clinical trials—despite being more closely representative of the patient population seen in real-world clinical practice ( Blonde et al., 2018 ; Singal et al., 2014 ). Real-world data sources can also provide a cost-effective means to assess outcomes in more challenging situations ( Dang, 2023 ). Real-world data can be used to investigate clinical outcomes among those with rare diseases (occurring in ≤ 1 in 2,000 people worldwide; Rare Diseases International, 2023 ) and longer-term clinical outcomes (e.g., 5- or 10-year survival); RCTs are not usually adequately powered to evaluate rare diseases or long-term outcomes as these may take too long to manifest and it is therefore not a feasible methodology ( Broglio & Berry, 2009 ; Mahendraratnam et al., 2019 ; Monti et al., 2018 ; Xia et al., 2019 ). Additionally, RWD can be used in place of a comparator arm by providing historical control data for single-arm trials, thus alleviating certain ethical concerns of RCTs (e.g., having a placebo arm or one that receives no intervention [observation arm]; Jahanshahi et al., 2021 ; Mahendraratnam et al., 2019 ; Thorlund et al., 2020 ; FDA, 2018 ).
REAL-WORLD EVIDENCE TO SUPPORT REGULATORY DECISIONS
The US Food and Drug Administration (FDA) recognizes the potential of RWD to improve the efficiency of drug approvals and has issued guidance for evaluating RWD and RWE to help support the approval of new and already approved drugs, as well as for use in post-marketing surveillance ( FDA, 2018 ). Between 2000 and 2019, the FDA accepted RWE that served as external controls in support of decision-making in 45 non-oncology product approvals ( Jahanshahi et al., 2021 ). It also accepted RWE to support decision-making when approving 13 indications for oncology treatments between 2015 and 2020 ( Arondekar et al., 2022 ). The median approval time for oncology treatments in expedited programs that included RWE in the application was 5.5 years, which was 1 year less than those that did not include RWE, suggesting that inclusion of RWE accelerated drug approval processes ( Arondekar et al., 2022 ).
Post-marketing pharmacovigilance is important for already approved products to determine whether there are rare adverse reactions or reactions that occur only after long-term exposure that should be included in the drug's label ( Forstag, 2019 ; FDA, 2018 ). Historically, these were based on case reports and journal publications ( Lavertu et al., 2021 ; FDA, 2018 ). More recently, RWD has been used in various pharmacovigilance studies, reviewed in Lavertu and colleagues (2021) . For example, RWE was used to confirm the safety of dosing schedules used in clinical practice that differed from the recommended administration of daratumumab for the treatment of patients with multiple myeloma. Daratumumab was indicated for one single (long) intravenous administration; however, due to the length of the infusion, it was often done over two separate infusions in clinical practice. A RWD study determined that this split dose was as safe as a single infusion ( Rifkin et al., 2019 ).
In the field of rare diseases, RWE has been an important support tool for regulatory and clinical decision-making ( Jahanshahi et al., 2021 ). Owing to the inherently small patient sample sizes associated with rare or ultra-rare diseases, standard RCTs are often not feasible ( Jahanshahi et al., 2021 ). Real-world evidence played a major role in the approval of the prescription gene therapy, onasemnogene abeparvovec-xioi (Zolgensma), for children under 2 years old with spinal muscular atrophy via the characterization of the natural history of the disease that allowed the researchers to associate the improvement in patient outcomes with the therapy ( FDA, 2019 ).
Real-world data have also been particularly useful for regulatory submissions where certain groups of patients may have been excluded from clinical trials. For example, men have been historically excluded from breast cancer RCTs as men account for less than 1% of newly diagnosed breast cancer cases ( American Cancer Society, 2023 ). In 2015, palbociclib (Ibrance), a cyclin-dependent kinase 4 and 6 (CDK4/6) inhibitor, received accelerated FDA approval to be used in combination with letrozole to treat postmenopausal women with hormone receptor-positive/human epidermal growth factor receptor 2–negative (HR+/HER2−) advanced or metastatic breast cancer (mBC) as initial endocrine-based therapy, followed by full FDA approval in 2017. In 2016, palbociclib received a second approval for the combination with fulvestrant to treat HR+/HER2– advanced or mBC in women with disease progression following endocrine therapy ( FDA, 2017 ). In 2019, palbociclib was then approved for use in men with HR+/HER2− mBC based on the results of the PALOMA-2 and PALOMA-3 trials, clinical trials of men treated with palbociclib for non-breast solid tumors, RWD derived from EHRs, insurance claims, the palbociclib global safety database, post-marketing reports from the FDA Adverse Event Reporting System, and published literature on male patients treated with palbociclib ( Wedam et al., 2020 ).
WHY SHOULD APPs UNDERSTAND RWE?
The use of RWE in health-care decisions has grown in recent years ( Arondekar et al., 2022 ; Jahanshahi et al., 2021 ). This has been further bolstered by the enactment of the 21st Century Cures Act in 2016 that called for the modernization of clinical trial design and the generation and storage of large quantities of patient health-related data ( Khozin et al., 2017 ; Schad & Thronicke, 2022 ; US Congress, 2016 ). Advanced practice providers may encounter RWD and RWE in their everyday practice and should understand how to interpret and apply this information to their practice. These data may be of particular interest to APPs because they have the potential to provide insight into patient populations that more closely match the demographically and clinically diverse population seen in clinics. Additionally, the documentation that APPs complete as part of EHRs may be later used in RWD studies, thus providing complete and accurate records that can aid in future health-care decisions based on RWE.
METHODOLOGY AND DATA SOURCES FOR RWD STUDIES
The study design and data source chosen to generate RWE depends on the research question of interest.
RWD Study Design
Data collected and used for RWD studies typically fall into one of two categories: prospective or retrospective ( FDA, 2018 ). Both prospective and retrospective RWD studies should have protocols that outline the methodology to collect and analyze the relevant data needed to answer specific research questions. Prospective data collection is preplanned. The study population is defined at the start of the study, and treatment and outcome data are collected from that point forward (i.e., prospectively; FDA, 2018 ). Contrastingly, retrospective studies utilize data generated prior to the initiation of the study ( FDA, 2018 ). Such data may have been originally generated for other purposes (i.e., clinical care that is not necessarily related to the study in question) and without research intent ( FDA, 2018 ). Therefore, some key information may not be available ( Berger et al., 2017 ; Khosla et al., 2018 ; Kim et al., 2018 ).
Some RWD study designs include pragmatic clinical trials, cohort, cross-sectional, and case-control studies ( Dang, 2023 ; Taur, 2022 ). Additionally, RWD studies can be defined by the data source (e.g., registry, EHR, or claims database analysis; Dang, 2023 ; Taur, 2022 ).
Pragmatic clinical trials resemble routine clinical practice and are often conducted within health-care practices where data are collected at the point of care and integrated into the health-care system ( Khozin et al., 2017 ; FDA, 2018 ). Pragmatic clinical trials may not always dictate specific treatment details and may involve refinement of the intervention throughout the trial ( Ford & Norrie, 2016 ). Suboptimal treatment adherence may occur as pragmatic clinical trials may not rely on study visits to collect outcome data ( Ford & Norrie, 2016 ). While pragmatic clinical trials may include treatment randomization ( FDA, 2018 ), treatment is often not masked or blinded to patients or health-care providers ( Ford & Norrie, 2016 ). Despite these limitations, pragmatic clinical trials have significant potential to generate RWE among broader patient populations with data collection practices that are already in place within health-care systems ( FDA, 2018 ).
Cohort studies can use prospectively or retrospectively collected data, and can provide information on incidence, natural history of disease, risk factors, course of disease, and treatment outcomes ( Dang, 2023 ). Cross-sectional studies can also use prospectively or retrospectively collected data. For cross-sectional studies, information is taken from a single point in time (i.e., treatment and outcome data are collected at the same time) and are useful for providing information on prevalence ( Dang, 2023 ; Setia, 2016 ; Taur, 2022 ). Case-control studies use retrospectively collected data to identify patients with a particular outcome and seek to determine factors that contributed to that outcome ( Dang, 2023 ; Taur, 2022 ). Patients without that outcome of interest are used for control data ( Taur, 2022 ).
RWD Sources
Real-world data can be derived from a variety of sources that each come with merits and caveats ( Gokhale et al., 2020 ). Sources of RWD can include product or disease registries, EHRs, administrative claims data, surveys, and wearable devices or through social media platforms ( Table 2 ). Selection of the ideal data source depends on the primary research question and the parameters and outcomes that are required for the analysis ( Gatto et al., 2022 ). As factors such as sample size and outcomes captured vary across data sources, it is critical for researchers to use RWD sources that best fit the needs of their analyses ( Gatto et al., 2022 ). Large sample sizes are necessary for the comparison of one or more treatments (comparative effectiveness; Tang et al., 2023 ). Furthermore, treatments with small effect sizes on measured outcomes require larger sample sizes than treatments with large effects; therefore, researchers may need a large database or multiple databases to assess treatments with small effect sizes ( Tang et al., 2023 ). While large, diverse, RWD sources may be applicable for many types of studies, there are instances where a rare outcome or a subset of patients may be the focus. In those instances, a small but robust registry that captures these rare outcomes or patients may be a better fit ( Gokhale et al., 2020 ).
Table 2
Sources of Real-World Data and the Types of Outcomes They Can Provide
Note. Information from Garrison et al. (2007) ; Gokhale et al. (2020) ; Tang et al. (2023) .
| Data source | Population | Data collected over time | Outcomes |
| Claims databases | People covered by a common payer | Longitudinal collection of resource use and associated payments may include: Demographics Treatments Healthcare visits Pharmacy Costs | Safety and effectiveness Treatment patterns Cost Utilization |
| Registries | People with a specific diagnosis, condition, or receiving a particular treatment, intervention or procedure | Clinical information for patients with an identified condition, which may include: Demographics Treatments Health status | Safety and effectiveness Treatment patterns Cost Utilization Estimated adherence Patient-reported outcomes |
| Electronic health records | People within a health-care system | Individual patient health data from clinician charts and records that may include: Demographics Treatments Health status Imaging Laboratory values | Safety and effectiveness Treatment patterns Utilization Patient-reported outcomes |
| Patient and physician surveys | People with a specific diagnosis, condition, treatment, or procedure, or physicians who treat those people | Data are collected at a single point in time and may include: Demographics Prescribing patterns Perceptions and attitudes | Treatment patterns Quality of life Estimated adherence Patient-reported outcomes |
| Patient-generated data (e.g., from wearable devices and health-related mobile applications, and social media) | Healthy and ill people | Data are collected at a single point in time or over a period of time and may include: Demographics Health status Perceptions and attitudes | Treatment patterns Quality of life Estimated adherence Patient-reported outcomes |
Statistical Methods Used in RWD Studies
Confounding is the existence of variables that may affect the outcome of interest ( Gokhale et al., 2020 ). Randomization is a key strength of RCTs and is implemented to help lessen confounding and to balance the patient characteristics in the arms of a clinical trial ( Blonde et al., 2018 ; Hariton & Locascio, 2018 ). Owing to lack of randomization in RWD studies, statistical methods must be used to account for many confounding variables ( Austin, 2014 ; Austin & Stuart, 2015 ).
Validated statistical approaches for adjustment include multiple imputation methods and propensity score (PS) techniques ( Austin, 2014 ; Austin & Stuart, 2015 ; Austin et al., 2021 ). Multiple imputation methods are valuable for dealing with missing data ( Austin et al., 2021 ), which can be prevalent in some RWD sources ( Haneuse et al., 2021 ). For example, variables such as body mass index may not be recorded in EHRs ( Haneuse et al., 2021 ), and patients may refuse to answer certain survey questions ( Austin et al., 2021 ). As it is not always practical to exclude patients from an analysis due to a single missing variable, multiple imputation methods use available data to calculate several possible values for the missing variable ( Austin et al., 2021 ). Statistical analysis is then performed on the resulting data.
Propensity score techniques, such as propensity score matching (PSM) and inverse probability of treatment weighting (IPTW), are used to attempt to balance baseline characteristics or variables between patient groups and thereby limit bias ( Austin & Stuart, 2015 ). The PS estimates the probability of receiving a particular treatment based on baseline (patient) characteristics that may affect the outcomes ( Austin, 2014 ; Gokhale et al., 2020 ). For example, two patients with the same PS are more likely to have similar baseline characteristics than patients with different PSs ( Austin, 2014 ; Austin & Stuart, 2015 ). With PSM, outcomes for patients with similar PSs are compared between treatments, excluding patients from the analysis who do not have a match ( Austin, 2014 ). With IPTW, a statistical weight is assigned to each patient based on their PSs so that, rather than excluding patients from the analysis, patients with similar statistical weights are compared between the cohorts ( Austin, 2014 ). The key variables used in PS analysis should be defined prior to data extraction. Both PSM and IPTW can be used in a single study to provide internal validity for the study.
WHAT MAKES A ROBUST/HIGH-QUALITY RWD STUDY?
Principles for evaluating the quality of observational studies that provide data on comparative effectiveness have been proposed ( Dreyer et al., 2010 ). Best practices for RWD studies of treatment effectiveness have also been outlined by a joint special task force of the International Society for Pharmacoeconomics and Outcomes Research and the International Society for Pharmacoepidemiology ( Berger et al., 2017 ), and even more recently, a structured template for planning and reporting on RWE study implementation (STaRT-RWE) was developed ( Wang et al., 2021 ). High-quality RWD studies are those that are designed with a research question and clinically meaningful outcomes ( Dreyer et al., 2010 ; Tang et al., 2023 ). A study plan should be documented in a protocol that includes detailed study parameters, including patient inclusion and exclusion criteria, definitions of diseases and conditions, and allowed treatment regimens ( Dreyer et al., 2010 ; Gokhale et al., 2020 ). The processes for data collection, including the types of codes (e.g., diagnostic codes) to define measures, the timing of measurements (e.g., index date, duration of follow-up) and outcome validation should be predefined ( Wang et al., 2021 ). In addition to a protocol, a statistical analysis plan allowing for analysis of the data and assessment of the outcomes should be in place prior to any data extraction.
The potential for bias, including selection bias, information bias, time-related biases, and prevalent-user biases, is present in all studies that use RWD; however, studies should be carefully designed to reduce the sources of bias that can occur ( Table 3 ; Dang, 2023 ; Gokhale et al., 2020 ; Wang & Schneeweiss, 2022 ). Selection bias in RWD can occur due to physicians choosing which treatment patients receive ( Gokhale et al., 2020 ). Detection bias is a specific type of selection bias that arises when the probability of an event being captured is more likely in one treatment group over another ( Dang, 2023 ; Rothman et al., 2008 ). Selection biases can be mitigated through statistical adjustments ( Blonde et al., 2018 ). Most other types of bias can be reduced by ensuring that the study design, including the data source chosen (i.e., is the source reliable and relevant), and the analysis plan are “fit for purpose” by taking into account any data source limitations and choosing an appropriate study design to control for sources of bias ( Gatto et al., 2022 ; Tang et al., 2023 ; Taur, 2022 ; Wang & Schneeweiss, 2022 ).
Table 3
Sources of Bias in Studies Using Real-World Data
Note. Information from Dang (2023) ; Gokhale et al. (2020) ; Rothman et al. (2008) ; Wang & Schneeweiss (2022) .
| Source of bias | Definition | Method or statistical approach to minimize bias |
| Selection bias | The selected population is not representative of the target population for which the conclusions are to be drawn that can arise due to physicians or patients (self-selection) choosing who gets a particular treatment | Identify and measure factors that influence selection in advance of the study Appropriate control selection Stratification Propensity score methods to balance cohorts |
| Information bias or measurement bias | Bias that occurs due to misclassification of a treatment or outcome or from inaccurate measurement | Quality control measures such as data editing to examine the data extracted for erroneous entries Multiple imputation methods for dealing with missing data |
| Immortal person-time bias | Occurs due to misclassification of the time before the treatment where the patients could not have experienced the outcome | Define the allowance for the time interval between exposure and disease onset that corresponds to a meaningful induction period (induction-time model) Stratification |
| Recall bias | Occurs due to selective recollection of events by patients or caregivers | Ensure that the time that lapsed between the exposure and recall is the same for both treatment and control groups |
| Prevalent-user bias | Patients are more tolerant of drug due to already being on the treatment before study follow-up start | Require a wash-out period or focus on new users/treatment-naive patients |
Additionally, personnel should be adequately trained to perform data extraction and, if possible, be blinded to the research question. Alternatively, data extraction can be performed by an approved automated system ( Rothman et al., 2008 ). Quality control measures should be used to assess and report missing data or erroneous entries and determine the cause and the potential effect on the results ( Dreyer et al., 2010 ; Rothman et al., 2008 ). When the study is published, enough information should be shared for the study to be replicated ( Berger et al., 2017 ; Dreyer et al., 2010 ). Where feasible, the study should be performed on multiple data sources to provide external validation ( Berger et al., 2017 ).
Limitations
Real-world data does not assess causation, but rather provides information about the association between a treatment and an outcome ( Khosla et al., 2018 ; FDA, 2018 ). This limitation is due to the nature of RWD as patient-level randomization is not possible; randomization is an effective means of mitigating some potential sources of bias ( Blonde et al., 2018 ; Hariton & Locascio, 2018 ). While there are study design recommendations and statistical approaches that can be applied in RWD studies to minimize the potential for bias and confounding effects, it is not possible to determine a causal relationship between interventions and outcomes ( Khosla et al., 2018 ; FDA, 2018 ) owing to the existence of unobserved variables. Despite quality-control measures, there is also the possibility of missing or erroneous data within the data sources, representing another limitation of RWD studies. Variation in the assessment of treatment response is a potential limitation. For example, disease response assessments in RWD are generally based on the provider's determinations instead of using standardized assessment criteria, such as the Response Evaluation Criteria in Solid Tumors (RECIST) and a blinded centralized review. Additionally, the results of RWD studies may not be generalizable to patients outside of the database used for that study; for example, a study conducted solely from a database of patient records in Asia may not be fully generalizable to a US population and vice versa.
PRACTICAL APPLICATION OF RWE: INFORMING TREATMENT DECISIONS
As previously noted, the data ascertained from RWD studies can help provide a more robust understanding of a therapy's effectiveness in real-world clinical practice. Palbociclib was the first CDK4/6 inhibitor to be approved by the FDA, receiving an accelerated approval in February 2015 for use in combination with letrozole for the treatment of postmenopausal women with estrogen receptor-positive (ER+)/HER2− advanced breast cancer; this approval was based on data from a phase II clinical trial. Since this approval, there are now more than 8 years of RWD collected following the integration of palbociclib into clinical practice. Here, we demonstrate the value and practical application of RWE using two palbociclib studies: the PALOMA-2 RCT ( Finn et al., 2016 ; Finn et al., 2022 ) and the P-REALITY X RWD study ( Rugo et al., 2022 ; Table 4 ).
Table 4
Complementary Data From the PALOMA-2 Clinical Trial and the P-REALITY X RWD Study
Note. aBC = advanced breast cancer; AI = aromatase inhibitor; CI = confidence interval; DOR = duration of response; ECOG PS = Eastern Cooperative Oncology Group performance status; EHRs = electronic health records; ER+/HER2? = estrogen-receptor positive/human epidermal growth factor 2-negative; HR = hazard ratio; HR+/HER2− = hormone-receptor positive/human epidermal growth factor 2 negative; HRQOL = health-related quality of life; mBC = metastatic breast cancer; OR = objective response; OS = overall survival; PFS = progression-free survival; PSM = propensity score matching; rwPFS = real-world progression-free survival; RWD = real-world data; sIPTW = stabilized inverse probability of treatment weighting. Information from Finn et al. (2016) ; Finn et al. (2022) ; Rugo et al. (2022) ; Rugo et al. (2019) .
| PALOMA-2 ( N = 666) | P-REALITY X ( N = 2,888) | |
| Type of study | Randomized (2:1), double-blind, multicenter, global, phase III clinical trial | Retrospective, cohort analysis of EHRs within the US Flatiron Health Analysis. Database of over 3 million patients |
| Key eligibility criteria | Postmenopausal women with ER+/HER2? aBC No prior treatment in the metastatic setting ECOG PS 0?2 | Men and postmenopausal women aged ≥ 18 years with HR+/HER2− mBC starting on palbociclib + AI or AI alone in the first-line from February 3, 2015, to March 31, 2020 (index date) Potential follow-up of 6 to 68 months from index date to study cutoff date, September 30, 2020 |
| Treatment arms | Palbociclib + letrozole vs. placebo + letrozole | Palbociclib + AI vs. AI alone |
| Primary endpoint | PFS | OS |
| Secondary endpoints | OS, OR, DOR, HRQOL, safety, pharmacokinetic effects, tissue biomarker assessments | rwPFS, subsequent treatments |
| Statistical methods/assumptions | Sample size calculated to detect ~44% improvement in median PFS from 9 months for the control arm to 13 months for the palbociclib arm Assuming a true HR of 0.69 in favor of the palbociclib arm (90% power with 1-sided α = 0.025) Median OS - 390 events required to detect HR of < 0.74 (80% power with 1-sided α = 0.025) The Kaplan-Meier method was used to estimate median PFS. HRs were estimated from Cox proportional hazards model | Sample size calculated to detect 25% improvement in median OS from 40 months for the control cohort to 50 months for the palbociclib cohort Assuming a true HR of 0.80 in favor of palbociclib (80% power with a 2-sided α = 0.05) Unadjusted analysis sIPTW as primary analysis to balance baseline demographics and clinical characteristics PSM conducted as sensitivity analysis |
| Median PFS | Primary endpoint Median follow-up 38 months Palbociclib + letrozole ( n = 444) 27.6 months Placebo + letrozole ( n = 222) 14.5 months HR = 0.56 (95% CI = 0.46?0.69), p < .0001 | Secondary endpoint (rwPFS) Median follow-up 24 months Unadjusted: Palbociclib + AI ( n = 1,324) 19.8 months AI alone ( n = 1,564) 13.9 months HR = 0.68 (95% CI = 0.62?0.76), p < .0001 After sIPTW: Palbociclib + AI ( n = 1,572) 19.3 months AI alone ( n = 1,137) 13.9 months HR = 0.70 (95% CI = 0.62–0.78, p < .0001 After PSM: Palbociclib + AI (n = 939) 19.8 months AI alone (n = 939) 14.9 months HR = 0.72 (95% CI = 0.63–0.82), p < .0001 |
| Median OS | Secondary endpoint Median follow-up 90 months Palbociclib + letrozole 53.9 months Placebo + letrozole 51.2 months HR = 0.956 (95% CI = 0.777?1.177), p = .338 | Primary endpoint Median follow-up 24 months Unadjusted: Palbociclib + AI 53.4 months AI alone 40.4 months HR = 0.67 (95% CI = 0.60?0.76), p < .0001 After sIPTW: Palbociclib + AI 49.1 months AI alone 43.2 months HR = 0.76 (95% CI = 0.65–0.87), p < .0001 After PSM: Palbociclib + AI 57.8 months AI alone 43.5 months HR = 0.72 (95% CI = 0.62–0.83), p < .0001 |
| Key takeaway | Palbociclib + letrozole compared with placebo + letrozole had statistically significantly prolonged median PFS. Median OS was numerically longer for palbociclib + letrozole compared with placebo + letrozole, but the result was not statistically significant | Palbociclib + AI compared with AI monotherapy was statistically significantly associated with prolonged median OS and median rwPFS before and after sIPTW and PSM |
The PALOMA-2 trial ( NCT01740427 ) was a prospective, randomized, double-blind, multicenter, global, phase III clinical trial that compared palbociclib + letrozole with placebo + letrozole in postmenopausal women with ER+/HER2− advanced breast cancer who had no prior treatment in the metastatic setting and an Eastern Cooperative Oncology Group performance status (ECOG PS) of 0 to 2 ( Finn et al., 2016 ). Patients were excluded if they had advanced, symptomatic, visceral spread and were at risk for life-threatening complications or known active uncontrolled or symptomatic central nervous system metastases (other inclusion/exclusion criteria applied). The primary endpoint of the trial was progression-free survival (PFS), which was assessed by a blinded, independent, central review panel using RECIST. Overall survival (OS) was assessed as a key secondary endpoint. Some of the other secondary endpoints were objective response and safety. The trial was specifically designed to detect a 44% improvement in the primary endpoint of median PFS with palbociclib + letrozole compared with the control arm (placebo + letrozole), with 90% power.
A total of 666 postmenopausal women were enrolled in PALOMA-2. The PALOMA-2 trial met its primary endpoint of PFS (with 194 events in the palbociclib + letrozole arm and 137 events in the placebo + letrozole arm) and demonstrated that patients treated with palbociclib + letrozole had a significantly longer median PFS of 27.6 months compared with 14.5 months for those treated with placebo + letrozole, resulting in a 44% reduction in the risk of disease progression or death (hazard ratio [HR], 0.56; 95% confidence interval [CI] = 0.46-0.69; p < .0001; Rugo et al., 2019 ). There was a numerically longer, but not statistically significant, difference in median OS (secondary endpoint) between those treated with palbociclib + letrozole vs. placebo + letrozole (53.9 vs. 51.2 months; p = .338) after 405 events and a follow-up period of 90 months ( Finn et al., 2022 ). As mentioned previously, one limitation of RCTs is that it can be difficult to assess endpoints that take a long time to manifest. For example, the long median survival after progression such as in PALOMA-2 could mean the OS results are confounded by sample size, treatment crossover, and the dilutionary effects of multiple subsequent treatments ( Broglio & Berry, 2009 ; Finn et al., 2022 ).
In contrast to PALOMA-2, the P-REALITY X ( NCT05361655 ) study had a retrospective design to evaluate clinical outcomes from both men and postmenopausal women with HR+/HER2− mBC treated with palbociclib + any aromatase inhibitor (AI) and those who received an AI alone between February 3, 2015, to March 31, 2020 ( Rugo et al., 2022 ). The study used a protocol and statistical analysis plan to extract data from EHRs of the US Flatiron Health Analysis Database, which included over 3 million patients, most of whom (> 90%) were primarily treated in community settings ( Rugo et al., 2022 ). Patients with clinical variables that may affect survival and treatment selection such as visceral disease, brain metastasis, and ECOG PS 0 to 4 were also included. The primary endpoint of P-REALITY X was OS, which was a strictly objective endpoint, as patients were either alive or not at the time of the assessment, with no possibility of bias in this determination. Real-world (rw) PFS was a secondary endpoint, assessed by physicians based on radiographic or pathological results, rather than requiring evaluations be conducted utilizing RECIST, and resulting in a potentially more subjective outcome than when assessed as an endpoint in PALOMA-2. Three methods were used in P-REALITY X for comparative analyses between treatment groups: (1) an unadjusted analysis that did not control for baseline demographic and clinical characteristics, (2) stabilized inverse probability of treatment weighting (sIPTW; primary analysis) to balance baseline characteristics and control for confounding variables, and (3) 1:1 PSM as a sensitivity analysis. Both sIPTW and PSM methodologies used PSs computed based on the following variables: age group, sex, race/ethnicity, practice type, disease stage at initial diagnosis, ECOG PS, bone disease, visceral disease, interval from initial breast cancer diagnosis to mBC diagnosis, and number of metastatic sites.
In P-REALITY X, data from 2,888 men and postmenopausal women with HR+/HER2− mBC were included, and the study met its primary endpoint of OS. In the primary analysis (sIPTW), patients treated with palbociclib + an AI had a median OS of 49.1 months compared with 43.2 months for those treated with an AI alone, resulting in a 24% reduction in the risk of death (HR, 0.76; 95% CI = 0.65-0.87; p < .0001). The secondary endpoint of rwPFS was also significantly extended to 19.3 months in the palbociclib + an AI arm compared with 13.9 months in the AI-alone arm, resulting in a 30% reduction in the risk of disease progression or death (HR, 0.70; 95% CI = 0.62-0.78; p < .0001) after sIPTW. Both primary and secondary endpoint results were confirmed by the PSM analysis ( Table 4 ), further lending internal validity to the study. Limitations of the P-REALITY X study include those that are inherent to any study that uses RWD, including the potential for missing or erroneous data entry, the inability to account for unobserved variables, and potential lack of generalizability to patient populations beyond the database used for the study ( Rugo et al., 2022 ).
While there are many differences between PALOMA-2 and P-REALITY X, including study design, inclusion and exclusion criteria, data-collection methods, sample sizes, and patient characteristics (i.e., median age, race), the two studies provide an example in which RWE was able to provide information that may fill in some of the existing data gaps. For example, both the clinical and demographic diversity of the patient population included in the P-REALITY X study differed from those in PALOMA-2: the patients in P-REALITY X were older, included men (in alignment with the US label) and were primarily treated in a community setting (90%) vs. an academic setting ( Rugo et al., 2022 ); and, importantly, more Black patients were included in P-REALITY X ( Rugo et al., 2022 ) (~8%, vs. 1.2%–1.8% in PALOMA-2; Finn et al., 2016 ). P-REALITY X also included patients with ECOG PS > 2, which were not included in PALOMA-2 ( Finn et al., 2016 ; Rugo et al., 2022 ). Fewer patients had visceral metastases (29% vs. 49% in PALOMA-2) and more had bone-only metastasis (39% vs. 23% in PALOMA-2; Finn et al., 2016 ; Rugo et al., 2022 ). There was also a notable difference in the sample sizes between the two studies, with data from 2,888 men and postmenopausal women included in P-REALITY X and 666 postmenopausal women enrolled in PALOMA-2.
The results of P-REALITY X and of PALOMA-2 together can provide some reassurance to APPs that prescribed medications can be effective in the patients that they care for in clinical practice, even when some of those patients' characteristics differ from those who were included in RCTs. This supports the integration of RWE into discussions with patients when considering therapeutic options.
WHY RWE IS IMPORTANT TO PATIENTS: SHARED DECISION-MAKING
Patients who are actively involved in their treatment decisions are more likely to adhere to their treatment plan ( Deniz et al., 2021 ). Patients want to know if the treatment will work for them ( Forstag, 2019 ). For patients and clinicians to make fully informed decisions about treatments, it is beneficial to understand the benefits and risks of potential treatments. Randomized controlled trials are intentionally designed to control for variability; however, in the real world, variability exists. As well-designed RWD studies are able to account for variability and factors that may moderate a treatment's effect ( Singal et al., 2014 ), they can contribute to the totality of evidence for a treatment and should be discussed with patients when making treatment decisions. This can be especially helpful if a patient has comorbidities or characteristics that were not well-represented or permitted in RCTs (e.g., age, race/ethnicity, cardiac disease).
The totality of the data for a specific treatment can help both the clinician and patient be more comfortable in understanding what the treatment experience could be for patients not well represented in clinical trials. This can be seen in a quick overview of the demographic data for the two studies described previously. Briefly, PALOMA-2 included postmenopausal women only, had a somewhat younger population, allowed an ECOG PS of 0 to 2, and the percentage of Black patients was < 2%. In contrast, the P-REALITY X RWD included a larger sample size, included men and postmenopausal women, was comprised of a slightly older population, allowed an ECOG PS of 0 to 4, and had a percentage of Black patients of approximately 8%. By looking at this simple comparison of demographic information, the P-REALITY X study included a larger and broader range of patients, allowing for greater generalizability. While there are similarities and differences in the two studies, taken together, these studies provide more data for consideration by both clinicians and patients when engaging in shared decision-making.
WHY IS THIS IMPORTANT TO APPs?
With the increase in RWE prevalence and use, APPs are likely to encounter this type of data and think about how to apply it to their practice. Understanding why RCTs and RWD studies are conducted and their strengths/limitations and similarities/differences, determining how generalizable each are to the broader population of patients with a particular disease, and how they can inform treatment decisions is critical. Importantly, knowing that much of RWD are derived from EHRs serves as a reminder to APPs that complete and accurate documentation is essential. Not only is this important for clinical care, but a patient's clinical information could also become part of a RWD study. While finding the needed time for thorough documentation can be challenging, entering relevant and comprehensive information can help make databases more robust and potentially contribute to RWE. Finally, the ability to assess the strengths of RWD studies is important in determining how and when the results can be applied to clinical practice. Internal discussion among APPs and oncologists may raise awareness of this type of research and show how RWE can be applied to selecting therapeutic options and be integrated into shared decision-making with patients.
Acknowledgment
Medical writing support, conducted in accordance with Good Publication Practice (GPP 2022) and the International Committee of Medical Journal Editors (ICMJE) guidelines, was provided by Heather Caballes, PhD, of Oxford PharmaGenesis Inc., Newtown, PA; with funding provided by Pfizer Inc., USA.
Footnotes
Joanne C Ryan is an employee and stockholder of Pfizer Inc. Kimberly Podsada has served as a speaker for Pfizer Inc.
References
American Cancer Society. (2023). Cancer facts & figures 2023. https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and-figures/2023/2023-cancer-facts-and-figures.pdf
Arondekar, B., Duh, M. S., Bhak, R. H., DerSarkissian, M., Huynh, L., Wang, K.,…Demetri, G. D. (2022). Real-world evidence in support of oncology product registration: A systematic review of new drug application and biologics license application approvals from 2015–2020. Clinical Cancer Research, 28(1), 27–35. 10.1158/1078-0432.CCR-21-2639
Austin, P. C. (2014). The use of propensity score methods with survival or time-to-event outcomes: Reporting measures of effect similar to those used in randomized experiments. Statistics in Medicine, 33(7), 1242–1258. 10.1002/sim.5984
Austin, P. C., & Stuart, E. A. (2015). Moving towards best practice when using inverse probability of treatment weighting (IPTW) using the propensity score to estimate causal treatment effects in observational studies. Statistics in Medicine, 34(28), 3661–3679. 10.1002/sim.6607
Austin, P. C., White, I. R., Lee, D. S., & van Buuren, S. (2021). Missing data in clinical research: A tutorial on multiple imputation. Canadian Journal of Cardiology, 37(9), 1322–1331. 10.1016/j.cjca.2020.11.010
Berger, M. L., Sox, H., Willke, R. J., Brixner, D. L., Eichler, H. G., Goettsch, W.,…Daniel Mullins, C. (2017). Good practices for real-world data studies of treatment and/or comparative effectiveness: Recommendations from the joint ISPOR-ISPE special task force on real-world evidence in health care decision making. Pharmacoepidemiology and Drug Safety, 26(9), 1033–1039. 10.1002/pds.4297
Blonde, L., Khunti, K., Harris, S. B., Meizinger, C., & Skolnik, N. S. (2018). Interpretation and impact of real-world clinical data for the practicing clinician. Advances in Therapy, 35(11), 1763–1774. 10.1007/s12325-018-0805-y
Broglio, K. R., & Berry, D. A. (2009). Detecting an overall survival benefit that is derived from progression-free survival. Journal of the National Cancer Institute, 101(23), 1642–1649. 10.1093/jnci/djp369
Cummings, J. (2018). Lessons learned from Alzheimer disease: Clinical trials with negative outcomes. Clinical and Translational Science, 11(2), 147–152. 10.1111/cts.12491
Dang, A. (2023). Real-world evidence: A primer. Pharmaceutical Medicine, 37(1), 25–36. 10.1007/s40290-022-00456-6
Deniz, S., Akbolat, M., Cimen, M., & Unal, O. (2021). The mediating role of shared decision-making in the effect of the patient-physician relationship on compliance with treatment. Journal of Patient Experience, 8, 23743735211018066. 10.1177/23743735211018066
Dreyer, N. A., Schneeweiss, S., McNeil, B. J., Berger, M. L., Walker, A. M., Ollendorf, D. A., & Gliklich, R. E. (2010). Grace principles: Recognizing high-quality observational studies of comparative effectiveness. The American Journal of Managed Care, 16(6). https://www.ajmc.com/view/ajmc_10jundreyer_467to471
FDA. (2017). Palbociclib (Ibrance). https://www.fda.gov/drugs/resources-information-approved-drugs/palbociclib-ibrance
FDA. (2018). Framework for FDA's real-world evidence program [FDA issued guidance]. https://www.fda.gov/media/120060/download
FDA. (2019). FDA approves innovative gene therapy to treat pediatric patients with spinal muscular atrophy, a rare disease and leading genetic cause of infant mortality. https://www.fda.gov/news-events/press-announcements/fda-approves-innovative-gene-therapy-treat-pediatric-patients-spinal-muscular-atrophy-rare-disease
Finn, R. S., Martin, M., Rugo, H. S., Jones, S., Im, S. A., Gelmon, K.,…Slamon, D. J. (2016). Palbociclib and letrozole in advanced breast cancer. New England Journal of Medicine, 375(20), 1925–1936. 10.1056/NEJMoa1607303
Finn, R. S., Rugo, H. S., Dieras, V. C., Harbeck, N., Im, S.-A., Gelmon, K. A.,…Slamon, D. J. (2022). Overall survival (OS) with first-line palbociclib plus letrozole (pal+let) versus placebo plus letrozole (pbo+let) in women with estrogen receptor–positive/human epidermal growth factor receptor 2–negative advanced breast cancer (ER+/HER2− ABC): Analyses from PALOMA-2. Journal of Clinical Oncology, 40(17_suppl), LBA1003. 10.1200/JCO.2022.40.17_suppl.LBA1003
Ford, I., & Norrie, J. (2016). Pragmatic trials. New England Journal of Medicine, 375(5), 454–463. 10.1056/NEJMra1510059
Forstag, E. H. (2019). In Shore C., Gee A. W., Kahn B., & Forstag E. H. (Eds.), Examining the impact of real-world evidence on medical product development: Proceedings of a workshop series. 10.17226/25352
Garrison, L. P., Jr., Neumann, P. J., Erickson, P., Marshall, D., & Mullins, C. D. (2007). Using real-world data for coverage and payment decisions: The ISPOR real-world data task force report. Value in Health, 10(5), 326–335. 10.1111/j.1524-4733.2007.00186.x
Gatto, N. M., Campbell, U. B., Rubinstein, E., Jaksa, A., Mattox, P., Mo, J., & Reynolds, R. F. (2022). The structured process to identify fit-for-purpose data: A data feasibility assessment framework. Clinical Pharmacology & Therapeutics, 111(1), 122–134. 10.1002/cpt.2466
Gokhale, M., Sturmer, T., & Buse, J. B. (2020). Real-world evidence: The devil is in the detail. Diabetologia, 63(9), 1694–1705. 10.1007/s00125-020-05217-1
Haneuse, S., Arterburn, D., & Daniels, M. J. (2021). Assessing missing data assumptions in EHR-based studies: A complex and underappreciated task. JAMA Network Open, 4(2), e210184. 10.1001/jamanetworkopen.2021.0184
Hariton, E., & Locascio, J. J. (2018). Randomised controlled trials - The gold standard for effectiveness research: Study design: Randomised controlled trials. BJOG: An International Journal of Obstetrics & Gynaecology, 125(13), 1716. 10.1111/1471-0528.15199
Jahanshahi, M., Gregg, K., Davis, G., Ndu, A., Miller, V., Vockley, J., Ollivier, C., Franolic, T., & Sakai, S. (2021). The use of external controls in FDA regulatory decision making. Therapeutic Innovation & Regulatory Science, 55(5), 1019–1035. 10.1007/s43441-021-00302-y
Khosla, S., White, R., Medina, J., Ouwens, M., Emmas, C., Koder, T., Male, G., & Leonard, S. (2018). Real world evidence (RWE) - A disruptive innovation or the quiet evolution of medical evidence generation? F1000Research, 7, 111. 10.12688/f1000research.13585.2
Khozin, S., Blumenthal, G. M., & Pazdur, R. (2017). Real-world data for clinical evidence generation in oncology. Journal of the National Cancer Institute, 109(11). 10.1093/jnci/djx187
Kim, H. S., Lee, S., & Kim, J. H. (2018). Real-world evidence versus randomized controlled trial: Clinical research based on electronic medical records. Journal of Korean Medical Sciences, 33(34), e213. 10.3346/jkms.2018.33.e213
Kurtin, S. E., & Taher, R. (2020). Clinical trial design and drug approval in oncology: A primer for the advanced practitioner in oncology. Journal of the Advanced Practitioner in Oncology, 11(7), 736–751. 10.6004/jadpro.2020.11.7.7
Lavertu, A., Vora, B., Giacomini, K. M., Altman, R., & Rensi, S. (2021). A new era in pharmacovigilance: Toward real-world data and digital monitoring. Clinical Pharmacology & Therapeutics, 109(5), 1197–1202. 10.1002/cpt.2172
Mahendraratnam, N., Mercon, K., Eckert, J., Romine, M., Kroetsch, A., Frank, K.,…McClellan, M. (2019). Adding real-world evidence to a totality of evidence approach for evaluating marketed product effectiveness [White Paper]. Duke Margolis Center for Health Policy. https://healthpolicy.duke.edu/sites/default/files/2020-08/Totality%20of%20Evidence%20Approach.pdf
Monti, S., Grosso, V., Todoerti, M., & Caporali, R. (2018). Randomized controlled trials and real-world data: Differences and similarities to untangle literature data. Rheumatology (Oxford), 57(57 Suppl 7), vii54–vii58. 10.1093/rheumatology/key109
O'Leary, C. P., & Cavender, M. A. (2020). Emerging opportunities to harness real-world data: An introduction to data sources, concepts, and applications. Diabetes, Obesity & Metabolism, 22(Suppl 3), 3–12. 10.1111/dom.13948
Rare Diseases International. (2023). Infographic-operational-description-of-rare-diseases. https://www.rarediseasesinternational.org/wp-content/uploads/2022/08/INFOGRAPHIC-OPERATIONAL-DESCRIPTION-OF-RARE-DISEASES.pdf
Rifkin, R., Singer, D., Aguilar, K. M., Baidoo, B., & Maiese, E. M. (2019). Daratumumab split first versus single dosing schedule among patients with multiple myeloma treated in a US community oncology setting: A retrospective observational study. Clinical Therapeutics, 41(5), 866–881 e867. 10.1016/j.clinthera.2019.03.013
Rohrig, B., du Prel, J. B., Wachtlin, D., Kwiecien, R., & Blettner, M. (2010). Sample size calculation in clinical trials: Part 13 of a series on evaluation of scientific publications. Deutsches Ärzteblatt International, 107(31–32), 552–556. 10.3238/arztebl.2010.0552
Rothman, K. J., Greenland, S., & Lash, T. (2008). Modern Epidemiology (Third Edition ed.). Lippincott Williams & Wilkins. http://students.aiu.edu/submissions/profiles/resources/onlineBook/a9c7D5_Modern_Epidemiology_3.pdf
Rugo, H. S., Brufsky, A., Liu, X., Li, B., McRoy, L., Chen, C.,…DeMichele, A. (2022). Real-world study of overall survival with palbociclib plus aromatase inhibitor in HR+/HER2− metastatic breast cancer. NPJ Breast Cancer, 8(1), 114. 10.1038/s41523-022-00479-x
Rugo, H. S., Finn, R. S., Dieras, V., Ettl, J., Lipatov, O., Joy, A. A.,…Slamon, D. J. (2019). Palbociclib plus letrozole as first-line therapy in estrogen receptor-positive/human epidermal growth factor receptor 2-negative advanced breast cancer with extended follow-up. Breast Cancer Research and Treatment, 174(3), 719–729. 10.1007/s10549-018-05125-4
Schad, F., & Thronicke, A. (2022). Real-world evidence-current developments and perspectives. International Journal of Environmental Research and Public Health, 19(16). 10.3390/ijerph191610159
Setia, M. S. (2016). Methodology series module 3: Cross-sectional studies. Indian Journal of Dermatology, 61(3), 261–264. 10.4103/0019-5154.182410
Singal, A. G., Higgins, P. D., & Waljee, A. K. (2014). A primer on effectiveness and efficacy trials. Clinical and Translational Gastroenterology, 5(1), e45. 10.1038/ctg.2013.13
Singh, G. K., & Jemal, A. (2017). Socioeconomic and racial/ethnic disparities in cancer mortality, incidence, and survival in the United States, 1950–2014: Over six decades of changing patterns and widening inequalities. Journal of Environmental and Public Health, 2017, 2819372. 10.1155/2017/2819372
Singh, H., Kanapuru, B., Smith, C., Fashoyin-Aje, L. A., Myers, A., Kim, G., & Pazdur, R. (2017). FDA analysis of enrollment of older adults in clinical trials for cancer drug registration: A 10-year experience by the U.S. Food and Drug Administration. Journal of Clinical Oncology, 35(15_suppl), 10009–10009. 10.1200/JCO.2017.35.15_suppl.10009
Subbiah, V. (2023). The next generation of evidence-based medicine. Nature Medicine, 29(1), 49–58. 10.1038/s41591-022-02160-z
Talarico, L., Chen, G., & Pazdur, R. (2004). Enrollment of elderly patients in clinical trials for cancer drug registration: A 7-year experience by the US Food and Drug Administration. Journal of Clinical Oncology, 22(22), 4626–4631. 10.1200/JCO.2004.02.175
Tang, M., Pearson, S. A., Simes, R. J., & Chua, B. H. (2023). Harnessing real-world evidence to advance cancer research. Current Oncology, 30(2), 1844–1859. 10.3390/curroncol30020143
Taur, S. R. (2022). Observational designs for real-world evidence studies. Perspectives in Clinical Research, 13(1), 12–16. 10.4103/picr.picr_217_21
Thorlund, K., Dron, L., Park, J. J. H., & Mills, E. J. (2020). Synthetic and external controls in clinical trials - a primer for researchers. Clinical Epidemiology, 12, 457–467. 10.2147/CLEP.S242097
Turner, B. E., Steinberg, J. R., Weeks, B. T., Rodriguez, F., & Cullen, M. R. (2022). Race/ethnicity reporting and representation in US clinical trials: A cohort study. The Lancet Regional Health – Americas, 11. 10.1016/j.lana.2022.100252
US Congress. (2016). 21st Century Cures Act. https://www.federalregister.gov/documents/2020/05/01/2020-07419/21st-century-cures-act-interoperability-information-blocking-and-the-onc-health-it-certification
Wang, S. V., Pinheiro, S., Hua, W., Arlett, P., Uyama, Y., Berlin, J. A.,…Schneeweiss, S. (2021). Start-RWE: Structured template for planning and reporting on the implementation of real-world evidence studies. BMJ, 372, m4856. 10.1136/bmj.m4856
Wang, S. V., & Schneeweiss, S. (2022). Assessing and interpreting real-world evidence studies: Introductory points for new reviewers. Clinical Pharmacology & Therapeutics, 111(1), 145–149. 10.1002/cpt.2398
Wedam, S., Fashoyin-Aje, L., Bloomquist, E., Tang, S., Sridhara, R., Goldberg, K. B,…Beaver, J. A. (2020). FDA approval summary: Palbociclib for male patients with metastatic breast cancer. Clinical Cancer Research, 26(6), 1208–1212. 10.1158/1078-0432.CCR-19-2580
Wouters, O. J., McKee, M., & Luyten, J. (2020). Estimated research and development investment needed to bring a new medicine to market, 2009-2018. JAMA, 323(9), 844–853. 10.1001/jama.2020.1166
Xia, A. D., Schaefer, C. P., Szende, A., Jahn, E., & Hirst, M. J. (2019). RWE framework: An interactive visual tool to support a real-world evidence study design. Drugs - Real World Outcomes, 6(4), 193–203. 10.1007/s40801-019-00167-6