Study Highlights
WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?
To our knowledge, the use of real‐world evidence (RWE) has been mainly reviewed using approved medicines and publicly available documents of several regulatory agencies worldwide.
WHAT QUESTION DID THIS STUDY ADDRESS?
What was the contribution of RWE to the assessment of efficacy/effectiveness by the Committee for Medicinal Products for Human Use and to the decision‐making process in recent marketing authorization applications (MAAs)?
WHAT DOES THIS STUDY ADD TO OUR KNOWLEDGE?
This study provides an in‐depth review of RWE used in recent centralized MAAs in Europe, including authorized, withdrawn, and refused products, and provides illustrative examples of its contribution to regulatory decision making.
HOW MIGHT THIS CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE?
RWE plays an increasing role in regulatory decisions for products for which randomized controlled trials are deemed unethical or unfeasible, such as medicines for very rare diseases or for “precision medicines.” The results may serve as a basis for future guidance for applicants and regulators in the use of RWE in medicines development.
Real‐world evidence (RWE) has been defined as the information derived from analysis of routinely collected data (referred to as real‐world data, RWD) relating to a patient's health status or the delivery of health care from a variety of sources other than traditional clinical trials. 1 While randomized controlled clinical trials (RCTs) are considered the gold standard to generate the initial safety and efficacy evidence of medicinal products, RWD may complement this knowledge during medicines' life cycles. 2 , 3 In particular in the setting of medicines for very rare diseases, or in “precision medicine” approaches where therapies target small patient populations and RCTs may not be feasible or ethical, RWD may provide a solution. 4 , 5 RWD can contextualize results of clinical trials by providing evidence on the natural disease course or standard of care, by serving as an external comparator arm for medicines evaluated in single‐arm trials, or by assessing long‐term effects or adverse events post authorization. 6 Furthermore, it is increasingly used to guide clinical trial designs, e.g., in selecting trial populations or calculating sample sizes. 7 Recent studies have shown the integration of RWE in marketing authorization applications (MAAs) over the past years. 8 , 9 , 10 Flynn et al . 9 found that 39.9% (63 of 158) of initial MAAs and 18.3% (28 of 153) of the extension of indication applications (EoIs) submitted in 2018 and 2019 to the European Medicines Agency (EMA) included real‐world data/evidence (RWD/RWE) to support the applicants' demonstration of their medicines' efficacy or safety profiles. The others found RWD/RWE for any purpose included in MAAs of authorized products ranging from 85.3% to even 100%. 8 , 10 However, this RWE does not always contribute to regulatory decision making. Of products authorized by the US Food and Drug Administration (FDA) from 2019 to 2021, RWE was considered critical in the assessment of the applicants' dossiers for 9% of the products and supportive in 65%, whereas it was not addressed at all in the assessment in 11% or was considered inadequate for decision making in 15% of the products. Reasons for inadequacy included lack of a prespecified study design and analysis plan and other issues that led to concerns regarding the reliability and relevancy of the RWE, which were similar to those identified in the 34 dossiers of products approved by the FDA from 1954 to 2020, for which RWE was included to support effectiveness specifically. 10 , 11 It was concluded that for RWE to reach its full potential, its role in regulatory effectiveness decision making should be further explored by publishing lessons learned from cases in which it is considered suitable or not. 11
As a follow‐up (“part II”) to the Flynn et al . 9 study, hereafter referred to as “part I,” we aimed to (i) characterize RWD/RWE submitted to EMA to support efficacy/effectiveness claims on medicinal products as part of MAAs and EoIs, and (ii) to analyze its contribution to the assessment of the dossiers by the responsible regulatory committee, i.e., the Committee for Medicinal Products for Human Use (CHMP). The results will inform opportunities and gaps in unlocking the full potential of RWE to contribute to the development, evaluation and monitoring of better medicines for patients. These findings will enlighten future guidance on the use of RWE in regulatory contexts.
METHODS
We performed a narrative synthesis of CHMP's appraisal of RWD/RWE to support efficacy/effectiveness claims in MAAs and EoIs.
Identification of relevant products
We identified those applications that included RWD/RWE to support efficacy/effectiveness claims, and/or to provide information on disease epidemiology to contextualize efficacy/effectiveness data from the original Flynn et al . data sets. This was a subset of the 63 MAAs (total n = 158) and 28 EoIs (total n = 153) that included RWD/RWE for any purpose submitted from January 1, 2018 to December 31, 2019. Description on data collection, extraction, and quality control of these data sets has been previously published. 9
Data collection
The two samples of products (MAAs and EoIs) resulting from this filtering exercise were extracted together with all the corresponding information already collected in part I, including all descriptive information to characterize RWD/RWE submitted as part of the dossiers. Previously collected administrative information of interest included product name, international nonproprietary name, application number, type of authorization (standard/conditional/exceptional), the therapeutic indication (anatomical therapeutic chemical classification, second level) and the legal basis. 12 The status of the MAA or EoI (authorized/refused/withdrawn) and orphan status (yes/no) were verified for each product and updated as applicable as of July 5, 2021. We consulted several source documents accessible at EMA for each product in order to confirm, supplement, or update the RWD/RWE characteristics as captured during part I in line with the part II scope, and to capture CHMP's view on applicants' proposals. Additionally, we extracted from these documents text fragments on the different variables and aspects of the appraisals that would help qualify the role of RWD/RWE in the regulatory assessment and decision making on the applications. Table 1 outlines the variables to describe and categorize the submitted RWD/RWE, while Table 2 presents the variables used to qualify CHMP appraisal on aspects of the RWD/RWE. Both tables provide source documents consulted for the extraction of the variables.
Table 1
Variables to characterize the submitted real‐world data/evidence, corresponding categories, and source files
CHMP, Committee of Medicinal Products for Human Use; RWD, real‐world data; RWE, real‐world evidence; SmPC, summary of product characteristics. a The categories are not necessarily mutually exclusive.
| RWD/RWE characteristics | |||
| Extracted data | Description | Categories a | Source files |
| Generated before authorization or post authorization | RWD/RWE generated before authorization or proposed to be generated post authorization as outlined in each study | Preauthorization Postauthorization | Most recent available CHMP assessment report (internal) European Public Assessment Reports (publicly available) Risk management plan (mainly for postauthorization studies; internal) |
| Main vs. supportive study | The “clinical efficacy” section in the CHMP assessment report template contains a section with “main study/studies” and “supportive study/studies.” This column describes in which of these the RWD/RWE was presented | Main study Supportive study | |
| Study objective(s) | Evidence/answer(s) the study aims to generate based on specific research question(s) | Efficacy/effectiveness Disease epidemiology Drug utilization Safety | |
| Purpose | Type of evidence and context brought by the RWD to reach the study objective(s) | Efficacy/Effectiveness—data collection Contextualization—natural history Contextualization—standard of care Disease epidemiology—data collection Drug utilization—data collection External comparator Safety—data collection | |
| Data source | Type of database from which RWD/RWE are derived | Literature Medical records Registries Research databases | |
| Study design | Type of study design in which RWD was used | Prospective cohort study Retrospective cohort study Cross‐sectional survey study |
Table 2
Variables to characterize CHMP's and SAWP's appraisal on real‐world data/evidence, corresponding categories, and source files
CHMP, Committee of Medicinal Products for Human Use; N/A, not applicable; RWD, real‐world data; RWE, real‐world evidence; SAWP, Scientific Advice Working Party; SmPC, summary of product characteristics.
| Extracted data | Description | Categories | Source files |
| CHMP appraisal of studies including RWD as applicable | |||
| CHMP position on added value of RWD/RWE | Overall CHMP view on the appropriateness of RWD to provide meaningful evidence to support the evaluation of the product's efficacy and benefit–risk balance in regulatory decision making | Supporting decision making Not supporting decision making Not addressed | Most recent available CHMP assessment report (internal) European Public Assessment Reports (publicly available) |
| CHMP comments on several aspects of the RWD/RWE | E.g., on relevance of the sample size (e.g., too small), inclusion/exclusion criteria applied or proposed, similarity to target population, definition of the exposure in terms of dose and measurement, end points definition or end points used, follow‐up time, comparability of study period, type of database (general practitioner, hospital, registry, etc.) and database itself, quality of the data (e.g., missing data), statistical considerations, bias and confounding, and the impact of the limitations on the acceptability of the evidence | Not appropriate/ inconclusive Appropriate | |
| RWD/RWE reference in SmPC | Was the RWD/RWE cited in the summary of product characteristics of the product? | Yes/no | European public assessment report Annex I Summary of product characteristics (publicly available) |
| SAWP advice on studies including RWD as applicable | |||
| Applicants' questions on the use of RWD/RWE | Qualitative description of the questions asked by the applicant on the use of RWD/RWE to support the relevant authorization application | N/A | Final advice letters (internal) Description of scientific advice procedures in assessment reports |
| SAWP/CHMP's advice on the use of RWD | Qualitative description of recommendations given by the SAWP/CHMP on the use of RWD/RWE to support the relevant authorization application | N/A |
Data analysis
We focused on RWD/RWE that was generated before authorization, and separately described the results for dossiers where RWD/RWE was presented as part of the main and/or supportive studies and identified opportunities and gaps in RWD/RWE and its assessment. Products' and RWD/RWE characteristics ( Table 1 ) were analyzed descriptively. An in‐depth review of the CHMP assessment reports was performed to further characterize the suitability and limitations of the submitted RWD/RWE as assessed by the committee, e.g., on sample size, study design, and handling bias ( Table 2 ). Based on the information extracted from the assessment report sections related to the CHMP discussion on clinical efficacy and the benefit–risk assessment, we categorized the RWD/RWE as “supporting decision making” when it was considered contributing to the efficacy evidence as part of the regulatory decision (e.g., “In general the historical controls are considered adequate for comparison with the study population” 13 ), “not supporting decision making” when it was considered inappropriate to contribute to the efficacy evidence as part of the regulatory decision (e.g., “The difference in OS [overall survival] observed in the comparison vs historical data cannot be accepted as evidence of efficacy but only as supporting information, due to the inability to control bias for comparisons to external controls” 14 ), or “not addressed” when nothing was mentioned on the submitted RWD/RWE in the evaluation.
We briefly described the results on proposals to generate RWD/RWE post authorization, and if and what scientific advice had been given by the Scientific Advice Working Party (SAWP) on the RWD/RWE proposals prior to the applications, and whether the RWD/RWE was included in the Summary of Product characteristics (SmPC) of the European Public Assessment Report (EPAR).
The data extraction, categorization, and analyses were performed by one researcher (E.B.) and cross‐checked by two other researchers (K.P. and C.J.J.). In case of discrepancies, these were discussed until consensus was reached. If necessary, the other coauthors (X.K., V.S., and P.G.M.M.) were consulted.
RESULTS
Of the 63 MAA and 28 EoI applications containing references to RWD/RWE identified in part I (ref. 9 ), 32 MAAs and 18 EoIs were identified as including RWD/RWE related to efficacy/effectiveness and disease epidemiology outcomes. Four additional EoIs were excluded: three because the mentioned postauthorization study had a disease epidemiology objective in the context of safety but not efficacy ( brigatinib , ixekizumab , and dulaglutide ), the other because this study measured effectiveness of risk minimization measures rather than effectiveness of the medicine ( dapagliflozin ). We therefore reviewed further 14 EoIs.
Initial MAAs
Description of the products
As of July 5, 2021, 23 of the 32 products had been authorized, 2 refused, and 7 withdrawn ( Table 3 ). Of the 23 authorized products, 15 received standard marketing authorization (MA), 5 conditional MA, and 3 MA under exceptional circumstances. Sixteen had received an orphan designation. The five most represented types of products were antineoplastic agents ( n = 7), vaccines ( n = 5), immunosuppressants ( n = 4), antivirals for systemic use ( n = 3), and other alimentary tract and metabolism products ( n = 3). Initial MAAs concerned 21 new active substances and 11 known active substances (also including hybrid and biosimilar applications).
Table 3
Administrative information on the products with real‐world data/evidence submitted in initial MAAs and EoIs
EoIs, extension of indication applications; MAAs, marketing authorization applications; N/A, not applicable.
| Marketing authorization applications ( n = 32) | Extension of indication applications ( n = 14) | |
| Authorization status | ||
| Authorized | 23 | 12 |
| Standard | 15 | N/A |
| Conditional | 5 | N/A |
| Exceptional circumstances | 3 | N/A |
| Refused | 2 | 2 |
| Withdrawn | 7 | 0 |
| Orphan status | ||
| Yes | 16 | 2 |
| No | 16 | 12 |
| Therapeutic areas | ||
| Antihemorrhagics | 1 | 4 |
| Antineoplastic agents | 7 | 2 |
| Antithrombotic agents | 1 | 0 |
| Antivirals for systemic use | 3 | 0 |
| Corticosteroids for systemic use | 1 | 0 |
| Diagnostic radiopharmaceuticals | 0 | 1 |
| Immunosuppressants | 4 | 3 |
| Other alimentary tract and metabolism products | 3 | 0 |
| Other drugs for disorders of the musculo‐skeletal system | 1 | 1 |
| Other hematological agents | 1 | 1 |
| Other nervous system drugs | 2 | 0 |
| Other respiratory system products | 1 | 1 |
| Psychoanaleptics | 1 | 0 |
| Psycholeptics | 1 | 1 |
| Vaccines | 5 | 0 |
| Legal basis | ||
| New active substance (Article 8(3), full or full‐mixed application (complete dossier)) | 21 | 9 |
| Known active substance (Article 8(3) full or full‐mixed application (complete dossier)) | 5 | 2 |
| Hybrid medicinal product application (Article 10(3)) | 4 | 1 |
| Similar biologic product application (Article 10(4)) | 2 | 0 |
| Complete application (stand‐alone) | 0 | 2 |
RWD/RWE characteristics
RWD/RWE generated preauthorization was submitted within 16 (16/32) MAAs, 20 (20/32) dossiers included a proposal to generate RWD/RWE post authorization, of which 4 (4/32) dossiers included both preauthorization and postauthorization RWE ( Figure 1 a ). Preauthorization‐generated RWD/RWE was submitted as part of the “main study/studies” as presented in the CHMP assessment report in eight (8/16) MAAs ( onasemnogene abeparvovec , ivacaftor/tezacaftor/elexacaftor , sodium oxybate , ivosidenib , idebenone, trientine dihydrochloride, melphalan, and enasidenib ), of which in three (3/16) MAAs ( trientine dihydrochloride , melphalan , and enasidenib ) RWD/RWE was included both in the “main study/studies” and “supportive study/studies.” Of these eight products, four were authorized, one was refused, and three were withdrawn. The RWD/RWE was considered supporting decision making in three MAAs ( Table 4 and Figure 1 a ). Of the eight MAAs (8/16) where RWD was presented as supportive evidence only ( aripiprazole , product X , hydroxycarbamide , product Y , emapalumab , edavarone , ravulizumab , and oral live cholera vaccine ), three were authorized, one was refused, and four were withdrawn.
Table 4
Initial MAAs including preauthorization RWE
AR, assessment report; CHMP, Committee for Medicinal Products for Human Use; RWD, real‐world data; RWE, real‐world evidence; SmPC, summary of product characteristics.
| RWE presented in CHMP ARs | International nonproprietary name | Indication | Product authorization status | Purpose of RWD/RWE | Data source(s) | CHMP appraisal on RWD/RWE | Details on strengths/limitations of RWD/RWE submitted by applicants |
| Main study/studies ( n = 5) | Onasemnogene abeparvovec | Treatment of patients with 5q spinal muscular atrophy (SMA) with a bi‐allelic mutation in the SMN1 gene and a clinical diagnosis of SMA Type 1, or patients with 5q SMA with a bi‐allelic mutation in the SMN1 gene and up to 3 copies of the SMN2 gene | Authorized | Efficacy/effectiveness data collection Safety data collection External comparator Contextualization (natural history) | Research databases (1,2) Registry (postauthorization) | Supporting decision making | Strengths The potential bias created by a difference in populations is not in favor of the drug (1) Historical controls considered adequate for comparison with the study population since this group is relatively homogeneous (1,2) Timepoints for efficacy analysis of supportive single‐arm trial matched with major efficacy end points in natural history study (1,2) Limitations For specific subgroup: follow‐up time and the heterogeneity of the natural history cohort make it not possible to draw conclusions on benefit at the time of initial approval (1,2) |
| Ivacaftor/tezacaftor/elexacaftor | Treatment of cystic fibrosis (CF) in patients aged 12 years and older who are homozygous for the F508del mutation in the cystic fibrosis transmembrane conductance regulator ( CFTR ) gene or heterozygous for the F508del in the CFTR gene with a minimal function (MF) mutation | Authorized | Efficacy/effectiveness data collection | Registry | Not supporting decision making | Limitations Data limited and subject to bias: individual data not present, genotypes unknown, modulator therapy unclear Effect size in real world not directly comparable to controlled setting Measurements at different timepoints/ different exposure times | |
| Sodium Oxybate | Substitution treatment for alcohol dependence within a framework of careful medical supervision along with continuous psychosocial support and social rehabilitation for patients resistant to existing interventions or in patients for whom existing therapies are contraindicated or not recommended | Refused | Efficacy/effectiveness data collection Drug utilization data collection | Survey | Not supporting decision making | Limitations No definite dosage recommendation is clearly deducible: exact applied dosage unclear Inclusion/exclusion criteria, baseline characteristics and outcomes missing, Lack of control group Confirmation of effectiveness only based on within‐subject comparisons Treatment duration not in line with recommendations in SmPC Cutoff date not appropriate No discussion on expected natural course of disease provided | |
| Ivosidenib | Treatment of relapsed or refractory acute myeloid leukemia (AML) with an isocitrate dehydrogenase‐1 (IDH1) R132 mutation in adult patients have received at least 2 prior regimens, including at least one standard intensive chemotherapy regimen, or are not candidates for standard intensive chemotherapy and have received at least one prior nonintensive regimen | Withdrawn | External comparator | Registry (1) Medical records (2) | Not supporting decision making | Limitations In general it is almost impossible to establish comparability of a treatment group and an external control group, especially where the disease is very heterogeneous Matching methods questioned (1) Bias suspected, but impact of bias not assessable (inability to control bias) (1,2) Outcome of “response to therapy” was only available without distinction in type of response (1) | |
| Idebenone | Treatment of respiratory dysfunction in patients with Duchenne muscular dystrophy not using glucocorticoids | Withdrawn | Efficacy/effectiveness data collection | Medical records | Not supporting decision making | Limitations Limited sample size Lack of representativeness of the study population End point changed without formal amendment End points not routinely measured Different start dates of individual patients Unexplained missing data | |
| Both main & supportive studies ( n = 3) | Trientine dihydrochloride | Treatment of Wilson's disease in patients intolerant to D‐penicillamine therapy, in adults and children aged 5 to ≤17 years | Authorized | Efficacy/effectiveness data collection Safety data collection | Medical records (from tertiary care centers) (1) Literature (2) | Supporting decision making | Strengths Inclusion/exclusion criteria deemed largely suited (1) Patient outcomes reported over long time periods (years/decades) (2) Limitations Required sample size not calculated (1) Knowledge gaps in dose‐exposure‐response (1,2) No primary end point defined (1) No standardized end point used (1,2) Unrestricted concomitant medications (1,2) Measurement timepoints: continuous analysis preferred over two or three timepoints (1) Protection against bias (selection bias) limited (1) |
| Melphalan | High‐dose used alone or in combination with other cytotoxic medicinal products and/or total body irradiation for treatment of multiple myeloma, malignant lymphoma (Hodgkin's, non‐Hodgkin's lymphoma), acute lymphoblastic and myeloblastic leukemia, childhood neuroblastoma, ovarian and mammary adenocarcinoma | Authorized | Efficacy/effectiveness data collection | Literature | Supporting decision making | Strengths Adult population representative for the target population Relevant end points Extrapolation of results obtained in United States / Asia to European population acceptable, because treatment protocols for hematological diseases are sufficiently comparable Limitations Several methodological shortcomings related to the retrospective nature, being lack of randomization, blinding, standardization of treatment regimens and predefined timing of outcome assessments Small sample size of subgroups Pediatric population not completely appropriate No power calculations performed Inclusion period too long: treatment protocols might have changed over time Not clear if prespecified statistical analysis plan was available, and if assessments were comparable between groups Publications do not provide all the details necessary for a rigorous assessment | |
| Enasidenib | Treatment of adult patients with relapsed or refractory acute myeloid leukemia (AML) with an isocitrate dehydrogenase 2 (IDH2) mutation | Withdrawn | Efficacy/effectiveness data collection External comparator | Literature (1) Registry (2) Medical records (3) | Not supporting decision making | Limitations Limited data available (1) High heterogeneity in populations influenced outcomes (1) Deviation of external comparator cohort from study population (1,2,3) Small sample size (2) Unavailability of individual data (aggregated data only) permits only comparison with indirect adjustment, and does not allow adequate quantification of benefit (2) Several types of possible bias, in particular concern for selection bias (2) | |
| Supportive study/ studies ( n = 8) | Ravulizumab | Treatment of paroxysmal nocturnal hemoglobinuria (PNH) in patients with hemolysis with clinical symptom(s) indicative of high disease activity or in patients who are clinically stable after having been treated with eculizumab for at least the past 6 months | Authorized | Efficacy/effectiveness Contextualization (standard of care) | Registry | Not addressed | Not applicable |
| Cholera vaccine, oral, live | Active immunization against disease caused by Vibrio cholerae serogroup O1 in adults and children aged 6 years and older | Authorized | Efficacy/effectiveness data collection | Literature | Not addressed | Not applicable | |
| Hydroxycarbamide | Prevention of vaso‐occlusive complications of sickle cell disease in patients over 2 years of age | Authorized | Efficacy/effectiveness data collection Safety data collection | Literature | Supporting decision making | CHMP appraisal based on all supportive studies together, not specifically on RWE (supportive studies could be used to extrapolate the indication to a subpopulation) | |
| Emapalumab | Treatment of pediatric patients with primary hemophagocytic lymphohistiocytosis (HLH) with refractory, recurrent or intolerance to conventional HLH therapy (combination of dexamethasone and etoposide) | Refused | Efficacy/effectiveness data collection Safety data collection External comparator | Literature (1) Compassionate Use Program (2) | Not supporting decision making | Limitations Historical controls data considered of little relevance for the understanding of the pivotal study also because of the differences in line of treatment, time frame and level of supportive care and lack of data on prognostic variables, which the applicant has not adjusted for in the analyses (1) Selection methods prone for selection bias (1) Statistical analysis plan not prespecified (1) Nonrandomized comparison between historical control and pivotal study indicated extremely wide confidence intervals (remaining uncertainty) (1) Data on compassionate use cases insufficiently robust to draw conclusions (2) Dosing regimen/concomitant treatments unclear “It should be noted that comparisons to any historical controls carry inherent biases which hinder adequate comparison” | |
| Aripiprazole | Treatment of adults with schizophrenia, moderate to severe manic episodes in bipolar I disorder and for the prevention of a new manic episode in adults who experienced predominantly manic episodes and whose manic episodes responded to aripiprazole treatment | Withdrawn | Efficacy/effectiveness data collection Safety data collection | Drug utilization studies | Not supporting decision making | Further details requested on real‐world studies, but these were not further addressed due to withdrawal | |
| Edaravone | Treatment of amyotrophic lateral sclerosis (ALS) | Withdrawn | Efficacy/effectiveness data collection Safety data collection | Registries | Not addressed | Only postauthorization RWE addressed | |
| Product X | Rare bleeding disorder | Withdrawn | Efficacy/effectiveness data collection Contextualization (natural history) | Medical records Registry (postauthorization) | Not addressed | Only postauthorization RWE addressed | |
| Product Y | Rare thromboembolic disorder | Withdrawn | External comparator | Literature | Supporting decision making | CHMP appraisal based on the overall study including the external comparator, not specifically on RWE |
CHMP appraisal on RWE
The overall CHMP appraisal as well as the mentioned strengths and limitations of the RWD/RWE submitted within the MAAs are summarized in Table 4 .
Description of cases where RWE was presented as main evidence
Several strengths were mentioned in the MAA of onasemnogene abeparvovec for the RWD that was considered supporting decision making, derived from research databases and used as historical comparators. The use of historical comparators was already discussed at an early stage in a scientific advice procedure. Considering the severity of the disease, the well‐characterized natural history, the high unmet medical need, and the promising results of the phase I trial, a phase III study without a concurrent control was considered acceptable, although some form of control was deemed necessary. The use of historical comparators from previously performed natural history studies was agreed upon, although careful matching of the cohorts and minimizing bias was recommended. 13 At the time of MAA, the CHMP considered the use of historical comparators adequate owing to the similar timing of outcome / end point assessments in the phase III study compared with the natural history cohort, as well as to the homogeneity of the cohorts. Moreover, any remaining potential bias from the difference in populations was considered not to favor the medicinal product as the historical control had less severe disease. A postauthorization registry study was already in an advanced planning stage at the time of MA. 13
RWD derived from a registry and medical records was proposed as an external comparator in the withdrawn procedure for ivosidenib . The evidence was not considered supporting decision making due to several limitations mentioned by the CHMP, including inappropriate matching methods to establish comparability of the treatment group and the external control group, partly caused by the heterogeneity of the underlying disease, the inability to exclude confounding, and a concern for selection bias. In a prior scientific advice procedure, the SAWP already pointed out that a historical control for the single‐arm pivotal study would be important, but that the data should be carefully matched at patient level to adjust for heterogeneity. It was concluded in the assessment that results from a controlled clinical trial are needed. 14
In the MAA for ivacaftor/tezacaftor/elexacaftor , RWD was initially requested by the CHMP to confirm efficacy in specific subpopulations. However, the CHMP considered the registry data not adequate, as the data were subject to bias, individual patient data were unavailable, and data on genotypes and modulator therapies were missing. Additionally, real‐world spirometry data collected were more heterogenous than the clinical study data due to the less standardized equipment used. The CHMP mentioned that, in general, an effect size estimated in the real‐world population is not necessarily comparable to the effect size shown in an RCT, because of, for example, differences in timing of outcome measurements and duration of exposure between clinical study protocols vs. real‐world practice. 15
The refused application of sodium oxybate to treat alcohol dependence was based on three RCTs and a retrospective patient survey. While reasons for refusal was based on the overall lack of robust efficacy data and risk of abuse, several drawbacks on the patient survey design were reported, including the lack of detailed descriptions of baseline variables and dose recommendations, absence of a control group, missing outcomes, and lack of discussion on the natural disease course. 16
The withdrawn application of idebenone included a retrospective cohort study, which, according to the CHMP, suffered from several issues regarding good clinical practice aspects, methodologies, and patient selection. Consequently, the interpretability and generalizability of the outcome data were considered limited, for which convincing conclusions on the long‐term positive effect could not be drawn, and consistency of effect size could not be claimed across the various sources of evidence. 17
From MAAs where RWD/RWE was included both as main and supportive efficacy data, such data were considered supporting decision making in the applications for trientine dihydrochloride and melphalan . Both were previously marketed active substances that were submitted as a hybrid and known active substance applications respectively. The provided evidence for trientine dihydrochloride consisted exclusively of a retrospective and prospective review of medical records and literature, since there was already 30 years of experience with this substance in the UK. The CHMP stated that this approach was acceptable in view of the applied legal basis, which was also in line with the previously provided scientific advice. Nevertheless, some shortcomings were identified with respect to the lack of control, blinding or randomization, the lack of protection against consequential bias, lack of sample size calculation, and the lack of definition of a primary end point for which they used an unvalidated five‐point outcome. 18 For melphalan , evidence was derived exclusively from literature on both interventional and observational studies. For the retrospective observational studies, it was reported that they did not provide all the necessary details for rigorous assessment and several methodological shortcomings related to the retrospective nature of the studies were identified, such as lack of randomization, blinding, standardization of treatment regimens and predefined timing of outcome assessments. Similar to the previous case, the mentioned limitations did not prevent the product from being authorized considering the totality of the evidence. 19
The MAA for enasidenib was withdrawn and the RWD used as external comparator were not supporting decision making due to the heterogeneity of the external cohort populations derived from literature, which deviated from the study population, and the limited data available. The registry data presented were aggregated (individual patient data were not available), were susceptible to confounding and bias, had limited sample size and deviated from the clinical study population, despite attempts to match the populations. 20
Description of cases where RWE was presented as supportive evidence
Of the eight MAAs (8/16), four did not include a description of RWD/RWE in the CHMP's appraisal. Of the four cases where RWD/RWE was addressed, it was derived from literature and was considered supporting decision making for two products ( hydroxycarbamide and product Y ). 21 However, as the CHMP appraisal was based on the overall supportive study package, the RWD was not addressed specifically. One application was later withdrawn for other reasons than the RWD provided. In the case of emapalumab , where the evidence was not considered supporting decision making, the historical comparator derived from literature was considered inappropriate due to deviation from the clinical study population, the risk of bias resulting from the used selection methods, and the lack of a predefined statistical analysis plan. Additionally, the CHMP concluded that the compassionate use program data did not provide sufficient information to draw any conclusions on the effectiveness. 22 In the case of aripiprazole , further details on the RWD were requested but not provided by the applicant that eventually withdrew the application. 23
Extension of indications
Description of the products
Of the 14 reviewed products that sought an EoI, 12 extensions were granted, 1 was refused, and 1 was withdrawn as of July 5, 2021 ( Table 5 and Figure 1 b ). The majority of EoIs concerned antihemorrhagic products ( n = 4) and immunosuppressants ( n = 4).
Table 5
EoIs including preauthorization RWE
AR, assessment report; CHMP, Committee for Medicinal Products for Human Use; PET, positron emission tomography; RCT, randomized controlled trial; RWD, real‐world data; RWE, real‐world evidence.
| RWE presented in CHMP ARs | International nonproprietary name | Extension of indication to | Product authorization status | Purpose of RWD/RWE | Data source(s) | CHMP appraisal on RWD/RWE | Details on strengths/limitations of RWD/RWE submitted by applicants |
| Main study/ studies ( n = 5) | Eptacog alfa (activated) | Patients with Glanzmann's thrombasthenia without antibodies to platelets, or where platelets are not readily available | Authorized | Efficacy/effectiveness data collection Safety data collection Contextualization (natural history) | Registry | Supporting decision making | Strengths Registries can obtain data over several years from a quite significant number of patients with a rare disease who are treated according to current practice at the treatment centers Limitations No pre‐defined comparable subgroups, for which treatment comparison was not possible (this was also not the purpose of the registry) Therapy is according to physician's choice, may be used in an open‐label manner and is, therefore, open to bias Possibility of underreporting of negative experiences as recognized by the MAH |
| Catridecacog | Treatment of bleeding episodes in patients with congenital factor XIII A‐subunit deficiency as well as minor surgery | Authorized | Efficacy/effectiveness data collection Safety data collection Drug utilization data collection | Registry | Supporting decision making | Strengths Follow‐up time is considered appropriate The very few inclusion and no exclusion criteria would reduce selection bias Limitations All treated patients were currently on long‐term prophylactic treatment (this restricted the indication) A number of biases and limitations would occur due to potential confounding factors (e.g., selection bias) and data collected in a real‐life setting (e.g., incorrectness of dose and bleeding evaluation by patient) | |
| Emicizumab | Routine prophylaxis of bleeding episodes in adults and children with hemophilia A with or without factor VIII inhibitors. In addition, two additional posology recommendations for adults and children with hemophilia A with and without factor VIII inhibitors are recommended | Authorized | External comparator Efficacy/effectiveness data collection (post‐authorization) | Medical records (1) Registry (post‐authorization) | Not supporting decision making | Limitations Intrapatient comparison not considered fair: noninterventional group had reduced compliance (1) | |
| Ataluren | Nonambulatory patients with Duchenne muscular dystrophy | Refused | Efficacy/effectiveness data collection Safety data collection External comparator (1) | Research database (1) Registry (post‐authorization) | Not supporting decision making | Limitations The statistical analysis plan was updated at the end of the study, for which data‐driven decisions cannot be excluded and efficacy results can only be considered exploratory (1) Matching with the historical control data was inappropriate and contained multiple sources of bias in favor of ataluren (1) Validity of the findings severely compromised due to the limited number of subjects completing the study, essentially due to patients switching to commercially available ataluren. It was not explained why subjects switching from the investigational product to the commercial product could not have been followed‐up for longer (1) Length in follow‐up time was different, which could have been the sole explanation for the differences in the ataluren and the external control group (1) | |
| Fluciclovine (18F) | Diagnosis and continuing assessment of glioma in adult patients | Withdrawn | Efficacy/effectiveness data collection | Medical records | Not supporting decision making | Limitations Retrospective nature of the analysis PET performance may have changed over the long time period Different dosages applied between populations Small sample included in analysis | |
| Supportive study/ studies ( n = 5) | Dexmedetomidine | For sedation of nonintubated adult patients prior to and/or during diagnostic or surgical procedures requiring sedation, i.e., procedural/awake sedation | Authorized | Efficacy/effectiveness data collection Safety data collection | Literature (including medical chart review) | Supporting decision making | CHMP appraisal based on all supportive studies together, not specifically on RWE |
| Ivacaftor | Patients with cystic fibrosis (CF) aged 6 years and older and weighing 25 kg or more who have an R117H mutation in the CFTR gene and patients with CF aged 12 months and older and weighing 7 kg to less than 25 kg who have an R117H mutation in the CFTR gene | Authorized | Efficacy/effectiveness data collection External comparator | Registry | Supporting decision making | Strengths Slight age difference in patient groups not considered problematic Limitations Patients in both cohorts could not be matched, which limits the accuracy of the conclusions that can be drawn from comparison Subgroup analysis not performed as they were not prespecified in the analysis plan Sample size limitations and low incidence and prevalence of some of events Precise treatment start dates and event occurrence dates may not be available Deaths and transplantations not included as end points Missing genetic data Changes in standard of care, lower visit frequency, and channeling bias/confounding by indication likely to have caused bias | |
| Anakinra | Treatment of familial Mediterranean fever (FMF) to be given in combination with colchicine | Authorized | Efficacy/effectiveness data collection Safety data collection | Literature | Supporting decision making | Strengths Results consistent across the RCT and the retrospective real‐world studies Supportive real‐world studies provided some long‐term data which supported the long‐term use of anakinra Limitations Most publications do not distinguish between efficacy results in colchicine‐resistant vs. in colchicine‐intolerant patients Efficacy mainly based on retrospective studies published in English and publications bias is expected with a potential overestimation of the treatment effect In 5 publications it was not reported whether the product was used alone or in combination with colchicine during the study periods | |
| Methotrexate | Treatment of mild to moderate Crohn's disease either alone or in combination with corticosteroids in patients refractory or intolerant to thiopurines | Authorized | Efficacy/effectiveness data collection Safety data collection | Literature | Not supporting decision making | Limitations Remaining data come only from observational studies found in peer reviewed data Pediatric studies provided do not include any RCT being only small open label studies, case series, or retrospective studies, for which CHMP considered that they were inadequate to provide reliable evidence of efficacy in this population Due to use in mild disease/second‐line settings and the open‐label designs, the submitted studies did not provide sufficient evidence for benefit | |
| Eltrombopag/ eltrombopag olamine | First‐line treatment of adult and pediatric patients aged 2 years and older with severe aplastic anemia for Revolade in combination with standard immunosuppressive therapy | Refused | Efficacy/effectiveness data collection | Literature | Not supporting decision making | Limitations Limited information available as the results and design of this study were available only as publication, for which no comparisons with the pivotal study could be performed |
RWD/RWE characteristics
RWD/RWE‐generated preauthorization was included within 10 (10/14) EoIs, while it was proposed to be submitted post authorization for seven (7/14) dossiers, of which three (3/14) included both preauthorization RWD/RWE and a proposal for postauthorization RWD/RWE ( Figure 1 b ).
Of the applications where RWD/RWE were generated before authorization, five (5/10) presented these data in the “main study/studies” ( fluciclovine (18F) , omalizumab, eptacog alfa (activated) , catridecacog , and ataluren ) and five (5/10) in the “supportive study/studies” as presented in the CHMP assessment report ( dexmedetomidine , ivacaftor , anakinra , methotrexate , eltrombopag / eltrombopag olamine ). Of the five products for which RWD/RWE was presented as main evidence (5/10), three were authorized, one was refused, and one was withdrawn. The RWD/RWE was considered supporting decision making in two and not supporting decision making in one of the authorized EoIs ( Table 5 ). All five EoIs (5/10) where RWD/RWE was presented as supportive evidence were authorized, of which the RWD/RWE were considered supporting decision making in three (3/5).
Description of cases where RWE was presented as main evidence
In two of the three authorized EoIs, the RWE was considered supporting the decision making (( activated ) eptacog alfa and catridecacog ). For ( activated ) eptacog alfa , RWD were used to provide efficacy data and natural history data derived from the Glanzmann's Thrombasthenia Registry, whereby the added value of the registry was acknowledged as follows: “An advantage of this registry design is that data can be obtained over several years from a quite significant number of patients with a rare disease who are treated according to practice which is current at the treatment centres.” 24 However, it was recognized that the observational nature of the study is not without limitations related to the data source itself, e.g., risk of bias such as underreporting of negative experience (reporting bias) and physician's choice of treatment leading to individual differences, and to the study design, e.g., no comparable subgroups were predefined ( Table 5 ). In the EoI of catridecacog , RWD from the Global Prospective Rare Bleeding Disorder Database registry showed efficacy in the proposed population to support the EoI. Again, the risk of bias (e.g., selection bias), together with confounding factors, e.g., incorrectness of dose and bleeding reporting by the patient, were mentioned as limitations of the use of registry data, although these did not prevent the EoIs from being granted. 25
RWD/RWE was not considered supporting the decision making in three (3/5) of the EoIs where RWD was included in the main studies. Of these, two (2/3) used RWD as external comparators. These were deemed inappropriate, because of insufficient matching between the RWD and the clinical study cohorts. The main discrepancy in the case of emicizumab was that “patients in the nonintervention study had markedly reduced compliance with treatment compared with the emicizumab‐exposed population whose treatment was administered in a health facility and under supervision.” 26 For the refused EoI of ataluren , several other limitations were mentioned, including the lack of a predefined statistical analysis plan, and the limited sample size and follow‐up time, which applied to both the performed clinical study and the external control RWD. The reason why these patients could no longer be followed up, was unexplained. 27 In the case of fluciclovine (18F) , a retrospective analysis of medical records was not considered supporting decision making, partly because of its retrospective design and partly because of the small sample size included. This application was later withdrawn. 28
Description of cases where RWE was used as supportive evidence
RWD/RWE supported decision making in three cases (3/5). An external comparator consisting of registry data for the EoI of ivacaftor was considered subject to several limitations, e.g., sample size limitations, misclassification of exposure and event timepoints, missing end points, differences in time period, and possible confounding by indication. However, the slight difference in the patient groups was not considered problematic. 29 For anakinra , the RWD derived from literature were considered supporting decision making, despite some literature‐related limitations. 30 For dexmedetomidine , the total supportive data package, that included not exclusively RWD, was considered supporting decision making, although the contribution of RWE was not addressed separately. 31 The EoI for methotrexate , for which the RWD was not supporting the decision making was still authorized. However, the EoI assessment report stated: “The pediatric studies provided do not include any RCT being only small open label studies, case series or retrospective studies, which does not support the use of MTX [Methotrexate] for the treatment of pediatric population with Crohn's Disease.” Therefore, the indication was only extended in the adult population. 32 The EoI of eltrombopag / eltrombopag olamine was refused, and the RWD could provide limited information as the results of the studies were only available as publications, and comparisons with the pivotal study could not be performed. 33
Postauthorization real‐world data
Twenty (20/32) MAAs and seven (7/14) EoI dossiers contained proposals to generate RWD post authorization. In the CHMP assessment reports, these studies based on RWD were either recommended as imposed studies (7/20), or were qualified as accepted (4/20) / not accepted (1/20) by the committee upon proposal from the applicant, or were addressed in the context of safety monitoring (4/20), or not addressed at all (4/20). Protocols that discuss the studies in detail were usually not available yet for the CHMP to comment on at the time of MA. For onasemnogene abeparvovec , the postauthorization study was already in an advanced planning stage, hence an extensive discussion was included in the CHMP report. 13 Another case where a postauthorization study was discussed more in detail was for edaravone , for which a registry study was proposed, but considered likely to be inadequate by the CHMP: “A protocol for a European registry has been submitted as a proposal to generate survival data. However, there is a major concern that the proposed registry study (cutoff year 2010 for inclusion of historical controls) conducted in two local centers (Ulm and Utrecht) will not be able to capture robust efficacy data in a representative EU population taking into consideration possible biases introduced by more recent changes in standard of care, potential off‐label use of medications and food supplements as well as the different approaches with respect to euthanasia in different EU countries.” 34 In seven MAAs ( betibeglogene autotemcel , Ebola vaccine (MVA‐BN‐Filo (recombinant)), Ebola vaccine (MVA‐BN‐Filo (recombinant), second regimen), ibalizumab, imlifidase, onasemnogene abeparvovec, rilpivirine ) the postauthorization efficacy studies were imposed as obligations or conditions of the MA. 15 , 35 , 36 , 37 , 38 , 39 , 40
Scientific advice on the use of RWE
For 6 of the 16 MAA dossiers, the preauthorization use of RWD/RWE had been discussed in a scientific advice procedure prior to MAA submission. As illustrated in the examples described above, the feedback provided by the SAWP on the overall appropriateness of the inclusion of RWD and the expected difficulties highlighted in these scientific advice procedures were generally in line with the final assessment of the MAA by the CHMP, although noting the efforts of applicants to address these difficulties, sometimes successfully. None of the retrievable scientific advice procedures discussed RWD/RWE in light of the specific EoI. However, in one case, the included study was discussed in a scientific advice prior to the initial authorization of the product as being a postauthorization study as part of the MAA.
RWE included in the SmPC
For three of the five (3/5) MAAs where RWD was considered supporting decision making ( Table 4 and Figure 1 a ), these data were included in the SmPC of the EPAR, and described in details for one ( melphalan ). 41
DISCUSSION
In 2018–2019 there were 32 MAAs and 14 EoIs where RWD/RWE was proposed to support efficacy/effectiveness out of a total of 158 MAAs and 153 EoIs submitted during this period. In a modest number of dossiers, i.e., 5 out of 16 MAAs and 5 out of 10 EoIs, the RWD/RWE was considered supporting the decision making by contributing to the preauthorization efficacy evidence of the MAA or EoI ( Figure 2 ). RWD/RWE proposed to be generated post authorization received only minimal attention in the benefit–risk evaluation, because protocols that discuss the studies in detail are usually not available yet for the CHMP to comment on at the time of MA. However, in certain cases, such as conditional MA, RWD/RWE may be considered key to addressing the missing pieces of evidence required post approval to convert conditional or exceptional MA into a standard MA. Here, studies using RWD may be imposed as specific obligations to provide complementary data to confirm the medicines' benefits continue to outweigh their risks.
The majority of products were antineoplastic agents and immunosuppressants, of which the first is in line with previous studies. 8 , 9 , 10 , 11 The proportion of medicinal products initially receiving an orphan designation was less pronounced in this study (16/32 MAAs and 2/14 EoIs) as compared with other studies, which might be due to shorter and more recent time periods and different definitions used. 11 , 42 Of the many possible sources of RWD, literature, medical records, and registries were most commonly used in this study. As was also recognized in the CHMP assessment, registries have the potential to obtain data from a relatively significant number of patients over the years in cases of rare diseases. 6 , 24
The main issues discussed with respect to RWE were around methodological weaknesses, including missing data, lack of population representativeness, small sample size, lack of an adequate or prespecified analysis plan, and the risk of several types of confounding and bias (mostly selection bias), which was in line with previous studies. 10 , 11 Also similar to these studies, in some MAAs' or EoIs' assessment RWD/RWE submitted in the procedures was not addressed at all. 10 , 11 The identified issues are not new and have been recognized by regulators working on RWE strategies, for example, in a report of the Heads of Medicines Agencies (HMA)–EMA Joint Big Data Task Force from 2019. Challenges highlighted in this report were, for example, extrapolating data from a precisely defined study population to wider patient population, i.e., representativeness of the disease, and the representativeness of the health care sector or geographical region in which the care is delivered. 43 A data quality framework for regulatory use has been proposed and was released for consultation in October 2022. 44 , 45 The EMA is piloting analyses of raw data from clinical trials. Equally where legally permitted, access to individual patient RWD in specific cases could increase regulators' trust and understanding of the data. 46
Impact of RWE on CHMP decision making
Isolating the exact impact of the RWE in regulatory decision making is difficult, as it constitutes only part of the total evidence package provided by the applicant. Although there was a distinction made between RWD/RWE being supporting or not within “main” and “supportive” studies, which may provide different degrees of impact, the CHMP assessment is based on the totality of evidence. Other studies included in the main or supportive evidence package, as well as the extensive knowledge on the mechanism of action (e.g., “extrapolation of efficacy from adults to the pediatric population has been accepted based on (i) similar systemic exposure as in adults or older pediatric patients at the selected doses, (ii) the mechanism of action of ivacaftor which acts similarly in adults and pediatric patients…” 29 ), or previous experience with the medicinal product outside the centralized EU application (e.g., in known active substance applications or hybrid applications) may influence the acceptability of the use of RWE, as was shown in several cases in our study. Other variables, such as characteristics of the disease population or study population, unequivocal natural course of the disease, the large observed effect size, or an existing high unmet medical need may play a role as well. 47
Since a dedicated section for RWE in assessment reports does not yet exist, and RWD are not always labeled as such (e.g., as “observational data,” “noninterventional data,” etc.), the complete assessment reports need to be manually and thoroughly screened to be able to identify the products for which RWD/RWE was included for efficacy. This makes the identification process time‐consuming and challenging to standardize. A structured approach of presenting RWD/RWE in the applications and assessment reports could aid in regulators' review of RWD in addition to monitoring the use in more recent and future procedures and in analyzing the data in a more structured way. 48 In general, in the appraisal, mainly limitations are emphasized, which results in a more negative presentation of regulators' view on RWE. Emphasizing the strengths as well would give a more complete view on when RWE is considered appropriate.
Opportunities to leverage the potential of RWD/RWE in medicines regulation
This study has shown that scientific advice procedures may be used as a platform to discuss the expected added value of RWD and difficulties already in an early planning stage, as the expected limitations of the RWD mentioned there by SAWP were usually in line with the remaining limitations mentioned in the final CHMP assessment, although noting the effort of applicants to address the SAWP advice as best as they could. Early interactions between applicants and regulators, as well as workshops and feasibility analyses focusing on how RWD can contribute to answering specific study questions and how abovementioned limitations could be minimized, are key to moving the appropriate use of RWE forward. 49 , 50
Additionally, improved guidance on how RWE should be incorporated in the evidence generation and in regulatory applications by applicants, as well as in the assessment reports by regulators are needed to facilitate the use of RWE in regulatory decision making. Current efforts, such as the establishment of the Methodologies Working Party composed of experts in (pharmaco)epidemiology as disclosed in the Big Data Steering Group Workplan 2022–2025 (ref. 44 ) will promote the development of a roadmap of new guidance linked to Big Data and RWD/RWE. Furthermore, application of existing guidance and streamlining it toward the different contexts of use of RWE (e.g., ICH (International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use) M11 for structured harmonized protocols, 51 the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP) Guide, 52 and the CHMP Guideline on Registry‐Based Studies 49 ) should already provide substantial direction toward the best use of RWD/RWE, and, therefore, expedites the availability of guidance. 53 Additionally, increased collaboration between regulatory agencies worldwide may promote harmonized definitions and guidance, as proposed in a roadmap resulting from the International Coalition of Medicines Regulatory Authorities' meeting on RWE. 54 As it is argued that RWD will increasingly be used to not only provide postauthorization safety evidence in rare disease, but also in a broader use of supporting and contextualizing clinical trial data, above mentioned efforts are of importance and will support the advancement of medicines development. 55 , 56
Strengths and limitations
This study provides an in‐depth view on the appraisal of RWD/RWE in European centralized procedures, not only for approved products, but also for products that are refused or withdrawn. The results should, however, be seen in light of some limitations. Since the data extraction is performed on internal assessment reports, not all information is publicly available, especially for the refused and withdrawn procedures, for which the published EPARs are usually less exhaustive. Although there is no formal approach to this type of data extraction and analysis, cross‐checks were performed and data were discussed by multiple researchers in case of uncertainties to minimize subjectivity. Finally, the impact of the RWE cannot be completely isolated as the final outcomes of the procedures are based on a complete evidence package. Despite these limitations, the results provide useful considerations for the use of RWE in future authorization applications.
CONCLUSIONS
In principle, RWE can be an endorsed type of evidence as part of the MA exercise as considered by CHMP as long as efforts are made to minimize its limitations. Additional efforts, such as specific guidance, are needed to optimize and facilitate increasing use of RWD not only to provide postauthorization safety data, but also to support and contextualize trial data, and to generate valuable knowledge that will strengthen medicine application dossiers and better inform regulatory decision making. Due to the heterogeneity of RWD and various possible scenarios where these can be used to provide RWE, the appraisal of these data by regulatory authorities still requires a case‐by‐case analysis to ensure RWD are fit for purpose in the specific settings. As all data sources come with their own opportunities and limitations, it is important to be aware of these when planning a real‐world study and to interact timely with regulators to find the most optimal approach.
FUNDING
This project has received funding from the Innovative Medicines Initiative (IMI) 2 Joint Undertaking under grant agreement no. 115974 (Biomarker Enterprise to Attack DKD; BEAT‐DKD). The Joint Undertaking receives support from the European Union's Horizon 2020 research and innovation program and European Federation of Pharmaceutical Industries and Associations (EFPIA) with JDRF. Any dissemination of results reflects only the author's view; the Joint Undertaking is not responsible for any use that may be made of the information it contains. The funding sources did not have a role in the design, conduct, or analysis of the study.
CONFLICT OF INTEREST
E.B.'s PhD project received funding through the Biomarker Enterprise to Attack DKD (BEAT‐DKD). All other authors declared no competing interests for this work.
AUTHOR CONTRIBUTIONS
E.B., K.P., C.J.J., X.K., V.S., and P.G.M.M. wrote the manuscript. E.B., K.P., C.J.J., X.K., V.S., and P.G.M.M. designed the research. E.B. performed the research and analyzed the data.
DISCLAIMER
Any dissemination of results reflects only the authors' view and may not be understood or quoted as being made on behalf of or reflecting the position of the Dutch Medicines Evaluation Board, the European Medicines Agency, or one of its committees or working parties.
ACKNOWLEDGMENTS
The findings of this study have been presented to and have been commented on by the following EMA committees: the CHMP and the SAWP.
References
1. Cave, A. , Kurz, X. & Arlett, P. Real‐world data for regulatory decision making: challenges and possible solutions for Europe. Clin. Pharmacol. Ther. 106, 36–39 (2019).
2. Arlett, P. , Kjær, J. , Broich, K. & Cooke, E. Real‐world evidence in EU medicines regulation: enabling use and establishing value. Clin. Pharmacol. Ther. 111, 21–23 (2022).
3. European Medicines Agency . EMA regulatory science to 2025: strategic reflection < https://www.ema.europa.eu/en/documents/regulatory‐procedural‐guideline/ema‐regulatory‐science‐2025‐strategic‐reflection_en.pdf > (2020). Accessed April 29, 2022.
4. Wu, J. , Wang, C. , Toh, S. , Pisa, F.E. & Bauer, L. Use of real‐world evidence in regulatory decisions for rare diseases in the United States—current status and future directions. Pharmacoepidemiol. Drug Saf. 29, 1213–1218 (2020).
5. Miksad, R.A. , Samant, M.K. , Sarkar, S. & Abernethy, A.P. Small but mighty: the use of real‐world evidence to inform precision medicine. Clin. Pharmacol. Ther. 106, 87–90 (2019).
6. Jonker, C.J. , Bakker, E. , Kurz, X. & Plueschke, K. Contribution of patient registries to regulatory decision making on rare diseases medicinal products in Europe. Front. Pharmacol. 13, 924648 (2022).
7. Baumfeld Andre, E. , Reynolds, R. , Caubel, P. , Azoulay, L. & Dreyer, N.A. Trial designs using real‐world data: the changing landscape of the regulatory approval process. Pharmacoepidemiol. Drug Saf. 29, 1201–1212 (2020).
8. Eskola, S.M. , Leufkens, H.G.M. , Bate, A. , De Bruin, M.L. & Gardarsdottir, H. Use of real‐world data and evidence in drug development of medicinal products centrally authorized in Europe in 2018–2019. Clin. Pharmacol. Ther. 111, 310–320 (2022).
9. Flynn, R. et al . Marketing authorization applications made to the european medicines agency in 2018–2019: what was the contribution of real‐world evidence? Clin. Pharmacol. Ther. 111, 90–97 (2022).
10. Purpura, C.A. , Garry, E.M. , Honig, N. , Case, A. & Rassen, J.A. The role of real‐world evidence in FDA‐approved new drug and biologics license applications. Clin. Pharmacol. Ther. 111, 135–144 (2022).
11. Mahendraratnam, N. , Mercon, K. , Gill, M. , Benzing, L. & McClellan, M.B. Understanding use of real‐world data and real‐world evidence to support regulatory decisions on medical product effectiveness. Clin. Pharmacol. Ther. 111, 150–154 (2022).
12. Directorate‐General for Health and Food Safety Directive 2001/83/EC of the European Parliament and of the Council of 6 November 2001 on the Community code relating to medicinal products for human use. Official J. L 311, 67–128 (2001).
13. European Medicines Agency . Zolgensma: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/zolgensma‐epar‐public‐assessment‐report_en.pdf > (2020). Accessed April 29, 2022.
14. European Medicines Agency . Tibsovo: EPAR—withdrawal public assessment report < https://www.ema.europa.eu/en/documents/withdrawal‐report/withdrawal‐assessment‐report‐tibsovo_en.pdf > (2020). Accessed July 31, 2022.
15. European Medicines Agency . Kaftrio: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/kaftrio‐epar‐public‐assessment‐report_en.pdf > (2020). Accessed April 29, 2022.
16. European Medicines Agency . Hopveus: EPAR—refusal public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/hopveus‐epar‐refusal‐public‐assessment‐report_en.pdf > (2020). Accessed April 29, 2022.
17. European Medicines Agency . Puldysa: EPAR—Withdrawal public assessment report < https://www.ema.europa.eu/en/documents/withdrawal‐report/withdrawal‐assessment‐report‐puldysa_en.pdf > (2020). Accessed July 26, 2022.
18. European Medicines Agency . Cufence: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/cufence‐epar‐public‐assessment‐report_en.pdf > (2019). Accessed April 29, 2022.
19. European Medicines Agency . Phelinun: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/phelinun‐epar‐public‐assessment‐report_en.pdf > (2020). Accessed April 29, 2022.
20. European Medicines Agency . Idhifa: EPAR—withdrawal public assessment report < https://www.ema.europa.eu/en/documents/withdrawal‐report/withdrawal‐assessment‐report‐idhifa_en.pdf > (2019). Accessed July 26, 2022.
21. European Medicines Agency . Xromi: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/xromi‐epar‐public‐assessment‐report_en.pdf > (2019). Accessed April 29, 2022.
22. European Medicines Agency . Gamifant: EPAR—refusal public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/gamifant‐epar‐refusal‐public‐assessment‐report_en.pdf > (2020). Accessed July 26, 2022.
23. European Medicines Agency . Abilify MyCite: EPAR—withdrawal public assessment report < https://www.ema.europa.eu/en/documents/withdrawal‐report/withdrawal‐assessment‐report‐abilify‐mycite_en.pdf > (2020). Accessed July 26, 2022.
24. European Medicines Agency . NovoSeven‐H‐C‐000074‐II‐0104: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/novoseven‐h‐c‐000074‐ii‐0104‐epar‐assessment‐report‐variation_en.pdf > (2018). Accessed April 29, 2022.
25. European Medicines Agency . NovoThirteen‐H‐C‐2284/II/0026/G: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/novothirteen‐h‐c‐2284/ii/0026/g‐epar‐assessment‐report‐variation_en.pdf > (2020). Accessed April 29, 2022.
26. European Medicines Agency . Hemlibra‐H‐C‐4406‐II‐0002: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/hemlibra‐h‐c‐4406‐ii‐0002‐epar‐assessment‐report‐variation_en.pdf > (2019). Accessed July 26, 2022.
27. European Medicines Agency . Translarna‐H‐C‐2720‐II‐0047: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/translarna‐h‐c‐2720‐ii‐0047‐epar‐assessment‐report‐variation_en.pdf > (2019). Accessed April 29, 2022.
28. European Medicines Agency . Axumin‐H‐C‐4197‐II‐0011: EPAR—withdrawal assessment report < https://www.ema.europa.eu/en/documents/withdrawal‐report/withdrawal‐assessment‐report‐axumin‐ii‐11_en.pdf > Accessed July 26, 2022.
29. European Medicines Agency . Kalydeco‐H‐C‐2494‐II‐0082: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/kalydeco‐h‐c‐2494‐ii‐0082‐epar‐assessment‐report‐variation_en.pdf > (2020). Accessed April 29, 2022.
30. European Medicines Agency . Kineret‐H‐C‐363‐II‐0070: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/kineret‐h‐c‐363‐ii‐0070‐epar‐assessment‐report‐variation_en.pdf > (2020). Accessed April 29, 2022.
31. European Medicines Agency . Dexdor‐H‐C‐2268‐II‐0026: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/dexdor‐h‐c‐2268‐ii‐0026‐epar‐assessment‐report‐variation_en.pdf > (2018). Accessed April 29, 2022.
32. European Medicines Agency . Nordimet‐H‐C‐3983‐II‐0016: EPAR—assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/nordimet‐h‐c‐3983‐ii‐0016‐epar‐assessment‐report‐variation_en.pdf > (2020). Accessed April 29, 2022.
33. European Medicines Agency . Revolade‐H‐C‐1110‐II‐0049: EPAR—refusal assessment report—variation < https://www.ema.europa.eu/en/documents/variation‐report/revolade‐h‐c‐1110‐ii‐0049‐epar‐assessment‐report‐variation_en.pdf > (2019). Accessed July 26, 2022.
34. European Medicines Agency . Withdrawal assessment report for Radicava < https://www.ema.europa.eu/en/documents/withdrawal‐report/withdrawal‐assessment‐report‐radicava_en.pdf > (2019). Accessed April 29, 2022.
35. European Medicines Agency . Zynteglo: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/zynteglo‐epar‐public‐assessment‐report_en.pdf > (2019). Accessed July 26, 2022.
36. European Medicines Agency . Mvabea: EPAR—Public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/mvabea‐epar‐public‐assessment‐report_en.pdf > (2020). Accessed July 26, 2022.
37. European Medicines Agency . Zabdeno: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/zabdeno‐epar‐public‐assessment‐report_en.pdf > (2020). Accessed July 26, 2022.
38. European Medicines Agency . Trogarzo: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/trogarzo‐epar‐public‐assessment‐report_en.pdf > (2019). Accessed July 26, 2022.
39. European Medicines Agency . Idefirix: EPAR—public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/idefirix‐epar‐public‐assessment‐report_en.pdf > (2020). Accessed July 26, 2022.
40. European Medicines Agency . Rekambys: EPAR—Public assessment report < https://www.ema.europa.eu/en/documents/assessment‐report/rekambys‐epar‐public‐assessment‐report_en.pdf > (2020). Accessed July 26, 2022.
41. European Medicines Agency . Phelinun: EPAR—product information < https://www.ema.europa.eu/en/documents/product‐information/phelinun‐epar‐product‐information_en.pdf > (2020). Accessed July 26, 2022.
42. Bolislis, W.R. , Fay, M. & Kühler, T.C. Use of real‐world data for new drug applications and line extensions. Clin. Ther. 42, 926–938 (2020).
43. Heads of Medicines Agencies, European Medicines Agency . HMA‐EMA Joint Big Data Taskforce Phase II report: ‘evolving data‐driven regulation’ < https://www.ema.europa.eu/en/documents/other/hma‐ema‐joint‐big‐data‐taskforce‐phase‐ii‐report‐evolving‐data‐driven‐regulation_en.pdf > (2019). Accessed September 30, 2022.
44. Heads of Medicines Agencies, European Medicines Agency . Big Data Steering Group: big data workplan 2022–2025 < https://www.ema.europa.eu/en/documents/work‐programme/workplan‐2022‐2025‐hma/ema‐joint‐big‐data‐steering‐group_en.pdf > (2022). Accessed September 30, 2022.
45. Heads of Medicines Agencies, European Medicines Agency . Draft data quality framework for eu medicines regulation < https://www.ema.europa.eu/en/documents/regulatory‐procedural‐guideline/data‐quality‐framework‐eu‐medicines‐regulation_en.pdf > (2022). Accessed October 14, 2022.
46. European Medicines Agency . Information about the raw data proof‐of‐concept pilot for industry: Access to raw data for marketing authorisation and post‐authorisation applications submitted to the EMA < https://www.ema.europa.eu/en/documents/other/information‐about‐raw‐data‐proof‐concept‐pilot‐industry_en.pdf > (2022). Accessed October 14, 2022.
47. Jahanshahi, M. et al . The use of external controls in FDA regulatory decision making. Ther. Innov. Regul. Sci. 55, 1019–1035 (2021).
48. Wang, S.V. et al . STaRT‐RWE: structured template for planning and reporting on the implementation of real world evidence studies. BMJ 372, m4856 (2021).
49. European Medicines Agency . Guideline on registry‐based studies < https://www.ema.europa.eu/en/documents/scientific‐guideline/guideline‐registry‐based‐studies_en‐0.pdf > (2021). Accessed July 26, 2022.
50. Hofer, M.P. et al . Impact of scientific advice from the European Medicines Agency. Nat. Rev. Drug Discov. 14, 302–303 (2015).
51. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use . Final Concept Paper ICH M11: Clinical electronic Structured Harmonised Protocol (CeSHarP) < https://database.ich.org/sites/default/files/M11_EWG_Concept_Paper.pdf > (2018). Accessed July 26, 2022.
52. European Network of Centres for Pharmacoepidemiology and Pharmacovigilance . ENCePP Guide on Methodological Standards in Pharmacoepidemiology < https://www.encepp.eu/standards_and_guidances/methodologicalGuide.shtml > (2022). Accessed July 26, 2022.
53. Burns, L. et al . Real‐World evidence for regulatory decision‐making: guidance from around the world. Clin. Ther. 44, 420–437 (2022).
54. International Coalition of Medicines Regulatory Authorities . ICMRA statement on international collaboration to enable real‐world evidence (RWE) for regulatory decision‐making < https://www.icmra.info/drupal/sites/default/files/2022‐07/icmra_statement_on_rwe.pdf > (2022). Accessed July 26, 2022.
55. Abrahami, D. , Pradhan, R. , Yin, H. , Honig, P. , Baumfeld Andre, E. & Azoulay, L. Use of real‐world data to emulate a clinical trial and support regulatory decision making: assessing the impact of temporality, comparator choice, and method of adjustment. Clin. Pharmacol. Ther. 109, 452–461 (2021).
56. Concato, J. & Corrigan‐Curay, J. Real‐world evidence—where are we now? N. Engl. J. Med. 386, 1680–1682 (2022).