Inconsistent approaches of the G-BA regarding acceptance of primary study endpoints as being relevant to patients - an analysis of three disease areas: oncological, metabolic, and infectious diseases.

Background

Since 2011, all new medicines in Germany are required to undergo an early benefit assessment (EBA) in comparison to a pre-specified appropriate comparator [ 1 ]. At market entry, the pharmaceutical manufacturer (PM) needs to submit an EBA dossier evaluating the additional benefit of the drug, based on available clinical trial data, to the Federal Joint Committee ( Gemeinsamer Bundesausschuss , G-BA).

The extent of additional benefit is classified by the G-BA as major (1), considerable (2), minor (3), non-quantifiable (4), no additional benefit (5), and less benefit (6) [ 2 ]. The benefit rating serves as the main basis for subsequent price negotiations for reimbursement between the PM and statutory health insurance providers.

For the determination of additional benefit the G-BA requests efficacy data in three different categories: mortality, morbidity and health-related quality of life (HRQoL) [ 2 , 3 ]. However, the G-BA only takes into account endpoints deemed to be patient-relevant. Importantly, the opinion of the G-BA on patient relevance of endpoints frequently diverges from those of regulatory authorities [ 4 – 8 ].

A previous analysis of endpoints in oncology indicated acceptance by the G-BA of endpoints related to mortality, while morbidity endpoints were largely disregarded [ 4 ]. The current analysis extends the scope beyond oncology to evaluate if the identified trends also apply to other disease areas. Benefit assessments in the three most frequently assessed therapeutic areas (with ≥10 finalised and evaluable assessments) were analysed: oncological, metabolic, and infectious diseases. For all primary endpoints (PEPs), patient relevance was evaluated and used to determine whether these PEPs would be taken into consideration in an EBA by the G-BA.

Methods

Analysis set : EBA dossiers submitted by the PMs, the corresponding G-BA appraisals, and oral hearing minutes, all obtained from the G-BA website, were used as sources of data [ 9 ]. Dossiers were not included in the analysis if the G-BA’s view on patient relevance of endpoints was not discernible. This could be due to i) no dossier being submitted by the PM, ii) inability to evaluate acceptance of PEP by the G-BA (due to use of inappropriate comparator, inadequate indirect comparison, or dossiers/studies being incomplete), or iii) if a more recent assessment for the same medicine and indication was available.

Analysis of patient relevance of primary endpoints in the benefit assessment by the G - BA : The resulting dataset was used to analyse whether PEPs used in the clinical studies and reported in the PM dossiers went on to be accepted as patient-relevant in the G-BA appraisals. PEPs were recorded as: patient-relevant when this was explicitly mentioned in the G-BA appraisal, or the corresponding data were used to justify an additional benefit, partially patient-relevant when the PEP was a composite or co-primary endpoint with both patient-relevant and non-patient-relevant components, or not relevant to patients when the G-BA clearly questioned the validity of the endpoint or the appraisal did not mention the PEP data or contained no clear statement of its inclusion.

Following determination of the G-BA’s view on patient relevance of each PEP, they were categorised as either symptomatic or asymptomatic. To ensure this categorisation of the endpoints was unbiased and meaningful, we utilised multiple sources of information integral to the EBA process to inform this procedure. Specifically, the categorisations were based on definitions and descriptions of the endpoints and the respective discussions within the dossiers, the assessments by the Institute for Quality and Efficiency in Healthcare ( Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen , IQWiG), and the G-BA appraisals. The oral hearing at the G-BA aims to provide clarification on critical and controversial topics. Key representatives of the relevant medical and scientific societies are always attending those hearings. Thus, the oral hearing minutes served as a source of expert opinion which we referred to for categorisation of certain endpoints along with supporting scientific statements.

The medicines were divided into therapeutic areas, as categorised by the G-BA, and only therapeutic areas with ≥10 finalised and evaluable EBAs as of 25 January 2016 were considered, which led to the inclusion of oncological, metabolic and infectious diseases in the final analysis set.

Results

Analysis set

Inclusion of therapeutic areas with ≥10 finalised and evaluable EBAs (oncological, metabolic, and infectious disease areas) resulted in 97 EBAs which were considered for evaluation. Of these, 65 (67 %) were evaluable according to the inclusion criteria. Table 1 summarises the number of EBAs included in the analysis and the main reasons for exclusion of EBAs from the analysis set by therapeutic area. Table 1 Inclusion vs exclusion of EBAs by therapeutic area Therapeutic area EBAs ( n ) Assessments in analysis set ( n ) Main reason for EBAs not being evaluable a Included Excluded No dossier submitted Data on PEP acceptance by G-BA not evaluable Newer assessment available (same medicine and indication) Oncological diseases 47 36 11 2 6 3 Metabolic diseases 34 15 19 3 14 2 Infectious diseases 16 14 2 0 2 0 Total 97 65 32 5 22 5 a EBAs were excluded from the analysis when no dossier was submitted by the PM, a newer assessment was available (same medicine and indication) or the G-BA’s viewpoint on the patient relevance of the PEP could not be discerned (due to incomplete dossiers or studies, use of inappropriate comparators, or inadequate indirect comparisons) EBA Early benefit assessment, G-BA Federal Joint Committee, n Number of EBAs, PEP Primary endpoint, PM Pharmaceutical manufacturer

A high proportion of EBAs in metabolic diseases (19 of 34, 56 %) were not evaluable, mainly because the G-BA objected to the methodological execution of the studies for reasons including use of inappropriate comparator or indirect comparison, and incomplete dossiers or studies. A list of EBAs and the respective PEPs evaluated in the analysis is presented in Table 2 . Table 2 EBAs included in the analysis set and respective primary endpoints Medicine Indication Primary endpoint (PEP) Therapeutic expert panel present at oral hearing Oncological diseases Abiraterone acetate Prostate carcinoma OS BDU, DGHO, DVPZ Abiraterone acetate a Prostate carcinoma OS, rPFS DGHO Afatinib c Non-small cell lung cancer PFS DGHO, DKG (Working Group on Thoracic Oncology of the AIO) Aflibercept Metastatic colorectal cancer OS DGVS Axitinib Renal cell carcinoma PFS DGHO Bosutinib Chronic myeloid leukaemia MCR DGHO Brentuximab vedotin Hodgkin-Lymphoma, anaplastic large cell lymphoma ORR DGHO Cabazitaxel Prostate carcinoma OS DGHO, DKG Cabozantinib Thyroid gland neoplasia PFS DGE (thyroid gland section), DGHO, DGN Crizotinib Non-small cell lung cancer PFS, ORR DGHO, POA, representatives from leading oncology centres Decitabine Myeloid leukaemia OS DGHO Enzalutamide Prostate carcinoma OS DGHO, DGU Enzalutamide a Prostate carcinoma OS, rPFS DGHO Eribulin Breast cancer OS DGHO Eribulin ac Breast cancer OS, rPFS DGHO Ibrutinib Chronic lymphocytic leukaemia, relapsed or refractory mantle cell lymphoma (MCL) ORR DGHO Idelalisib Chronic lymphocytic leukaemia, follicular lymphoma PFS DGHO, GLSG, representative from the University Hospital Gießen Ipilimumab Melanoma OS DGHO Lenvatinib Thyroid gland neoplasia PFS DGHO Nintedanib Non-small cell lung cancer PFS DGHO, DKG (Working Group on Thoracic Oncology of the AIO), representative from the LungenClinic Grosshansdorf Nivolumab Melanoma OS, PFS ADO, DGHO Obinutuzumab Chronic lymphocytic leukaemia PFS DGHO Olaparib Ovarian cancer PFS DGHO Pertuzumab Breast cancer PFS, ORR AGO, DGHO Pomalidomide Multiple myeloma PFS DGHO, representatives from the University Hospitals of Heidelberg, Tübingen and Würzburg Radium-223-dichloride Prostate carcinoma OS DGHO Ramucirumab Stomach cancer OS DGHO, DGVS Regorafenib Colorectal cancer OS DGHO Ruxolitinib d Chronic myeloproliferative diseases ≥35 % reduction in spleen volume DGHO Ruxolitinib a Polycythaemia vera Haematocrit control without phlebotomy and ≥35 % reduction in spleen volume DGHO Siltuximab Multicentric Castleman’s disease Durable tumour & symptomatic response (complete and partial response) DGHO Sipuleucel-T Prostate carcinoma OS DGHO, representative from the University Hospital Tübingen, Department of Urology Trastuzumab emtansine Breast cancer OS, PFS AGO, DGHO Vandetanib b Thyroid gland neoplasia PFS DGHO Vemurafenib c Melanoma OS, PFS ADO, DGHO Vismodegib Basal cell cancer ORR DGHO, DGMKG, German Society of Dermatology Metabolic diseases Albiglutide Diabetes mellitus type 2 HbA1c DDG, Diabetes Research Group HZM, representative from the University Hospital Carl Gustav Carus (Dresden) Dulaglutide Diabetes mellitus type 2 HbA1c BVND, DDG, Diabetes Research Group HZM Eliglustat Gaucher disease type 1 Stable health status (decrease in spleen and liver volume, Hb, thrombocytes) and %-change in spleen volume ASIM, DGVS, representative from the Charité University Medicine Berlin Elosulfase alfa Mucopolysaccharidose type IVA (Morquio A syndrome) 6MWT ZSE Wiesbaden Insulin degludec a Diabetes mellitus type 1 HbA1c DDG Ivacaftor Cystic fibrosis FEV1% - Ivacaftor a Cystic fibrosis FEV1% - Linagliptin b Diabetes mellitus type 2 HbA1c BVND, DDG, Diabetes Research Group HZM Pasireotide Pituitary gland dysfunction mUFC ≤ ULN - Pasireotide a Acromegaly Biochemical control - Saxagliptin Diabetes mellitus type 2 HbA1c DDG, Working Group on Pharmacoepidemiology, Diabetes Research Group HZM, Diabetes Centre Bad Lauterberg Saxagliptin/metformin Diabetes mellitus type 2 HbA1c BVND Sitagliptin Diabetes mellitus type 2 HbA1c DDG, Working Group on Pharmacoepidemiology, Diabetes Research Group HZM, Diabetes Centre Bad Lauterberg Sitagliptin/metformin Diabetes mellitus type 2 HbA1c DDG, Diabetes Centre Bad Lauterberg, Diabetes Research Group HZM, Working Group on Pharmacoepidemiology Vildagliptin b Diabetes mellitus type 2 HbA1c BVND, DDG Infectious diseases Boceprevir Chronic hepatitis C SVR bng, DGVS, German Liver Foundation Daclatasvir Chronic hepatitis C SVR bng, dagnä, DGIM, DGVS Dasabuvir Chronic hepatitis C SVR bng, dagnä, DGIM, DGVS Dolutegravir HIV infection VR dagnä, DAIG Dolutegravir/abacavir/lamivudine HIV infection VR - Elvitegravir/cobicistat/emtricitabin/tenofovir-disoproxil HIV infection VR dagnä, DAIG Emtricitabine/rilpivirine/tenofovirdisoproxil HIV infection VR dagnä, DAIG Fidaxomicin Clostridium infection Overall cure DGHO, DGVS Ledipasvir/sofosbuvir Chronic hepatitis C SVR bng, dagnä, DGIM, DGVS Ombitasvir/paritaprevir/ritonavir Chronic hepatitis C SVR bng, dagnä, DGIM, DGVS Rilpivirine HIV infection VR DAIG, dagnä Simeprevir Chronic hepatitis C SVR bng, dagnä, DGI, DGIM, DGVS Sofosbuvir Chronic hepatitis C SVR bng, dagnä, DGIM Telaprevir Hepatitis C SVR bng, DGVS, German Liver Foundation a New therapeutic indication, b Re-assessment for the same indication, c Re-assessment after expiration of G-BA appraisal, d Ruxolitinib is an orphan drug, but has undergone a regular EBA process after having reached sales of >€50 million per year 6MWT 6-minute walking test, ADO Working Group on Dermatological Oncology, AGO Working Group on Gynaecologic Oncology, AIO Working Group on Internal Oncology, ASIM Working Group on Congenital Metabolic Disorders in Internal Medicine, BDU Professional Association of German Urologists, bng Federal Association of Registered Gastroenterologists, BVND Federal Association of Registered Diabetologists, dagnä German Working Group of Registered Doctors in the Care of HIV-infected Persons, DAIG German AIDS Society, DDG German Diabetes Society, DGE German Society of Endocrinology, DGHO German Society of Hematology and Medical Oncology, DGI German Society of Infectious Diseases, DGIM German Society of Internal Medicine, DGMKG German Society of Oral and Maxillofacial Surgery, DGN German Society of Nuclear Medicine, DGU German Society of Urology, DGVS German Society of Gastroenterology, Digestive and Metabolic Diseases, DKG German Cancer Society, DVPZ Umbrella Organisation of Prostate Centres in Germany, EBA Early benefit assessment, FEV1 Forced expiratory volume in 1 second, GLSG German Low Grade Lymphoma Study Group, Hb Haemoglobin, HbA1c Glycated haemoglobin, HIV Human immunodeficiency virus, HZM Helmholtz Centre Munich, POA Working Group on Pulmonary Oncology, MCR Major cytogenic response, mUFC Median urinary free cortisol, ORR Objective response rate, OS Overall survival, PFS Progression-free survival, rPFS Radiographic progression-free survival, SVR Sustained viral response, ULN Upper limit of normal, VR Viral response, ZSE Centre for Rare Diseases

Analysis of patient relevance of primary endpoints in the benefit assessment by G-BA

All therapeutic areas

This analysis set consisted of 65 PEPs obtained from 65 EBA dossiers. The G-BA deemed the PEPs as relevant to patients in 31 (48 %) of 65 EBAs, not relevant to patients in 25 (38 %), and partially patient-relevant in nine (14 %) assessments (refer to sections on individual therapeutic areas for more details on specific PEPs).

Analysis by endpoint category showed that the majority of PEPs were morbidity endpoints (47 of 65, 72 %). Mortality PEPs were used in 11 of 65 PM submissions (17 %), all of which concerned oncological diseases and were deemed patient-relevant by the G-BA (Fig. 1a ). Morbidity PEPs were considered patient-relevant in 20 of 47 (43 %) EBAs and partially patient-relevant in two cases (4 %). In seven EBAs the PEP was a composite endpoint of mortality and morbidity and in each case deemed partially patient-relevant. Six of these cases were in oncological diseases and one was in the metabolic disease area (see also sections on individual therapeutic areas). No assessment included HRQoL as a PEP. Fig. 1 Patient relevance of PEPs according to the G-BA by endpoint dimension. a All three therapeutic areas, b oncological diseases, c metabolic diseases, and d infectious diseases. HRQoL was not used as PEP for any assessment, therefore it is not shown in the graphs

Oncological diseases

In oncological diseases 36 of 47 PM submissions were analysed (Table 1 ). Mortality PEPs were reported in 11 of the 36 EBAs (all overall survival [OS]) and regarded as patient-relevant in the benefit assessment by the G-BA in all cases (Fig. 1b ). Fifteen of 19 (79 %) morbidity PEPs (mostly progression-free survival [PFS]) were deemed not patient-relevant. The three patient-relevant morbidity PEPs were major cytogenic response (MCR) (bosutinib, chronic myeloid leukaemia), reduction of spleen volume by >35 % (ruxolitinib, chronic myeloproliferative diseases), and haematocrit control without phlebotomy and reduction of spleen volume by >35 % (ruxolitinib, polycythaemia vera). For one composite morbidity endpoint, namely partial and complete durable tumour and symptomatic response, only complete response was considered patient-relevant and included in the G-BA appraisal (siltuximab, multicentric Castleman’s disease). Six assessments had co-primary endpoints, comprising mortality and morbidity (all OS and PFS). In all these cases only OS was considered in the appraisal.

Metabolic diseases

In metabolic diseases, 34 PM submissions were available, 15 of which were analysed (Table 1 ). All PEPs were morbidity endpoints, apart from one composite mortality/morbidity PEP (Fig. 1c ) and three morbidity PEPs were deemed patient-relevant in the benefit assessments. One of these PEPs was glycated haemoglobin (HbA1c) in type 1 diabetes (insulin degludec), which was explicitly specified by the G-BA as a validated and patient-relevant endpoint. The others were 6-minute walk test (6MWT) in type IVA mucopolysaccharidose (elosulfase alfa), and median urinary free cortisol level (mUFC) in pituitary gland dysfunction (pasireotide).

In the ten EBAs where PEPs were not considered patient-relevant by the G-BA, PEPs were HbA1c and FEV1 (forced expiratory volume in 1 second) in type 2 diabetes and cystic fibrosis, respectively. In eight benefit assessments (albiglutide, dulaglutide, linagliptin [re-assessment], saxagliptin, saxagliptin/metformin, sitagliptin, sitagliptin/metformin, vildagliptin [re-assessment]) the PEP HbA1c was not explicitly mentioned as patient-relevant and not considered in the G-BA benefit evaluation, and therefore assumed to be not relevant to patients according to the evaluation criteria of this analysis stated above. In the two assessments of ivacaftor, FEV1 data was not taken into consideration by the G-BA as there were different opinions within the G-BA regarding its patient relevance.

In two EBAs the PEP was only partially accepted as patient-relevant by the G-BA. For eliglustat (Gaucher disease type 1) the PEP was a co-primary endpoint, consisting of two endpoints. The first was the composite morbidity endpoint stable health status (spleen volume, liver volume, haemoglobin and thrombocyte count) and the second percentage change in spleen volume, where only the change in spleen volume was deemed patient-relevant. For pasireotid (acromegaly) the PEP was biochemical control (defined as a combination of reduction of the mean growth hormone [GH] level below 2.5 μg/l and normalisation of the age- and sex-adjusted insulin-like growth factor 1 [IGF-1] level after 24 weeks). The PM defined biochemical control as a composite mortality/morbidity endpoint. However the G-BA did not accept the mortality component due to lack of validation.

Infectious diseases

In infectious diseases 14 of 16 PM submissions were included in the analysis set (Table 1 ). There were no mortality PEPs in infectious diseases. All 14 PEPs concerned morbidity (Fig. 1d ), and all were accepted as patient-relevant by the G-BA. In most cases (13 of 14) the PEP was viral response (sustained viral response [SVR] in hepatitis C or viral response [VR] in HIV-infected patients), and in one case, the PEP was overall cure (fidaxomicin, clostridium infection).

Acceptance of symptomatic vs asymptomatic PEPs

PEPs were categorised as symptomatic or asymptomatic according to the definition of the PEP and based on the appraisal documents as well as scientific discussions during the hearing process. A list of therapeutic area expert panels attending the oral hearings is shown in Table 2 . Table 3 summarises each evaluated PEP and the results of the categorisation by therapeutic area (see Additional file 1 : Table S1 for a table listing the rationale of categorisation for each endpoint). Table 3 G-BA acceptance of symptomatic vs asymptomatic morbidity PEPs as (A) patient-relevant and (B) non-patient-relevant Oncological diseases Metabolic diseases Infectious diseases (A) Morbidity PEPs accepted as patient-relevant by the G-BA Symptomatic Complete durable tumour & symptomatic response a 6MWT Overall cure ≥35 % reduction in spleen volume a Reduction in spleen volume a - Asymptomatic Haematocrit control without phlebotomy a HbA1c (Type 1 diabetes) Viral response (VR, SVR) MCR mUFC - - Biochemical control (mean GH <2.5 μg/L and normalisation of IGF-1) - (B) Morbidity PEPs not accepted as patient-relevant by the G-BA Symptomatic Partial durable tumour & symptomatic response a - - Asymptomatic PFS a HbA1c (Type 2 diabetes) - ORR Haemoglobin level a - - Thrombocyte count a - - Reduction in liver volume a - - FEV1 - a This PEP was a component of a co-primary endpoint in at least one dossier 6MWT 6-minute walking test, FEV1 Forced expiratory volume in 1 second, G-BA Federal Joint Committee, GH Growth hormone, HbA1c Glycated haemoglobin, IGF-1 Insulin-like growth factor 1, MCR Major cytogenic response, mUFC Median urinary free cortisol, ORR Objective response rate, OS Overall survival, PEP Primary endpoint, PFS Progression-free survival, SVR Sustained viral response, VR Viral response

Asymptomatic PEPs mostly comprised laboratory parameters (e.g. HbA1c in diabetes) or endpoints involving imaging outcomes (e.g. PFS or objective response rate [ORR]) [ 9 ]. Symptomatic endpoints included outcomes directly experienced by the patient, for example OS in oncological conditions and 6MWT in mucopolysaccharidose [ 9 ]. For other PEPs (complete/partial durable tumour and symptomatic response, overall cure), symptomaticity was self-explanatory by their definitions [ 9 ]. Regarding the symptomaticity of the PEPs related to reduction in spleen or liver volume, an independent Gaucher disease type 1 expert (a member of the German Working Group for Congenital Metabolic Disorders in Internal Medicine [ Arbeitsgemeinschaft für angeborene Stoffwechselstörungen in der Inneren Medizin , ASIM]) stated in the minutes of the oral hearing procedure for eliglustat that the spleen volume in patients prior to treatment is typically increased 15-fold to ca. 2 L [ 9 ]. Since the measured reduction with eliglustat treatment was around 30 % and therefore easily palpable we assigned the reduction in spleen volume as a symptomatic endpoint. A similar reasoning applies to the reduction of ≥35 % of spleen volume in the EBA of ruxolitinib in the indications chronic myeloproliferative diseases and polycythaemia vera. In contrast, a closer analysis of the data for Gaucher disease showed that the relative enlargement and reduction in liver volume before and after eliglustat treatment were too small to be palpable and thus deemed not symptomatic.

Subsequently, we compared the consideration of symptomatic vs asymptomatic PEPs by the G-BA in evaluating additional benefit. For three PEPs the oral hearing minutes were consulted in order to categorise them as symptomatic or asymptomatic, as described above. Symptomatic endpoints were mostly regarded as patient-relevant (Table 3 A), apart from partial durable tumour and symptomatic response (siltuximab, multicentric Castleman’s disease) (Table 3 B). Asymptomatic endpoints (mainly laboratory parameters and endpoints assessed by imaging techniques) were largely deemed not relevant to patients. This had considerable impact on benefit assessment of oncology medications, since PFS and ORR are widely used PEPs which are generally accepted as patient-relevant by the European Medicines Agency (EMA), but were not deemed patient-relevant by the G-BA (Table 3 B). In metabolic diseases HbA1c was only considered patient-relevant in type 1 diabetes. In contrast, asymptomatic viral response endpoints (VR and SVR) in infectious diseases (mainly hepatitis C and HIV infection) were readily accepted as patient-relevant by the G-BA.

Discussion

This analysis demonstrates that specific morbidity PEPs from the EBA are categorically excluded by the G-BA across major disease areas, irrespective of indication and disease stage. Typically, these are PEPs used in pivotal studies for marketing authorisation. The respective clinical studies were specifically designed to show clinical benefit based on these endpoints. The aim of the EBA is to assess the additional benefit of a new medicine compared to the prespecified comparative treatment. This generally differs from the aim of regulatory authorities, which is to evaluate the benefit vs risk profile of a new medicine. Nevertheless, the methods and standards of evidence-based medicine apply to both evaluation procedures. Furthermore, primary endpoints of pivotal studies which led to EMA approval of the investigated drug should be accepted as patient-relevant, irrespective of whether ‘benefit’ or ‘additional benefit’ is under consideration. To suggest that these primary study endpoints are not relevant to patients, as the G-BA does, creates a major dilemma in clinical development and implicitly questions the ethical conduct of studies.

Our analysis by outcome dimension showed a strong dominance of morbidity over mortality PEPs (47 vs 11). While the mortality PEPs were accepted as patient-relevant without exception, less than half of the morbidity PEPs were deemed patient-relevant. Interestingly, the acceptance varied by therapeutic area, revealing a disadvantage for oncological and metabolic disease indications, where morbidity PEPs were predominantly regarded as not patient-relevant. Considering that the majority of these PEPs were accepted by regulatory authorities for marketing authorisation, the classification of such a large proportion of PEPs as not relevant to patients by the G-BA highlights marked differences in data interpretation by regulatory and health technology assessment (HTA) bodies. A similar finding was also demonstrated in a comparative analysis of parallel scientific advice of different European HTA bodies and the EMA [ 10 ].

The PEPs that were most frequently dismissed as not patient-relevant were PFS in oncological diseases and HbA1c in metabolic diseases (except for type 1 diabetes). Despite opposing views on patient relevance of PFS within the G-BA (e.g. afatinib, axitinib, crizotinib, eribuline) [ 9 ] data on this endpoint were systematically excluded from benefit assessments, based on the justification that PFS evaluation utilises asymptomatic endpoint assessment techniques such as imaging. Conversely, in infectious diseases, the PEP SVR was accepted as patient-relevant in hepatitis C despite being an asymptomatic surrogate endpoint lacking formal validation. PEPs in neurological diseases were also evaluated but ultimately excluded from the analysis set as only six assessments were evaluable. Here, morbidity PEPs were largely accepted as patient-relevant by the G-BA (5 of 6 EBAs).

The results of our analysis clearly demonstrate inconsistency in the G-BA’s approach to judging the patient relevance of PEPs between disease areas. For example, in infectious diseases, the G-BA has shown some flexibility in accepting the asymptomatic endpoint SVR as patient-relevant. On the other hand, in oncological and metabolic diseases some asymptomatic endpoints (PFS in cancers and HbA1c in type 2 diabetes) are categorically dismissed, without taking into consideration the different disease profiles of the indications. This is in contrast to the EMA, which adopts a broader approach when evaluating patient relevance and takes indication, difficulty in obtaining mortality data, and priority of accelerating patient access into account.

We welcome the flexibility the G-BA has shown regarding the acceptance of SVR as patient-relevant in hepatitis C, but identify a need to expand this flexibility to other endpoints. The general dismissal of PFS is questionable since in some cancers, such as ovarian cancer, it has been suggested that improved PFS is associated with clinical benefit and is a valid surrogate for extended OS [ 11 ]. Nevertheless the G-BA did not accept PFS as patient-relevant in the EBA of olaparib for the treatment of ovarian cancer [ 9 ].

The success of novel anticancer drugs in recent years has led to a classification shift of various oncological conditions from acutely fatal to chronic disease. Similar to chronic diseases such as diabetes, it is challenging to obtain mortality data showing significant differences in those oncological diseases with slow progression and low death rates. For these reasons, PFS has gained importance as a PEP in clinical trials because it is an early detectable and meaningful endpoint for disease progression [ 12 , 13 ].

Similarly to PFS, HbA1c is routinely used as a surrogate endpoint in diabetes mellitus and widely accepted by regulatory authorities [ 14 ]. For example the EMA states in its “Note for guidance on clinical investigation of medicinal products in the treatment of diabetes mellitus” that HbA1c is the most widely accepted measure of overall, long-term blood glucose control in type 1 and type 2 diabetes, and therefore an appropriate primary study endpoint [ 15 ]. Moreover, this guidance states that reduction of HbA1c is directly related to a reduced risk of developing vascular complications. The G-BA, however, only accepts HbA1c as a patient-relevant endpoint in type 1 diabetes, but not in type 2.

For the patient relevance of morbidity endpoints, clarification of its definition and determination criteria by the G-BA are urgently needed, particularly in oncological diseases and type 2 diabetes, as currently no clear criteria for patient relevance are listed in the G-BA rules of procedure [ 2 ]. Representatives from patient advocacy groups and other external experts are consulted at several stages of the assessment procedure, potentially also concerning the patient relevance of endpoints, but their influence on the benefit appraisal is not transparent and has been criticised as insufficient [ 16 , 17 ]. In a recent press release from the German Society for Hematology and Medical Oncology ( Deutsche Gesellschaft für Hämatologie und Medizinische Onkologie , DGHO) a DGHO board member criticised the fact that market authorisation, EBA, and treatment guidelines often come to different conclusions, despite being based on the same clinical data, making it difficult for doctors to make treatment decisions [ 18 ].

Alignment of study requirements between the G-BA and regulatory authorities is necessary in order to optimise trial design and reduce patient sample sizes, particularly in rare indications, in order to allow timely access to new treatments. A recent publication has identified considerable heterogeneity in regulatory and HTA approaches, even among different European HTA bodies [ 10 ]. Regulatory authorities have adapted their approval pathways for innovative and promising new medicines to facilitate early patient access to new treatments, for example via conditional approval or adaptive pathways [ 10 ]. In order to effectively establish early access to medicines, HTA procedures need to follow the EMA’s footsteps and provide harmonised, transparent, flexible, conditional, and adaptive methods that adopt the level of evidence accepted by medicines agencies [ 10 ].

The UK National Institute for Health and Clinical Excellence (NICE) uses a less restrictive approach in their HTA procedure, where appropriate methods may be used to extrapolate data from less than ideal study designs regarding study type, study duration, patient population, choice of comparator, and type of outcomes [ 19 ]. In contrast to the approach adopted by the G-BA, where data that do not fully comply with requirements are largely rejected, NICE is more receptive to data derived from clinical trials which are for instance non-randomised, non-comparative or employ modelling.

Our analysis has revealed that asymptomatic endpoints were frequently disregarded by the G-BA, while symptomatic endpoints were largely deemed patient-relevant. However, acceptance of asymptomatic endpoints varied by therapeutic area. A major achievement in modern medicine is the ability to capture aspects of morbidity prior to their transformation in symptomatic disease in diagnostic and prognostic examinations. For example, in the assessment of axitinib for renal cell carcinoma, PFS was used to assess the delay in onset of metastases by imaging [ 9 ]. It is evident that early detection of metastases before the onset of symptomatic pain or even vertebral fractures is essential and of utmost relevance to the patient. In this case, the strict insistence on symptomatic endpoints may lead to a delay in complementary treatment options, such as bisphosphonates, which could prevent occurrence of vertebral fractures [ 20 ]. This would contradict the underlying ethical principles of diagnosis and therapy in this and many other oncological diseases.

Currently, the G-BA demands patient-relevant outcome data for the three endpoint categories mortality, morbidity and HRQoL, and weighs each with equal importance for all assessments. However, a number of reasons may lead to attenuation of mortality data, for instance, i) clinical advancement of available therapies, ii) change in disease status from ‘untreatable’ to ‘chronic’, and iii) differences in the course of diseases themselves. Given the differences in mortality and progression between different diseases, for example some forms of cancer vs non-fatal chronic diseases, equal weighting of endpoint categories is not informative. Instead, differential weighting of each endpoint category in the EBA, taking into account disease type and severity/stage, as outlined in Fig. 2 would be advisable. Fig. 2 Adaptable and disease-specific weighting of endpoint categories

Limitation

For each EBA evaluated, only the respective PEP was included, as this is the parameter which provides the most clinically relevant and convincing evidence, based on the statistical principles of clinical trials [ 21 ], and clinical trials are explicitly designed to show efficacy in terms of the PEP. A more comprehensive approach would also include secondary endpoints. Nevertheless, our approach sufficiently highlights the clear inconsistencies that exist between endpoint acceptance by regulatory authorities and the G-BA.

Conclusions

There is a need for the G-BA to define the morbidity endpoints that are considered patient-relevant, particularly in indications such as diabetes and oncological diseases. This will allow PMs to target clinical trial designs towards requirements of the G-BA at an early clinical development stage to ensure an efficient pipeline for bringing innovative and efficacious treatments to patients. In addition, further harmonisation between regulatory bodies and G-BA with regards to acceptance of PEPs as relevant to patients is urgently required. In particular, this applies to morbidity-related PEPs that are not (yet) symptomatic and tangible to patients.

Additional file

Additional file 1: Table S1. Rationale of categorisation of PEPs as symptomatic and asymptomatic. List of the rationale of categorisation as symptomatic and asymptomatic for all PEPs included in the analysis. (DOCX 14 kb)

Abbreviations

6-minute walking test

Working Group of Dermatological Oncology

Working Group of Gynaecological Oncology

Working Group of Internal Oncology

The Act on the Reform of the Market for Medicinal Products

Working Group for Congenital Metabolic Disorders in Internal Medicine

Professional Association of German Urologists

Federal Association of Registered Gastroenterologists

Federal Association of Registered Diabetologists

German Working Group of Registered Doctors in the Care of HIV-infected Persons

German AIDS Society

German Diabetes Society

German Society of Endocrinology

German Society of Hematology and Medical Oncology

German Society of Infectiology

German Society of Internal Medicine

German Society of Oral and Maxillofacial Surgery

German Society of Nuclear Medicine

German Society of Urology

German Society of Gastoenterology, Digestive and Metabolic Diseases

German Cancer Society

Umbrella Organisation of Prostate Centres Germany

Early benefit assessment

European Medicines Agency

Forced expiratory volume in 1 second

Federal Joint Committee

Growth hormone

German Low Grade Lymphoma Study Group

Haemoglobin

Glycated haemoglobin

Human immunodeficiency virus

Health related quality of life

Health technology assessment

Helmholtz Centre Munich

Insulin-like growth factor 1

Institute for Quality and Efficiency in Healthcare

Major cytogenic response

Median urinary free cortisol

National Institute for Health and Clinical Excellence

Objective response rate

Overall survival

Primary endpoint

Progression-free survival

Pharmaceutical manufacturer

Working Group on Pulmonary Oncology

Radiographic progression-free survival

Sustained viral response

Upper limit of normal

Viral response

Centre for Rare Diseases

Acknowledgements

Manuscript preparation was supported by medical writing services from nspm ltd, Meggen, Switzerland, with financial support from Roche Pharma AG.

Funding

Not applicable.

Availability of data and material

Datasets were derived from the website of the G-BA ( https://www.g-ba.de/informationen/nutzenbewertung/ ), a public domain resource. All data analysed during the current study are included in this published article and its supplementary information file.

Authors’ contributions

JR and TS designed the research and analysed the data. TS, GI, VA and JR participated in the interpretation of the data and the writing of the manuscript. TS, GI, VA and JR read and approved the final manuscript.

Competing interests

TS, GI and JR are employees of Roche Pharma AG, Grenzach-Wyhlen, Germany. VA has no conflicts of interest.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

References

1. Bundesiministerium für Gesundheit. [Gesetz zur Neuordnung des Arzneimittelmarktes - AMNOG]. 2011. http://www.bundesgesundheitsministerium.de/service/begriffe-von-a-z/a/arzneimittelmarktneuordnungsgesetz-amnog.html . Accessed 15 Sep 2016.

2. G-BA. [Verfahrensordnung des Gemeinsamen Bundesausschusses]. 2016. https://www.g-ba.de/downloads/62-492-1243/VerfO_2016-03-17_iK-2016-08-23.pdf . Accessed 15 Sep 2016.

3. Deutscher Bundestag. [Sozialgesetzbuch (SGB) Fünftes Buch (V) - Gesetzliche Krankenversicherung - (Artikel 1 des Gesetzes v. 20. Dezember 1988, BGBl. I S. 2477, letzte Änderung durch Artikel 3 des Gesetzes vom 20. Dezember 2012 (BGBI. I S. 2781))]. 1988. http://www.gesetze-im-internet.de/bundesrecht/sgb_5/gesamt.pdf . Accessed 15 Sep 2016.

4. Ruof J Knoerzer D Duenne AA Dintsios CM Staab T Schwartz FW Analysis of endpoints used in marketing authorisations versus value assessments of oncology medicines in Germany Health Policy 2014 118 2 242 254 10.1016/j.healthpol.2014.08.004 25194474

5. Ruof J Fluckiger O Andre N Early benefit assessments in oncology in Germany: how can a clinically relevant endpoint not be relevant to patients? Drugs R D 2015 15 3 221 226 10.1007/s40268-015-0100-1 26286202 PMC4561053

6. Dabisch I Dethling J Dintsios CM Drechsler M Kalanovic D Kaskel P Patient relevant endpoints in oncology: current issues in the context of early benefit assessment in Germany Health Econ Rev 2014 4 1 2 9 10.1186/2191-1991-4-2 24460706 PMC3901346

7. Kvitkina T, ten Haaf A, Reken S, McGauran N, Wieseler B. Patient-relevant outcomes and surrogates in the early benefit assessment of drugs: first experiences [Patientenrelevante Endpunkte und Surrogate in der frühen Nutzenbewertung von Arzneimitteln: erste Erfahrungen]. Z Evid Fortbild Qual Gesundhwesen. 2014;108(8-9):528–38. 10.1016/j.zefq.2014.06.015 25523852

8. Hörn H Nink K McGauran N Wieseler B Early benefit assessment of new drugs in Germany - results from 2011 to 2012 Health Policy 2014 116 2-3 147 153 10.1016/j.healthpol.2013.12.008 24472328

9. G-BA. Overview of products [Übersicht der Wirkstoffe]. 2016. http://www.g-ba.de/informationen/nutzenbewertung/ . Accessed 15 Sep 2016.

10. Tafuri G Pagnini M Moseley J Massari M Petavy F Behring A How aligned are the perspectives of EU regulators and HTA bodies? A comparative analysis of regulatory-HTA parallel scientific advice Br J Clin Pharmacol 2016 82 4 965 73 10.1111/bcp.13023 27245362 PMC5137821

11. Lebwohl D Kay A Berg W Baladi JF Zheng J Progression-free survival: gaining on overall survival as a gold standard and accelerating drug development Cancer J 2009 15 5 386 394 10.1097/PPO.0b013e3181b9c5ec 19826358

12. Sargent DJ Hayes DF Assessing the measure of a new drug: is survival the only thing that matters? J Clin Oncol 2008 26 12 1922 1923 10.1200/JCO.2007.14.8064 18421044

13. Axelrod C The significance of progression-free survival as a clinical end point in oncology Drug Benefit Trends 2010 22 39 45

14. European Medicines Agency (EMA). Guideline on clinical investigation of medicinal products in the treatment or prevention of diabetes mellitus (CPMP/EWP/1080/00 Rev. 1). 2012. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2012/06/WC500129256.pdf . Accessed 15 Sep 2016.

15. Committee for proprietary medicinal products (CPMP). Note for guidance on clinical investigation of medicinal products in the treatment of diabetes mellitus (Report No.: CPMP/EWP/1080/00). 2002. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2009/09/WC500003262.pdf . Accessed 15 Sep 2016.

16. Bleß H-H, de Millas C, Kern B, Seidlitz C, Talaschus A. Auswirkungen von Stellungnahmen der Wissenschaft in der Nutzenbewertung von Arzneimitteln. 2016. https://www.mynewsdesk.com/material/document/55081/download . Accessed 15 Sep 2016.

17. Deutsche Gesellschaft für Hämatologie und medizinische Onkologie (DGHO). Nutzenbewertung von Arzneimitteln der Onkologie und Hämatologie - eine Standortbestimmung. 2013. http://www.dgho.de/informationen/gesundheitspolitische-schriftenreihe/dgho_gpsr_arzneimittelnutzenbewertung.pdf . Accessed 15 Sep 2016.

18. Deutsche Gesellschaft für Hämatologie und medizinische Onkologie (DGHO). DGHO erweitert Informationsplattform Onkopedia um den Bereich Arzneimittel Bewertung. 2016. https://www.dgho.de/informationen/presse/pressemitteilungen/dgho-erweitert-informationsplattform-onkopedia-um-den-bereich-arzneimittel-bewertung . Accessed 15 Sep 2016.

19. Ivandic V Requirements for benefit assessment in Germany and England - overview and comparison Health Econ Rev 2014 4 1 12 10.1186/s13561-014-0012-8 26054401 PMC4884042

20. Novartis. Zometa Summary of Product Characteristics. 2016. http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000336/WC500051730.pdf . Accessed 15 Sep 2016.

21. ICH Expert Working Group. ICH Harmonised tripartite guideline - Statistical principles for clinical trials - E9. 1998. http://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E9/Step4/E9_Guideline.pdf . Accessed 15 Sep 2016. 10532877