Immunotherapy with check-point inhibitors (CPI) in adult malignancies: a protocol for the systematic review of the quality of economic analyses.

Background

Rationale

Immuno-oncology has led to a paradigm shift in the field of cancer care. Since its inception, this new class of systemic cancer therapy has received at least 26 Food and Drug Administration (FDA) approvals across 17 different cancer types, with 940 more agents being studied in clinical development as of September 2017 [ 1 ].

Contrary to traditional cytotoxic chemotherapy, CPIs enhance the body’s immune system to induce antitumor activity [ 2 ]. This allows for comparatively more tolerable adverse events and increased efficacy in certain tumor types. CPIs have been integrated within the treatment pathways of many solid and hematologic malignancies, such as lung and head and neck cancer, urothelial and renal cell carcinoma, and lymphoma. Furthermore, CPIs are under development for use in many more tumor types.

Since healthcare in countries such as Canada and the UK is administered under a publicly funded system, governments have insisted that, prior to approval for funding, new therapies must not only demonstrate clinical effectiveness, but also cost-effectiveness as well. Countries without single-payer systems are also facing the need to account for the rising costs of care. The American Society of Clinical Oncology (ASCO) has recently focused attention on the importance of considering financial impacts in addition to other criteria when assessing new therapies [ 3 ]. Thus, along with clinical effectiveness, evaluations of the economic effectiveness of immunotherapies, and in particular check-point inhibitors (CPIs), is of importance.

Unlike the validated approaches for the assessment of clinical effectiveness, such as randomized controlled trials, the assessment of cost-effectiveness is currently not as rigorously scrutinized. Some analyses combine real-world patient-level data with quality of life data collected from randomized controlled trials in single disease sites [ 4 ]. Other studies combine multiple disease sites across different countries with inherently different perspectives and healthcare payment systems [ 5 , 6 ]. Decision-makers often use incremental cost-effectiveness ratios (ICERs) to compare new immunotherapy regimens with existing therapies; however, this method is nuanced and plagued by inaccuracies, such that experts in the field are calling into question the true validity of such analyses [ 7 ]. Others have proposed the addition of a standardized scale for grading financial toxicity, parallel to the Common Terminology Criteria for Adverse Events, a system commonly used to evaluate new agents in clinical trials [ 8 ].

In an attempt to standardize health economic studies, the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) has developed an aggregate guideline for authors and reviewers to take into consideration. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement was developed by international experts, and jointly endorsed by 10 journals including the British Medical Journal in 2013 [ 9 , 10 ].

Previous studies have adopted the CHEERS checklist as their reference for evaluating the quality of published health economic studies in multiple settings [ 11 – 13 ]. In particular, the quality of economic analyses of oral cancer drugs [ 14 ], multiple myeloma therapies [ 15 ], and adjuvant breast cancer radiotherapy [ 16 ], have been assessed in this way. These studies have revealed that adherence to the CHEERS standards was quite variable ranging from 50% to nearly complete compliance.

Although many regulatory bodies have approved immunotherapies based on existing economic analyses, there has not been a review of the quality and reporting of health economic studies conducted on this new class of agents.

Objectives

Thus, we propose to systematically review the methodologic and reporting quality of cost-effectiveness and cost-utility studies comparing new CPIs used in immune-oncology regimens, including anti-PD1 agents (nivolumab and pembrolizumab), anti-PDL1 agents (atezolizumab, avelumab, and durvalumab), and the anti-CTLA4 agents (ipilimumab and tremelimumab), to established therapies in adults with solid and hematologic malignancies.

Methods

Protocol and registration

This review will not be registered with the international prospective register of systematic reviews (PROSPERO) as it does not meet the eligibility criterion of addressing an outcome of direct patient or clinical relevance.

This protocol was developed based on the Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) 2015 checklist (Additional file 1 : Figure S1) [ 17 , 18 ], and any protocol amendments will be described, dated, and a rationale provided.

Inclusion and exclusion criteria

Only studies meeting the following criteria will be included: Articles reporting on the results of economic analyses with cost-effectiveness, cost-utility, or cost-benefit analyses. This includes pre-planned analyses within randomized control trials, meta-analyses, or any study design using appropriate economic modeling. Articles focusing on the methodological aspects of economic analyses or reporting only on cost-of-illness or budget impacts will be excluded. Articles comparing at least two treatment arms, one of which could be placebo, in adults (at least 18 years old or older, or if children were included, where the data for treatment in adults is separately reported and analyzed) with hematologic and solid malignancies, where at least one treatment arm used one of the following immunotherapeutic agents, which were approved at the time of study design: ○Anti-PD1 agents (nivolumab and pembrolizumab) ○Anti-PDL1 agents (atezolizumab, avelumab, and durvalumab) ○Anti-CTLA4 agents (ipilimumab and tremelimumab) Articles published in English between 1 January 2007 and 31 December 2018, inclusively. This period was chosen as it captures all reports on the above-mentioned therapeutic agents in human trials. Full text and original articles. Abstracts, conference proceedings, editorials, letters, reviews, and systematic reviews will be excluded. In the event of multiple publications reporting an analysis of the same underlying data, all articles meeting the inclusion criteria will be included in the analysis given the focus on methodological and reporting quality, rather than the conclusion reached by the underlying studies.

Information sources, data management, and study selection

A systematic search will be performed in Ovid MEDLINE® (1946 to present), OVID Embase (1974 to present), Cost-effectiveness Analysis Registry (1926 to 2015), Evidence-Based Medicine Reviews, and NIHR-HTA (1989 to present) with the search strategy detailed in Table 1 . Literature search results will be uploaded to EPPI-Reviewer 4 [ 19 ], in which two reviewers will independently assess the titles and abstracts of all articles retrieved from the database search. The full-text of articles potentially meeting the inclusion criteria will then be assessed for inclusion by the same two independent reviewers. Neither author will be blinded to the authors, institution, or journal of publication. Any disagreements will be resolved through discussion and consensus between the two reviewers, and a third reviewer will be asked to adjudicate on any cases where consensus cannot be reached. Data will then be extracted from all full-text articles selected for analysis according to the data extraction form detailed in Table 2 . Table 1 Search strategy Search term 1. Immunotherapy/ 2. immunotherap*.ti,ab. 3. immuno therap*.ti,ab. 4. immuno-oncology.ti,ab. 5. immunooncology.ti,ab. 6. immuno oncology.ti,ab. 7. immuno-therap*.ti,ab. 8. chemoimmunotherap*.ti,ab. 9. chemo-immunotherap*.ti,ab 10. checkpoint inhibitor*.ti,ab. 11. check-point inhibitor*.ti,ab. 12. check point inhibitor*.ti,ab. 13. check-point block*.ti,ab. 14. checkpoint block*.ti,ab. 15. check point block*.ti,ab. 16. check-point therap*.ti,ab. 17. checkpoint therap*.ti,ab. 18. check point therap*.ti,ab. 19. pd-l1.ti,ab. 20. pd-1.ti,ab. 21. ctla-4.ti,ab. 22. exp CTLA-4 Antigen/ 23. Cytotoxic T-Lymphocyte Associated Antigen 4.ti,ab. 24. Cytotoxic T-Lymphocyte Antigen 4.ti,ab. 25. exp Programmed cell death 1 receptor/ 26. Programmed cell death 1 receptor.ti,ab. 27. Programmed cell death 1 protein.ti,ab. 28. Pd 1 protein.ti,ab. 29. PDCD1 protein.ti,ab 30. CD279 antigen.ti,ab. 31. nivolumab.ti,ab. 32. Opdivo.ti,ab. 33. pembrolizumab.ti,ab. 34. Keytruda.ti,ab. 35. durvalumab.ti,ab. 36. Imfinzi.ti,ab. 37. tremelimumab.ti,ab. 38. Exp ipilimumab/ 39. Yervoy.ti,ab. 40. atezolizumab.ti,ab. 41. Tecentriq.ti,ab. 42. avelumab.ti,ab. 43. Bavencio.ti,ab. 44. Economics/ 45. exp Costs and Cost Analysis/ 46. Economics, Medical/ 47. Economics, Nursing/ 48. Economics, Pharmaceutical/ 49. exp Economics, Hospital/ 50. exp “Fees and Charges”/ 51. exp Budgets/ 52. economic*.ti,ab. 53. cost*.ti,ab. 54. pric*.ti,ab. 55. fees.ti,ab. 56. pharmacoeconomic.ti,ab. 57. pharmaco-economic.ti,ab. 58. expendit*.ti,ab. 59. (value adj2 mone*).ti,ab. 60. exp models, economic/ 61. exp Neoplasms/ 62. cancer*.ti,ab. 63. neoplas*.ti,ab. 64. tumor.ti,ab. 65. tumor.ti,ab. 66. malignan*.ti,ab. 67. or/1–43 68. or/44–60 69. or/61–66 70. 67 and 68 and 69 Table 2 Data extraction form Variable name Allowable values Explanation Baseline trial characteristics ID Valid PubMed ID number Pubmed ID Journal Name of journal Journal of publication Year 2007–2018 Year of publication Tumor site Non-small cell lung cancer Melanoma Renal cell carcinoma Squamous cell carcinoma of the head and neck Urothelial carcinoma Classical Hodgkin lymphoma Other The tumor type for which treatment with a check-point inhibitor was analyzed. Check_point_1 Nivolumab Name of the first check-point inhibitor being compared. Pembrolizumab Atezolizumab Avelumab Durvalumab Ipilimumab Tremelimumab Check_point_2 Nivolumab Name of 2nd check-point inhibitor being compared, if applicable. Pembrolizumab Atezolizumab Avelumab Durvalumab Ipilimumab Tremelimumab Comparator_1 Placebo Name of the 1st non-check-point-inhibitor agent or chemotherapeutic regimen being compared, if applicable. Name of other agent or regimen Comparator_2 Placebo Name of the 2nd non-check-point-inhibitor agent or chemotherapeutic regimen being compared, if applicable. Name of other agent or regimen Funding Industry/government Funding source declared by article Country List of country names List of all the countries in which the economic analysis applied as reported by the article. Multiple_countries Y/N Did the economic analysis apply to multiple countries? CHEERS criteria Title C/I As detailed in CHEERS recommendations [ 9 ] Abstract C/PC/I As detailed in CHEERS recommendations [ 9 ] Background & Objectives C/PC/I As detailed in CHEERS recommendations [ 9 ] Target population C/PC/I As detailed in CHEERS recommendations [ 9 ] Setting_Location C/PC/I As detailed in CHEERS recommendations [ 9 ] Perspective C/PC/I As detailed in CHEERS recommendations [ 9 ] Comparators C/PC/I As detailed in CHEERS recommendations [ 9 ] Time horizon C/PC/I As detailed in CHEERS recommendations [ 9 ] Discount rate C/PC/I As detailed in CHEERS recommendations [ 9 ] Health outcomes C/PC/I As detailed in CHEERS recommendations [ 9 ] Effectiveness C/PC/I As detailed in CHEERS recommendations [ 9 ] Preference for outcomes C/PC/I/NA As detailed in CHEERS recommendations [ 9 ] Resources and cost C/PC/I As detailed in CHEERS recommendations [ 9 ] Resource quantities C/PC/I As detailed in CHEERS recommendations [ 9 ] Choice of model C/PC/I As detailed in CHEERS recommendations [ 9 ] Assumptions C/PC/I As detailed in CHEERS recommendations [ 9 ] Analytic methods C/PC/I As detailed in CHEERS recommendations [ 9 ] Study parameters C/PC/I As detailed in CHEERS recommendations [ 9 ] ICERs C/PC/I As detailed in CHEERS recommendations [ 9 ] Uncertainty C/PC/I As detailed in CHEERS recommendations [ 9 ] Heterogeneity C/PC/I/NA As detailed in CHEERS recommendations [ 9 ] Limitations C/PC/I As detailed in CHEERS recommendations [ 9 ] Funding source C/I As detailed in CHEERS recommendations [ 9 ] Conflict of Interest C/I As detailed in CHEERS recommendations [ 9 ] C complete; PC partially complete; I incomplete; NA not applicable

Synthesis of results

A qualitative description of the quality of included articles will be performed and data extracted (Table 2 ) will be provided in text and table form. All extracted data will be presented by article in table form. A random effects model will be used to assess the association of journal impact factor, year of publication, funding source, tumor site, trial or model-based study, and check-point-inhibitor studied on compliance with CHEERS criteria. The significance, magnitude, and confidence interval of associations will be presented in table form. Significance will be set at a two-sided p value of ≤ 0.05. All data will be analyzed using STATA 14 (StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP). As the current study focuses on the quality and reporting of economic analyses of CPIs, rather than the reported results, and the high heterogeneity expected of the included studies, no attempts will be made to quantitatively synthesize or analyze the results of the underlying studies, nor the risk of bias of the results, of the included articles. Wherever possible, all results will be presented in accordance with the PRISMA guidelines [ 20 , 21 ].

Discussion

The described systematic review will aim to provide an overview of the quality of economic analyses on the use of novel check-point inhibitor therapies for the treatment of malignancies in adult patients. Any systemic or recurrent deficiencies in methodological or reporting quality will be described, and barriers to improvement explored, in an attempt to better characterize the quality of the evidence base on which financial toxicity assessments and funding policies are determined.

Additional file

Additional file 1: PRISMA-P 2015 checklist. (DOCX 33 kb)

Abbreviations

American Society of Clinical Oncology

Consolidated Health Economic Evaluation Reporting Standards

Check-point inhibitors

Cytotoxic T-lymphocyte-associated protein 4

Food and Drug Administration

Incremental cost-effectiveness ratio

International Society for Pharmacoeconomic and Outcomes Research

National Institute for Health Research–Health Technology Assessment

Programmed death-1

Programmed cell death ligand

Preferred Reporting Items for Systematic review and Meta-Analysis

Preferred Reporting Items for Systematic review and Meta-Analysis Protocols

International prospective register of systematic reviews

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Acknowledgements

None.

Author contributions

DS developed the search strategy. YW and PC drafted the manuscript, and developed and piloted the data extraction form. All authors contributed to the conception of the systematic review, development of the protocol, read, provided feedback and edits, and approved the final manuscript.

Funding

This protocol and the planned study were not sponsored, and no funding has or will be received. YW receives fellowship funding from the University of British Columbia Clinical Investigator Program which was not involved in the development of the present protocol.

Availability of data and materials

The datasets generated and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

References

1. Tang J Shalabi A Vm H-L Comprehensive analysis of the clinical immuno-oncology landscape Ann Oncol 2018 29 1 84 91 10.1093/annonc/mdx755 29228097

2. Postow MA Sidlow R Hellmann MD Immune-related adverse events associated with immune checkpoint blockade N Engl J Med 2018 378 2 158 168 10.1056/NEJMra1703481 29320654

3. Meropol NJ Schrag D Smith TJ Mulvey TM Langdon RM Blum D American society of clinical oncology guidance statement: the cost of cancer care J Clin Oncol 2009 27 23 3868 3874 10.1200/JCO.2009.23.1183 19581533

4. Matter-Walstra K Schwenkglenks M Aebi S Dedes K Diebold J Pietrini M A cost-effectiveness analysis of nivolumab versus docetaxel for advanced nonsquamous nsclc including pd-l1 testing J Thorac Oncol 2016 11 11 1846 1855 10.1016/j.jtho.2016.05.032 27311996

5. Vouk K Benter U Amonkar MM Marocco A Stapelkamp C Pfersch S Cost and economic burden of adverse events associated with metastatic melanoma treatments in five countries J Med Econ 2016 19 9 900 912 10.1080/13696998.2016.1184155 27123564

6. Tartari F Santoni M Burattini L Mazzanti P Onofri A Berardi R Economic sustainability of anti-pd-1 agents nivolumab and pembrolizumab in cancer patients: recent insights and future challenges Cancer Treat Rev 2016 48 20 24 10.1016/j.ctrv.2016.06.002 27310708

7. Gafni A Birch S Incremental cost-effectiveness ratios (icers): the silence of the lambda Soc Sci Med 2006 62 9 2091 2100 10.1016/j.socscimed.2005.10.023 16325975

8. Khera N Reporting and grading financial toxicity J Clin Oncol 2014 32 29 3337 3338 10.1200/JCO.2014.57.8740 25199760

9. Husereau D Drummond M Petrou S Carswell C Moher D Greenberg D Consolidated health economic evaluation reporting standards (cheers) statement BMJ 2013 346 f1049 10.1136/bmj.f1049 23529982

10. Husereau D Drummond M Petrou S Carswell C Moher D Greenberg D Consolidated health economic evaluation reporting standards (cheers)--explanation and elaboration: a report of the ispor health economic evaluation publication guidelines good reporting practices task force Value Health 2013 16 2 231 250 10.1016/j.jval.2013.02.002 23538175

11. Meregaglia M Cairns J Economic evaluations of follow-up strategies for cancer survivors: a systematic review and quality appraisal of the literature Expert Rev Pharmacoecon Outcomes Res 2015 15 6 913 929 10.1586/14737167.2015.1087316 26449255

12. Edwards K Jones N Newton J Foster C Judge A Jackson K The cost-effectiveness of exercise-based cardiac rehabilitation: a systematic review of the characteristics and methodological quality of published literature Health Econ Rev 2017 7 1 37 10.1186/s13561-017-0173-3 29052044 PMC5648728

13. Iannazzo S Iliza AC Perrault L Disease-modifying therapies for multiple sclerosis: a systematic literature review of cost-effectiveness studies Pharmacoeconomics 2018 36 2 189 204 10.1007/s40273-017-0577-2 29032493

14. Al Kadour A Wa M Al-Badriyeh D Pharmacoeconomics evaluations of oral anticancer agents: systematic review of characteristics, methodological trends, and reporting quality Value Health Reg Issues 2018 16 46 60 10.1016/j.vhri.2018.05.003 30144775

15. Aguiar PM Lima TM Storpirtis S Systematic review of the economic evaluations of novel therapeutic agents in multiple myeloma: what is the reporting quality? J Clin Pharm Ther 2016 41 2 189 197 10.1111/jcpt.12384 27009796

16. Monten C Veldeman L Verhaeghe N Lievens Y A systematic review of health economic evaluation in adjuvant breast radiotherapy: quality counted by numbers Radiother oncol 2017 125 2 186 192 10.1016/j.radonc.2017.08.034 28923574

17. Shamseer L Moher D Clarke M Ghersi D Liberati A Petticrew M Preferred reporting items for systematic review and meta-analysis protocols (prisma-p) 2015: elaboration and explanation BMJ 2015 350 g7647 10.1136/bmj.g7647 25555855

18. Moher D Shamseer L Clarke M Ghersi D Liberati A Petticrew M Preferred reporting items for systematic review and meta-analysis protocols (prisma-p) 2015 statement Syst Rev 2015 4 1 1 10.1186/2046-4053-4-1 25554246 PMC4320440

19. Thomas J, Brunton J, Graziosi S. Eppi-reviewer 4.0: software for research synthesis. London: Social science research unit, Institute of education, University of london; 2010. Available from: https://eppi.ioe.ac.uk/cms/default.aspx?tabid=1913 . Accessed 31 May 2019.

20. Moher D Liberati A Tetzlaff J Altman DG Prisma Group Preferred reporting items for systematic reviews and meta-analyses: the prisma statement J Clin Epidemiol 2009 62 10 1006 1012 10.1016/j.jclinepi.2009.06.005 19631508

21. Liberati A Altman DG Tetzlaff J Mulrow C Gøtzsche PC JPA I The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration J Clin Epidemiol 2009 62 10 e1 34 10.1016/j.jclinepi.2009.06.006 19631507