Strengths and limitations of this study
For the first time, our systematic reviews and meta-analyses will provide an exhaustive review of autologous cancer cell vaccine studies and will summarise their efficacy in solid and haematological malignancies.
By separating our focus into two systematic reviews, focusing on haematological and solid malignancies, respectively, we will be able to provide concerted summaries, analysis and conclusions highly relevant for each target audience.
Single arm studies with no comparator groups may represent a major limitation for our study, however we have developed a comprehensive plan to address this limitation with a modified Institute of Health Economics tool for assessing risk of bias in such studies.
Introduction
Autologous cancer cell vaccines are a personalised cancer immunotherapy platform
Cancer vaccines are re-emerging as promising therapies with the explosion of antibody mediated immunotherapies, such as checkpoint blockade. Success of immunotherapies is often associated with T-cell responses to mutated tumour epitopes in both preclinical and clinical settings. 1–4 Delivery of one or more tumour-derived peptides alongside appropriate adjuvants is one widely used method of priming this adaptive antitumour response. However, increased mutation rates prevalent in tumours may lead to downregulation of these antigens or reduce their presentation on major histocompatibility complex class I (MHC-I) allowing tumours to escape an oligoclonal generated response. 5 6 As such, for effective cancer therapy, the need to target a broad repertoire of these tumour associated antigens has been identified. 5–7 Although advances in next-generation sequencing has allowed identification of these neoepitopes, more than 95% are unique to each individual, making it difficult to tailor such a therapy suitable for a broad range of patients, and therefore development of alternative personalised cancer vaccines are being undertaken. 4
A polyclonal but antigen agnostic approach includes the use of autologous whole cell tumour vaccines or tumour lysates, which provide the entire repertoire of a patients’ tumour antigens including mutated neoepitopes without the need for identification of individual antigens. 8 9 Briefly, tumour cells are treated to render them non-proliferative via rapid freeze-thaw cycles, heat-shock or irradiation or lysed to produce tumour lysates. 10 11 An analysis of 173 published studies covering over 3000 patients revealed that those receiving whole tumour cell or lysate-based vaccines had objective responses of 8.1%, compared with only 3.6% in those receiving antigen-specific therapy. 8 However, it is worth noting that this analysis included a broad range of therapies, including dendritic cell-based vaccines as well as allogeneic and autologous whole-cell tumour vaccines. While lack of adjuvanticity or issues during manufacturing may be attributed for the difference in efficacy between preclinical and clinical studies, an immunosuppressive human tumour microenvironment may also be a key barrier to a therapeutic effect. 6
Autologous cell vaccines show renewed promise as the field of cancer immunotherapy progresses
Studies involving vaccination of patients with their own cancer cells (autologous cell vaccine) have consistently shown that survival is significantly better in patients that mount an immune response against their tumour cells, suggesting that when an immune response is generated, prognosis is improved. 10 12–16 The strong immunological rationale for cytokine-based whole cell vaccines continues to drive the development of novel experimental approaches and currently two companies, Gradalis and Vaccinogen are pursuing phase III clinical trials based on promising clinical responses in ovarian and colon cancer. 17 18 Specifically, Gradalis is undertaking a phase III randomised clinical trials of Vigil, an autologous ovarian cancer cell vaccine engineered to express granulocyte-macrophage colony-stimulating factor (GM-CSF) in combination with a knock-down of transforming growth factor β (TGFβ; bifunctional shRNA against furin ), in resected patients with ovarian cancer based on promising phase II data. 19 20 Vaccinogen is undertaking a phase IIIb confirmatory study of OncoVAX, an autologous colon cancer vaccine mixed with live Bacillus Calmette Guérin (BCG), in stage II patients with colon cancer following a significant improvement in overall and disease-free survival in this subgroup in a prior phase III study. 21 22
While therapy with haematological malignancies are in earlier phases, successes have been observed in patients with acute myeloid leukaemia (AML), chronic lymphocytic leukaemia (CLL) and multiple myeloma. 23 Combination therapy of GM-CSF-secreting K562 cells with autologous AML cells improved relapse-free survival in patients also receiving autologous haematopoietic cell transplantation and adoptive therapy. 24 A similar treatment strategy (autologous tumour cells with GM-CSF-secreting K562 cells) in CLL patients following hematopoietic stem cell transplantation (HCT) resulted in the development of T-cell immunity against autologous cells compared with patients undergoing HCT alone. 25
Moreover, with the rapidly expanding number of immunotherapies, such as immune checkpoint inhibitors, designed to reverse intratumoural immune suppression, cancer vaccines are further being re-evaluated for their potential in combination trials. 26 27
Research objectives
These systematic reviews and meta-analyses will review the evidence from controlled and uncontrolled interventional studies of autologous cell vaccines administered to patients with cancer. Our reviews will aim to quantify the efficacy of this treatment in terms of clinical response rates, disease free-survival and overall survival of patients. We will also explore any safety concerns related to the treatment and correlative endpoints that predict efficacy. Separate reviews on the use of these vaccines in (1) solid and (2) haematological malignancies will be prepared given the inherent differences in the preparation and treatment of autologous cancer cell vaccines prepared from solid or haematological cancer cells.
Protocol
Our systematic review protocol is reported according to the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols guidelines. 28 The reviews will be carried out in accordance with methodological recommendations from the Cochrane Collaboration, and our protocol has been registered at the International Prospective Registry of Systematic Reviews.
Methods and analysis
Eligibility criteria
Population
Patients with cancer of any age (paediatric or adult) will be included. Patients with any type of cancer and any cancer stage will be included (distinguished only based on solid or haematological malignancies for each review). For studies on patients with solid tumour as well as patients with haematological malignancies, outcomes will only be reported from the patients with the indication of interest for a given systematic review (eg, for the systematic review focusing on haematological malignancies).
Intervention
Any autologous whole cell tumour vaccine (ie, any whole cell product made from the patient’s own cancer cells and re-administered with or without an immunogen). We will be including autologous cell vaccines administered in both the adjuvant setting and the palliative setting.
Comparators
No comparator required.
Outcomes
The primary outcomes of interest in this review are clinical response (complete or overall/objective response) and safety outcomes (adverse events (AEs)). Secondary outcomes include immune response, disease-free survival and overall survival. Tertiary outcomes include correlative biological assays that predict efficacy, health-related quality of life (HRQoL) and health utility measures.
Study design
We will be including any interventional trial (randomised, non-randomised and quasi-randomised trials). Observational studies, case reports and case series will be excluded. Unpublished grey literature, abstracts, conference abstracts, commentaries, letters, reviews and editorials will also be excluded.
Information sources
We will search Medline (OVID interface, including In-Process and Epub Ahead of Print), Embase (OVID interface) and the Cochrane Central Register of Controlled Trials (Wiley interface) from 1947 to 30 July 2018. The search will also be updated prior to submission of the review for publication. ClinicalTrials.gov will be searched to identify ongoing and completed trials. In addition, we will examine reference lists of included studies or relevant reviews identified through the search.
Search strategy
A search strategy will be created in collaboration with an information specialist and a clinical expert in the field, in order to identify all potentially relevant studies. The literature strategies will be developed using keywords related to cancer and autologous whole cell vaccines. A Peer Review of Electronic Search Strategies will be performed by a second information specialist who is not associated with the project. Full-text articles in any language will be considered, and there will be no restriction on year of publication. The search strategy used in Medline can be found in online supplementary appendix 1 . At time of revision of this protocol for publication, we have completed full text review and are embarking on a pilot trial for data extraction ( table 1 ).
Table 1
Timeline of systematic review process
| June 2018 | July 2018 | August 2018 | September 2018 | October 2018 | November 2018 | December 2018 | January 2019 | February 2019 | March 2019 | April 2019 | May 2019 | June 2019 | July 2019 | August 2019 | September 2019 | October 2019 | November 2019 | December 2019 | January 2020 | February 2020 | March 2020 | April 2020 | May 2020 | June 2020 | July 2020 | August 2020 | September 2020 | October 2020 | |
| Write protocol | |||||||||||||||||||||||||||||
| Perform literature search | |||||||||||||||||||||||||||||
| Screen abstracts | |||||||||||||||||||||||||||||
| Full text review | |||||||||||||||||||||||||||||
| Register protocol on PROSPERO | |||||||||||||||||||||||||||||
| Submit protocol for publication | |||||||||||||||||||||||||||||
| Data extraction pilot | |||||||||||||||||||||||||||||
| Data extraction | |||||||||||||||||||||||||||||
| Repeat search on publications since 30 July to 2018 | |||||||||||||||||||||||||||||
| Data analysis | |||||||||||||||||||||||||||||
| Draft manuscripts | |||||||||||||||||||||||||||||
| Submit manuscripts for publication |
Selection process
The literature search results will be uploaded to Distiller Systematic Review Software (Distiller, Evidence Partners, Ottawa, Canada). Distiller is a cloud-based software program that allows for transparent and reproducible work required for an accurate review. Two reviewers will independently screen the titles and abstracts from the search results using the predefined eligibility criteria. For all titles that appear to meet the inclusion criteria or where there is any uncertainty, we will access the full text. Two reviewers will then assess the eligibility of the full-text articles. Discrepancies between the reviewers will be resolved by discussion or with a third-party member if a consensus cannot be established. The study selection process will be documented and reported using a selection flow diagram, as recommended by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 29 ( online supplementary appendix 2 ).
Data collection process
Standardised drafts of data extraction forms will be designed to extract all data items of interest from the included studies. The drafts of the data extraction form will be used to inform the development of the online abstraction tool using DistillerSR. Data will be extracted independently by two reviewers. Disagreements between reviewers will be resolved by discussion or with a third-party member if a consensus cannot be reached.
Data items
Data pertaining to the publication details (authors, year of publication, journal, location), study population characteristics (age, sex, type of malignancy, stage of cancer, previous treatments), intervention characteristics (method of autologous tumour cell harvest, feasibility of vaccine creation (ie, number of patients treated over number of patients harvested), dose and number of treatments, schedule of administration, manipulation of autologous cells (eg, radiation, freezing), adjuvant coadministration, product release assay used for safety and potency, central vs disseminated manufacturing of the vaccine), study design (methods, setting, sample size, number of centres), outcomes of interest and risk of bias details will be collected.
Outcomes and prioritisation
Primary outcomes
Clinical response will be defined by the type of disease (ie, solid or haematological malignancy). Response and the definition of response will be collected for each study. If complete response is not feasible, secondary response outcomes (when available) will be reported using best overall response. Best overall response will be defined according to the response evaluation criteria in solid tumours guidelines where patients will be assigned to one of the following categories: complete response, partial response, stable disease, progression or non-evaluable for response. 30 31 Outcomes for haematological malignancies will be collected based on standard criteria.
We will also capture and report any provided safety data including AEs to evaluate the safety of autologous cell vaccines. Definition of AEs (if provided) will be collected for each study and AEs will be grouped by severity and the organ system affected. Data on serious AEs will also be collected and will include events that are life threatening or result in death, hospitalisation or prolongation of hospitalisation (not for routine procedures) and/or persistent or serious disability or incapacity. 32
Secondary outcomes
Adaptive immune responses to autologous tumours when characterised by standard Interferon-γ enzyme-linked immunosorbent spot (ELISPOT), delayed-type hypersensitivity responses or flow cytometric analysis of intracellular cytokine production following antigen specific stimulation will be recorded and reported.
Additionally we will report disease-free survival (length of time after treatment initiation that the patient survives without any detectable cancer), progression-free survival the length of time during and after the treatment of a disease that a patient lives with the disease but it does not get worse) and overall survival (time from the start of treatment to the time of death from any cause).
Tertiary outcomes
Tertiary outcomes of interest include patient experience, HRQoL and healthy utility measures. Tertiary outcomes collected at any time point during the study will be considered.
Patient experience: a combination of different dimensions including patient satisfaction, expectations and outcomes throughout the duration of clinical treatment.
HRQoL : a multidimensional concept encompassing an individual’s self-perceived health status.
Health utility: health utility reflects the preference values patients ascribe to their overall health status. It is a global measure of health status that summarises the effect of treatment into a value between 0 (death) and 1 (perfect health).
Due to the variety of measures for patient experience in clinical trials, all validated measures of HRQoL and health utility will be considered.
Risk of bias assessment
Randomised controlled trials (RCTs) that met inclusion criteria will be assessed for risk of bias using the Cochrane Risk of Bias tool for RCTs. 33 Non-RCTs will be assessed for risk of bias using the appropriate Cochrane Risk Of Bias In Non-randomized Studies tool. 34 As no standard tool exists to assess the risk of bias for single-arm interventional studies, we will use a modified Institute of Health Economics (IHE) risk of bias tool for case series studies with items incorporated from the Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. 35 This modified IHE tool has been previously employed by our group and includes, as previously described, ‘assessment of study objective, design, study population, intervention and cointerventions, outcome measures (eg, blinding, incomplete outcome data such as participants lost to follow-up, selective outcome reporting), statistical analysis, results and conclusions and conflicts of interest’. 31 Risk of bias will be assessed by two independent reviewers. Disagreements will be resolved first by discussion of by consulting a third-party member. Graphic representations of risk of bias within and across studies will be conducted using RevMan V.5.3 (Cochrane Collaboration, Oxford, UK).
Data analysis
Studies will be pooled using Comprehensive Meta-Analyst (V.3; Biostat, Engelwood, NJ, USA). Dichotomous outcomes will be analysed using a random effects meta-analysis based on the DerSimonian Laird model, and reported as proportions or ORs with accompanying 95% confidence intervals (CIs). Continuous outcomes will be analysed using a random effects inverse variance meta-analysis, and reported as weighted mean difference or standardised mean difference (with 95% CI), depending on the nature of the data available. Time to event data will be presented as HRs, with accompanying 95% CIs. Non-quantitative data will be presented descriptively.
Statistical heterogeneity will be assessed using the Cochrane I 2 statistic, as well as the χ 2 test of the Cochrane Q statistic, depending on the analysis method. The thresholds for interpretation of I 2 are as follows: 0%–40% low heterogeneity, 30%–60% moderate heterogeneity, 50%–90% may represent substantial heterogeneity and 75%–100% is considerable heterogeneity. 36 If there is considerable heterogeneity (75%–100%) then sources of heterogeneity will be explored through subgroup and sensitivity analyses. The presence of publication bias will be investigated using funnel plots.
Dealing with missing data
The authors of the individual studies included in our review will be contacted to obtain relevant missing data.
Subgroup and sensitivity analyses
Planned subgroup analyses include analysis by disease indication, analysis by therapy indication (presence or absence of active disease at time of vaccination), type of vaccine adjuvant used (GM-CSF vs other), use of an infectious agent, single versus multidose regimen, fresh versus frozen vaccine and vaccines manufactured at a central versus several distributed locations.
Reporting of review
The findings of these systematic reviews will be reported according to the PRISMA statement. 29 The completed checklist will be provided as supplementary material ( online supplementary appendix 2 ).
Confidence in cumulative evidence
Grading of Recommendations, Assessment, Development and Evaluations (GRADE) will be used to systematically and comprehensively evaluate the quality of intervention. This is recognised as a highly effective method that compares intervention efficacy and quality to clinical recommendations. The quality of evidence will be assessed across the domains of risk of bias, consistency, directness, precision and publication bias, and will be assigned one of four GRADE scores (0–4) representing high, moderate, low or very low-quality evidence. 37 High quality evidence reflects a high degree of confidence in the estimate of effect, and vice versa.
Patient and public involvement
There was no patient or public involvement in the planning or reporting of this protocol.
Ethics and dissemination
Ethics approval is not required for this systematic review protocol as the review will solely use previously published literature. Results will be submitted to peer-reviewed journals for publication and presented to relevant stakeholders and scientific meetings.
Amendments
If amendments to the protocol are required, description, rationale and date of amendments will be posted to PROSPERO.
Supplementary Material
Reviewer comments
Author's manuscript
Twitter: @manojlalu
STK, JM, ML and RCA contributed equally.
Contributors: ML is the guarantor. Conceptualisation: ML, DAF, RCA, NK, J-SD, MAK, NF. Methodology: JM, ML, DAF. Writing (original draft): STK, JM. Writing (review and editing): STK, JM, NF, ML, DAF, RCA, NK, J-SD, MAK, DB. Supervision and funding acquisition: ML, DAF, RCA, NK and J-SD. Project administration: JM and NF.
Funding: The study is funded through a BioCanRx Catalyst Grant (RCA and J-SD).
Competing interests: None declared.
Patient and public involvement: Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication: Not required.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
1 Lennerz V , Fatho M , Gentilini C , et al The response of autologous T cells to a human melanoma is dominated by mutated neoantigens . Proc Natl Acad Sci U S A 2005 ; 102 : 16013 – 8 . 10.1073/pnas.0500090102 16247014 PMC1266037
2 Lauss M , Donia M , Harbst K , et al Mutational and putative neoantigen load predict clinical benefit of adoptive T cell therapy in melanoma . Nat Commun 2017 ; 8 : 8 . 10.1038/s41467-017-01460-0 29170503 PMC5701046
3 van Rooij N , van Buuren MM , Philips D , et al Tumor exome analysis reveals neoantigen-specific T-cell reactivity in an ipilimumab-responsive melanoma . J Clin Oncol 2013 ; 31 : e439 – 42 . 10.1200/JCO.2012.47.7521 24043743 PMC3836220
4 Castle JC , Kreiter S , Diekmann J , et al Exploiting the mutanome for tumor vaccination . Cancer Res 2012 ; 72 : 1081 – 91 . 10.1158/0008-5472.CAN-11-3722 22237626
5 Hu Z , Ott PA , Wu CJ Towards personalized, tumour-specific, therapeutic vaccines for cancer . Nat Rev Immunol 2018 ; 18 : 168 – 82 . 10.1038/nri.2017.131 29226910 PMC6508552
6 Mellman I , Coukos G , Dranoff G Cancer immunotherapy comes of age . Nature 2011 ; 480 : 480 – 9 . 10.1038/nature10673 22193102 PMC3967235
7 Britten CM , Singh-Jasuja H , Flamion B , et al The regulatory landscape for actively personalized cancer immunotherapies . Nat Biotechnol 2013 ; 31 : 880 – 2 . 10.1038/nbt.2708 24104749
8 Neller MA , López JA , Schmidt CW Antigens for cancer immunotherapy . Semin Immunol 2008 ; 20 : 286 – 95 . 10.1016/j.smim.2008.09.006 18951039
9 Keenan BP , Jaffee EM Whole cell vaccines--past progress and future strategies . Semin Oncol 2012 ; 39 : 276 – 86 . 10.1053/j.seminoncol.2012.02.007 22595050 PMC3356993
10 Chiang CL-L , Coukos G , Kandalaft LE Whole tumor antigen vaccines: where are we? Vaccines 2015 ; 3 : 344 – 72 . 10.3390/vaccines3020344 26343191 PMC4494356
11 Chiang CL-L , Benencia F , Coukos G Whole tumor antigen vaccines . Semin Immunol 2010 ; 22 : 132 – 43 . 10.1016/j.smim.2010.02.004 20356763 PMC3119500
12 de Gruijl TD , van den Eertwegh AJM , Pinedo HM , et al Whole-Cell cancer vaccination: from autologous to allogeneic tumor- and dendritic cell-based vaccines . Cancer Immunol Immunother 2008 ; 57 : 1569 – 77 . 10.1007/s00262-008-0536-z 18523771 PMC2491427
13 Baars A , Claessen AM , van den Eertwegh AJ , et al Skin tests predict survival after autologous tumor cell vaccination in metastatic melanoma: experience in 81 patients . Ann Oncol 2000 ; 11 : 965 – 70 . 10.1023/A:1008363601515 11038032
14 Harris JE , Ryan L , Hoover HC , et al Adjuvant active specific immunotherapy for stage II and III colon cancer with an autologous tumor cell vaccine: eastern cooperative Oncology Group study E5283 . J Clin Oncol 2000 ; 18 : 148 . 10.1200/JCO.2000.18.1.148 10623705
15 Senzer N , Barve M , Kuhn J , et al Phase I trial of "bi-shRNAi(furin)/GMCSF DNA/autologous tumor cell" vaccine (FANG) in advanced cancer . Mol Ther 2012 ; 20 : 679 – 86 . 10.1038/mt.2011.269 22186789 PMC3293620
16 Campbell CT , Gulley JL , Oyelaran O , et al Humoral response to a viral glycan correlates with survival on PROSTVAC-VF . Proc Natl Acad Sci U S A 2014 ; 111 : E1749 – 58 . 10.1073/pnas.1314722111 24733910 PMC4035954
18 Vaccinogen Inc OncoVAX , 2015 Available: http://www.vaccinogeninc.com/oncovax [Accessed 28 Mar 2016 ].
19 U.S. National Library of Medicine NCT01061840 : trial of Bi-shRNA-furin and granulocyte macrophage colony stimulating factor (GMCSF) augmented autologous tumor cell vaccine for advanced cancer , 2010 Available: https://clinicaltrials.gov/ct2/show/NCT01061840
20 Oh J , Barve M , Matthews CM , et al Phase II study of Vigil® DNA engineered immunotherapy as maintenance in advanced stage ovarian cancer . Gynecol Oncol 2016 ; 143 : 504 – 10 . 10.1016/j.ygyno.2016.09.018 27678295
21 U.S. National Institutes of Health NCT02448173 : a multicenter study of active specific immunotherapy with OncoVax® in patients with stage II colon cancer . Available: https://clinicaltrials.gov/ct2/show/NCT02448173 [Accessed 27 Jun 2018 ].
22 Vermorken JB , Claessen AM , van Tinteren H , et al Active specific immunotherapy for stage II and stage III human colon cancer: a randomised trial . Lancet 1999 ; 353 : 345 – 50 . 10.1016/S0140-6736(98)07186-4 9950438
23 Nahas MR , Rosenblatt J , Lazarus HM , et al Anti-Cancer vaccine therapy for hematologic malignancies: an evolving era . Blood Rev 2018 ; 32 : 312 – 25 . 10.1016/j.blre.2018.02.002 29475779
24 Borrello IM , Levitsky HI , Stock W , et al Granulocyte-macrophage colony-stimulating factor (GM-CSF)-secreting cellular immunotherapy in combination with autologous stem cell transplantation (ASCT) as postremission therapy for acute myeloid leukemia (AML) . Blood 2009 ; 114 : 1736 – 45 . 10.1182/blood-2009-02-205278 19556425 PMC2738565
25 Burkhardt UE , Hainz U , Stevenson K , et al Autologous CLL cell vaccination early after transplant induces leukemia-specific T cells . J Clin Invest 2013 ; 123 : 3756 – 65 . 10.1172/JCI69098 23912587 PMC3754265
26 Parmiani G , Maccalli C , Maio M Integrating immune checkpoint blockade with Anti-Neo/Mutated antigens reactivity to increase the clinical outcome of immunotherapy . Vaccines 2015 ; 3 : 420 – 8 . 10.3390/vaccines3020420 26343195 PMC4494352
27 Kleponis J , Skelton R , Zheng L Fueling the engine and releasing the break: combinational therapy of cancer vaccines and immune checkpoint inhibitors . Cancer Biol Med 2015 ; 12 : 201 – 8 . 10.7497/j.issn.2095-3941.2015.0046 26487965 PMC4607816
28 Moher D , Shamseer L , Clarke M , et al Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement . Syst Rev 2015 ; 4 : 1 . 10.1186/2046-4053-4-1 25554246 PMC4320440
29 Moher D , Liberati A , Tetzlaff J , et al Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . BMJ 2009 ; 339 : b2535 . 10.1136/bmj.b2535 19622551 PMC2714657
30 Nishino M , Jagannathan JP , Ramaiya NH , et al Revised RECIST guideline version 1.1: what oncologists want to know and what radiologists need to know . AJR Am J Roentgenol 2010 ; 195 : 281 – 9 . 10.2214/AJR.09.4110 20651182
31 Grigor EJM , Fergusson DA , Haggar F , et al Efficacy and safety of chimeric antigen receptor T-cell (CAR-T) therapy in patients with haematological and solid malignancies: protocol for a systematic review and meta-analysis . BMJ Open 2017 ; 7 : e019321 . 10.1136/bmjopen-2017-019321 PMC5988064 29288188
32 CFR Code of federal regulations title 21 volume 5 (21CFR312.32). US FDA , 2018 Available: https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfcfr/CFRSearch.cfm?fr=312.32 [Accessed 23 Oct 2018 ].
33 Higgins JPT , Altman DG , Gøtzsche PC , et al The Cochrane collaboration's tool for assessing risk of bias in randomised trials . BMJ 2011 ; 343 : d5928 . 10.1136/bmj.d5928 22008217 PMC3196245
34 Sterne JA , Hernán MA , Reeves BC , et al ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions . BMJ 2016 ; 355 : i4919 . 10.1136/bmj.i4919 27733354 PMC5062054
35 Guo B , Moga C , Harstall C , et al A principal component analysis is conducted for a case series quality appraisal checklist . J Clin Epidemiol 2016 ; 69 : 199 – 207 . 10.1016/j.jclinepi.2015.07.010 26307459
36 Higgins JP , Green S Cochrane Handbook for systematic reviews of interventions . 5.02 edn , 2009 .
37 Guyatt GH , Oxman AD , Vist GE , et al Grade: an emerging consensus on rating quality of evidence and strength of recommendations . BMJ 2008 ; 336 : 924 – 6 . 10.1136/bmj.39489.470347.AD 18436948 PMC2335261