Development and Validation of a Prediction Model for 1-Year Mortality in Patients With a Hematologic Malignancy Admitted to the ICU.

Текст сохранён. Часть иллюстраций пока доступна только в первоисточнике; загрузка будет проверена повторно.

KEY POINTS

Question: We aimed to develop and validate a prediction model for 1-year mortality in hematologic malignancy patients acutely admitted to the ICU.

Findings: Our model included 13 predictors and was streamlined to 8 predictors for clinical usability. Key predictors were active disease, age, previous hematopoietic stem cell transplantation, mechanical ventilation, lowest platelet count, acute kidney injury, maximum heart rate, and type of malignancy. Internal–external validation by center demonstrated acceptable discrimination ( C -statistic 0.70).

Meaning: This model improved prediction of 1-year mortality in hematologic malignancy patients in the ICU, offering potential clinical value in shared decision-making regarding the continuation of ICU care and end-of-life considerations.

In the last decades, there has been a reduction in the death rate of patients with a hematologic malignancy who required ICU admission ( 1 – 5 ). Nevertheless, mortality rates remain substantial in this patient category, necessitating accurate risk estimates to support decision-making regarding ICU management. Existing prediction models for the general ICU population, such as Acute Physiology and Chronic Health Evaluation II (APACHE II), may inaccurately predict the mortality risk in patients with a hematologic malignancy because this specific diagnosis was not incorporated in their development ( 1 , 6 – 9 ).

Several factors are postulated to be associated with mortality in hematologic oncology patients admitted to the ICU. These include the need for mechanical ventilation, the presence of active disease, the specific type of hematologic malignancy, hematopoietic stem cell transplantation (HSCT), total bilirubin levels, platelet count, creatinine levels or the presence of acute kidney injury (AKI), use of vasopressors, and sepsis ( 5 , 7 , 10 – 13 ). Integrating these factors into a prognostic model could be helpful for clinical decision-making regarding the continuation of ICU treatment, and optimal utilization of ICU facilities, and it may provide patients and their families with realistic information on the probability of survival of an ICU episode and thereafter ( 13 ).

As of our current understanding, there is no validated model available that incorporates all postulated factors to predict the long-term survival of ICU patients with a hematologic malignancy. A recent study introduced an adjusted version of the modified early warning score (MEWS) pre-ICU to predict in-hospital mortality and identify deteriorating patients with a hematologic malignancy who might benefit from ICU admission ( 14 ). Another retrospective study compared prediction models for in-hospital mortality in patients with hematologic malignancies admitted to the ICU, acting as a proof-of-concept study for support vector machine modeling ( 15 ). This present study aimed to develop and validate a multifactorial model to predict 1-year mortality in patients with a hematologic malignancy after being admitted to the ICU for 24 hours.

MATERIALS AND METHODS

Study Population

This multicenter cohort study used data from the HEMA-ICU Study Group cohort, as described previously ( 3 ). In reporting this study, we adhered to the Transparent Reporting of a Multivariable Model for Individual Prognosis or Diagnosis guidelines for prognostic modeling studies ( Supplement 2 , http://links.lww.com/CCX/B341 ) ( 16 ). The cohort consisted of prospectively collected data from the Dutch National Intensive Care Evaluation Registry and the Diagnosis Treatment Combination Healthcare Cost and Utilization databases from five University Medical Centers in The Netherlands from December 2002 to August 2015. Dutch regulations encourage the inclusion of all ICU admissions in this registry. The registration captured data of the first 24 h postadmittance to the ICU, excluding clinical data before ICU admittance. The participating centers were The Amsterdam University Medical Center (location AMC), Erasmus Medical Center Rotterdam, Leiden University Medical Center, Radboud University Medical Center Nijmegen, and the University Medical Center Groningen. The study encompassed only hematologic malignancy patients with an acute medical or surgical indication who were acutely admitted to the ICU for the first time. We excluded patients discharged alive within 24 hours following ICU admission to dismiss those admitted to the ICU solely for diagnostic procedures.

Ethical Considerations

The institutional review board of the University Medical Center Groningen granted ethical approval on August 23, 2016 (reference number METc 2016.396); Title: Patients with a Hematologic Malignancy Admitted on the Intensive Care (HEMA-ICU). The study followed the Helsinki Declaration of 1975.

Outcome and Predictors

The primary outcome of our study was overall 1-year mortality following ICU admission, which we considered the most relevant endpoint to address the difficulty of early mortality after ICU discharge. The extensive outpatient follow-up in this cohort supported the use of manual chart review as an accurate method for establishing 1-year mortality.

A literature search and expert opinion guided the selection of 22 potential predictors of mortality in hematologic malignancy patients admitted to the ICU. We collected the following predictors 24 hours after ICU admittance: age, sex, emergency surgical intervention, sepsis, maximum heart rate, maximum respiratory rate, Glasgow Coma Scale, lowest WBC count, lowest platelet count, bilirubin plasma level, vasopressor use, mechanical ventilation, highest fraction-inspired oxygen (F io 2 ), partial arterial oxygen pressure (Pa o 2 ), Pa o 2 /F io 2 ratio, AKI, cerebrovascular accident (CVA), cardiopulmonary resuscitation. Through manual chart review, we gathered data for the variables, type of hematologic malignancy, active disease status, prior allogeneic HSCT, and presence of neutropenia. Supplement 1 ( http://links.lww.com/CCX/B341 ) offers a detailed description and definition of the potential predictors ( 3 ).

Model Development

First, we winsorized potential predictors for extreme values using three times the interquartile range for the upper and lower limits and labeled implausible values as missing ( 17 ). We considered several transformations (polynomials, restricted cubic splines, logarithmic) for continuous variables and included a transformation when the fit improved by 50% as determined by the chi-square test, coupled with a p value of less than 0.05. We compared models with transformations against a baseline model without any and preferred the baseline model for its simplicity and interpretability. We opted for transformations only when they offered clear improvements in performance.

We did not consider statistical interaction terms for the conciseness of the model. When necessary to ensure adequate group size, we combined categories within categorical variables according to their frequency distributions.

Where possible, we completed missing values by a search in the electronic patient record. We examined the data for remaining missing values and analyzed the different mechanisms of missingness, including missing completely at random (MCAR), missing at random (MAR), and missing not a random. Because a complete case analysis would lead to a loss of information, we decided to use Multivariate Imputation by Chained Equations to fill in missing values when the supposed mechanism was MCAR or MAR. We created 20 imputed datasets and combined model estimates over these datasets ( 17 ).

We specified a complete model from 22 predictors across the 20 imputed datasets and performed univariable and multivariable logistic regression analyses. We considered our sample size sufficient for this number of predictors; with over 400 patients surviving after 1 year and 22 variables, the event per variable ratio exceeded 18. For model reduction, we applied a stepwise backward selection method with Akaike’s Information Criterion. This is equivalent to selection with a p value of less than 0.157 for predictors with 1 degree of freedom.

We evaluated model performance in terms of discrimination and calibration. For discrimination, we used the C -statistic, which is equivalent to the area under the receiver operator characteristic curve for predicting binary outcomes. Calibration assessment involved calibration plots and quantitative measures such as calibration intercept, calibration slope, and Brier score. To improve the model’s generalizability, we reduced overfitting by a bootstrapping procedure. This involved estimating the optimism in the initial model via 200 bootstraps and shrinking the regression coefficients in the final model accordingly.

We also performed backward selection with p values of 0.05, 0.01, and 0.005 to retain only the most significant predictors, aiming to develop a potentially clinically more relevant model. The performance of these models in comparison to our original model determined the optimal balance of predictor inclusion and model efficacy.

We conducted an internal–external cross-validation procedure by the center to obtain an impression of external validity. In this procedure, we omitted each medical center from the development dataset once and repeated the modeling steps: transformation, imputation, backward selection, and shrinking by 200 bootstraps on the data of the four remaining centers to develop a new model in each iteration. Next, we validated this model on the data of the left-out center. The final reported model used the completed dataset to maximize the use of the available information ( 17 ). Decision curve analysis evaluated clinical usefulness. Because of the high heterogeneity inpatient age and potential life wishes, we considered a threshold probabilities range of 60–100% predicted 1-year mortality as clinically relevant for a decision regarding discontinuation of ICU care. We compared these decision curves with the APACHE II score and a currently proposed greater than or equal to two organ failure decision model ( 9 ). We created a nomogram as the final model presentation. In a nomogram each predictor value corresponds to a regression weight and the total sum of points equals the linear predictor ( 17 , 18 ). Analyses were performed in R statistical software (4.3.2; R Core Team 2023) with packages: tableone, rms, mice, psfmi, and The Calibration Curve package.

RESULTS

Study Population

Our dataset included 1097 patients admitted to the ICU between 2002 and 2015. Of these, 682 (62%; 95% CI, 59–65) died within 1 year of admission. Table 1 contains a summary of patient characteristics and missing data. In total, there were 24,134 datapoints (1097 × 22), of which 2135 missing (8.8%). The highest number of missing values occurred in the predictors: Glasgow Coma Scale (437 missing—40%), bilirubin level (357 missing—33%), and F io 2 (323 missing—29%). We classified the missingness of the Glasgow Coma Scale as not at random, noting a pattern where missing values presumably corresponded with the magnitude of the predictor. Consequently, we chose not to impute these missing values. The primary outcome had no missing values, so there was no censoring. Given this complete follow-up, we conducted a logistic regression analysis.

The mean age was 55 ( sd 15) years. The mean APACHE II score was 22 ( sd 8), indicating a critically ill study population. According to the APACHE II score, the expected postoperative in-hospital mortality would be 30%, and the nonoperative expected mortality 40% ( 19 ).

In our population, 30-day mortality accounted for 480 cases (44%), and in-hospital mortality comprised 501 cases (46%). An additional 181 deaths (27%) occurred between hospital discharge and 1 year. The discriminative ability of APACHE II for 1-year mortality was modest, with a C -statistic of 0.61 (95% CI, 0.57–0.65). Also, the calibration of APACHE II in predicting 1-year mortality was inaccurate ( Fig. E1 , http://links.lww.com/CCX/B341 ).

Model Development

Multiple transformations for continuous variables and 1-year mortality indicated mainly that WBC count in the lower range (0–8 × 10 9 /L) and higher range (> 15 × 10 9 /L) had an association with mortality. However, the WBC transformation did not enhance the model’s performance. Consequently, we opted not to include it in the final model, prioritizing simplicity and interpretability over unnecessary complexity without evident benefits.

Transformations also indicated that platelet counts within the lower range (0–150 × 10 9 /L) possibly had a stronger association with mortality than platelet counts in the higher range (> 150 × 10 9 /L). However, due to the absence of a 50% improvement in the chi-square test, we opted not to implement this transformation ( Fig. E2 , http://links.lww.com/CCX/B341 ). In a univariable analysis factors most strongly associated with 1-year mortality included mechanical ventilation, lowest platelet count, and maximum heart rate ( Table E1 , http://links.lww.com/CCX/B341 ). Mechanical ventilation, active disease, and age were the most important predictors in a multivariable analysis (Table E1, http://links.lww.com/CCX/B341 ). After stepwise backward selection, 13 predictors were associated with 1-year mortality. The eight strongest predictors were mechanical ventilation (χ 2 53.5), lowest platelet count (χ 2 15.7), active disease (χ 2 18.2), HSCT (χ 2 11.1), AKI (χ 2 8.6), maximum heart rate (χ 2 17.2), age (χ 2 16.0), and type of hematologic malignancy (χ 2 11.0). Other predictors were CVA, urgent surgery, lowest WBC, bilirubin, and maximum F io 2 (Table E1, http://links.lww.com/CCX/B341 ).

The 13-predictor model demonstrated satisfactory discrimination with a pooled C -statistic of 0.70 (95% CI, 0.63–0.77) at internal–external validation across centers. The 8-predictor model had the same discrimination at internal–external validation across centers (pooled C -statistic 0.70; 95% CI, 0.63–0.77). Both the 13-predictor model ( Fig. E3 A , http://links.lww.com/CCX/B341 ) and 8-predictor model ( Fig. E3 B , http://links.lww.com/CCX/B341 ) displayed reasonable calibration across centers. The calibration of the 8-predictor model (intercept −0.02 slope 0.90, Brier 0.07) was also comparable to that of the 13-predictor model (intercept −0.03 slope 0.91, Brier 0.10) ( Fig. 1 ).

The decision curve analysis ( Fig. 2 ) demonstrated net benefit for the 13- and 8-predictor model for our clinically relevant chosen threshold probability of 60–100% predicted mortality, offering more net benefit than the APACHE II, or a greater than or equal to two organ failure decision model. The nomogram ( Fig. 3 ) displays the relative importance of the thirteen predictors, an 8-predictor nomogram can be found in the supplements ( Fig. E4 , http://links.lww.com/CCX/B341 ). These nomograms can be used to quickly predict the 1-year risk of death for an individual patient with a hematologic malignancy 24 h after admittance to the ICU.

For instance, for a patient on a mechanical ventilator, we draw a vertical line from “yes” to the line marked “points,” resulting in a score of 5.8 points for a ventilated patient. This process is repeated for all 13 predictors, with the total points corresponding to the 1-year mortality probability.

Consider a 47-year-old patient (age 2.5 points) with an acute myeloid leukemia (4 points) in the premedical history, who is mechanically ventilated (5.8 points) with 40% F io 2 (1 point), without active disease (0 points), and no allogeneic HSTC (0 points), no urgent surgery (3 points), no CVA (0 points), a platelet count of 40 (10 9 /L) (7 points), bilirubin of 70 (1.5 points) a maximum heart rate of 140 beats per minute (5 points), lowest WBC 10 (1 point), and no AKI. The total points amount to 30.8, corresponding to a 1-year mortality probability of around 65%. If this person had an active disease (additional 3.5 points), the total points would increase to 34.3, and the probability of dying within 1 year would rise to 77%.

DISCUSSION

In this study, we propose a concise clinical prediction model based on 13 predictors to predict 1-year mortality for patients with hematologic malignancy who are acutely admitted to the ICU. This model can be used 24 h after ICU admission and outperforms the APACHE II model for this population and time period. For parsimony, clinical relevance, and usefulness, we streamlined the 13-predictor model via backward selection with a p value of 0.005 and found 8 variables to be highly significant predictors of 1-year mortality. These were mechanical ventilation, lowest platelet count, active disease, previous allogeneic HSCT, AKI, age, type of malignancy, and maximum heart rate within the first 24 hours after ICU admission.

In clinical settings when a quick impression of prognosis is wanted, the 8-variable model might be preferred over the 13-variable model for simplicity. Although the 13-variable model provides a slightly higher predictive accuracy this comes at the cost of increased model complexity and potential overfitting. The 8-variable model, with its clear focus on the most significant predictors, may offer better generalizability in that way.

The 1-year mortality rate of 62% observed in this cohort aligns with previously reported ICU mortality rates of patients with a hematologic malignancy ( 1 , 6 ). Our model demonstrated satisfactory performance, although it also indicates that predicting longer-term outcomes, such as 1-year mortality, is challenging. The calibration was adequate overall and superior to the APACHE II score. To our knowledge, this is the first prediction model specifically for patients with a hematologic malignancy admitted to the ICU. One recent study presented a prediction model based on the MEWS; however, their model only predicted in-hospital mortality, as does APACHE II ( 14 ). Also, the model is based solely on single-center data, and a split sample analysis was used. This could have led to bias and overfitting. The difference with our study is that we developed a model to predict 1-year mortality 24 hours after ICU admission and used multicenter data over a more extended period. Instead of a split sample, we evaluated performance through internal–external validation and used all the available data for the final model. Although our C -statistic is marginally lower, it might present a more accurate reflection of reality. In an earlier retrospective study, we created a classification and regression tree analysis for 1-year survival based on a combination of organ failure. Although a regression tree is a simple and intuitive prediction model, it requires dichotomizing continuous variables, which can lead to loss of information ( 3 , 17 ).

Additional strengths of our model are the selection of predictors based on previous research ( 5 , 7 , 10 , 12 , 13 ) and the use of a multicenter dataset with consecutive patients in different hospitals. This reflects a real-world population of patients with a hematologic malignancy admitted to the ICU, limits the risk of overfitting, and increases the robustness of the proposed model ( 17 , 20 ). Our methodology is more interpretable than more complex machine learning methods like neural networks and random forests. Such approaches may be promising in fields like image analysis but tend to have limited advantages in classical prediction problems, primarily because of their substantial data requirements ( 21 ). Finally, the decision curve analysis showed net benefit in our proposed clinically relevant threshold probability of 60–100% predicted 1-year mortality. This threshold probability aligns with a current proposed treatment decision model of ICU care for patients with a hematologic malignancy, where greater than or equal to two organ failures lead to a time-limited trial of critical care or discontinuation of critical care ( 9 ). In the future, our model could help identify patients with a poor prognosis during a time-limited trial of critical care.

Clinical prediction models are becoming increasingly important in facilitating shared clinical decision-making and personalized medicine in the current complex medical environment. Our model aims to help ICU clinicians optimize their daily practice regarding patients with hematologic malignancy in the ICU. However, before its implementation, external validation is necessary and we should realize that performance may depend on local practice ( 22 ).

Our cohort covers 2002 until 2015, and this era may not reflect the current situation where Chimere Antigen Receptor T cell treatment evolved. Otherwise, we note no significant changes in the treatment of patients with hematologic malignancies or the treatment of these patients during ICU stay. Another limitation is that there were missing variables in some key predictors, such as maximum heart rate and lowest platelet count. Although we used an advanced imputation method, the complete recording might have contributed to a better final prediction model.

CONCLUSIONS

One-year mortality of patients with a hematologic malignancy acutely admitted to the ICU can be reasonably predicted using 13 predictors of which the 8 most important ones are: active disease, age, previous allogeneic HSCT, mechanical ventilation, lowest platelet count, AKI, type of malignancy and maximum heart rate within the first 24 hours after ICU admission. After further validation, the nomogram may support shared decision-making regarding the potential discontinuation of ICU care 24 hours after admittance.

Supplementary Material

Footnotes

Dr. Boldingh was involved in conceptualization, data curation, formal analysis, methodology—creation of the model, visualization, and original draft preparation. Drs. Arbous was conceptualization, investigation—data collection, writing—review and editing. Drs. Biemond, Blijlevens, van Bommel, Hilkens, Kusadasi, Muller, and de Vries were involved in data curation, investigation—data collection, writing—review and editing. Steyerberg was conceptualization, methodology—the creation of the model, writing—review and editing. van den Bergh was conceptualization, investigation—data collection, writing—review and editing, and supervision.

Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal’s website ( http://journals.lww.com/ccejournal ).

The authors have disclosed that they do not have any potential conflicts of interest.

REFERENCES

1. Asdahl PH, Christensen S, Kjærsgaard A, et al. : One-year mortality among non-surgical patients with hematological malignancies admitted to the intensive care unit: A Danish nationwide population-based cohort study. Intensive Care Med 2020; 46:756–765

2. Kondakci M, Reinbach MC, Germing U, et al. : Interaction of increasing ICU survival and admittance policies in patients with hematologic neoplasms: A single center experience with 304 patients. Eur J Haematol 2019; 102:265–274

3. de Vries VA, Müller MCA, Arbous MS, et al. ; HEMA-ICU Study Group: Long-term outcome of patients with a hematologic malignancy and multiple organ failure admitted at the intensive care. Crit Care Med 2019; 47:e120–e128

4. Azoulay E, Mokart D, Pène F, et al. : Outcomes of critically ill patients with hematologic malignancies: Prospective multicenter data from France and Belgium—a groupe de recherche respiratoire en réanimation onco-hématologique study. J Clin Oncol 2013; 31:2810–2818

5. Lemiale V, Pons S, Mirouse A, et al. : Sepsis and septic shock in patients with malignancies: A Groupe de Recherche Respiratoire en Réanimation Onco-Hématologique Study. Crit Care Med 2020; 48:822–829

6. Ferreyro BL, Scales DC, Wunsch H, et al. : Critical illness in patients with hematologic malignancy: A population-based cohort study. Intensive Care Med 2021; 47:1104–1114

7. Al-Zubaidi N, Shehada E, Alshabani K, et al. : Predictors of outcome in patients with hematologic malignancies admitted to the intensive care unit. Hematol Oncol Stem Cell Ther 2018; 11:206–218

8. de Vries VA, Müller MCA, Sesmu Arbous M, et al. ; HEMA-ICU Study Group: Time trend analysis of long term outcome of patients with haematological malignancies admitted at Dutch intensive care units. Br J Haematol 2018; 181:68–76

9. Bird GT, Farquhar-Smith P, Wigmore T, et al. : Outcomes and prognostic factors in patients with haematological malignancy admitted to a specialist cancer intensive care unit: A 5 yr study. Br J Anaesth 2012; 108:452–459

10. Bisbal M, Darmon M, Saillard C, et al. : Hepatic dysfunction impairs prognosis in critically ill patients with hematological malignancies: A post-hoc analysis of a prospective multicenter multinational dataset. J Crit Care 2021; 62:88–93

11. Biard L, Darmon M, Lemiale V, et al. : Center effects in hospital mortality of critically ill patients with hematologic malignancies. Crit Care Med 2019; 47:809–816

12. Irie H, Otake T, Kawai K, et al. : Prognostic factors in critically ill patients with hematological malignancy admitted to the general intensive care unit: A single-center experience in Japan. J Anesth 2017; 31:736–743

13. Richards S, Wibrow B, Anstey M, et al. : Determinants of 6-month survival of critically ill patients with an active hematologic malignancy. J Crit Care 2016; 36:252–258

14. Lee JR, Jung YK, Kim HJ, et al. : Derivation and validation of modified early warning score plus SpO 2 /FiO 2 score for predicting acute deterioration of patients with hematological malignancies. Korean J Intern Med 2020; 35:1477–1488

15. Verplancke T, Van Looy S, Benoit D, et al. : Support vector machine versus logistic regression modeling for prediction of hospital mortality in critically ill patients with haematological malignancies. BMC Med Inform Decis Mak 2008; 8:56.

16. Collins GS, Reitsma JB, Altman DG, et al. : Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. BMJ 2015; 350:g7594.

17. Steyerberg EW: Clinical prediction models: A practical approach to development, validation, and updating. Second edition. ed. Cham, Switzerland, Springer, 2019, p^pp

18. Harrel EW, Jr: Regression Modeling Strategies. 6.7-1 ed. 09-September-2023, p^pp

19. Knaus WA, Draper EA, Wagner DP, et al. : APACHE II: A severity of disease classification system. Crit Care Med 1985; 13:818–829

20. Ioannidis JP: Why most published research findings are false. PLoS Med 2005; 2:e124.

21. Christodoulou E, Ma J, Collins GS, et al. : A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. J Clin Epidemiol 2019; 110:12–22

22. Steyerberg EW, Nieboer D, Debray TPA, et al. : Assessment of heterogeneity in an individual participant data meta-analysis of prediction models: An overview and illustration. Stat Med 2019; 38:4290–4309