Машинное обучение для прогнозирования заболеваний с высоким коэффициентом риска у пациентов с раком.

Introduction

Infections during cancer treatment are leading causes of treatment delays, dose modifications, premature termination of treatment, and in the worst cases infection‐related deaths [ 1 , 2 , 3 ]. Serious infections may lead to suboptimal treatment intensity and ultimately inferior outcomes with shorter overall survival [ 4 , 5 , 6 ]. Infections often result in hospitalisation of patients with cancer, but recent studies suggest that some patients could be safely treated in outpatient settings [ 7 ]. This is reflected in updated guidelines from the American Society of Clinical Oncology, which recommend selection of patients with solid or haematological malignancies for outpatient treatment based on clinical judgement and risk stratification [ 8 ]. The most widely used stratification methods include the Multinational Association for Supportive Care in Cancer (MASCC) risk index for febrile neutropenia, Talcott's rules, and the Clinical Index of Stable Febrile Neutropenia (CISNE) score [ 9 , 10 , 11 ]. All of these, however, have important limitations. First, they do not account for complex relationships between predictive variables; and second, they only consider a small fraction of the vast amount of data available from electronic health records (EHR) [ 12 ]. Third, variable dichotomisations (e.g., age < 60) are performed despite several studies showing that use of continuous variables results in better model performance [ 13 , 14 ]. By addressing these limitations, performance of risk stratification methods may be improved, resulting in safer patient selection for outpatient treatment [ 9 ]. With the accessible EHRs and high‐level computing power available today, machine learning (ML)‐based prediction frameworks are feasible and are uniquely positioned to address the above‐mentioned limitations [ 15 ]. ML‐based prediction has already proven to produce promising classification models and has been successfully implemented in clinical practice for infections and in‐hospital mortality [ 16 , 17 , 18 ]. Limiting infection‐related hospitalisations (IRHs) or facilitating early discharge of patients admitted with infections would reduce the number of hospital beds occupied by patients with non‐serious infections. In addition to improving quality of life for patients, this could redirect limited healthcare resources to provide better care for those at high risk of serious infections. In the present work, three ML‐based approaches were introduced for risk stratification of IRHs occurring in patients with haematological and solid malignancies undergoing anticancer treatment.

Materials and Methods

Study Population

Adult patients (≥ 18 years) diagnosed with and treated for lymphoma, multiple myeloma (MM), chronic lymphocytic leukaemia (CLL), colorectal cancer, or lung cancer between 2013 and 2023 in the North Denmark Region were identified for the study. The inclusion and exclusion criteria varied according to cancer type (SM Data S1 ). Patients with MM were included from the beginning of treatment until the end of follow‐up. Patients with lymphoma or CLL were included from the beginning of treatment until 365 days after the end of treatment but were re‐included upon relapse for an additional 365 days (except when treatment was considered at the level of best supportive care or minimal treatment, SM Data S2 ). For patients with colorectal or lung cancer, inclusion was from the beginning of neoadjuvant treatment until the date of potential surgery. Patients receiving subsequent adjuvant treatment were included 30 days after surgery and up to 365 days after the final treatment. This allowed for exclusion of potential postoperative complications. Patients with multiple cancer diagnoses were included based on their first diagnosis.

Data

Baseline data (age, sex, and ECOG performance status (PS)) were retrieved from the five Danish cancer registries: the Danish Lymphoma Registry, the Danish National Multiple Myeloma Registry, the Danish National Chronic Lymphocytic Leukaemia Registry, the Danish Lung Cancer Registry and the Danish Colorectal Cancer Group Database [ 19 , 20 , 21 , 22 , 23 ]. Structured data from the EHR during treatment and up to 5 years before the date of diagnosis were included. These data included vital parameters, hospital contacts (including procedure and diagnostic codes), microbiology data, and blood test results. Information on treatment for patients with lung and colorectal cancer was not available in the registries but was collected from structured administrative data sources using ICD‐10 codes (SM Data S3 ). For the haematological cancers, treatment information was retrieved from the cancer registries.

Access to data was granted by the North Denmark Region (ID: 2023–002807). Data were analysed in pseudonymised form and accessed through a secure hospital server. Storage of data was registered in the North Denmark Region (ID: F2023‐002).

Definition of Serious Infection‐Related Hospitalisation

Serious IRHs (S‐IRHs) were defined by the presence of one or more of the following conditions: positive blood culture, death, admission to the intensive care unit (ICU), or sepsis defined as a quick Sequential Organ Failure Assessment (qSOFA) score ≥ 2 [ 24 ] during hospitalisation. If none of these conditions were met, the hospitalisation was labelled as non‐serious IRH (NS‐IRH). IRHs were identified using discharge diagnosis codes based on the ICD‐10 system (SM Data S4 ).

Machine Learning Methodology

Three models were developed: ML with the prediction point set at the date of admission, ML 48 with the prediction point set at 48 h after admission to include initial data from hospitalisation, and ML 48,reduced , a pruned version of ML 48 (SM Data S5 ). Whereas ML could act as decision support for inpatients versus outpatient treatments, ML 48 could support decisions related to early discharge after a short observation (up to 48 h). The reduced model version, ML 48,reduced , was developed as a more operational and implementable version of ML 48 . All models were trained using different ML methodologies including, but not limited to, XGBoost, logistic regression, and Gradient Boosting [ 25 ].

Data after the prediction point were blinded to the model. Furthermore, for ML 48 and ML 48,reduced , admissions were excluded if a positive blood culture or admission to the ICU was present during the 48 h prior to the prediction point. A 70:15:15 training/validation/test split was applied, with all model development (tuning and feature selection) conducted on the training and validation sets only. Final performance metrics presented in this study are reported exclusively on the held‐out test set. To prevent patient‐level data leakage, data were split at the patient level, ensuring that all hospitalisations for each patient were either in the training, validation, or test set. Confidence intervals (CIs) were computed using bootstrapping with 100 replicates.

Features were created by aggregating summary statistics across multiple look‐back windows, designed to capture both short‐ and long‐term patient history. Look‐back windows and variables were selected based on data availability, with varying thresholds applied across different data modalities. Further details are provided in SM S6 .

Feature Selection

Due to the high number of potential features from the available data, a data‐driven feature selection process was chosen. This approach allows a general and automated process for identifying relevant features and avoids the need to predefine the number of variables to include. First, features with > 90% missingness were removed or if mutual information between the feature and the outcome was less than 1%. After this, features were imputed using k‐nearest neighbour imputation. The final features were selected using a Least Absolute Shrinkage and Selection Operator (Lasso) regression, where all features with a Lasso coefficient ≠ 0 were selected. The selected features and percentages of missing data can be found in SM Data S7 .

For ML 48,reduced the Boruta feature selection [ 26 ] algorithm was applied to further reduce the number of features. Further details on the methods are provided in SM Data S8 . The Boruta algorithm ran until all features were either accepted or rejected.

Sensitivity Analysis

To assess the performance of the models across different clinical subgroups, two sensitivity analyses were performed. The model performance was tested in PS subgroups (0–1 vs ≥ 2). Second, the performance for each cancer type was analysed. Finally, 60‐day survival was estimated using Kaplan–Meier estimates, with the end point being death. Follow‐up was calculated from the prediction point (admission time for ML , 48 h after admission for ML 48 ).

Results

Patient Characteristics

A total of 9874 patients were diagnosed with one of the included cancers in the North Denmark Region from 2013 to 2023 (lung cancer [ n = 4338; 43.9%], colorectal cancer [ n = 2722; 28.0%], lymphoma [ n = 1407; 14.2%], multiple myeloma [ n = 894; 9.0%], CLL [ n = 513; 5.1%]). Based on the inclusion criteria, 5377 patients were included (lung cancer [ n = 2384; 44.3%], colorectal cancer [ n = 1774; 32.9%], lymphoma [ n = 713; 14.2%], multiple myeloma [ n = 369; 6.8%], CLL [ n = 137; 2.5%]). Most exclusions occurred due to absence of relevant treatment (CONSORT diagram, SM Data S9 ). The median age was 68 years (IQR: 61–74 years), 53.5% were male (Table 1 ).

Characteristics of IRHs

A total of 2691 IRHs occurred (1892 in the training set, 394 in the validation set, 405 in the test set); 691 (21.5%) were classified as S‐IRHs (493 in the training set, 98 in the validation set, 100 in the test set). The most prevalent diagnosis codes for IRH were febrile neutropenia (FN) (30.5%), pneumonia (27.4%), and bacterial infection (10.2%). The distribution of IRHs and S‐IRHs by cancer type was as follows: Lung cancer (IRH: n = 1290; 47.6%, S‐IRH: n = 373; 53.9%), MM (IRH: n = 592; 22.2%, S‐IRH: n = 119; 17.2%), lymphoma (IRH: n = 486; 18.1%, S‐IRH: n = 112; 16.2%), colorectal cancer (IRH: n = 252; 9.2%, S‐IRH: n = 74; 10.7%), and CLL (IRH: n = 71; 2.6%, S‐IRH: n = 13; 1.8%). The causes of S‐IRHs were sepsis ( n = 266; 38.4%), death ( n = 214; 30.9%), positive blood culture ( n = 184; 26.6%), and admission to the ICU ( n = 27; 3.9%) (SM Data S10 and Data S11 ).

ML

The ML model (using only data available up to the IRH) achieved a ROC‐AUC of 0.79 (95% CI: 0.74–0.84). At approximately ~85% sensitivity, the specificity was 44.9% (95% CI: 40.1%–49.8%) (Table 2 ). At this sensitivity, and out of a total of 405 hospitalisations, the model correctly predicted 87 S‐IRHs, 137 NS‐IRHs, and falsely classified 168 NS‐IRHs as S‐IRHs and 13 S‐IRHs as NS‐IRHs. To reduce the risk of missed S‐IRHs, analyses were also performed using a sensitivity of 90%. This resulted in a specificity of 38.6% (95% CI: 32.6%–43.8%), reduced the number of S‐IRHs that were misclassified as NS‐IRHs to 10, but increased the number of NS‐IRHs classified as S‐IRHs to 188.

ML 48

A total of 2124 NS‐IRHs (1731 in the training set, 359 in the validation set, 372 in the test set) and 338 S‐IRHs (231 in the training set, 54 in the validation set, 54 in the test set) were available from the 48‐h time point. The ML 48 model achieved a ROC‐AUC of 0.76 (95% CI: 0.69%–0.81%). At a sensitivity of 85%, the specificity was 50.9% (95% CI: 44.4%–56.1%). At this sensitivity, the model correctly predicted 46 S‐IRHs and 162 NS‐IRHs and falsely classified 156 NS‐IRHs as S‐IRHs and 8 S‐IRHs as NS‐IRHs. To decrease the number of missed S‐IRHs, the analysis was again performed using a sensitivity of 90%. This resulted in a specificity of 40.2% (95% CI: 35.7%–44.8%), reduced the number of S‐IRHs misclassified as NS‐IRHs to 4 but increased the number of NS‐IRHs classified as S‐IRHs to 194.

ML 48 ,reduced

In ML 48,reduced, 10 features were identified using the stricter feature selection process. Using this iteration of the model resulted in a slight decrease in performance, with a ROC‐AUC of 0.75% (95% CI: 0.68%–0.79%). At a sensitivity of 85% it achieved a specificity of 46.5% (95% CI: 41.8%–51.8%). The specificity increased to 59.7% (95% CI: 53.8%–66.7%) when the sensitivity was decreased to 75%. As a further test of ML 48,reduced the feature selection process was restricted to only include features in the 3‐day look‐back window. This setup used 10 features and achieved similar predictive performance, with a ROC‐AUC of 0.75 (95% CI: 0.69%–0.82%). In this instance, the specificity was 46.5% (95% CI: 41.8%–51.6%) at 85% sensitivity and 59.7% (95% CI: 53.8%–66.7%) at 75% sensitivity. Results from a pruned version of ML can be found in SM Data S12 .

Feature Explanation

SHAP values revealed distinct patterns of feature importance across the three models. For ML , the most influential features were the mean number of positive blood cultures within 3 days, minimum neutrophil count within 60 days, and number of sepsis episodes (defined as qSOFA ≥ 2 at any time) within 7 days (Figure 1 ). The dominant look‐back window was the 3 days prior to hospitalisation.

In ML 48 , the most influential features were maximum qSOFA score within 3 days, maximum carbamide level within 30 days, and minimum LDH within 60 days (Figure 2 ). The most informative time window was again 3 days (SM Data S13 ).

For ML 48,reduced , the top features included count of positive urine tests within 7 days, maximum qSOFA score within 3 days, and mean albumin within 3 days (Figure 3 ). SHAP analysis of ML 48,reduced further restricted to only 3‐day look‐back windows, showed similar feature importance as ML 48,reduced .

In sensitivity analyses, ML achieved the best performance for patients diagnosed with lymphoma. For these patients, the ROC‐AUC was 0.86 (95% CI: 0.76–0.94), with a specificity of 60.0% (95% CI: 45.2%–72.2%) at a specificity of 85%. The worst performance was seen in lung cancer patients with an ROC‐AUC of 0.73 and a specificity of 31.1% (95% CI: 23.7%–38.8%).

For patients with PS 0–1, the model achieved a ROC‐AUC of 0.78 (95% CI: 0.72–0.84) versus 0.77 (95% CI: 0.63–0.56) for PS 2–4. The specificities were 49.5% (95% CI: 43.4%–56.5%) for PS 0–1 versus 32.2% (95% CI: 21.2%–44.6%) for PS 2–4 (SM Data S14 ).

Outcomes According to Predicted IRH Seriousness

Analyses of 60‐day survival after the patients' first hospitalisation by IRH classifications were performed. For ML , the 60‐day survival rate was 36.9% (95% CI: 25.3%–53.9%) for correctly classified S‐IRHs and 94.0% (95% CI: 88.5%–99.8%) for correctly classified NS‐IRHs. For incorrectly classified S‐IRHs, the 60‐day survival rate was 72.6% (95% CI: 64.8%–81.3%) and 50.0% (95% CI: 29.9%–84.4%) for incorrectly classified NS‐IRHs (Figure 4a ). In ML 48 , the 60‐day survival rate was 39.1% (95% CI: 23.5%–65.1%) for correctly classified S‐IRHs and 92.5% (95% CI: 87.0%–94.4%) for correctly classified NS‐IRHs. Conversely, the 60‐day survival rate was 63.0% (95% CI: 54.9%–72.3%) for incorrectly classified S‐IRHs and 50.0% (95% CI: 18.8%–100%) for incorrectly classified NS‐IRHs (Figure 4b ). The number of patients at risk in the test sets was too small for similar survival analyses focusing on subsequent hospitalisations.

Discussion

In the present study, three ML‐based risk stratification models for IRHs in cancer patients were introduced. Across 2691 IRHs, the best performing model ( ML ) achieved a ROC‐AUC of 0.79, while the model shifting the prediction point by 48 h ( ML 48 ) reached 0.76 and the feature‐reduced model achieved 0.75. Performance varied by cancer type and PS, with the strongest discriminatory power for lymphoma and patients with PS 0–1. The models also distinguished survival outcomes, as patients predicted to have S‐IRH had substantially lower 60‐day survival rates than those predicted to have an NS‐IRH.

Due to demographic changes with ageing populations, combined with improved treatment options for many cancers resulting in improved survival rates, the incidence and prevalence of cancer are increasing [ 27 , 28 ]. Consequently, the incidence of treatment‐related complications such as infections will continue to increase and add to resource use in an already strained healthcare system. IRHs are costly due to frequent hospitalisations, but resource use could be reduced by better risk stratification. The IRHs in this study accounted for 13,305 hospital days, assuming a mean hospital stay of 5 days.

The developed ML models in the present study achieved a ROC‐AUC of approximately ~0.79–0.81 with a high sensitivity (~85%) and a modest specificity (40.6%–50.9%). A study from 2021 used ML to predict in‐hospital mortality in 126,013 FN admissions (both in patients with and without malignancies) and achieved a ROC‐AUC of 0.92 [ 17 ]. Similar performance was found in a study predicting lung infections in 502 patients with lung cancer (outcome frequency: 19.5%; ROC‐AUC: 0.88), a study predicting bloodstream infections in 19,225 patients undergoing chemotherapies at a tertiary cancer hospital (outcome frequency: 1.3%; ROC‐AUC: 0.91), and a study predicting complications during FN episodes using 513 patients (outcome frequencies: 12,8%, 18.4% and 19.6%; ROC‐AUC: 0.85–0.91) [ 18 , 29 , 30 ]. Unlike these studies, which targeted either single cancers or single outcomes, this study focuses on multiple cancers and a broad set of relevant clinical endpoints. This adds complexity to the prediction models and could contribute to the comparatively lower performance. Future studies should incorporate previous infections into the model, as FN episodes have been shown to be associated with higher risks of mortality and admission to the ICU and may also affect hospitalisation outcomes [ 31 ].

ML models that utilise large volumes of data are not straightforward to implement, as many included features depend on the local data structure. EHR data structure can vary substantially from hospital to hospital, even within countries, which hampers generalisability and validation work for complex ML‐based models. For that reason, the present study explored the ML 48,reduced inwhich the number of explanatory variables was limited.

Explainability of features is one of the greatest barriers to acceptability and implementation of clinical ML‐based decision support tools [ 32 ]. The reason for a prediction should be transparent for the clinician as well as explainable to patients [ 33 ]. Complex models like the ones developed in the present study and in prior studies have weaknesses in this regard. The ML and ML 48 models used over 60 features, and it is impossible to explain what flagged patients as high risk in individual patients. Despite the use of a high number of features, the SHAP analyses showed that well‐known and expected clinical factors such as previous BSI and sepsis episodes as well as relevant laboratory tests, were important contributing factors to the predictions. It would therefore be necessary to incorporate the model into local EHR system, where automatic retrieval of features would be possible.

The developed models in the present study provide examples of decision support tools for different time points in the patient trajectory. The ML model made predictions at the time of admission and would be relevant to support decisions related to inpatient versus outpatient treatment. In contrast, ML 48 predicts outcomes after 48 h after admission and could be used to support decisions related to early discharge of hospitalised patients. Importantly, using such models comes with risks, and they are not suitable for stand‐alone tools. Ultimately, the decision to treat patients in outpatient settings or pursue early discharge is a clinical judgement, supported by available information. Wrong predictions have consequences as shown with the low 60‐day survival of S‐IRHs that were misclassified as NS‐IRHs.

The sequential application of LASSO and Boruta was chosen to apply their complementary strengths. LASSO reduced the feature space through linear shrinkage, capturing linear relationships, whereas the Boruta algorithm captured non‐linear interactions. When reducing the number of features to 10 in ML 48,reduced , a slight decrease in performance was observed. However, the features corresponding to the class assigned by the model were those expected from a clinical perspective (e.g., LDH, respiratory frequency and albumin). The smaller number of features and the clinical relevance of those retained make it a strong candidate for implementation, as predictions are easier to explain and validate than comparison with the more comprehensive ML and ML 48 models. This reflects an inherent trade‐off between predictive power and clinical feasibility. While the full models offer higher discriminatory power, the reduced models are better suited for clinical integration, particularly in settings where automated EHR‐based feature extraction is not possible.

Despite the models utilising data from multiple cancer modalities, no cancer‐type specific feature impacted the predictions made by the models substantially. This indicates that the model is not driven by diagnosis‐specific variables but rather by shared infection‐related patterns across cancers, making it broadly applicable. Developing a model on a more general cohort has some technical advantages. First, pooling patients from different diagnoses increases the sample size, which can help improve stability and lower the risk of overfitting compared to creating models on smaller datasets [ 34 , 35 ]. Second, training a shared model can learn generalisable predictors of risk of S‐IRH risk that go beyond predictors for each type of cancer, which has been demonstrated in multi‐oncology studies showing better discriminatory performance when training across heterogeneous populations [ 34 , 36 , 37 ]. Moreover, in a recent paper employing a comparable methodology to that used in the present study—but restricted to lymphoma patients—lower performance was observed compared to all models introduced here [ 38 ]. This further underscores the advantage of larger and more heterogeneous datasets for predictive modelling.

While the sensitivity analysis revealed performance differences between the cancer subgroups, there was nothing hinting to the model relying on features from specific cancer types. The model performed better among patients with a good PS, which could be attributed to a higher representation of this group in the dataset. Likewise, the performance was highest for lymphoma and lowest for lung cancer. The sensitivity analyses were limited by the small test set, especially for the CLL subgroup only containing 14 hospitalisations for ML and 6 for ML 48 .

One major limitation of this study is the lack of external validation. To partially address this, a held‐out test set was employed to evaluate final model performance. While this can improve the robustness of the reported results, it cannot substitute the need for external validation on an independent cohort to prove sufficient generalisability across healthcare settings. Hence, the models introduced in the present paper still require external validation before potential implementation in clinical practice. Even if external validation demonstrates a reduced performance at a new site, the solution may not be to deploy the model unchanged; studies have shown that local calibration or fine‐tuning may be beneficial. [ 39 ]. The method of labelling IRHs based on ICD‐10 codes could be a limitation. Although ICD‐10 codes for infections have an acceptable positive predictive value in Denmark (85.0%) (95% CI: 75.0%–95%) [ 40 ], some cases of infections will be missed and some (likely few) cases of registered infections were likely misclassified. Lastly, the composite outcome used in the present study is both a weakness and a strength. It provides a single, clinically relevant indicator of high‐risk IRHs in cancer patients, by integrating multiple serious complications into one outcome. This approach supports early identification of patients who may be safely discharged as well as those who require closer monitoring. However, it does not allow the model to distinguish the severity or nature of the individual complications. Thus, while the composite outcome offers a broad definition of “high‐risk,” it does so at the cost of clinical granularity regarding each specific event.

This work demonstrates that ML can add meaningful value to risk stratification of IRHs in patients with cancer. Key lesson from the present study is that the models have the potential to support decisions related to admission and early discharge of patients with IRHs, but the models require simplification if they are to be validated to the standard required for medical devices and seamlessly integrated with EHRs outside the development setting.

Author Contributions

Mathias Holmsgaard Eskesen: methodology, investigation, conceptualization, writing – review and editing. Alexander Djupnes Fuglkjær: investigation, conceptualization, writing – review and editing, writing – original draft. Laurids Østergaard Poulsen: methodology, resources, project administration, funding acquisition, conceptualization, investigation, writing – original draft. Mikkel Runason Simonsen: writing – original draft, validation, conceptualization, project administration, funding acquisition. Carsten Utoft Niemann: funding acquisition, software, formal analysis, conceptualization, writing – review and editing. Mikkel Werling: conceptualization, writing – review and editing, writing – original draft, visualization. Paw Jensen: investigation, writing – original draft, writing – review and editing, conceptualization, validation. Kirstine Kobberøe Søgaard: funding acquisition, investigation, writing – review and editing, writing – original draft, conceptualization. Frederik Christensen: writing – original draft, methodology, funding acquisition, visualization, conceptualization. Izabela Ewa Nielsen: conceptualization, visualization, formal analysis, investigation. Tarec Christoffer El‐Galaly: validation, formal analysis, writing – review and editing, conceptualization, funding acquisition.

Funding

This work was supported by Sygeforsikringen Danmark (grant number 2022‐0086). The Danish Data Science Academy (grant number 2023‐1210). This work was supported by the Danish Data Science Academy, which is funded by the Novo Nordisk Foundation (NNF21SA0069429) and (NNF22OC0078228) and VILLUM FONDEN (40516).

Ethics Statement

The study was registered in the North Denmark Region where the study was designed and conducted (ID: F2023‐002). According to Danish law, ethical approval is not mandated for retrospective data studies. Access to health care records was granted by the North Denmark Region (ID: 2023–002807). Data were pseudonymised and analysed on a closed server hosted by the North Denmark Region.

Conflicts of Interest

Carsten Utoft Niemann received research funding and/or consultancy fees from Abbvie, AstraZeneca, Janssen, Beigene, Genmab, Octapharma, Takeda, MSD, Lilly, CSL Behring, Galapagos, Synamics, Novo Nordisk Foundation. All other authors declare no competing interests.

Supporting information

Data S1: Table showing inclusion and exclusion criteria for each included cancer type.

Data S2: Treatments of interest for haematological patients.

Data S3: Treatments of interest for lung and colorectal cancer patients.

Data S4: ICD‐10 codes used to define infection related hospitalizations.

Data S5: Flowchart of data processing.

Data S6: Description of datasets and specification of lookback windows and aggregations.

Data S7: Further elaboration for methods.

Data S8: Features used for ML, ML48, and MLreduced along with percentage missing in each dataset.

Data S9: cam472116‐sup‐0001‐Supinfo. docx.

Data S10: Most prevalent infections stratified by cancer type.

Data S11: The reason for labelling serious infection‐related hospitalisation (S‐IRH), stratified by cancer type.

Data S12: Results for pruned version of ML.

Data S13: Distribution of most look back window.

Data S14: Results of sensitivity analysis including confusion matrices for ML and ML48. All performance is presented at ~85% sensitivity for the models. The performance is presented based on CatBoost for ML and XGBoost for ML48. All performance is calculated based on held‐out test set.

Table S1: Description of inclusion and exclusion criteria for different cancer types.

Table S4: Included infections used for locating infection related hospitalizations along with ICD‐10 code.

Figure S5: Flowchart of data processing.

Table S6: Description of included datasets, along with lookback windows and used statistical summaries. TOKS = The National Early Warning Score 2, qSOFA = The Quick Organ Failure Assessment Score (qSOFA).

Table S8: ‐1: Name and corresponding missing percentage for features used in ML.

Table S8: ‐2: Name and corresponding missing percentage for features used in ML48.

Table S8: ‐3: Name and corresponding missing percentage for features used in ML48, reduced.

Figure S9: Consort diagram for inclusion and exclusion of patients.

Table S10: The five most prevalent reasons to an infection related hospitalisation stratified by cancer type. For all haematological cancers, it is seen that febrile neutropenia was the most prevalent, whereas pneumonia was for lung cancer and bacterial infections for colorectal cancer.

Table S11: Causes of infection related hospitalizations being deemed serious. For example, 161 infection related hospitalizations were serious for lung cancer patients, due to sepsis during hospitalisation.

Table S12: results for pruned version of ML.

Figure S12: SHAP values for pruned version of ML.

Figure S13: ‐1: SHAP importance for each lookback window for ML.

Figure S13: ‐2: SHAP importance for each data source for ML48.

Data Availability Statement

The authors cannot share individual level patient data due to Danish law.

References

1 M. Elhadi , A. Khaled , and A. Msherghi , “ Infectious Diseases as a Cause of Death Among Cancer Patients: A Trend Analysis and Population‐Based Study of Outcome in the United States Based on the Surveillance, Epidemiology, and End Results Database ,” Infect Agent Cancer 16 ( 2021 ): 72 , 10.1186/s13027-021-00413-z . 34972537 PMC8719405

2 M. Forster , A. M. Deal , A. Page , et al., “ Dose Delay, Dose Reduction, and Early Treatment Discontinuation in Black and White Women Receiving Chemotherapy for Nonmetastatic Breast Cancer ,” Oncologist 29 , no. 10 ( 2024 ): e1246 – e1259 , 10.1093/oncolo/oyae150 . 38913986 PMC11449010

3 A. Taha , I. Vinograd , A. Sakhnini , et al., “ The Association Between Infections and Chemotherapy Interruptions Among Cancer Patients: Prospective Cohort Study ,” Journal of Infection 70 , no. 3 ( 2015 ): 223 – 229 , 10.1016/j.jinf.2014.10.008 . 25444974

4 R. Attiê , L. T. D. Chinen , E. M. Yoshioka , M. C. F. Silva , and V. C. C. de Lima , “ Acute Bacterial Infection Negatively Impacts Cancer Specific Survival of Colorectal Cancer Patients ,” World Journal of Gastroenterology: WJG 20 , no. 38 ( 2014 ): 13930 – 13935 , 10.3748/wjg.v20.i38.13930 . 25320529 PMC4194575

5 A. Delgado and A. K. Guddati , “ Infections in Hospitalised Cancer Patients ,” World Journal of Oncology 12 , no. 6 ( 2021 ): 195 – 205 , 10.14740/wjon1410 . 35059079 PMC8734501

6 Y. Zheng , Y. Chen , K. Yu , et al., “ Fatal Infections Among Cancer Patients: A Population‐Based Study in the United States ,” Infectious Disease and Therapy 10 , no. 2 ( 2021 ): 871 – 895 , 10.1007/s40121-021-00433-7 . PMC8116465 33761114

7 R. Rivas‐Ruiz , M. Villasis‐Keever , G. Miranda‐Novales , O. D. Castelán‐Martínez , and S. Rivas‐Contreras , “ Outpatient Treatment for People With Cancer Who Develop a Low‐Risk Febrile Neutropaenic Event ,” Cochrane Database of Systematic Reviews 3 , no. 3 ( 2019 ): CD009031 , 10.1002/14651858.CD009031.pub2 . 30887505 PMC6423292

8 R. A. Taplitz , E. B. Kennedy , E. J. Bow , et al., “ Outpatient Management of Fever and Neutropenia in Adults Treated for Malignancy: American Society of Clinical Oncology and Infectious Diseases Society of America Clinical Practice Guideline Update ,” Journal of Clinical Oncology 36 , no. 14 ( 2018 ): 1443 – 1453 , 10.1200/JCO.2017.77.6211 . 29461916

9 A. Carmona‐Bayonas , J. Gómez , E. González‐Billalabeitia , et al., “ Prognostic Evaluation of Febrile Neutropenia in Apparently Stable Adult Cancer Patients ,” British Journal of Cancer 105 , no. 5 ( 2011 ): 612 – 617 , 10.1038/bjc.2011.284 . 21811253 PMC3188929

10 J. Klastersky , M. Paesmans , E. B. Rubenstein , et al., “ The Multinational Association for Supportive Care in Cancer Risk Index: A Multinational Scoring System for Identifying Low‐Risk Febrile Neutropenic Cancer Patients ,” Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology 18 , no. 16 ( 2000 ): 3038 – 3051 , 10.1200/JCO.2000.18.16.3038 . 10944139

11 J. A. Talcott , R. D. Siegel , R. Finberg , and L. Goldman , “ Risk Assessment in Cancer Patients With Fever and Neutropenia: A Prospective, Two‐Center Validation of a Prediction Rule ,” Journal of Clinical Oncology: Official Journal of the American Society of Clinical Oncology 10 , no. 2 ( 1992 ): 316 – 322 , 10.1200/JCO.1992.10.2.316 . 1732432

12 T. Hernandez‐Boussard , D. W. Blayney , and J. D. Brooks , “ Leveraging Digital Data to Inform and Improve Quality Cancer Care ,” Cancer Epidemiology, Biomarkers & Prevention 29 , no. 4 ( 2020 ): 816 – 822 , 10.1158/1055-9965.EPI-19-0873 . PMC7195903 32066619

13 J. Biccler , S. Eloranta , P. de Nully Brown , et al., “ Simplicity at the Cost of Predictive Accuracy in Diffuse Large B‐Cell Lymphoma: A Critical Assessment of the R‐IPI, IPI, and NCCN‐IPI ,” Cancer Medicine 7 , no. 1 ( 2018 ): 114 – 122 , 10.1002/cam4.1271 . 29239133 PMC5773951

14 O. Naggara , J. Raymond , F. Guilbert , D. Roy , A. Weill , and D. G. Altman , “ Analysis by Categorising or Dichotomising Continuous Variables Is Inadvisable: An Example From the Natural History of Unruptured Aneurysms ,” AJNR. American Journal of Neuroradiology 32 , no. 3 ( 2011 ): 437 – 440 , 10.3174/ajnr.A2425 . 21330400 PMC8013096

15 O. Ali , W. Abdelbaki , A. Shrestha , E. Elbasi , M. A. A. Alryalat , and Y. K. Dwivedi , “ A Systematic Literature Review of Artificial Intelligence in the Healthcare Sector: Benefits, Challenges, Methodologies, and Functionalities ,” Journal of Innovation and Knowledge 8 , no. 1 ( 2023 ): 100333 , 10.1016/j.jik.2023.100333 .

16 R. Agius , C. Brieghel , M. A. Andersen , et al., “ Machine Learning Can Identify Newly Diagnosed Patients With CLL at High Risk of Infection ,” Nature Communications 11 ( 2020 ): 363 , 10.1038/s41467-019-14225-8 . PMC6969150 31953409

17 X. Du , J. Min , C. P. Shah , R. Bishnoi , W. R. Hogan , and D. J. Lemas , “ Predicting In‐Hospital Mortality of Patients With Febrile Neutropenia Using Machine Learning Models ,” International Journal of Medical Informatics 139 ( 2020 ): 104140 , 10.1016/j.ijmedinf.2020.104140 . 32325370 PMC7255942

18 T. Sun , J. Liu , H. Yuan , X. Li , and H. Yan , “ Construction of a Risk Prediction Model for Lung Infection After Chemotherapy in Lung Cancer Patients Based on the Machine Learning Algorithm ,” Frontiers in Oncology 14 ( 2024 ): 1403392 , 10.3389/fonc.2024.1403392 . 39184040 PMC11341396

19 B. Arboe , P. Josefsson , J. Jørgensen , et al., “ Danish National Lymphoma Registry ,” Clinical Epidemiology 8 ( 2016 ): 577 – 581 , 10.2147/CLEP.S99470 . 27822102 PMC5094596

20 C. da Cunha‐Bang , C. Geisler , L. Enggaard , et al., “ The Danish National Chronic Lymphocytic Leukemia Registry ,” Clinical Epidemiology 8 ( 2016 ): 561 – 565 , 10.2147/CLEP.S99486 . 27822100 PMC5094649

21 P. Gimsing , M. Holmstrøm , T. W. Klausen , et al., “ The Danish National Multiple Myeloma Registry ,” Clinical Epidemiology 8 ( 2016 ): 583 – 587 , 10.2147/CLEP.S99463 . 27822103 PMC5094522

22 P. Ingeholm , I. Gögenür , and L. Iversen , “ Danish Colorectal Cancer Group Database ,” Clinical Epidemiology 8 ( 2016 ): 465 – 468 , 10.2147/CLEP.S99481 . 27822086 PMC5094575

23 E. Jakobsen and T. Rasmussen , “ The Danish Lung Cancer Registry ,” Clinical Epidemiology 8 ( 2016 ): 537 – 541 , 10.2147/CLEP.S99458 . 27822096 PMC5094603

24 M. M. Churpek , A. Snyder , X. Han , et al., “ Quick Sepsis‐Related Organ Failure Assessment, Systemic Inflammatory Response Syndrome, and Early Warning Scores for Detecting Clinical Deterioration in Infected Patients Outside the Intensive Care Unit ,” American Journal of Respiratory and Critical Care Medicine 195 , no. 7 ( 2017 ): 906 – 911 , 10.1164/rccm.201604-0854OC . 27649072 PMC5387705

25 T. Chen and C. Guestrin , “XGBoost: A Scalable Tree Boosting System” . Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discov. Data Min.

26 M. B. Kursa , A. Jankowski , and W. R. Rudnicki , “ Boruta – A System for Feature Selection ,” Fundamenta Informaticae 101 , no. 4 ( 2010 ): 271 – 285 , 10.3233/FI-2010-288 .

27 S. A. Seabury , D. P. Goldman , C. N. Gupta , et al., “ Quantifying Gains in the War on Cancer due to Improved Treatment and Earlier Detection ,” Forum Health Econ Policy 19 , no. 1 ( 2016 ): 141 – 156 , 10.1515/fhep-2015-0028 . 31419891

28 C. Xie , X. Huang , D. Lin , et al., “ Long‐Term Trend of Future Cancer Onset: A Model‐Based Prediction of Cancer Incidence and Onset Age by Region and Gender ,” Preventive Medicine 177 ( 2023 ): 107775 , 10.1016/j.ypmed.2023.107775 . 37951546

29 R. Padmanabhan , A. Elomri , R. Y. Taha , H. El Omri , H. Elsabah , and A. El Omri , “ Prediction of Multiple Clinical Complications in Cancer Patients to Ensure Hospital Preparedness and Improved Cancer Care ,” International Journal of Environmental Research and Public Health 20 , no. 1 ( 2022 ): 526 , 10.3390/ijerph20010526 . 36612856 PMC9819091

30 A. Villani Júnior , M. P. Freire , F. Lazar Neto , et al., “ Prediction of Bacterial and Fungal Bloodstream Infections Using Machine Learning in Patients Undergoing Chemotherapy ,” Eur J Cancer Oxf Engl 1990 223 ( 2025 ): 115516 , 10.1016/j.ejca.2025.115516 . 40382858

31 T. Aagaard , J. Reekie , M. Jørgensen , et al., “ Mortality and Admission to Intensive Care Units After Febrile Neutropenia in Patients With Cancer ,” Cancer Medicine 9 , no. 9 ( 2020 ): 3033 – 3042 , 10.1002/cam4.2955 . 32144897 PMC7196064

32 M. Hassan , A. Kushniruk , and E. Borycki , “ Barriers to and Facilitators of Artificial Intelligence Adoption in Health Care: Scoping Review ,” JMIR Human Factors 11 , no. 1 ( 2024 ): e48633 , 10.2196/48633 . 39207831 PMC11393514

33 Y. Okada , Y. Ning , and M. E. H. Ong , “ Explainable Artificial Intelligence in Emergency Medicine: An Overview ,” Clinical and Experimental Emergency Medicine 10 , no. 4 ( 2023 ): 354 – 362 , 10.15441/ceem.23.145 . 38012816 PMC10790070

34 E. J. Oh , R. B. Parikh , C. Chivers , and J. Chen , “ Two‐Stage Approaches to Accounting for Patient Heterogeneity in Machine Learning Risk Prediction Models in Oncology ,” JCO Clinical Cancer Informatics 5 ( 2021 ): 1015 – 1023 , 10.1200/CCI.21.00077 . 34591602 PMC8812620

35 J. Tolksdorf , M. W. Kattan , S. A. Boorjian , et al., “ Multi‐Cohort Modeling Strategies for Scalable Globally Accessible Prostate Cancer Risk Tools ,” BMC Medical Research Methodology 19 ( 2019 ): s1 , 10.1186/s12874-019-0839-0 . PMC6792191 31615451

36 R. B. Parikh , J. S. Hasler , Y. Zhang , et al., “ Development of Machine Learning Algorithms Incorporating Electronic Health Record Data, Patient‐Reported Outcomes, or Both to Predict Mortality for Outpatients With Cancer ,” JCO Clinical Cancer Informatics 6 ( 2022 ): e2200073 , 10.1200/CCI.22.00073 . 36480775 PMC10166444

37 H. Smith , M. Sweeting , T. Morris , and M. J. Crowther , “ A Scoping Methodological Review of Simulation Studies Comparing Statistical and Machine Learning Approaches to Risk Prediction for Time‐To‐Event Data ,” Diagnostic and Prognostic Research 6 ( 2022 ): 10 , 10.1186/s41512-022-00124-y . 35650647 PMC9161606

38 A. D. Fuglkjaer , D. K. Kilic , M. H. Eskesen , et al., “ Machine Learning for Prediction of High‐Risk Hospitalisations in Lymphoma Patients: A Danish Population‐Based Study ,” European Journal of Haematology 115 , no. 4 ( 2025 ): 443 – 453 , 10.1111/ejh.70012 . 40728147 PMC12505830

39 J. Yang , A. A. S. Soltan , and D. A. Clifton , “ Machine Learning Generalisability Across Healthcare Settings: Insights From Multi‐Site COVID‐19 Screening ,” npj Digital Medicine 5 ( 2022 ): 1 – 8 , 10.1038/s41746-022-00614-9 . 35672368 PMC9174159

40 L. Holland‐Bill , H. Xu , H. T. Sørensen , et al., “ Positive Predictive Value of Primary Inpatient Discharge Diagnoses of Infection Among Cancer Patients in the Danish National Registry of Patients ,” Annals of Epidemiology 24 , no. 8 ( 2014 ): 593 – 597.e18 , 10.1016/j.annepidem.2014.05.011 . 25084702