Human Versus Machine in Survival Prediction for Metastatic Spinal Cord Compression: A Retrospective Cohort Study.

Introduction

Accurate prediction of life expectancy in patients with metastatic spinal cord compression (MSCC) is essential for guiding treatment selection, balancing surgical risk, and setting realistic patient expectations [ 1 ]. In this vulnerable patient population, survival projections directly influence treatment options [ 2 ], with thresholds such as six and 12 months often serving as critical decision points for surgical intervention, radiotherapy, or palliative care strategies [ 3 ]. However, survival estimation remains inherently challenging [ 4 ], influenced by a complex interplay of oncological, functional, and systemic factors [ 5 , 6 ].

Traditional prognostic tools, such as the revised Tokuhashi score, have historically provided structured frameworks to guide decision-making [ 6 ]. While such scoring systems facilitate systematic evaluation, they were developed prior to the widespread adoption of targeted therapies and immunotherapy and may not fully account for contemporary oncological advances [ 6 ]. Notably, recent studies have suggested only modest predictive accuracy for Tokuhashi-based estimations, with overall survival prediction rates approximating 60-70% [ 6 ]. Although clinical judgment by oncologists incorporates nuanced assessments beyond formal scoring, recent studies have demonstrated persistent limitations in predictive accuracy, particularly in forecasting intermediate survival periods [ 7 ].

Recent efforts to integrate artificial intelligence (AI) and machine learning (ML) into oncology prognostication have shown promise in areas such as tumor detection and other image-based classification tasks [ 8 - 10 ]. However, while most AI research in spinal oncology has centered around imaging classification tasks, structured prediction models based on clinical data remain underexplored [ 11 - 12 ]. Moreover, external validation of AI models has revealed substantial variability in performance, underscoring concerns regarding generalizability across diverse patient populations [ 9 , 13 , 14 ].

Importantly, few studies have critically addressed the specific difficulty of predicting survival in the intermediate range (six to 12 months), a group in which clinical decisions are often most uncertain, and where misclassification can have profound consequences [ 2 , 3 ]. Existing literature has largely emphasized overall accuracy metrics, without dissecting the variable performance across survival strata [ 6 , 7 ]. Other evaluations of prognostic modelling have demonstrated the limitations of utilising only global discrimination metrics without analysis of subgroups [ 14 ].

Using a real-world cohort of anonymized clinical cases encompassing oncological, imaging, and functional parameters, the primary objective of this study was to compare the accuracy of survival prediction in MSCC across four prognostic approaches: oncologist clinical judgment, surgeon-calculated Tokuhashi scores, ChatGPT-assisted literature-based predictions, and ChatGPT-assisted Tokuhashi-based predictions. We specifically assessed performance within clinically relevant survival categories (<6 months, six to 12 months, and >12 months). Secondary exploratory objectives were to examine areas of relative agreement and difficulty across these prognostic approaches and to explore potential predictors of survival using logistic regression and machine learning analyses. Through this approach, we aimed to provide a clinically relevant assessment of current and emerging prognostic strategies in metastatic spinal disease.

Materials and methods

Study design and patient selection

We conducted a retrospective cohort study at the Centre for Spinal Studies and Surgery, Queen’s Medical Centre, Nottingham, United Kingdom. Consecutive patients with MSCC referred to our tertiary spinal oncology service between January 2020 and December 2022 were included. A total of 100 patients (n = 100) were analyzed. Referral data routinely included the oncology team's initial management plan and predicted life expectancy at the time of referral.

As all data were fully anonymized and retrospective in nature, institutional policy determined that formal ethical approval was not required.

Data collection

Clinical data were summarized into standardized PowerPoint slides (Microsoft Corp., USA) for each patient. Each slide included patient demographics particularly age and gender, presenting complaint, past medical history, primary tumor histology and systemic metastases, imaging findings including CT chest/abdomen/pelvis and MRI whole spine, biopsy results whenever they were available, Karnofsky Performance Status, serum calcium levels, Spinal Instability Neoplastic Score (SINS), clinical neurological examination findings, current medications, and previous chemotherapy and radiotherapy treatments. All slides were anonymized to exclude any identifiable patient information. Figure 1 illustrates an example of the anonymized clinical slide used for prognostic assessment.

Prognostic estimation

Pertaining to prognostication, for each patient, three prognostic assessments were documented. The first assessment was the oncologist-predicted life expectancy as recorded at the time of referral. The second assessment was the Tokuhashi scores calculated independently by spinal fellows based on the clinical data provided, using the revised Tokuhashi scoring criteria. The third prognostication assessment was ChatGPT-based estimations, obtained in two stages. The first stage was literature-based prediction: ChatGPT-4.5 (OpenAI, San Francisco, CA, USA; accessed March 2025) was first prompted to predict life expectancy after reviewing the anonymized slide and performing a literature-based interpretation. The second stage was Tokuhashi-based prediction in which ChatGPT was then prompted to independently calculate a Tokuhashi score from the clinical information and predict survival based on published Tokuhashi thresholds. Predictions were generated across at least five separate chat sessions.

To ensure reproducibility and minimize variability in AI responses, ChatGPT was queried using a standardized prompt format following each anonymized clinical slide, requesting life expectancy estimation based on structured clinical analysis. To further enhance consistency and accurate interpretation, one example slide was initially used to calibrate ChatGPT’s recognition of abbreviations, clinical tables, and formatting conventions before formal predictions were conducted. Importantly, neither the oncologist-predicted life expectancy nor the surgeon-calculated Tokuhashi scores were included in the slides reviewed by ChatGPT. Cases with missing variables were excluded; therefore, ChatGPT predictions were based only on slides containing complete study variables, and no imputation of missing data was performed.

The prompts used are available in the Appendix.

Survival data

Actual survival outcomes were collected by reviewing hospital records, oncology clinic notes, and death registry data. Survival was categorized into three groups: <6 months, six to 12 months, and >12 months.

Statistical analysis

Prediction performance was evaluated by comparing estimated survival categories to actual survival outcomes. Performance metrics included overall accuracy, recall (sensitivity), precision (positive predictive value), contingency table-based analysis, and Cohen’s kappa coefficients for inter-method agreement.

In addition, logistic regression, decision tree, random forest, and XGBoost machine learning models were developed to identify the most influential predictors of survival based on the clinical variables. Feature importance analysis was performed to rank predictors within each model.

The chi-square test was performed using the full contingency table of the predicted versus actual survival categories across all methods to assess differences in prediction distributions. A p-value < 0.05 was considered statistically significant. All statistical analyses were performed using Python (v3.10), scikit-learn (v1.3), and SciPy.

Results

Patient cohort

A total of 100 patients (n = 100) with MSCC were included in the analysis. The cohort comprised both male and female patients across a wide age range, with prostate, breast, and lung cancers representing the most common primary tumors. All patients had radiologically confirmed MSCC at presentation. The baseline patient characteristics are summarized in Table 1 and Figure 2 .

Overall, most patients demonstrated preserved neurological function and a high burden of systemic metastatic disease. Regarding survival outcomes, 56 (56%) patients survived <6 months, 13 (13%) survived six to 12 months, and 31 (31%) survived >12 months.

Prediction performance

Overall prediction performance across methods is summarized in Table 2 . ChatGPT Tokuhashi-based prediction showed the highest overall accuracy, followed by surgeon-calculated Tokuhashi, oncologist clinical judgment, and ChatGPT literature-based prediction. Differences in overall prediction accuracy between methods were assessed using the chi-square test (χ² = 6.38, p = 0.095), with statistical significance defined as p < 0.05. No significant differences were observed. Agreement with actual survival, measured by Cohen’s kappa, was fair for oncologist judgment (κ = 0.26) and ChatGPT Tokuhashi (κ = 0.22), and slight for surgeon Tokuhashi (κ = 0.16) and ChatGPT literature-based estimates (κ = 0.10).

When stratified by actual survival outcomes, important differences emerged. The distribution of predicted survival categories across methods and actual survival groups is presented in Table 3 . Recall by survival group across methods is illustrated in Figure 3 .

For short-term survival (<6 months), surgeon-calculated Tokuhashi and ChatGPT Tokuhashi provided the highest recall (70% and 68%, respectively), markedly outperforming oncologist judgment (30%) and ChatGPT literature-based prediction (32%). For intermediate survival (six to 12 months), predictive performance was poor across all methods, with recall ranging from 23% to 54%. No method demonstrated consistent reliability within this subgroup. For long-term survival (>12 months), oncologists performed better (74% recall), whereas Tokuhashi-based predictions, both human and AI-assisted, showed lower sensitivity (19% and 32%, respectively). ChatGPT literature-based predictions demonstrated moderate performance (48% recall). Notably, misclassification was most frequent among patients who survived 6-12 months, confirming this intermediate group as the most difficult to predict reliably, regardless of prognostic approach; however, this should be interpreted cautiously given the small subgroup size. This subgroup represents a critical clinical challenge, as treatment decisions-particularly regarding surgical intervention-are highly sensitive to survival estimates in this timeframe.

Prediction model coefficients for survival categories are detailed in Table 4 . Functional status, as measured by Karnofsky Performance Status, was associated with longer survival, while the presence and extent of metastatic disease showed the strongest association with short-term survival. Age and MSCC demonstrated comparatively smaller effects across survival categories.

Exploratory machine learning feature importance

Machine learning exploratory analyses using decision tree, random forest, and XGBoost models were utilized to capture potential non-linear relationships and interactions between predictors. Across the exploratory machine learning analyses, functional status (Karnofsky score) and age appeared to be the most influential predictors of survival within this cohort, as shown in Table 5 and Table 6 . By contrast, traditional oncological factors such as primary tumor histology, number of vertebral metastases, and presence of MSCC showed minimal predictive value.

Discussion

Accurate survival prediction in MSCC remains a central challenge in spine oncology. In this study, we critically compared survival estimations made by oncologists, surgeon-calculated Tokuhashi scores, and ChatGPT-assisted predictions using a real-world cohort of MSCC patients. Our findings highlight that while prediction accuracy remains modest across all methods, Tokuhashi-based predictions, both human and AI-assisted, outperformed oncologist judgment for short-term survival (<6 months), while for long-term survival (>12 months), oncologists performed better. Intermediate-term survival (six to 12 months) continues to represent a major predictive gap. Although differences in overall prediction accuracy were observed between methods, these did not reach statistical significance. By contrast, the distribution of predicted survival categories differed significantly across survival groups and methods. This emphasizes substantial variability in predictive performance across methods depending on the survival category. This suggests that while overall prediction accuracy of the various methods may be comparable, clinically significant differences emerge when performance is examined within specific survival groups.

These results align with recent findings from Cox et al. [ 7 ], who demonstrated that oncologists predict short-term survival with reasonable sensitivity but struggle significantly in the intermediate range. Similarly, earlier work by Quraishi et al. [ 6 ] questioned the contemporary applicability of Tokuhashi scoring systems developed before advances in systemic therapies. More recent studies have demonstrated that traditional scoring systems, such as Tokuhashi and Tomita, show reduced predictive accuracy as oncologic management has evolved with modern systemic therapies [ 15 - 16 ]. Despite these historical limitations, our data suggest that structured scoring systems retain clinical value for identifying patients with poor prognosis, particularly when applied consistently. ChatGPT-assisted predictions based on Tokuhashi criteria achieved comparable performance to spinal surgeons, emphasizing the potential utility of AI models to support-but not replace-clinical decision-making.

An important limitation is that the different prognostic approaches were not based on identical inputs. Oncologists likely had access to broader longitudinal oncological context, including treatment response, disease trajectory, and remaining systemic therapy options, whereas surgeon-calculated Tokuhashi scores and ChatGPT-based predictions were derived only from the anonymized clinical slides. This may partly explain the better oncologist performance for longer-term survival, while also highlighting the difficulty of prognostication in MSCC.

The slight differences observed between ChatGPT-calculated Tokuhashi scores and surgeon-calculated scores likely reflect differences in rule-based implementation versus clinician-applied judgment. While ChatGPT systematically applied the scoring rules based purely on provided data, surgeons may have incorporated implicit clinical judgment, particularly in ambiguous cases (e.g., borderline functional status or mixed metastatic patterns). However, the small absolute difference in prediction accuracy of ChatGPT-calculated Tokuhashi scores and surgeon-calculated (53% vs. 49%) and similar recall rates for <6 months survival (68% vs. 70%) suggest that these discrepancies were minor and unlikely to be clinically meaningful.

While logistic regression analyses demonstrated strong associations between metastatic cancer burden and survival category, tree-based machine learning models, including random forest, decision tree, and XGBoost, consistently identified Karnofsky Performance Status as the strongest predictor of survival, with age being the second strongest predictor. This difference reflects methodological differences between the statistical measures rather than true inconsistency. Logistic regression coefficients quantify linear log-odds associations per unit change. They can emphasize variables with strong directional effects, such as the presence and count of metastatic cancer. On the other hand, tree-based models capture nonlinear relationships and variable interactions. This led to Karnofsky Performance Status emerging as the dominant variable in the prediction models. We also acknowledge that heterogeneity in primary tumor type may have influenced prediction performance, particularly in the >12-month survival group, where more favorable tumor biology and greater responsiveness to systemic therapy may have contributed to longer survival. Overall, these findings, which should be interpreted cautiously given the small sample size, class imbalance, and lack of external validation, are still consistent with prior observations [ 6 , 16 ].

While AI applications in spinal oncology are expanding, most efforts to date have concentrated on imaging classification tasks rather than structured clinical survival prediction [ 17 - 18 ]. In addition, even advanced AI models encounter difficulties when predicting complex survival endpoints, particularly when intermediate time frames are considered [ 19 - 20 ]. Our study addresses this critical gap by applying AI models to real-world structured clinical data in a prognostic context.

Clinically, our findings underscore the persistent difficulty in accurately predicting survival between six and 12 months - a range where surgical decision-making often hinges most critically on life expectancy. The modest performance of all methods in this subgroup reinforces the need for cautious, multidisciplinary evaluation rather than sole reliance on scoring systems or emerging AI tools. At present, AI-assisted prediction can serve as a supportive adjunct to structured human judgment but should not supplant it.

Future research should focus on integrating dynamic, longitudinal clinical data and developing hybrid models that combine oncological, functional, and patient-reported outcomes to improve prognostication. External validation across multiple institutions and prospective testing in clinical decision workflows will be essential to move beyond theoretical performance into meaningful clinical impact. Future studies with larger cohorts may also benefit from tumour-specific or continuous survival modelling to better characterize heterogeneity within the longer-survival group.

Several limitations warrant acknowledgment. First, while data were anonymized and presented consistently, minor ambiguities inherent to real-world clinical documentation may have influenced predictions. Second, although ChatGPT applied literature-based reasoning and Tokuhashi calculation without bias, its predictions are still bounded by the limitations of the data and scoring systems provided. Third, the study cohort, while reflective of typical MSCC referrals, originates from a single tertiary center and may not fully capture broader oncological variability. Fourth, we did not prospectively retain a structured item-level record of individual revised Tokuhashi score components for both surgeon- and ChatGPT-derived assessments. Therefore, although the underlying clinical variables were available on the anonymized slides and the surgeon-calculated Tokuhashi score was removed before presentation to ChatGPT, formal post hoc domain-level comparison between the two approaches could not be performed reliably. In addition, the relatively small sample size may limit the power to detect differences in overall accuracy despite statistical tests being performed. Finally, although a standardized prompt framework was used, large language model outputs may still vary according to prompt phrasing and session context.

Conclusions

Structured prognostic frameworks, particularly Tokuhashi scoring, continue to provide meaningful guidance in estimating short-term survival among patients with MSCC. Augmentation with AI-assisted calculations, such as those generated by ChatGPT, achieved comparable performance to experienced spinal surgeons in predicting short survival, highlighting the potential role of AI tools as clinical adjuncts. For long-term survival, oncologists performed better. However, intermediate survival prediction remains a critical unmet need across all modalities, reinforcing the necessity for multidisciplinary evaluation and cautious interpretation of prognostic estimates. Functional status and patient age emerged as more reliable survival predictors than traditional oncological variables, suggesting that future prognostic strategies should prioritize dynamic functional metrics over categorical tumor-based scores. Prospective validation and integration of longitudinal clinical data will be essential to refine and personalize survival prediction in spinal oncology.

Disclosures

Human subjects: Informed consent for treatment and open access publication was obtained or waived by all participants in this study.

Animal subjects: All authors have confirmed that this study did not involve animal subjects or tissue.

Conflicts of interest: In compliance with the ICMJE uniform disclosure form, all authors declare the following:

Payment/services info: All authors have declared that no financial support was received from any organization for the submitted work.

Financial relationships: All authors have declared that they have no financial relationships at present or within the previous three years with any organizations that might have an interest in the submitted work.

Other relationships: All authors have declared that there are no other relationships or activities that could appear to have influenced the submitted work.

Author Contributions

Concept and design: Jonathan Dimitry, Rawan Masarwa, Khalid Salem, Nasir A. Quraishi, Elie Najjar, Vaishnavi Sharma

Acquisition, analysis, or interpretation of data: Jonathan Dimitry, Rawan Masarwa, Khalid Salem, Nasir A. Quraishi, Elie Najjar, Vaishnavi Sharma

Drafting of the manuscript: Jonathan Dimitry, Rawan Masarwa, Khalid Salem, Nasir A. Quraishi, Elie Najjar, Vaishnavi Sharma

Critical review of the manuscript for important intellectual content: Jonathan Dimitry, Rawan Masarwa, Khalid Salem, Nasir A. Quraishi, Elie Najjar, Vaishnavi Sharma

Supervision: Elie Najjar

Appendices

ChatGPT prompts used for this study

Familiarization/Calibration Prompt

You will be shown anonymized clinical summary slides for patients with metastatic spinal disease.

Before formal predictions begin, this example slide is provided only so that you can familiarize yourself with the structure and formatting of the clinical summaries, including common abbreviations, clinical tables, imaging summaries, and layout conventions.

Please review the slide and confirm that you understand the abbreviations, format, and structure of the information presented.

Do not use this example for analysis in the study dataset. Do not retain or refer back to this case when assessing subsequent patients.

Literature-Based Survival Prediction Prompt

You are reviewing an anonymized clinical summary of a patient with metastatic spinal cord compression.

Based only on the information provided in this slide, estimate the patient’s life expectancy using clinical reasoning and general medical knowledge from the literature. Do not calculate or use a Tokuhashi score unless explicitly asked.

Please consider the following factors where relevant: age, primary tumour type, extent of metastatic disease, neurological status, functional status/Karnofsky Performance Status, imaging findings, prior or current oncological treatment, and other major comorbid or clinical features shown on the slide.

After reviewing the case, assign the patient to one of the following survival categories only: 1) <6 months, 2) six to 12 months, and 3) >12 months.

Please provide 1) the predicted survival category and 2) a brief explanation for the prediction based only on the slide content

Tokuhashi-Based Survival Prediction Prompt

You are reviewing an anonymized clinical summary of a patient with metastatic spinal cord compression.

Based only on the information provided in this slide, independently calculate the revised Tokuhashi score and then assign the predicted survival category according to the Tokuhashi framework.

Please 1) identify the relevant Tokuhashi variables from the slide, 2) calculate the total revised Tokuhashi score, and 3) assign the patient to one of the following survival categories only: <6 months, six to 12 months, and >12 months.

Please provide 1) the individual Tokuhashi components, 2) the total Tokuhashi score, 3) the predicted survival category, and 4) a brief explanation based only on the slide content.

References

1 Direct decompressive surgical resection in the treatment of spinal cord compression caused by metastatic cancer: a randomised trial Lancet Patchell R Tibbs P Regine W 643 648 366 2005 16112300 10.1016/S0140-6736(05)66954-1

2 Surgical strategy for spinal metastases Spine (Phila Pa 1976) Tomita K Kawahara N Kobayashi T Yoshida A Murakami H Akamaru T 298 306 26 2001 11224867 10.1097/00007632-200102010-00016

3 A revised scoring system for preoperative evaluation of metastatic spine tumor prognosis Spine (Phila Pa 1976) Tokuhashi Y Matsuzaki H Oda H Oshima M Ryu J 2186 2191 30 2005 16205345 10.1097/01.brs.0000180401.06919.a5

4 WHO are the best candidates for decompressive surgery and spine stabilization in patients with metastatic spinal cord compression?: a new scoring system Spine (Phila Pa 1976) Lei M Li J Liu Y Jiang W Liu S Zhou S 1469 1476 41 2016 26937605 10.1097/BRS.0000000000001538 PMC5001136

5 A validated survival score for patients with metastatic spinal cord compression from non-small cell lung cancer BMC Cancer Rades D Douglas S Veninga T Schild SE 302 12 2012 22817686 10.1186/1471-2407-12-302 PMC3411487

6 Accuracy of the revised Tokuhashi score in predicting survival in patients with metastatic spinal cord compression (MSCC) Eur Spine J Quraishi NA Manoharan SR Arealis G Khurana A Elsayed S Edwards KL Boszczyk BM 0 6 22 Suppl 1 2013 10.1007/s00586-012-2649-5 PMC3578515 23328875

7 Six-month survivorship prediction in spinal metastatic patients by oncologists shows reliable prognostication Global Spine J Cox K Ahmed H Saha P 961 968 15 2025 38009792 10.1177/21925682231218712 PMC11877491

8 Dermatologist-level classification of skin cancer with deep neural networks Nature Esteva A Kuprel B Novoa RA Ko J Swetter SM Blau HM Thrun S 115 118 542 2017 28117445 10.1038/nature21056 PMC8382232

9 A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis Lancet Digit Health Liu X Faes L Kale AU 271 1 2019 10.1016/S2589-7500(19)30123-2 33323251

10 Machine learning applications in cancer prognosis and prediction Comput Struct Biotechnol J Kourou K Exarchos TP Exarchos KP Karamouzis MV Fotiadis DI 8 17 13 2015 25750696 10.1016/j.csbj.2014.11.005 PMC4348437

11 Artificial intelligence and machine learning in spine research JOR Spine Galbusera F Casaroli G Bassani T 0 2 2019 10.1002/jsp2.1044 PMC6686793 31463458

12 Artificial learning and machine learning applications in spine surgery: a systematic review Global Spine J Lopez CD Boddapati V Lombardi JM 1561 1572 12 2022 35227128 10.1177/21925682211049164 PMC9393994

13 Key challenges for delivering clinical impact with artificial intelligence BMC Med Kelly CJ Karthikesalingam A Suleyman M Corrado G King D 195 17 2019 31665002 10.1186/s12916-019-1426-2 PMC6821018

14 Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal BMJ Wynants L Van Calster B Collins GS 0 369 2020 10.1136/bmj.m1328 PMC7222643 32265220

15 The Oswestry Risk Index: an aid in the treatment of metastatic disease of the spine Bone Joint J Balain B Jaiswal A Trivedi JM Eisenstein SM Kuiper JH Jaffray DC 210 216 95-B 2013 23365031 10.1302/0301-620X.95B2.29323

16 Predicting survival for metastatic spine disease: a comparison of nine scoring systems Spine J Ahmed AK Goodwin CR Heravi A 1804 1814 18 2018 29567516 10.1016/j.spinee.2018.03.011

17 Using a machine learning approach to predict outcome after surgery for degenerative cervical myelopathy PLoS One Merali ZG Witiw CD Badhiwala JH Wilson JR Fehlings MG 0 14 2019 10.1371/journal.pone.0215133 PMC6448910 30947300

18 Machine learning and neurosurgical outcome prediction: a systematic review World Neurosurg Senders JT Staples PC Karhade AV 476 486 109 2018 28986230 10.1016/j.wneu.2017.09.149

19 Machine learning algorithms for objective remission and clinical outcomes with thiopurines J Crohns Colitis Waljee AK Sauder K Patel A 801 810 11 2017 28333183 10.1093/ecco-jcc/jjx014 PMC5881698

20 Big data and machine learning in radiation oncology: state of the art and future prospects Cancer Lett Bibault JE Giraud P Burgun A 110 117 382 2016 27241666 10.1016/j.canlet.2016.05.033