1
|
Ton A, Wishart D, Ball JR, Shah I, Murakami K, Ordon MP, Alluri RK, Hah R, Safaee MM. The Evolution of Risk Assessment in Spine Surgery: A Narrative Review. World Neurosurg 2024; 188:1-14. [PMID: 38677646 DOI: 10.1016/j.wneu.2024.04.117] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/17/2024] [Revised: 04/17/2024] [Accepted: 04/18/2024] [Indexed: 04/29/2024]
Abstract
BACKGROUND Risk assessment is critically important in elective and high-risk interventions, particularly spine surgery. This narrative review describes the evolution of risk assessment from the earliest instruments focused on general surgical risk stratification, to more accurate and spine-specific risk calculators that quantified risk, to the current era of big data. METHODS The PubMed and SCOPUS databases were queried on October 11, 2023 using search terms to identify risk assessment tools (RATs) in spine surgery. A total of 108 manuscripts were included after screening with full-text review using the following inclusion criteria: 1) study population of adult spine surgical patients, 2) studies describing validation and subsequent performance of preoperative RATs, and 3) studies published in English. RESULTS Early RATs provided stratified patients into broad categories and allowed for improved communication between physicians. Subsequent risk calculators attempted to quantify risk by estimating general outcomes such as mortality, but then evolved to estimate spine-specific surgical complications. The integration of novel concepts such as invasiveness, frailty, genetic biomarkers, and sarcopenia led to the development of more sophisticated predictive models that estimate the risk of spine-specific complications and long-term outcomes. CONCLUSIONS RATs have undergone a transformative shift from generalized risk stratification to quantitative predictive models. The next generation of tools will likely involve integration of radiographic and genetic biomarkers, machine learning, and artificial intelligence to improve the accuracy of these models and better inform patients, surgeons, and payers.
Collapse
Affiliation(s)
- Andy Ton
- Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Danielle Wishart
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Jacob R Ball
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Ishan Shah
- Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Kiley Murakami
- Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Matthew P Ordon
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - R Kiran Alluri
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Raymond Hah
- Department of Orthopedic Surgery, Keck School of Medicine, University of Southern California, Los Angeles, California, USA
| | - Michael M Safaee
- Department of Neurological Surgery, Keck School of MedicineUniversity of Southern California, Los Angeles, California, USA.
| |
Collapse
|
2
|
Karabacak M, Schupper A, Carr M, Margetis K. A machine learning-based approach for individualized prediction of short-term outcomes after anterior cervical corpectomy. Asian Spine J 2024; 18:541-549. [PMID: 39113482 PMCID: PMC11366553 DOI: 10.31616/asj.2024.0048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/05/2024] [Revised: 03/20/2024] [Accepted: 04/15/2024] [Indexed: 09/03/2024] Open
Abstract
STUDY DESIGN A retrospective machine learning (ML) classification study for prognostic modeling after anterior cervical corpectomy (ACC). PURPOSE To evaluate the effectiveness of ML in predicting ACC outcomes and develop an accessible, user-friendly tool for this purpose. OVERVIEW OF LITERATURE Based on our literature review, no study has examined the capability of ML algorithms to predict major shortterm ACC outcomes, such as prolonged length of hospital stay (LOS), non-home discharge, and major complications. METHODS The American College of Surgeons' National Surgical Quality Improvement Program database was used to identify patients who underwent ACC. Prolonged LOS, non-home discharges, and major complications were assessed as the outcomes of interest. ML models were developed with the TabPFN algorithm and integrated into an open-access website to predict these outcomes. RESULTS The models for predicting prolonged LOS, non-home discharges, and major complications demonstrated mean areas under the receiver operating characteristic curve (AUROC) of 0.802, 0.816, and 0.702, respectively. These findings highlight the discriminatory capacities of the models: fair (AUROC >0.7) for differentiating patients with major complications from those without, and good (AUROC >0.8) for distinguishing between those with and without prolonged LOS and non-home discharges. According to the SHapley Additive Explanations analysis, single- versus multiple-level surgery, age, body mass index, preoperative hematocrit, and American Society of Anesthesiologists physical status repetitively emerged as the most important variables for each outcome. CONCLUSIONS This study has considerably enhanced the prediction of postoperative results after ACC surgery by implementing advanced ML techniques. A major contribution is the creation of an accessible web application, highlighting the practical value of the developed models. Our findings imply that ML can serve as an invaluable supplementary tool to stratify patient risk for this procedure and can predict diverse postoperative adverse outcomes.
Collapse
Affiliation(s)
- Mert Karabacak
- Department of Neurosurgery, Mount Sinai Health System, New York, NY, USA
| | - Alexander Schupper
- Department of Neurosurgery, Mount Sinai Health System, New York, NY, USA
| | - Matthew Carr
- Department of Neurosurgery, Mount Sinai Health System, New York, NY, USA
| | | |
Collapse
|
3
|
Karabacak M, Bhimani AD, Schupper AJ, Carr MT, Steinberger J, Margetis K. Machine learning models on a web application to predict short-term postoperative outcomes following anterior cervical discectomy and fusion. BMC Musculoskelet Disord 2024; 25:401. [PMID: 38773464 PMCID: PMC11110429 DOI: 10.1186/s12891-024-07528-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Accepted: 05/15/2024] [Indexed: 05/23/2024] Open
Abstract
BACKGROUND The frequency of anterior cervical discectomy and fusion (ACDF) has increased up to 400% since 2011, underscoring the need to preoperatively anticipate adverse postoperative outcomes given the procedure's expanding use. Our study aims to accomplish two goals: firstly, to develop a suite of explainable machine learning (ML) models capable of predicting adverse postoperative outcomes following ACDF surgery, and secondly, to embed these models in a user-friendly web application, demonstrating their potential utility. METHODS We utilized data from the National Surgical Quality Improvement Program database to identify patients who underwent ACDF surgery. The outcomes of interest were four short-term postoperative adverse outcomes: prolonged length of stay (LOS), non-home discharges, 30-day readmissions, and major complications. We utilized five ML algorithms - TabPFN, TabNET, XGBoost, LightGBM, and Random Forest - coupled with the Optuna optimization library for hyperparameter tuning. To bolster the interpretability of our models, we employed SHapley Additive exPlanations (SHAP) for evaluating predictor variables' relative importance and used partial dependence plots to illustrate the impact of individual variables on the predictions generated by our top-performing models. We visualized model performance using receiver operating characteristic (ROC) curves and precision-recall curves (PRC). Quantitative metrics calculated were the area under the ROC curve (AUROC), balanced accuracy, weighted area under the PRC (AUPRC), weighted precision, and weighted recall. Models with the highest AUROC values were selected for inclusion in a web application. RESULTS The analysis included 57,760 patients for prolonged LOS [11.1% with prolonged LOS], 57,780 for non-home discharges [3.3% non-home discharges], 57,790 for 30-day readmissions [2.9% readmitted], and 57,800 for major complications [1.4% with major complications]. The top-performing models, which were the ones built with the Random Forest algorithm, yielded mean AUROCs of 0.776, 0.846, 0.775, and 0.747 for predicting prolonged LOS, non-home discharges, readmissions, and complications, respectively. CONCLUSIONS Our study employs advanced ML methodologies to enhance the prediction of adverse postoperative outcomes following ACDF. We designed an accessible web application to integrate these models into clinical practice. Our findings affirm that ML tools serve as vital supplements in risk stratification, facilitating the prediction of diverse outcomes and enhancing patient counseling for ACDF.
Collapse
Affiliation(s)
- Mert Karabacak
- Department of Neurosurgery, Mount Sinai Health System, 1468 Madison Ave, New York, NY, 10029, USA
| | - Abhiraj D Bhimani
- Department of Neurosurgery, Mount Sinai Health System, 1468 Madison Ave, New York, NY, 10029, USA
| | - Alexander J Schupper
- Department of Neurosurgery, Mount Sinai Health System, 1468 Madison Ave, New York, NY, 10029, USA
| | - Matthew T Carr
- Department of Neurosurgery, Mount Sinai Health System, 1468 Madison Ave, New York, NY, 10029, USA
| | - Jeremy Steinberger
- Department of Neurosurgery, Mount Sinai Health System, 1468 Madison Ave, New York, NY, 10029, USA
| | - Konstantinos Margetis
- Department of Neurosurgery, Mount Sinai Health System, 1468 Madison Ave, New York, NY, 10029, USA.
| |
Collapse
|
4
|
Adida S, Legarreta AD, Hudson JS, McCarthy D, Andrews E, Shanahan R, Taori S, Lavadi RS, Buell TJ, Hamilton DK, Agarwal N, Gerszten PC. Machine Learning in Spine Surgery: A Narrative Review. Neurosurgery 2024; 94:53-64. [PMID: 37930259 DOI: 10.1227/neu.0000000000002660] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2023] [Accepted: 07/06/2023] [Indexed: 11/07/2023] Open
Abstract
Artificial intelligence and machine learning (ML) can offer revolutionary advances in their application to the field of spine surgery. Within the past 5 years, novel applications of ML have assisted in surgical decision-making, intraoperative imaging and navigation, and optimization of clinical outcomes. ML has the capacity to address many different clinical needs and improve diagnostic and surgical techniques. This review will discuss current applications of ML in the context of spine surgery by breaking down its implementation preoperatively, intraoperatively, and postoperatively. Ethical considerations to ML and challenges in ML implementation must be addressed to maximally benefit patients, spine surgeons, and the healthcare system. Areas for future research in augmented reality and mixed reality, along with limitations in generalizability and bias, will also be highlighted.
Collapse
Affiliation(s)
- Samuel Adida
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Andrew D Legarreta
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Joseph S Hudson
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - David McCarthy
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Edward Andrews
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Regan Shanahan
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Suchet Taori
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Raj Swaroop Lavadi
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Thomas J Buell
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - D Kojo Hamilton
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| | - Nitin Agarwal
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
- Department of Neurosurgery, University of Pittsburgh Medical Center, Pittsburgh , Pennsylvania , USA
| | - Peter C Gerszten
- Department of Neurosurgery, University of Pittsburgh School of Medicine, Pittsburgh , Pennsylvania , USA
| |
Collapse
|
5
|
Levy AS, Bhatia S, Merenzon MA, Andryski AL, Rivera CA, Daggubati LC, Di L, Shah AH, Komotar RJ, Ivan ME. Exploring the Landscape of Machine Learning Applications in Neurosurgery: A Bibliometric Analysis and Narrative Review of Trends and Future Directions. World Neurosurg 2024; 181:108-115. [PMID: 37839564 DOI: 10.1016/j.wneu.2023.10.042] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Revised: 10/08/2023] [Accepted: 10/08/2023] [Indexed: 10/17/2023]
Abstract
BACKGROUND The field of neurosurgery has consistently represented an area of innovation and integration of technology since its inception. As such, machine learning (ML) has found its way into applications within neurosurgery relatively rapidly. Through this bibliometric review and cluster analysis, we seek to identify trends and emerging applications of ML within neurosurgery. METHODS A bibliometric analysis was carried out in the Web of Science database on publications from January 2000 to March 2023. The full data set of the 200 most cited publications including title, author information, journal, citation count, keywords, and abstracts for each publication was evaluated in CiteSpace. CiteSpace was used to elucidate publication characteristics, trends, and topic clusters via collaborate network analysis using the Kamada-Kawai algorithm. RESULTS The 25 most cited titles were included in our analysis. Harvard University and its affiliates represented the top institution, contributing nearly 25% of publications in the literature. WORLD NEUROSURGERY was the journal with the highest net citation count of 747 (29%). Collaborative network analysis generated 12 unique clusters, the largest of which was machine learning, followed by feature importance and deep brain stimulation. CONCLUSION This review highlights the most impactful articles pertaining to ML in the field of neurosurgery. ML has been applied into several sub-specialties within neurosurgery to optimize patient care, with special attention to outcome predictors, patient selection, and surgical decision making.
Collapse
Affiliation(s)
- Adam S Levy
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA.
| | - Shovan Bhatia
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA
| | - Martin A Merenzon
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA
| | - Allie L Andryski
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA
| | - Cameron A Rivera
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA
| | - Lekhaj C Daggubati
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA
| | - Long Di
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA
| | - Ashish H Shah
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA
| | - Ricardo J Komotar
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA; Sylvester Cancer Center, University of Miami Health System, Miami, Florida, USA
| | - Michael E Ivan
- Department of Neurological Surgery, University of Miami, Miller School of Medicine, Miami, Florida, USA; Sylvester Cancer Center, University of Miami Health System, Miami, Florida, USA
| |
Collapse
|
6
|
Ghanem M, Ghaith AK, El-Hajj VG, Bhandarkar A, de Giorgio A, Elmi-Terander A, Bydon M. Limitations in Evaluating Machine Learning Models for Imbalanced Binary Outcome Classification in Spine Surgery: A Systematic Review. Brain Sci 2023; 13:1723. [PMID: 38137171 PMCID: PMC10741524 DOI: 10.3390/brainsci13121723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Revised: 12/12/2023] [Accepted: 12/15/2023] [Indexed: 12/24/2023] Open
Abstract
Clinical prediction models for spine surgery applications are on the rise, with an increasing reliance on machine learning (ML) and deep learning (DL). Many of the predicted outcomes are uncommon; therefore, to ensure the models' effectiveness in clinical practice it is crucial to properly evaluate them. This systematic review aims to identify and evaluate current research-based ML and DL models applied for spine surgery, specifically those predicting binary outcomes with a focus on their evaluation metrics. Overall, 60 papers were included, and the findings were reported according to the PRISMA guidelines. A total of 13 papers focused on lengths of stay (LOS), 12 on readmissions, 12 on non-home discharge, 6 on mortality, and 5 on reoperations. The target outcomes exhibited data imbalances ranging from 0.44% to 42.4%. A total of 59 papers reported the model's area under the receiver operating characteristic (AUROC), 28 mentioned accuracies, 33 provided sensitivity, 29 discussed specificity, 28 addressed positive predictive value (PPV), 24 included the negative predictive value (NPV), 25 indicated the Brier score with 10 providing a null model Brier, and 8 detailed the F1 score. Additionally, data visualization varied among the included papers. This review discusses the use of appropriate evaluation schemes in ML and identifies several common errors and potential bias sources in the literature. Embracing these recommendations as the field advances may facilitate the integration of reliable and effective ML models in clinical settings.
Collapse
Affiliation(s)
- Marc Ghanem
- Mayo Clinic Neuro-Informatics Laboratory, Mayo Clinic, Rochester, MN 55902, USA; (M.G.); (A.K.G.); (V.G.E.-H.); (A.B.); (M.B.)
- Department of Neurological Surgery, Mayo Clinic, Rochester, MN 55902, USA
- School of Medicine, Lebanese American University, Byblos 4504, Lebanon
| | - Abdul Karim Ghaith
- Mayo Clinic Neuro-Informatics Laboratory, Mayo Clinic, Rochester, MN 55902, USA; (M.G.); (A.K.G.); (V.G.E.-H.); (A.B.); (M.B.)
- Department of Neurological Surgery, Mayo Clinic, Rochester, MN 55902, USA
| | - Victor Gabriel El-Hajj
- Mayo Clinic Neuro-Informatics Laboratory, Mayo Clinic, Rochester, MN 55902, USA; (M.G.); (A.K.G.); (V.G.E.-H.); (A.B.); (M.B.)
- Department of Neurological Surgery, Mayo Clinic, Rochester, MN 55902, USA
- Department of Clinical Neuroscience, Karolinska Institutet, 17177 Stockholm, Sweden
| | - Archis Bhandarkar
- Mayo Clinic Neuro-Informatics Laboratory, Mayo Clinic, Rochester, MN 55902, USA; (M.G.); (A.K.G.); (V.G.E.-H.); (A.B.); (M.B.)
- Department of Neurological Surgery, Mayo Clinic, Rochester, MN 55902, USA
| | - Andrea de Giorgio
- Artificial Engineering, Via del Rione Sirignano, 80121 Naples, Italy;
| | - Adrian Elmi-Terander
- Department of Clinical Neuroscience, Karolinska Institutet, 17177 Stockholm, Sweden
- Department of Surgical Sciences, Uppsala University, 75236 Uppsala, Sweden
| | - Mohamad Bydon
- Mayo Clinic Neuro-Informatics Laboratory, Mayo Clinic, Rochester, MN 55902, USA; (M.G.); (A.K.G.); (V.G.E.-H.); (A.B.); (M.B.)
- Department of Neurological Surgery, Mayo Clinic, Rochester, MN 55902, USA
| |
Collapse
|
7
|
Khazanchi R, Bajaj A, Shah RM, Chen AR, Reyes SG, Kurapaty SS, Hsu WK, Patel AA, Divi SN. Using Machine Learning and Deep Learning Algorithms to Predict Postoperative Outcomes Following Anterior Cervical Discectomy and Fusion. Clin Spine Surg 2023; 36:143-149. [PMID: 36920355 DOI: 10.1097/bsd.0000000000001443] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/16/2022] [Accepted: 01/25/2023] [Indexed: 03/16/2023]
Abstract
STUDY DESIGN A retrospective cohort study from a multisite academic medical center. OBJECTIVE To construct, evaluate, and interpret a series of machine learning models to predict outcomes related to inpatient health care resource utilization for patients undergoing anterior cervical discectomy and fusion (ACDF). SUMMARY OF BACKGROUND DATA Reducing postoperative health care utilization is an important goal for improving the delivery of surgical care and serves as a metric for quality assessment. Recent data has shown marked hospital resource utilization after ACDF surgery, including readmissions, and ED visits. The burden of postoperative health care use presents a potential application of machine learning techniques, which may be capable of accurately identifying at-risk patients using patient-specific predictors. METHODS Patients 18-88 years old who underwent ACDF from 2011 to 2021 at a multisite academic center and had preoperative lab values within 3 months of surgery were included. Outcomes analyzed included 90-day readmissions, postoperative length of stay, and nonhome discharge. Four machine learning models-Extreme Gradient Boosted Trees, Balanced Random Forest, Elastic-Net Penalized Logistic Regression, and a Neural Network-were trained and evaluated through the Area Under the Curve estimates. Feature importance scores were computed for the highest-performing model per outcome through model-specific metrics. RESULTS A total of 1026 cases were included in the analysis cohort. All machine learning models were predictive for outcomes of interest, with the Random Forest algorithm consistently demonstrating the strongest average area under the curve performance, with a peak performance of 0.84 for nonhome discharge. Important features varied per outcome, though age, body mass index, American Society of Anesthesiologists classification >2, and medical comorbidities were highly weighted in the studied outcomes. CONCLUSIONS Machine learning models were successfully applied and predictive of postoperative health utilization after ACDF. Deployment of these tools can assist clinicians in determining high-risk patients. LEVEL OF EVIDENCE III.
Collapse
Affiliation(s)
- Rushmin Khazanchi
- Department of Orthopaedic Surgery, Northwestern University Feinberg School of Medicine, Chicago, IL
| | | | | | | | | | | | | | | | | |
Collapse
|