Analysis and Review of Automated Risk Calculators Used to Predict Postoperative Complications After Orthopedic Surgery

Prediction of 30-Day Mortality Following Revision Total Hip and Knee Arthroplasty: Machine Learning Algorithms Outperform CARDE-B, 5-Item, and 6-Item Modified Frailty Index Risk Scores

BACKGROUND

Although risk calculators are used to prognosticate postoperative outcomes following revision total hip and knee arthroplasty (TJA), machine learning (ML) based predictive tools have emerged as a promising alternative for improved risk stratification. This study aimed to compare the predictive ability of ML models for 30-day mortality following revision TJA to that of traditional risk-assessment indices such as the CARDE-B score (congestive heart failure, albumin (<3.5 mg/dL), renal failure on dialysis, dependence for daily living, elderly (>65 years of age), and body mass index of <25 kg/m2), 5-item (5MFI), and 6-item modified frailty index (6MFI).

METHODS

Adult patients undergoing revision TJA between 2013 and 2020 were selected from the ACS-NSQIP database and randomly split 80:20 to compose the training and validation cohorts. There were three ML models - extreme gradient boosting (XGB), random forest (RF), and elastic-net penalized logistic regression (NEPLR) - that were developed and evaluated using discrimination, calibration metrics, and accuracy. The discrimination of CARDE-B, 5MFI, and 6MFI scores was assessed individually and compared to that of ML models.

RESULTS

All models were equally accurate (Brier score = 0.005) and demonstrated outstanding discrimination with similar areas (AUC) under the receiver operating characteristic curve (XGB = 0.94, RF = NEPLR = 0.93). The NEPLR was the best-calibrated model overall (slope = 0.54, intercept = -0.004). The CARDE-B had the highest discrimination among the scores (AUC = 0.89), followed by 6MFI (AUC = 0.80), and 5MFI (AUC = 0.68). Albumin < 3.5 mg/dL and BMI (< 30.15) were the most important predictors of 30-day mortality following revision TJA.

CONCLUSIONS

The ML models outperform traditional risk-assessment indices in predicting postoperative 30-day mortality after revision TJA. Our findings highlight the utility of ML for risk stratification in a clinical setting. The identification of hypoalbuminemia and BMI as prognostic markers may allow patient-specific perioperative optimization strategies to improve outcomes following revision TJA.

The Evolution of Risk Assessment in Spine Surgery: A Narrative Review

BACKGROUND

Risk assessment is critically important in elective and high-risk interventions, particularly spine surgery. This narrative review describes the evolution of risk assessment from the earliest instruments focused on general surgical risk stratification, to more accurate and spine-specific risk calculators that quantified risk, to the current era of big data.

METHODS

The PubMed and SCOPUS databases were queried on October 11, 2023 using search terms to identify risk assessment tools (RATs) in spine surgery. A total of 108 manuscripts were included after screening with full-text review using the following inclusion criteria: 1) study population of adult spine surgical patients, 2) studies describing validation and subsequent performance of preoperative RATs, and 3) studies published in English.

RESULTS

Early RATs provided stratified patients into broad categories and allowed for improved communication between physicians. Subsequent risk calculators attempted to quantify risk by estimating general outcomes such as mortality, but then evolved to estimate spine-specific surgical complications. The integration of novel concepts such as invasiveness, frailty, genetic biomarkers, and sarcopenia led to the development of more sophisticated predictive models that estimate the risk of spine-specific complications and long-term outcomes.

CONCLUSIONS

RATs have undergone a transformative shift from generalized risk stratification to quantitative predictive models. The next generation of tools will likely involve integration of radiographic and genetic biomarkers, machine learning, and artificial intelligence to improve the accuracy of these models and better inform patients, surgeons, and payers.

MBSAQIP risk calculator use in bariatric surgery is associated with a reduction in serious complications: a retrospective analysis of 210,710 patients

BACKGROUND

The Metabolic and Bariatric Accreditation and Quality Improvement Program (MBSAQIP) Bariatric Surgical Risk/Benefit Calculator was developed to provide patient-specific information to assist surgical decision-making. To date, no study has characterized which patients are being evaluated with this tool.

OBJECTIVE

We sought to characterize the use and impact of the MBSAQIP calculator.

SETTING

MBSAQIP collects data from 955 centers in North America.

METHODS

The 2021 MBSAQIP database was evaluated for the use of the calculator on preoperative counseling for patients undergoing bariatric surgery. Patient characteristics, operative techniques, and outcomes were compared with bivariate analysis. Multivariable modeling evaluated factors including use of the calculator independently associated with serious complications and mortality.

RESULTS

Our study included 210,710 patients, 35,158 (16.7%) of whom were evaluated using the calculator. Patients with whom the calculator was used preoperatively were older (43.8 ± 11.6 yr versus 43.6 ± 11.7 yr; P < .001) and were more likely to have insulin-dependent diabetes, hypertension, gastroesophageal reflux disease, renal insufficiency, and sleep apnea. More patients underwent Roux-en-Y gastric bypass in the calculator cohort compared with the cohort that did not use the calculator (29.6% versus 28.6%; P < .003). The rate of serious complication was significantly less in the calculator cohort (3.1% versus 3.4%; P < .030). Multivariable modeling evaluating serious complications showed that use of the calculator was independently associated with reduced risk of serious complications (odds ratio .87, CI .82-.93, P < .001) but was not associated with mortality.

CONCLUSION

The use of the risk calculator may help to reduce the incidence of complications by opening a dialogue between healthcare professionals and patients, setting realistic expectations, and identifying modifiable risk factors.

A Comparison of the Caprini Score With an Institutional Risk Assessment Tool for Prediction of Venous Thromboembolism After Total Joint Arthroplasty at an Urban Tertiary Care Health Safety Net Hospital

Background

Patients undergoing total joint arthroplasty (TJA) are at increased risk for venous thromboembolism (VTE). Prediction tools such as the Caprini Risk Assessment Model (RAM) have been developed to identify patients at higher risk. However, studies have reported heterogeneous results when assessing its efficacy for TJA. Patients treated in an urban health safety net hospital have increased medical complexity, advanced degenerative joint disease, and severe disability prior to TJA increasing the risk of VTE. We hypothesize that use of a tool designed to account for these conditions-the Boston Medical Center (BMC) VTE score-will more accurately predict VTE in this patient population.

Methods

A retrospective case-control study was performed including subjects 18 years of age and older who underwent primary or revision TJA in an urban academic health safety net hospital. Patients with hemiarthroplasties, simultaneous bilateral TJA, and TJA after acute trauma were excluded. A total of 80 subjects were included: 40 who developed VTE after TJA (VTE+) and 40 who did not develop VTE (controls). Subjects were matched by age, gender, and surgical procedure.

Results

There was a statistically significant difference between the mean BMC VTE score for VTE+ and controls (4.40 and 3.13, respectively, P = .036). Conversely, there was no statistical difference between the mean Caprini scores for VTE+ and controls (9.50 and 9.35, respectively, P = .797).

Conclusions

In a health safety-net patient population, an institutional RAM-the BMC VTE score-was found to be more predictive of VTE than the modified Caprini RAM following TJA. The BMC-VTE score should be externally validated to confirm its reliability in VTE prediction in similar patient populations.

Validation and Generalizability of Machine Learning Models for the Prediction of Discharge Disposition Following Revision Total Knee Arthroplasty

BACKGROUND

Postoperative discharge to facilities account for over 33% of the $ 2.7 billion revision total knee arthroplasty (TKA)-associated annual expenditures and are associated with increased complications when compared to home discharges. Prior studies predicting discharge disposition using advanced machine learning (ML) have been limited due to a lack of generalizability and validation. This study aimed to establish ML model generalizability by externally validating its prediction for nonhome discharge following revision TKA using national and institutional databases.

METHODS

The national and institutional cohorts comprised 52,533 and 1,628 patients, respectively, with 20.6 and 19.4% nonhome discharge rates. Five ML models were trained and internally validated (five-fold cross-validation) on a large national dataset. Subsequently, external validation was performed on our institutional dataset. Model performance was assessed using discrimination, calibration, and clinical utility. Global predictor importance plots and local surrogate models were used for interpretation.

RESULTS

The strongest predictors of nonhome discharge were patient age, body mass index, and surgical indication. The area under the receiver operating characteristic curve increased from internal to external validation and ranged between 0.77 and 0.79. Artificial neural network was the best predictive model for identifying patients at risk for nonhome discharge (area under the receiver operating characteristic curve = 0.78), and also the most accurate (calibration slope = 0.93, intercept = 0.02, and Brier score = 0.12).

CONCLUSION

All five ML models demonstrated good-to-excellent discrimination, calibration, and clinical utility on external validation, with artificial neural network being the best model for predicting discharge disposition following revision TKA. Our findings establish the generalizability of ML models developed using data from a national database. The integration of these predictive models into clinical workflow may assist in optimizing discharge planning, bed management, and cost containment associated with revision TKA.

The Use of Race in Risk Assessment Tools Contributes to Systemic Racism

Many patients suffer from hip or knee osteoarthritis and elect to pursue total joint arthroplasty (TJA). Though perioperative risk is an inherent component of surgery, calculators that assess the risk of complications following TJA can help both surgeons and patients make informed decisions about the risk of surgery and aid in shared decision-making discussions. The inclusion of race in a risk calculator for readmission after TJA is flawed and unacceptable because a patient's race does not increase their risk of a complication after total joint replacement.

Risk Factors for Postoperative Complications After Surgical Treatment of Type B and C Injuries of the Thoracolumbar Spine

BACKGROUND

Unstable thoracolumbar spinal injuries benefit from surgical fixation. However, perioperative complications significantly affect outcomes in surgicallytreated spine patients. We evaluated associations between risk factors and postoperative complications in patients surgically treated for thoracolumbar spine fractures.

METHODS

We conducted a retrospective multicenter study collating data from 21 spine centers across 9 countries on the treatment of AOSpine types B and C injuries of the thoracolumbar spine treated via a posterior approach. Comparative analysis was performed between patients with postoperative complications and those without. Univariate and multivariable analyses were performed.

RESULTS

Among 535 patients, at least 1 complication occurred in 43%. The most common surgical complication was surgical-site infection (6.9%), while the most common medical complication was urinary tract infection (13.8%). Among 136 patients with American Spinal Injury Association (ASIA) Impairment Scalelevel A disability, 77.9% experienced at least 1 complication. The rate of complications also rose sharply among patients waiting >3 days for surgery (P<0.001), peaking at 68.4% among patients waiting ≥30 days. On multivariable analysis, significant predictors of complications were surgery at a governmental hospital (odds ratio = 3.38, 95% confidence interval = 1.73-6.60), having ≥1 comorbid illness (2.44, 1.61-3.70), surgery delayed due to health instability (2.56, 1.50-4.37), and ASIA Impairment Scalelevel A (3.36, 1.78-6.35), while absence of impairment (0.39, 0.22-0.71), ASIAlevel E (0.39, 0.22-0.67) and, unexpectedly, delay caused by operating room unavailability (0.60, 0.36-0.99) were protective.

CONCLUSIONS

Types B and C thoracolumbar spine injuries are associated with a high risk of postoperative complications, especially common at governmental hospitals, and among patients with comorbidity, health instability, longer delays to surgery, and worse preoperative neurologic status.

Development and internal validation of predictive models to assess risk of post-acute care facility discharge in adults undergoing multi-level instrumented fusions for lumbar degenerative pathology and spinal deformity

PURPOSE

To develop a model for factors predictive of Post-Acute Care Facility (PACF) discharge in adult patients undergoing elective multi-level (≥ 3 segments) lumbar/thoracolumbar spinal instrumented fusions.

METHODS

The State Inpatient Databases acquired from the Healthcare Cost and Utilization Project from 2005 to 2013 were queried for adult patients who underwent elective multi-level thoracolumbar fusions for spinal deformity. Outcome variables were classified as discharge to home or PACF. Predictive variables included demographic, pre-operative, and operative factors. Univariate and multivariate logistic regression analyses informed development of a logistic regression-based predictive model using seven selected variables. Performance metrics included area under the curve (AUC), sensitivity, and specificity.

RESULTS

Included for analysis were 8866 patients. The logistic model including significant variables from multivariate analysis yielded an AUC of 0.75. Stepwise logistic regression was used to simplify the model and assess number of variables needed to reach peak AUC, which included seven selected predictors (insurance, interspaces fused, gender, age, surgical region, CCI, and revision surgery) and had an AUC of 0.74. Model cut-off for predictive PACF discharge was 0.41, yielding a sensitivity of 75% and specificity of 59%.

CONCLUSIONS

The seven variables associated significantly with PACF discharge (age > 60, female gender, non-private insurance, primary operations, instrumented fusion involving 8+ interspaces, thoracolumbar region, and higher CCI scores) may aid in identification of adults at risk for discharge to a PACF following elective multi-level lumbar/thoracolumbar spinal fusions for spinal deformity. This may in turn inform discharge planning and expectation management.

Predictive Models for Length of Stay and Discharge Disposition in Elective Spine Surgery: Development, Validation, and Comparison to the ACS NSQIP Risk Calculator

STUDY DESIGN

A retrospective study at a single academic institution.

OBJECTIVE

The purpose of this study is to utilize machine learning to predict hospital length of stay (LOS) and discharge disposition following adult elective spine surgery, and to compare performance metrics of machine learning models to the American College of Surgeon's National Surgical Quality Improvement Program's (ACS NSQIP) prediction calculator.

SUMMARY OF BACKGROUND DATA

A total of 3678 adult patients undergoing elective spine surgery between 2014 and 2019, acquired from the electronic health record.

METHODS

Patients were divided into three stratified cohorts: cervical degenerative, lumbar degenerative, and adult spinal deformity groups. Predictive variables included demographics, body mass index, surgical region, surgical invasiveness, surgical approach, and comorbidities. Regression, classification trees, and least absolute shrinkage and selection operator (LASSO) were used to build predictive models. Validation of the models was conducted on 16% of patients (N=587), using area under the receiver operator curve (AUROC), sensitivity, specificity, and correlation. Patient data were manually entered into the ACS NSQIP online risk calculator to compare performance. Outcome variables were discharge disposition (home vs. rehabilitation) and LOS (days).

RESULTS

Of 3678 patients analyzed, 51.4% were male (n=1890) and 48.6% were female (n=1788). The average LOS was 3.66 days. In all, 78% were discharged home and 22% discharged to rehabilitation. Compared with NSQIP (Pearson R2 =0.16), the predictions of poisson regression ( R2 =0.29) and LASSO ( R2 =0.29) models were significantly more correlated with observed LOS ( P =0.025 and 0.004, respectively). Of the models generated to predict discharge location, logistic regression yielded an AUROC of 0.79, which was statistically equivalent to the AUROC of 0.75 for NSQIP ( P =0.135).

CONCLUSION

The predictive models developed in this study can enable accurate preoperative estimation of LOS and risk of rehabilitation discharge for adult patients undergoing elective spine surgery. The demonstrated models exhibited better performance than NSQIP for prediction of LOS and equivalent performance to NSQIP for prediction of discharge location.

Predicting Length of Stay using machine learning for total joint replacements performed at a rural community hospital

Background

Predicting patient’s Length of Stay (LOS) before total joint replacement (TJR) surgery is vital for hospitals to optimally manage costs and resources. Many hospitals including in rural areas use publicly available models such as National Surgical Quality Improvement Program (NSQIP) calculator which, unfortunately, performs suboptimally when predicting LOS for TJR procedures.

Objective

The objective of this research was to develop a Machine Learning (ML) model to predict LOS for TJR procedures performed at a Perioperative Surgical Home implemented rural community hospital for better accuracy and interpretation than the NSQIP calculator.

Methods

A total of 158 TJR patients were collected and analyzed from a rural community hospital located in Montana. A random forest (RF) model was used to predict patient’s LOS. For interpretation, permuted feature importance and partial dependence plot methods were used to identify the important variables and their relationship with the LOS.

Results

The root mean square error for the RF model (0.7) was lower than the NSQIP calculator (1.21). The five most important variables for predicting LOS were BMI, Duke Activity Status-Index, diabetes, patient’s household income, and patient’s age.

Conclusion

This pilot study is the first of its kind to develop an ML model to predict LOS for TJR procedures that were performed at a small-scale rural community hospital. This pilot study contributes an approach for rural hospitals, making them more independent by developing their own predictions instead of relying on public models.

A scoping review of complication prediction models in spinal surgery: An analysis of model development, validation and impact

Background

Predictive analytics are being used increasingly in the field of spinal surgery with the development of models to predict post-surgical complications. Predictive models should be valid, generalizable, and clinically useful. The purpose of this review was to identify existing post-surgical complication prediction models for spinal surgery and to determine if these models are being adequately investigated with internal/external validation, model updating and model impact studies.

Methods

This was a scoping review of studies pertaining to models for the prediction of post-surgical complication after spinal surgery published over 10 years (2010-2020). Qualitative data was extracted from the studies to include study classification, adherence to Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines and risk of bias (ROB) assessment using the Prediction model study Risk Of Bias Assessment Tool (PROBAST). Model evaluation was determined using area under the curve (AUC) when available. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement was used as a basis for the search methodology in four different databases.

Results

Thirty studies were included in the scoping review and 80% (24/30) included model development with or without internal validation. Twenty percent (6/30) were exclusively external validation studies and only one study included an impact analysis in addition to model development and internal validation. Two studies referenced the TRIPOD guidelines and there was a high ROB in 100% of the studies using the PROBAST tool.

Conclusions

The majority of post-surgical complication prediction models in spinal surgery have not undergone standardized model development and internal validation or adequate external validation and impact evaluation. As such there is uncertainty as to their validity, generalizability, and clinical utility. Future efforts should be made to use existing tools to ensure standardization in development and rigorous evaluation of prediction models in spinal surgery.

Neural network models accurately predict discharge disposition after revision total knee arthroplasty?

PURPOSE

Based on the rising incidence of revision total knee arthroplasty (TKA), bundled payment models may be applied to revision TKA in the near future. Facility discharge represents a significant cost factor for those bundled payment models; however, accurately predicting discharge disposition remains a clinical challenge. The purpose of this study was to develop and validate artificial intelligence algorithms to predict discharge disposition following revision total knee arthroplasty.

METHODS

A retrospective review of electronic patient records was conducted to identify patients who underwent revision total knee arthroplasty. Discharge disposition was defined as either home discharge or non-home discharge, which included rehabilitation and skilled nursing facilities. Four artificial intelligence algorithms were developed to predict this outcome and were assessed by discrimination, calibration and decision curve analysis.

RESULTS

A total of 2228 patients underwent revision TKA, of which 1405 patients (63.1%) were discharged home, whereas 823 patients (36.9%) were discharged to a non-home facility. The strongest predictors for non-home discharge following revision TKA were American Society of Anesthesiologist (ASA) score, Medicare insurance type and revision surgery for peri-prosthetic joint infection, non-white ethnicity and social status (living alone). The best performing artificial intelligence algorithm was the neural network model which achieved excellent performance across discrimination (AUC = 0.87), calibration and decision curve analysis.

CONCLUSION

This study developed four artificial intelligence algorithms for the prediction of non-home discharge disposition for patients following revision total knee arthroplasty. The study findings show excellent performance on discrimination, calibration and decision curve analysis for all four candidate algorithms. Therefore, these models have the potential to guide preoperative patient counselling and improve the value (clinical and functional outcomes divided by costs) of revision total knee arthroplasty patients.

LEVEL OF EVIDENCE

IV.

Nomogram to predict postoperative complications in elderly with total hip replacement

BACKGROUND

By analyzing the risk factors of postoperative complications in elderly patients with hip replacement, We aimed to develop a nomogram model based on preoperative and intraoperative variables and verified the sensitivity and specificity for risk stratification of postoperative complications in elderly with total hip replacement patients.

AIM

To develop a nomogram model for risk stratification of postoperative complications in elderly with total hip replacement patients.

METHODS

A total of 414 elderly patients who underwent surgical treatment for total hip replacement hospitalized at the Affiliated Hospital of Guangdong Medical University from March 1, 2017 to August 31, 2019 were included into this study. Univariate and multivariate logistic regression were conducted to identify independent risk factors of postoperative complication in the 414 patients. A nomogram was developed by R software and validated to predict the risk of postoperative complications.

RESULTS

Multivariate logistic regression analysis revealed that age (OR = 1.05, 95%CI: 1.00-1.09), renal failure (OR = 0.90, 95%CI: 0.83-0.97), Type 2 diabetes (OR = 1.05, 95%CI: 1.00-1.09), albumin (ALB) (OR = 0.91, 95%CI: 0.83-0.99) were independent risk factors of postoperative complication in elderly patients with hip replacement (P < 0.05). For validation of the nomogram, receive operating characteristic curve revealed that the model predicting postoperative complication in elderly patients with hip replacement was the area under the curve of 0.8254 (95%CI: 0.78-0.87), the slope of the calibration plot was close to 1 and the model passed Hosmer-Lemeshow goodness of fit test (χ² = 10.16, P = 0.4264), calibration in R E_max = 0.176, E_avg = 0.027, which all demonstrated that the model was of good accuracy.

CONCLUSION

The nomogram predicting postoperative complications in patients with total hip replacement constructed based on age, type 2 diabetes, renal failure and ALB is of good discrimination and accuracy, which was of clinical significance.

Development of a model to predict the probability of incurring a complication during spine surgery

PURPOSE

Predictive models in spine surgery are of use in shared decision-making. This study sought to develop multivariable models to predict the probability of general and surgical perioperative complications of spinal surgery for lumbar degenerative diseases.

METHODS

Data came from EUROSPINE's Spine Tango Registry (1.2012-12.2017). Separate prediction models were built for surgical and general complications. Potential predictors included age, gender, previous spine surgery, additional pathology, BMI, smoking status, morbidity, prophylaxis, technology used, and the modified Mirza invasiveness index score. Complete case multiple logistic regression was used. Discrimination was assessed using area under the receiver operating characteristic curve (AUC) with 95% confidence intervals (CI). Plots were used to assess the calibration of the models.

RESULTS

Overall, 23'714/68'111 patients (54.6%) were available for complete case analysis: 763 (3.2%) had a general complication, with ASA score being strongly predictive (ASA-2 OR 1.6, 95% CI 1.20-2.12; ASA-3 OR 2.98, 95% CI 2.19-4.07; ASA-4 OR 5.62, 95% CI 3.04-10.41), while 2534 (10.7%) had a surgical complication, with previous surgery at the same level being an important predictor (OR 1.9, 95%CI 1.71-2.12). Respectively, model AUCs were 0.74 (95% CI, 0.72-0.76) and 0.64 (95% CI, 0.62-0.65), and calibration was good up to predicted probabilities of 0.30 and 0.25, respectively.

CONCLUSION

We developed two models to predict complications associated with spinal surgery. Surgical complications were predicted with less discriminative ability than general complications. Reoperation at the same level was strongly predictive of surgical complications and a higher ASA score, of general complications. A web-based prediction tool was developed at https://sst.webauthor.com/go/fx/run.cfm?fx=SSTCalculator .

The epidemiology of Medical Emergency Team calls for orthopedic patients in a teaching hospital: A retrospective cohort study

BACKGROUND

Patients undergoing orthopedic surgery are at risk of post-operative complications and needing Medical Emergency Team (MET) review. We assessed the frequency of, and associations with MET calls in orthopedic patients, and whether this was associated with increased in-hospital morbidity and mortality.

METHODS

Retrospective cohort study of patients admitted over four years to a University teaching hospital using hospital administrative and MET call databases.

RESULTS

Amongst 6344 orthopedic patients, 55.8% were female, the median (IQR) age and Charlson comorbidity index were 66 years (47-79) and 3 (1-5), respectively. Overall, 54.5% of admissions were emergency admissions, 1130 (17.8%) were non-operative, and 605 (9.5%) patients received a MET call. The strongest independent associations with receiving a MET call was the operative procedure, especially hip and knee arthroplasty. Common MET triggers were hypotension (37.5%), tachycardia (25.0%) and tachypnoea (9.1%). Patients receiving a MET call were at increased risk of anemia, delirium, pressure injury, renal failure and wound infection. The mortality of patients who received a MET call was 9.8% compared with 0.8% for those who did not. After adjusting for pre-defined co-variates, requirement for a MET call was associated with an adjusted odd-ratio of 9.57 (95%CI 3.1-29.7) for risk of in-hospital death.

CONCLUSIONS

Approximately 10% of orthopedic patients received a MET call, which was most strongly associated with major hip and knee arthroplasty. Such patients are at increased risk of morbidity and in-hospital mortality. Further strategies are needed to more pro-actively manage at-risk orthopedic patients.