Validation and Generalizability of Machine Learning Models for the Prediction of Discharge Disposition Following Revision Total Knee Arthroplasty

Artificial intelligence and machine learning in knee arthroplasty

BACKGROUND

Artificial intelligence (AI) and its subset, machine learning (ML), have significantly impacted clinical medicine, particularly in knee arthroplasty (KA). These technologies utilize algorithms for tasks such as predictive analytics and image recognition, improving preoperative planning, intraoperative navigation, and postoperative complication anticipation. This systematic review presents AI-driven tools' clinical implications in total and unicompartmental KA, focusing on enhancing patient outcomes and operational efficiency.

METHODS

A systematic search was conducted across multiple databases including Cochrane Central Register of Controlled Trials, Embase, OVID Medline, PubMed, and Web of Science, following the PRISMA guidelines for studies published in the English language till March 2024. Inclusion criteria targeted adult human models without geographical restrictions, specifically related to total or unicompartmental KA.

RESULTS

A total of 153 relevant studies were identified, covering various aspects of ML application for KA. Topics of studies included imaging modalities (n = 28), postoperative primary KA complications (n = 26), inpatient status (length of stay, readmissions, and cost) (n = 24), implant configuration (n = 14), revision (n = 12), patient-reported outcome measures (PROMs) (n = 11), function (n = 11), procedural communication (n = 8), total knee arthroplasty/unicompartmental knee arthroplasty prediction (n = 6), outpatient status (n = 4), perioperative efficiency (n = 4), patient satisfaction (n = 3), opioid usage (n = 3). A total of 66 ML models were described, with 48.7% of studies using multiple approaches.

CONCLUSION

This review assesses ML applications in knee arthroplasty, highlighting their potential to improve patient outcomes. While current algorithms and AI show promise, our findings suggest areas for enhancement in predictive performance before widespread clinical adoption.

Utilization of Machine Learning Models to More Accurately Predict Case Duration in Primary Total Joint Arthroplasty

BACKGROUND

Accurate operative scheduling is essential for the appropriation of operating room esources. We sought to implement a machine learning model to predict primary total hip arthroplasty (THA) and total knee arthroplasty (TKA) case time.

METHODS

A total of 10,590 THAs and 12,179 TKAs between July 2017 and December 2022 were retrospectively identified. Cases were chronologically divided into training, validation, and test sets. The test set cohort included 1,588 TKAs and 1,204 THAs. There were four ML algorithms developed: linear ridge regression (LR), random forest, XGBoost, and explainable boosting machine. Each model's case time estimate was compared to the scheduled estimate measured in 15-minute "wait" time blocks ("underbooking") and "excess" time blocks ("overbooking"). Surgical case time was recorded, and SHAP values were assigned to patient characteristics, surgical information, and the patient's medical condition to understand feature importance.

RESULTS

The most predictive model input was "median previous 30 procedure case times." The XGBoost model outperformed the other models in predicting both TKA and THA case times. The model reduced TKA 'excess time blocks' by 85 blocks (P < 0.001) and 'wait time blocks' by 96 blocks (P < 0.001). The model did not significantly reduce 'excess time blocks' in THA (P = 0.89) but did significantly reduce 'wait time blocks' by 134 blocks (P < 0.001). In total, the model improved TKA operative booking by 181 blocks (2,715 minutes) and THA operative booking by 138 blocks (2,070 minutes).

CONCLUSIONS

Machine learning outperformed a traditional method of scheduling total joint arthroplasty cases. The median time of the prior 30 surgical cases was the most influential on scheduling case time accuracy. As ML models improve, surgeons should consider ML utilization in case scheduling; however, prior 30 surgical cases may serve as an adequate alternative.

Predicting 30-day reoperation following primary total knee arthroplasty: machine learning model outperforms the ACS risk calculator

The ACS risk calculator (ARC) has proven less effective in predicting patient-specific risk of early reoperation after primary total knee arthroplasty (TKA), compromising care quality and cost efficiency. This study compared the performance of a machine learning (ML) model and ARC in predicting 30-day reoperation after primary TKA using a national-scale dataset. Data of 366,151 TKAs were acquired from the ACS-NSQIP database. A random forest model was derived using ARC build-in parameters from the training dataset via techniques of hyperparameter optimization and cross-validation. The predictive performance of random forest and ARC was evaluated by metrics of discrimination, calibration, and clinical utility using the testing dataset. The ML model demonstrated good discrimination and calibration (AUC: 0.72, slope: 1.18, intercept: - 0.14, Brier score: 0.012), outperforming ARC in all metrics (AUC: 0.51, slope: - 0.01, intercept: 0.01, Brier score: 0.135) including clinical utility measured by decision curve analyses. Age (> 67 years) and BMI (> 34 kg/m2) were the important predictors of reoperation. This study suggests the superiority of ML models in identifying individualized 30-day reoperation risk following TKA. ML models may be an adjunct prediction tool in enhancing patient-specific risk stratification and postoperative care management.

Comparing prediction accuracy for 30-day readmission following primary total knee arthroplasty: the ACS-NSQIP risk calculator versus a novel artificial neural network model

BACKGROUND

Unplanned readmission, a measure of surgical quality, occurs after 4.8% of primary total knee arthroplasties (TKA). Although the prediction of individualized readmission risk may inform appropriate preoperative interventions, current predictive models, such as the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) surgical risk calculator (SRC), have limited utility. This study aims to compare the predictive accuracy of the SRC with a novel artificial neural network (ANN) algorithm for 30-day readmission after primary TKA, using the same set of clinical variables from a large national database.

METHODS

Patients undergoing primary TKA between 2013 and 2020 were identified from the ACS-NSQIP database and randomly stratified into training and validation cohorts. The ANN was developed using data from the training cohort with fivefold cross-validation performed five times. ANN and SRC performance were subsequently evaluated in the distinct validation cohort, and predictive performance was compared on the basis of discrimination, calibration, accuracy, and clinical utility.

RESULTS

The overall cohort consisted of 365,394 patients (trainingN = 362,559; validationN = 2835), with 11,392 (3.1%) readmitted within 30 days. While the ANN demonstrated good discrimination and calibration (area under the curve (AUC)ANN = 0.72, slope = 1.32, intercept = -0.09) in the validation cohort, the SRC demonstrated poor discrimination (AUCSRC = 0.55) and underestimated readmission risk (slope = -0.21, intercept = 0.04). Although both models possessed similar accuracy (Brier score: ANN = 0.03; SRC = 0.02), only the ANN demonstrated a higher net benefit than intervening in all or no patients on the decision curve analysis. The strongest predictors of readmission were body mass index (> 33.5 kg/m2), age (> 69 years), and male sex.

CONCLUSIONS

This study demonstrates the superior predictive ability and potential clinical utility of the ANN over the conventional SRC when constrained to the same variables. By identifying the most important predictors of readmission following TKA, our findings may assist in the development of novel clinical decision support tools, potentially improving preoperative counseling and postoperative monitoring practices in at-risk patients.

Predicting extended hospital stay following revision total hip arthroplasty: a machine learning model analysis based on the ACS-NSQIP database

INTRODUCTION

Prolonged length of stay (LOS) following revision total hip arthroplasty (THA) can lead to increased healthcare costs, higher rates of readmission, and lower patient satisfaction. In this study, we investigated the predictive power of machine learning (ML) models for prolonged LOS after revision THA using patient data from a national-scale patient repository.

MATERIALS AND METHODS

We identified 11,737 revision THA cases from the American College of Surgeons National Surgical Quality Improvement Program database from 2013 to 2020. Prolonged LOS was defined as exceeding the 75th value of all LOSs in the study cohort. We developed four ML models: artificial neural network (ANN), random forest, histogram-based gradient boosting, and k-nearest neighbor, to predict prolonged LOS after revision THA. Each model's performance was assessed during training and testing sessions in terms of discrimination, calibration, and clinical utility.

RESULTS

The ANN model was the most accurate with an AUC of 0.82, calibration slope of 0.90, calibration intercept of 0.02, and Brier score of 0.140 during testing, indicating the model's competency in distinguishing patients subject to prolonged LOS with minimal prediction error. All models showed clinical utility by producing net benefits in the decision curve analyses. The most significant predictors of prolonged LOS were preoperative blood tests (hematocrit, platelet count, and leukocyte count), preoperative transfusion, operation time, indications for revision THA (infection), and age.

CONCLUSIONS

Our study demonstrated that the ML model accurately predicted prolonged LOS after revision THA. The results highlighted the importance of the indications for revision surgery in determining the risk of prolonged LOS. With the model's aid, clinicians can stratify individual patients based on key factors, improve care coordination and discharge planning for those at risk of prolonged LOS, and increase cost efficiency.

Predicting 30-day unplanned hospital readmission after revision total knee arthroplasty: machine learning model analysis of a national patient cohort

Revision total knee arthroplasty (TKA) is associated with a higher risk of readmission than primary TKA. Identifying individual patients predisposed to readmission can facilitate proactive optimization and increase care efficiency. This study developed machine learning (ML) models to predict unplanned readmission following revision TKA using a national-scale patient dataset. A total of 17,443 revision TKA cases (2013-2020) were acquired from the ACS NSQIP database. Four ML models (artificial neural networks, random forest, histogram-based gradient boosting, and k-nearest neighbor) were developed on relevant patient variables to predict readmission following revision TKA. The length of stay, operation time, body mass index (BMI), and laboratory test results were the strongest predictors of readmission. Histogram-based gradient boosting was the best performer in distinguishing readmission (AUC: 0.95) and estimating the readmission probability for individual patients (calibration slope: 1.13; calibration intercept: -0.00; Brier score: 0.064). All models produced higher net benefit than the default strategies of treating all or no patients, supporting the clinical utility of the models. ML demonstrated excellent performance for the prediction of readmission following revision TKA. Optimization of important predictors highlighted by our model may decrease preventable hospital readmission following surgery, thereby leading to reduced financial burden and improved patient satisfaction.

Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review

» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.

The Predictive Abilities of Machine Learning Algorithms in Patients with Thoracolumbar Spinal Cord Injuries

OBJECTIVES

The goal of this study is to implement machine learning (ML) algorithms to predict mortality, non-home discharge, prolonged length of stay (LOS), prolonged length of intensive care unit stay (ICU-LOS), and major complications in patients diagnosed with thoracolumbar spinal cord injury, while creating a publicly accessible online tool.

METHODS

The American College of Surgeons Trauma Quality Program database was used to identify patients with thoracolumbar spinal cord injury. Feature selection was performed with the Least Absolute Shrinkage and Selection Operator algorithm. Five ML algorithms, including TabPFN, TabNet, XGBoost, LightGBM, and Random Forest, were used along with the Optuna optimization library for hyperparameter tuning.

RESULTS

A total of 147,819 patients were included in the analysis. For each outcome, we determined the best model for deployment in our web application based on the area under the receiver operating characteristic (AUROC) values. The top performing algorithms were as follows: LightGBM for mortality with an AUROC of 0.885, TabPFN for non-home discharge with an AUROC of 0.801, LightGBM for prolonged LOS with an AUROC of 0.673, Random Forest for prolonged ICU-LOS with an AUROC of 0.664, and LightGBM for major complications with an AUROC of 0.73.

CONCLUSIONS

ML models demonstrate good predictive ability for in-hospital mortality and non-home discharge, fair predictive ability for major complications and prolonged ICU-LOS, but poor predictive ability for prolonged LOS. We have developed a web application that allows these models to be accessed.