Internal and External Validation of the Generalizability of Machine Learning Algorithms in Predicting Non-home Discharge Disposition Following Primary Total Knee Joint Arthroplasty

Artificial intelligence and machine learning in knee arthroplasty

BACKGROUND

Artificial intelligence (AI) and its subset, machine learning (ML), have significantly impacted clinical medicine, particularly in knee arthroplasty (KA). These technologies utilize algorithms for tasks such as predictive analytics and image recognition, improving preoperative planning, intraoperative navigation, and postoperative complication anticipation. This systematic review presents AI-driven tools' clinical implications in total and unicompartmental KA, focusing on enhancing patient outcomes and operational efficiency.

METHODS

A systematic search was conducted across multiple databases including Cochrane Central Register of Controlled Trials, Embase, OVID Medline, PubMed, and Web of Science, following the PRISMA guidelines for studies published in the English language till March 2024. Inclusion criteria targeted adult human models without geographical restrictions, specifically related to total or unicompartmental KA.

RESULTS

A total of 153 relevant studies were identified, covering various aspects of ML application for KA. Topics of studies included imaging modalities (n = 28), postoperative primary KA complications (n = 26), inpatient status (length of stay, readmissions, and cost) (n = 24), implant configuration (n = 14), revision (n = 12), patient-reported outcome measures (PROMs) (n = 11), function (n = 11), procedural communication (n = 8), total knee arthroplasty/unicompartmental knee arthroplasty prediction (n = 6), outpatient status (n = 4), perioperative efficiency (n = 4), patient satisfaction (n = 3), opioid usage (n = 3). A total of 66 ML models were described, with 48.7% of studies using multiple approaches.

CONCLUSION

This review assesses ML applications in knee arthroplasty, highlighting their potential to improve patient outcomes. While current algorithms and AI show promise, our findings suggest areas for enhancement in predictive performance before widespread clinical adoption.

Racial and Ethnic Disparities in Predictive Accuracy of Machine Learning Algorithms Developed Using a National Database for 30-Day Complications Following Total Joint Arthroplasty

BACKGROUND

While predictive capabilities of machine learning (ML) algorithms for hip and knee total joint arthroplasty (TJA) have been demonstrated in previous studies, their performance in racial and ethnic minority patients has not been investigated. This study aimed to assess the performance of ML algorithms in predicting 30-days complications following TJA in racial and ethnic minority patients.

METHODS

A total of 267,194 patients undergoing primary TJA between 2013 and 2020 were identified from a national outcomes database. The patient cohort was stratified according to race, with further substratification into Hispanic or non-Hispanic ethnicity. There were two ML algorithms, histogram-based gradient boosting (HGB), and random forest (RF), that were modeled to predict 30-days complications following primary TJA in the overall population. They were subsequently assessed in each racial and ethnic subcohort using discrimination, calibration, accuracy, and potential clinical usefulness.

RESULTS

Both models achieved excellent (Area under the curve (AUC) > 0.8) discrimination (AUCHGB = AUCRF = 0.86), calibration, and accuracy (HGB: slope = 1.00, intercept = -0.03, Brier score = 0.12; RF: slope = 0.97, intercept = 0.02, Brier score = 0.12) in the non-Hispanic White population (N = 224,073). Discrimination decreased in the White Hispanic (N = 10,429; AUC = 0.75 to 0.76), Black (N = 25,116; AUC = 0.77), Black Hispanic (N = 240; AUC = 0.78), Asian non-Hispanic (N = 4,809; AUC = 0.78 to 0.79), and overall (N = 267,194; AUC = 0.75 to 0.76) cohorts, but remained well-calibrated. We noted the poorest model discrimination (N = 1,870; AUC = 0.67 to 0.68) and calibration in the American-Indian cohort.

CONCLUSIONS

The ML algorithms demonstrate an inferior predictive ability for 30-days complications following primary TJA in racial and ethnic minorities when trained on existing healthcare big data. This may be attributed to the disproportionate underrepresentation of minority groups within these databases, as demonstrated by the smaller sample sizes available to train the ML models. The ML models developed using smaller datasets (e.g., in racial and ethnic minorities) may not be as accurate as larger datasets, highlighting the need for equity-conscious model development.

LEVEL OF EVIDENCE

III; retrospective cohort study.

Machine learning models based on a national-scale cohort accurately identify patients at high risk of deep vein thrombosis following primary total hip arthroplasty

BACKGROUND

The occurrence of deep venous thrombosis (DVT) following total hip arthroplasty (THA) poses a substantial risk of morbidity and mortality, highlighting the need for preoperative risk stratification and prophylaxis initiatives. However, there exists a paucity of big-data-driven predictive models for DVT risk following elective hip arthroplasty. Therefore, this study aimed to develop and assess machine learning (ML) models in predicting DVT risk following THA using a national patient cohort.

HYPOTHESIS

We hypothesized that machine learning models would accurately predict patient-specific DVT risk in patients undergoing elective total hip arthroplasty.

PATIENTS AND METHODS

The ACS-NSQIP national database was queried to identify 70,733 THA patients from 2013 to 2020, including 317 patients (0.45%) with DVT. Artificial neural network, random forest, histogram-based gradient boosting, k-nearest neighbor, and support vector machine algorithms were trained and utilized to predict the risk of DVT following THA. Model performance was assessed using discrimination, calibration, and potential clinical utility.

RESULTS

Histogram-based gradient boosting demonstrated the best prediction performance with an area under the receiver operating curve of 0.93 (discrimination), a slope of 0.92 (closely aligned with actual outcomes), an intercept of 0.18 (minimal prediction bias), and a Brier score of 0.010 (high accuracy). The model also demonstrated clinical utility with greater net benefit than alternative decision criteria in the decision curve analysis. Length of stay, international normalized ratio, age, and partial thromboplastin time were the strongest predictors of DVT after primary THA.

DISCUSSION

Machine learning models demonstrated excellent predictive performance in terms of discrimination, calibration, and decision curve analysis. Further research is warranted in terms of external validation to realize the potential of these algorithms as a valuable adjunct tool for risk stratification in patients undergoing THA.

LEVEL OF EVIDENCE

III; Retrospective study.

Predicting 30-day reoperation following primary total knee arthroplasty: machine learning model outperforms the ACS risk calculator

The ACS risk calculator (ARC) has proven less effective in predicting patient-specific risk of early reoperation after primary total knee arthroplasty (TKA), compromising care quality and cost efficiency. This study compared the performance of a machine learning (ML) model and ARC in predicting 30-day reoperation after primary TKA using a national-scale dataset. Data of 366,151 TKAs were acquired from the ACS-NSQIP database. A random forest model was derived using ARC build-in parameters from the training dataset via techniques of hyperparameter optimization and cross-validation. The predictive performance of random forest and ARC was evaluated by metrics of discrimination, calibration, and clinical utility using the testing dataset. The ML model demonstrated good discrimination and calibration (AUC: 0.72, slope: 1.18, intercept: - 0.14, Brier score: 0.012), outperforming ARC in all metrics (AUC: 0.51, slope: - 0.01, intercept: 0.01, Brier score: 0.135) including clinical utility measured by decision curve analyses. Age (> 67 years) and BMI (> 34 kg/m2) were the important predictors of reoperation. This study suggests the superiority of ML models in identifying individualized 30-day reoperation risk following TKA. ML models may be an adjunct prediction tool in enhancing patient-specific risk stratification and postoperative care management.

Comparing prediction accuracy for 30-day readmission following primary total knee arthroplasty: the ACS-NSQIP risk calculator versus a novel artificial neural network model

BACKGROUND

Unplanned readmission, a measure of surgical quality, occurs after 4.8% of primary total knee arthroplasties (TKA). Although the prediction of individualized readmission risk may inform appropriate preoperative interventions, current predictive models, such as the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) surgical risk calculator (SRC), have limited utility. This study aims to compare the predictive accuracy of the SRC with a novel artificial neural network (ANN) algorithm for 30-day readmission after primary TKA, using the same set of clinical variables from a large national database.

METHODS

Patients undergoing primary TKA between 2013 and 2020 were identified from the ACS-NSQIP database and randomly stratified into training and validation cohorts. The ANN was developed using data from the training cohort with fivefold cross-validation performed five times. ANN and SRC performance were subsequently evaluated in the distinct validation cohort, and predictive performance was compared on the basis of discrimination, calibration, accuracy, and clinical utility.

RESULTS

The overall cohort consisted of 365,394 patients (trainingN = 362,559; validationN = 2835), with 11,392 (3.1%) readmitted within 30 days. While the ANN demonstrated good discrimination and calibration (area under the curve (AUC)ANN = 0.72, slope = 1.32, intercept = -0.09) in the validation cohort, the SRC demonstrated poor discrimination (AUCSRC = 0.55) and underestimated readmission risk (slope = -0.21, intercept = 0.04). Although both models possessed similar accuracy (Brier score: ANN = 0.03; SRC = 0.02), only the ANN demonstrated a higher net benefit than intervening in all or no patients on the decision curve analysis. The strongest predictors of readmission were body mass index (> 33.5 kg/m2), age (> 69 years), and male sex.

CONCLUSIONS

This study demonstrates the superior predictive ability and potential clinical utility of the ANN over the conventional SRC when constrained to the same variables. By identifying the most important predictors of readmission following TKA, our findings may assist in the development of novel clinical decision support tools, potentially improving preoperative counseling and postoperative monitoring practices in at-risk patients.

Developing a decision support tool to predict delayed discharge from hospitals using machine learning

BACKGROUND

The growing demand for healthcare services challenges patient flow management in health systems. Alternative Level of Care (ALC) patients who no longer need acute care yet face discharge barriers contribute to prolonged stays and hospital overcrowding. Predicting these patients at admission allows for better resource planning, reducing bottlenecks, and improving flow. This study addresses three objectives: identifying likely ALC patients, key predictive features, and preparing guidelines for early ALC identification at admission.

METHODS

Data from Nova Scotia Health (2015-2022) covering patient demographics, diagnoses, and clinical information was extracted. Data preparation involved managing outliers, feature engineering, handling missing values, transforming categorical variables, and standardizing. Data imbalance was addressed using class weights, random oversampling, and the Synthetic Minority Over-Sampling Technique (SMOTE). Three ML classifiers, Random Forest (RF), Artificial Neural Network (ANN), and eXtreme Gradient Boosting (XGB), were tested to classify patients as ALC or not. Also, to ensure accurate ALC prediction at admission, only features available at that time were used in a separate model iteration.

RESULTS

Model performance was assessed using recall, F1-Score, and AUC metrics. The XGB model with SMOTE achieved the highest performance, with a recall of 0.95 and an AUC of 0.97, excelling in identifying ALC patients. The next best models were XGB with random oversampling and ANN with class weights. When limited to admission-only features, the XGB with SMOTE still performed well, achieving a recall of 0.91 and an AUC of 0.94, demonstrating its effectiveness in early ALC prediction. Additionally, the analysis identified diagnosis 1, patient age, and entry code as the top three predictors of ALC status.

CONCLUSIONS

The results demonstrate the potential of ML models to predict ALC status at admission. The findings support real-time decision-making to improve patient flow and reduce hospital overcrowding. The ALC guideline groups patients first by diagnosis, then by age, and finally by entry code, categorizing prediction outcomes into three probability ranges: below 30%, 30-70%, and above 70%. This framework assesses whether ALC status can be accurately predicted at admission or during the patient's stay before discharge.

Prediction of 30-Day Mortality Following Revision Total Hip and Knee Arthroplasty: Machine Learning Algorithms Outperform CARDE-B, 5-Item, and 6-Item Modified Frailty Index Risk Scores

BACKGROUND

Although risk calculators are used to prognosticate postoperative outcomes following revision total hip and knee arthroplasty (total joint arthroplasty [TJA]), machine learning (ML) based predictive tools have emerged as a promising alternative for improved risk stratification. This study aimed to compare the predictive ability of ML models for 30-day mortality following revision TJA to that of traditional risk-assessment indices such as the CARDE-B score (congestive heart failure, albumin (< 3.5 mg/dL), renal failure on dialysis, dependence for daily living, elderly (> 65 years of age), and body mass index (BMI) of < 25 kg/m2), 5-item modified frailty index (5MFI), and 6MFI.

METHODS

Adult patients undergoing revision TJA between 2013 and 2020 were selected from the American College of Surgeons National Surgical Quality Improvement Program database and randomly split 80:20 to compose the training and validation cohorts. There were 3 ML models - extreme gradient boosting, random forest, and elastic-net penalized logistic regression (NEPLR) - that were developed and evaluated using discrimination, calibration metrics, and accuracy. The discrimination of CARDE-B, 5MFI, and 6MFI scores was assessed individually and compared to that of ML models.

RESULTS

All models were equally accurate (Brier score = 0.005) and demonstrated outstanding discrimination with similar areas under the receiver operating characteristic curve (AUCs, extreme gradient boosting = 0.94, random forest = NEPLR = 0.93). The NEPLR was the best-calibrated model overall (slope = 0.54, intercept = -0.004). The CARDE-B had the highest discrimination among the scores (AUC = 0.89), followed by 6MFI (AUC = 0.80), and 5MFI (AUC = 0.68). Albumin < 3.5 mg/dL and BMI (< 30.15) were the most important predictors of 30-day mortality following revision TJA.

CONCLUSIONS

The ML models outperform traditional risk-assessment indices in predicting postoperative 30-day mortality after revision TJA. Our findings highlight the utility of ML for risk stratification in a clinical setting. The identification of hypoalbuminemia and BMI as prognostic markers may allow patient-specific perioperative optimization strategies to improve outcomes following revision TJA.

Predicting extended hospital stay following revision total hip arthroplasty: a machine learning model analysis based on the ACS-NSQIP database

INTRODUCTION

Prolonged length of stay (LOS) following revision total hip arthroplasty (THA) can lead to increased healthcare costs, higher rates of readmission, and lower patient satisfaction. In this study, we investigated the predictive power of machine learning (ML) models for prolonged LOS after revision THA using patient data from a national-scale patient repository.

MATERIALS AND METHODS

We identified 11,737 revision THA cases from the American College of Surgeons National Surgical Quality Improvement Program database from 2013 to 2020. Prolonged LOS was defined as exceeding the 75th value of all LOSs in the study cohort. We developed four ML models: artificial neural network (ANN), random forest, histogram-based gradient boosting, and k-nearest neighbor, to predict prolonged LOS after revision THA. Each model's performance was assessed during training and testing sessions in terms of discrimination, calibration, and clinical utility.

RESULTS

The ANN model was the most accurate with an AUC of 0.82, calibration slope of 0.90, calibration intercept of 0.02, and Brier score of 0.140 during testing, indicating the model's competency in distinguishing patients subject to prolonged LOS with minimal prediction error. All models showed clinical utility by producing net benefits in the decision curve analyses. The most significant predictors of prolonged LOS were preoperative blood tests (hematocrit, platelet count, and leukocyte count), preoperative transfusion, operation time, indications for revision THA (infection), and age.

CONCLUSIONS

Our study demonstrated that the ML model accurately predicted prolonged LOS after revision THA. The results highlighted the importance of the indications for revision surgery in determining the risk of prolonged LOS. With the model's aid, clinicians can stratify individual patients based on key factors, improve care coordination and discharge planning for those at risk of prolonged LOS, and increase cost efficiency.

A systematic literature review of predicting patient discharges using statistical methods and machine learning

Discharge planning is integral to patient flow as delays can lead to hospital-wide congestion. Because a structured discharge plan can reduce hospital length of stay while enhancing patient satisfaction, this topic has caught the interest of many healthcare professionals and researchers. Predicting discharge outcomes, such as destination and time, is crucial in discharge planning by helping healthcare providers anticipate patient needs and resource requirements. This article examines the literature on the prediction of various discharge outcomes. Our review discovered papers that explore the use of prediction models to forecast the time, volume, and destination of discharged patients. Of the 101 reviewed papers, 49.5% looked at the prediction with machine learning tools, and 50.5% focused on prediction with statistical methods. The fact that knowing discharge outcomes in advance affects operational, tactical, medical, and administrative aspects is a frequent theme in the papers studied. Furthermore, conducting system-wide optimization, predicting the time and destination of patients after discharge, and addressing the primary causes of discharge delay in the process are among the recommendations for further research in this field.

Predicting 30-day unplanned hospital readmission after revision total knee arthroplasty: machine learning model analysis of a national patient cohort

Revision total knee arthroplasty (TKA) is associated with a higher risk of readmission than primary TKA. Identifying individual patients predisposed to readmission can facilitate proactive optimization and increase care efficiency. This study developed machine learning (ML) models to predict unplanned readmission following revision TKA using a national-scale patient dataset. A total of 17,443 revision TKA cases (2013-2020) were acquired from the ACS NSQIP database. Four ML models (artificial neural networks, random forest, histogram-based gradient boosting, and k-nearest neighbor) were developed on relevant patient variables to predict readmission following revision TKA. The length of stay, operation time, body mass index (BMI), and laboratory test results were the strongest predictors of readmission. Histogram-based gradient boosting was the best performer in distinguishing readmission (AUC: 0.95) and estimating the readmission probability for individual patients (calibration slope: 1.13; calibration intercept: -0.00; Brier score: 0.064). All models produced higher net benefit than the default strategies of treating all or no patients, supporting the clinical utility of the models. ML demonstrated excellent performance for the prediction of readmission following revision TKA. Optimization of important predictors highlighted by our model may decrease preventable hospital readmission following surgery, thereby leading to reduced financial burden and improved patient satisfaction.

The Investigation of Preoperative Factors Associated With Postoperative Outcomes Following Total Knee Arthroplasty for Osteoarthritis: A Scoping Review

This study aimed to investigate preoperative factors associated with non-home discharges from acute care hospitals in patients undergoing total knee arthroplasty (TKA) due to osteoarthritis. It was a scoping review focused on patients who received their first unilateral TKA for osteoarthritis. The research targeted observational studies that examined the destinations of patients post-surgery based on preoperative factors, with a literature search conducted in April 2023. Out of 3,255 identified papers, 28 met the eligibility criteria. A total of 26 preoperative factors were identified as potentially related to discharge destinations, including age, gender, comorbidities, and obesity. By selecting an appropriate discharge destination based on preoperative factors, there may be potential for more efficient use of medical resources. Future studies should consider preoperative factors in the context of national healthcare systems and lengths of hospital stay.

Enhancing clinical decision-making: An externally validated machine learning model for predicting isocitrate dehydrogenase mutation in gliomas using radiomics from presurgical magnetic resonance imaging

Background

Glioma, the most prevalent primary brain tumor, poses challenges in prognosis, particularly in the high-grade subclass, despite advanced treatments. The recent shift in tumor classification underscores the crucial role of isocitrate dehydrogenase (IDH) mutation status in the clinical care of glioma patients. However, conventional methods for determining IDH status, including biopsy, have limitations. Exploring the use of machine learning (ML) on magnetic resonance imaging to predict IDH mutation status shows promise but encounters challenges in generalizability and translation into clinical practice because most studies either use single institution or homogeneous datasets for model training and validation. Our study aims to bridge this gap by using multi-institution data for model validation.

Methods

This retrospective study utilizes data from large, annotated datasets for internal (377 cases from Yale New Haven Hospitals) and external validation (207 cases from facilities outside Yale New Haven Health). The 6-step research process includes image acquisition, semi-automated tumor segmentation, feature extraction, model building with feature selection, internal validation, and external validation. An extreme gradient boosting ML model predicted the IDH mutation status, confirmed by immunohistochemistry.

Results

The ML model demonstrated high performance, with an Area under the Curve (AUC), Accuracy, Sensitivity, and Specificity in internal validation of 0.862, 0.865, 0.885, and 0.713, and external validation of 0.835, 0.851, 0.850, and 0.847.

Conclusions

The ML model, built on a heterogeneous dataset, provided robust results in external validation for the prediction task, emphasizing its potential clinical utility. Future research should explore expanding its applicability and validation in diverse global healthcare settings.