Validation of Machine Learning Model Performance in Predicting Blood Transfusion After Primary and Revision Total Hip Arthroplasty

Racial and Ethnic Disparities in Predictive Accuracy of Machine Learning Algorithms Developed Using a National Database for 30-Day Complications Following Total Joint Arthroplasty

BACKGROUND

While predictive capabilities of machine learning (ML) algorithms for hip and knee total joint arthroplasty (TJA) have been demonstrated in previous studies, their performance in racial and ethnic minority patients has not been investigated. This study aimed to assess the performance of ML algorithms in predicting 30-days complications following TJA in racial and ethnic minority patients.

METHODS

A total of 267,194 patients undergoing primary TJA between 2013 and 2020 were identified from a national outcomes database. The patient cohort was stratified according to race, with further substratification into Hispanic or non-Hispanic ethnicity. There were two ML algorithms, histogram-based gradient boosting (HGB), and random forest (RF), that were modeled to predict 30-days complications following primary TJA in the overall population. They were subsequently assessed in each racial and ethnic subcohort using discrimination, calibration, accuracy, and potential clinical usefulness.

RESULTS

Both models achieved excellent (Area under the curve (AUC) > 0.8) discrimination (AUCHGB = AUCRF = 0.86), calibration, and accuracy (HGB: slope = 1.00, intercept = -0.03, Brier score = 0.12; RF: slope = 0.97, intercept = 0.02, Brier score = 0.12) in the non-Hispanic White population (N = 224,073). Discrimination decreased in the White Hispanic (N = 10,429; AUC = 0.75 to 0.76), Black (N = 25,116; AUC = 0.77), Black Hispanic (N = 240; AUC = 0.78), Asian non-Hispanic (N = 4,809; AUC = 0.78 to 0.79), and overall (N = 267,194; AUC = 0.75 to 0.76) cohorts, but remained well-calibrated. We noted the poorest model discrimination (N = 1,870; AUC = 0.67 to 0.68) and calibration in the American-Indian cohort.

CONCLUSIONS

The ML algorithms demonstrate an inferior predictive ability for 30-days complications following primary TJA in racial and ethnic minorities when trained on existing healthcare big data. This may be attributed to the disproportionate underrepresentation of minority groups within these databases, as demonstrated by the smaller sample sizes available to train the ML models. The ML models developed using smaller datasets (e.g., in racial and ethnic minorities) may not be as accurate as larger datasets, highlighting the need for equity-conscious model development.

LEVEL OF EVIDENCE

III; retrospective cohort study.

Predictive analysis of economic and clinical outcomes in total knee arthroplasty: Identifying high-risk patients for increased costs and length of stay

PURPOSE

The purpose of this study was to predict high-risk patients who experience significant increases in hospital charges and length of stay (LOS) following specific postoperative complications.

METHODS

This study analyzed over two million patients from the Nationwide Inpatient Sample database undergoing elective total knee arthroplasty (TKA) for primary osteoarthritis. Baseline demographics, clinical characteristics and incidence of postoperative complications were examined. A neural network model was utilized to predict high-risk patients who fall into the top 25% for both LOS and total hospital charges after complications such as sepsis or surgical site infection (SSI).

RESULTS

The most common complications were blood loss anaemia (14.6%), acute kidney injury (1.6%) and urinary tract infection (0.9%). Patients with complications incurred significantly higher total charges (mean $66,804) and longer LOS (mean 2.9 days) compared to those without complications (mean $58,545 and 2.1 days, respectively). The neural network model demonstrated strong predictive performance, with an area under the curve of 0.83 for the training set and 0.78 for the testing set. Key complications like sepsis and SSIs significantly impacted hospital charges and LOS. For example, a 57-year-old patient with diabetes and sepsis had a 100% probability of being in the top 25% for both total charges and LOS.

CONCLUSION

Postoperative complications in TKA patients significantly increase hospital charges and LOS. The neural network model effectively predicted high-risk patients after specific complications occurred, offering a potential tool for improving patient management and resource allocation.

LEVELS OF EVIDENCE

Level III.

Predicting 30-day reoperation following primary total knee arthroplasty: machine learning model outperforms the ACS risk calculator

The ACS risk calculator (ARC) has proven less effective in predicting patient-specific risk of early reoperation after primary total knee arthroplasty (TKA), compromising care quality and cost efficiency. This study compared the performance of a machine learning (ML) model and ARC in predicting 30-day reoperation after primary TKA using a national-scale dataset. Data of 366,151 TKAs were acquired from the ACS-NSQIP database. A random forest model was derived using ARC build-in parameters from the training dataset via techniques of hyperparameter optimization and cross-validation. The predictive performance of random forest and ARC was evaluated by metrics of discrimination, calibration, and clinical utility using the testing dataset. The ML model demonstrated good discrimination and calibration (AUC: 0.72, slope: 1.18, intercept: - 0.14, Brier score: 0.012), outperforming ARC in all metrics (AUC: 0.51, slope: - 0.01, intercept: 0.01, Brier score: 0.135) including clinical utility measured by decision curve analyses. Age (> 67 years) and BMI (> 34 kg/m2) were the important predictors of reoperation. This study suggests the superiority of ML models in identifying individualized 30-day reoperation risk following TKA. ML models may be an adjunct prediction tool in enhancing patient-specific risk stratification and postoperative care management.

Prediction of risk of blood transfusion in patients with cirrhosis

BACKGROUND

Blood transfusion is usually required for cirrhotic patients with bleeding which is related to high risk of mortality. Identifying cirrhotic patients at high risk of bleeding and needing blood transfusions would benefit these patients, yet this remains an unmet need.

OBJECTIVES

This study aims to enhance blood transfusion management for patients with cirrhosis by developing a predictive model to assess the risk of transfusion.

METHODS

We enrolled a cohort of 711 patients diagnosed with cirrhosis at The First Affiliated Hospital of Nanjing Medical University and divided into training set ( n  = 537) and validation set ( n  = 174). Each participant underwent a comprehensive clinical assessment. Data on prothrombin time (PT), platelet counts, and inflammatory markers were collected. Univariate and multivariate logistic regression analyses were performed to identify independent predictors. The nomogram was constructed. Model performance was evaluated through receiver operating characteristic curve analysis.

RESULTS

The study successfully identified PT, platelet counts, and the mentioned inflammatory markers as significant predictors of the need for transfusion. The resulting nomogram demonstrated high predictive accuracy, with area under the curve values of 0.85 in the training set and 0.83 in the validation set.

CONCLUSION

The developed nomogram for predicting the need for blood transfusion in patients with cirrhosis shows promising effectiveness for clinical application. This tool can significantly contribute to optimizing transfusion practices, potentially improving patient care and outcomes through more personalized and efficient transfusion strategies.

Predicting Early Hospital Discharge Following Revision Total Hip Arthroplasty: An Analysis of a Large National Database Using Machine Learning

BACKGROUND

Revision total hip arthroplasty (rTHA) was recently removed from the Medicare inpatient-only list. However, appropriate candidate selection for outpatient rTHA remains paramount. The purpose of this study was to evaluate the utility of a large national database using machine learning (ML) and traditional multivariable logistic regression (MLR) models in predicting early hospital discharge (EHD) (< 24 hours) following rTHA. Furthermore, this study aimed to use the trained ML models, cross-referenced with traditional MLR, to determine key perioperative variables predictive of EHD following rTHA.

METHODS

Data were obtained from a large national database from 2021. Patients who had unilateral rTHA procedures were included. Demographic, preoperative, and operative variables were analyzed as inputs for the models. An ML regression model and various ML techniques were used to predict EHD and were compared using the area under the curve, calibration, Brier score, and decision curve analysis. Feature importance was identified from the overall best-performing model. Of the 3,097 patients in this study, 866 (27.96%) underwent EHD.

RESULTS

The random forest model performed the best overall and identified aseptic surgical indication, operative time < three hours, absence of anemia (hematocrit < 40% in men and < 35% in women), neuraxial anesthesia type, White race, men, independent functional status, body mass index > 20, age < 75 years, and the presence of home support as factors predictive of EHD. Each of these variables was also significant in the MLR model.

CONCLUSIONS

Each ML model and MLR displayed good performance and identified clinically important variables for determining candidates for EHD following rTHA. Machine learning (ML) techniques such as random forest may allow clinicians to accurately risk stratify their patients preoperatively to optimize resources and improve patient outcomes.

Predicting prolonged length of stay following revision total knee arthroplasty: A national database analysis using machine learning models

BACKGROUND

As the number of revision total knee arthroplasty (TKA) continues to rise, close attention has been paid to factors influencing postoperative length of stay (LOS). The aim of this study is to develop generalizable machine learning (ML) algorithms to predict extended LOS following revision TKA using data from a national database.

METHODS

23,656 patients undergoing revision TKA between 2013 and 2020 were identified using the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) database. Patients with missing data and those undergoing re-revision or conversion from unicompartmental knee arthroplasty were excluded. Four ML algorithms were applied and evaluated based on their (1) ability to distinguish between at-risk and not-at-risk patients, (2) accuracy, (3) calibration, and (4) clinical utility.

RESULTS

All four ML predictive algorithms demonstrated good accuracy, calibration, clinical utility, and discrimination, with all models achieving a similar area under the curve (AUC) (AUCLR=AUCRF=AUCHGB=0.75, AUCANN=0.74). The most important predictors of prolonged LOS were found to be operative time, preoperative diagnosis of sepsis, and body mass index (BMI).

CONCLUSIONS

ML models developed in this study demonstrated good performance in predicting extended LOS in patients undergoing revision TKA. Our findings highlight the importance of utilizing nationally representative patient data for model development. Prolonged operative time, preoperative sepsis, BMI, and elevated preoperative serum creatinine and BUN were noted to be significant predictors of prolonged LOS. Knowledge of these associations may aid with patient-specific preoperative planning, discharge planning, patient counseling, and cost containment with revision TKA.

Prediction of 30-Day Mortality Following Revision Total Hip and Knee Arthroplasty: Machine Learning Algorithms Outperform CARDE-B, 5-Item, and 6-Item Modified Frailty Index Risk Scores

BACKGROUND

Although risk calculators are used to prognosticate postoperative outcomes following revision total hip and knee arthroplasty (total joint arthroplasty [TJA]), machine learning (ML) based predictive tools have emerged as a promising alternative for improved risk stratification. This study aimed to compare the predictive ability of ML models for 30-day mortality following revision TJA to that of traditional risk-assessment indices such as the CARDE-B score (congestive heart failure, albumin (< 3.5 mg/dL), renal failure on dialysis, dependence for daily living, elderly (> 65 years of age), and body mass index (BMI) of < 25 kg/m2), 5-item modified frailty index (5MFI), and 6MFI.

METHODS

Adult patients undergoing revision TJA between 2013 and 2020 were selected from the American College of Surgeons National Surgical Quality Improvement Program database and randomly split 80:20 to compose the training and validation cohorts. There were 3 ML models - extreme gradient boosting, random forest, and elastic-net penalized logistic regression (NEPLR) - that were developed and evaluated using discrimination, calibration metrics, and accuracy. The discrimination of CARDE-B, 5MFI, and 6MFI scores was assessed individually and compared to that of ML models.

RESULTS

All models were equally accurate (Brier score = 0.005) and demonstrated outstanding discrimination with similar areas under the receiver operating characteristic curve (AUCs, extreme gradient boosting = 0.94, random forest = NEPLR = 0.93). The NEPLR was the best-calibrated model overall (slope = 0.54, intercept = -0.004). The CARDE-B had the highest discrimination among the scores (AUC = 0.89), followed by 6MFI (AUC = 0.80), and 5MFI (AUC = 0.68). Albumin < 3.5 mg/dL and BMI (< 30.15) were the most important predictors of 30-day mortality following revision TJA.

CONCLUSIONS

The ML models outperform traditional risk-assessment indices in predicting postoperative 30-day mortality after revision TJA. Our findings highlight the utility of ML for risk stratification in a clinical setting. The identification of hypoalbuminemia and BMI as prognostic markers may allow patient-specific perioperative optimization strategies to improve outcomes following revision TJA.

Generalizability of machine learning models predicting 30-day unplanned readmission after primary total knee arthroplasty using a nationally representative database

Unplanned readmission after primary total knee arthroplasty (TKA) costs an average of US $39,000 per episode and negatively impacts patient outcomes. Although predictive machine learning (ML) models show promise for risk stratification in specific populations, existing studies do not address model generalizability. This study aimed to establish the generalizability of previous institutionally developed ML models to predict 30-day readmission following primary TKA using a national database. Data from 424,354 patients from the ACS-NSQIP database was used to develop and validate four ML models to predict 30-day readmission risk after primary TKA. Individual model performance was assessed and compared based on discrimination, accuracy, calibration, and clinical utility. Length of stay (> 2.5 days), body mass index (BMI) (> 33.21 kg/m2), and operation time (> 93 min) were important determinants of 30-day readmission. All ML models demonstrated equally good accuracy, calibration, and discriminatory ability (Brier score, ANN = RF = HGB = NEPLR = 0.03; ANN, slope = 0.90, intercept =  - 0.11; RF, slope = 0.93, intercept =  - 0.12; HGB, slope = 0.90, intercept =  - 0.12; NEPLR, slope = 0.77, intercept = 0.01; AUCANN = AUCRF = AUCHGB = AUCNEPLR = 0.78). This study validates the generalizability of four previously developed ML algorithms in predicting readmission risk in patients undergoing TKA and offers surgeons an opportunity to reduce readmissions by optimizing discharge planning, BMI, and surgical efficiency.

Development and Validation of a Machine Learning Algorithm to Predict the Risk of Blood Transfusion after Total Hip Replacement in Patients with Femoral Neck Fractures: A Multicenter Retrospective Cohort Study

OBJECTIVE

Total hip arthroplasty (THA) remains the primary treatment option for femoral neck fractures in elderly patients. This study aims to explore the risk factors associated with allogeneic blood transfusion after surgery and to develop a dynamic prediction model to predict post-operative blood transfusion requirements. This will provide more accurate guidance for perioperative humoral management and rational allocation of medical resources.

METHODS

We retrospectively analyzed data from 829 patients who underwent total hip arthroplasty for femoral neck fractures at three third-class hospitals between January 2017 and August 2023. Patient data from one hospital were used for model development, whereas data from the other two hospitals were used for external validation. Logistic regression analysis was used to screen the characteristic subsets related to blood transfusion. Various machine learning algorithms, including logistic regression, SVA (support vector machine), K-NN (k-nearest neighbors), MLP (multilayer perceptron), naive Bayes, decision tree, random forest, and gradient boosting, were used to process the data and construct prediction models. A 10-fold cross-validation algorithm facilitated the comparison of the predictive performance of the models, resulting in the selection of the best-performing model for the development of an open-source computing program.

RESULTS

BMI (body mass index), surgical duration, IBL (intraoperative blood loss), anticoagulant history, utilization rate of tranexamic acid, Pre-Hb, and Pre-ALB were included in the model as well as independent risk factors. The average area under curve (AUC) values for each model were as follows: logistic regression (0.98); SVA (0.91); k-NN (0.87) MLP, (0.96); naive Bayes (0.97); decision tree (0.87); random forest (0.96); and gradient boosting (0.97). A web calculator based on the best model is available at: (https://nomo99.shinyapps.io/dynnomapp/).

CONCLUSION

Utilizing a computer algorithm, a prediction model with a high discrimination accuracy (AUC > 0.5) was developed. The logistic regression model demonstrated superior differentiation and reliability, thereby successfully passing external validation. The model's strong generalizability and applicability have significant implications for clinicians, aiding in the identification of patients at high risk for postoperative blood transfusion.

Development of machine learning models to predict perioperative blood transfusion in hip surgery

BACKGROUND

Allogeneic Blood transfusion is common in hip surgery but is associated with increased morbidity. Accurate prediction of transfusion risk is necessary for minimizing blood product waste and preoperative decision-making. The study aimed to develop machine learning models for predicting perioperative blood transfusion in hip surgery and identify significant risk factors.

METHODS

Data of patients undergoing hip surgery between January 2013 and October 2021 in the Peking Union Medical College Hospital were collected to train and test predictive models. The primary outcome was perioperative red blood cell (RBC) transfusion within 72 h of surgery. Fourteen machine learning algorithms were established to predict blood transfusion risk incorporating patient demographic characteristics, preoperative laboratory tests, and surgical information. Discrimination, calibration, and decision curve analysis were used to evaluate machine learning models. SHapley Additive exPlanations (SHAP) was performed to interpret models.

RESULTS

In this study, 2431 hip surgeries were included. The Ridge Classifier performed the best with an AUC = 0.85 (95% CI, 0.81 to 0.88) and a Brier score = 0.21. Patient-related risk factors included lower preoperative hemoglobin, American Society of Anesthesiologists (ASA) Physical Status > 2, anemia, lower preoperative fibrinogen, and lower preoperative albumin. Surgery-related risk factors included longer operation time, total hip arthroplasty, and autotransfusion.

CONCLUSIONS

The machine learning model developed in this study achieved high predictive performance using available variables for perioperative blood transfusion in hip surgery. The predictors identified could be helpful for risk stratification, preoperative optimization, and outcomes improvement.

Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review

» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.

Risk of blood transfusion following total hip arthroplasty: A national database study from 367,894 patients

Purpose

A growing elderly population in the United States coupled with improvement in surgical techniques have resulted in more elderly individuals undergoing total hip arthroplasty (THA). As such, risk factors associated with increased risk of blood transfusion following THA, which has been linked to various detrimental outcomes, must be better understood. This study aims to identify co-morbidities associated with blood transfusion following THA.

Methods

Using the Nationwide Inpatient Sample (NIS) database, we selected patients that received a THA from 2016 to 2019 using ICD-10CMP codes. Patients were classified into a "blood transfusion" or "no transfusion" groups and data pertaining to demographics, co-morbidities, and events during hospital stays were compared between the groups.

Results

Our study dataset included 367,894 patients from the NIS database that underwent a THA from 2016 to 2019. 12,900 (3.5 %) patients received a blood transfusion after their THA and were classified as "blood transfusion group." The remaining 354,994 patients were classified as the "no transfusion group." Elective admission was found to decrease the odds of a blood transfusion following a THA (compared to nonelective THA: odd's ratio 0.283; p value < 0.001). Multivariate analysis demonstrated sickle cell disease, liver cirrhosis, and dialysis exhibited the greatest increase in odds of blood transfusion after a THA by 4.81- (p < 0.001), 3.02- (p < 0.001), and 2.22-fold (p < 0.001), respectively. Looking at patient demographics, male sex increased odds of postoperative transfusion by 1.99 (p < 0.001) while Caucasian ethnicity decreased odds of postoperative transfusion by 0.65 (p < 0.001).

Conclusion

Blood transfusion has a low occurrence in the early post-operative period following THA (3.6 % of patients). Sickle cell disease, liver cirrhosis, dialysis, SLE, and heart pathologies were the comorbidities found to be most significantly associated with an increased risk of blood transfusion after a THA. Additionally, both mortality and non-elective admissions were significantly more prevalent in the "blood transfusion" group.

Precision medicine for traumatic cervical spinal cord injuries: accessible and interpretable machine learning models to predict individualized in-hospital outcomes

BACKGROUND CONTEXT

A traumatic spinal cord injury (SCI) can cause temporary or permanent motor and sensory impairment, leading to serious short and long-term consequences that can result in significant morbidity and mortality. The cervical spine is the most commonly affected area, accounting for about 60% of all traumatic SCI cases.

PURPOSE

This study aims to employ machine learning (ML) algorithms to predict various outcomes, such as in-hospital mortality, nonhome discharges, extended length of stay (LOS), extended length of intensive care unit stay (ICU-LOS), and major complications in patients diagnosed with cervical SCI (cSCI).

STUDY DESIGN

Our study was a retrospective machine learning classification study aiming to predict the outcomes of interest, which were binary categorical variables, in patients diagnosed with cSCI.

PATIENT SAMPLE

The data for this study were obtained from the American College of Surgeons (ACS) Trauma Quality Program (TQP) database, which was queried to identify patients who suffered from cSCI between 2019 and 2021.

OUTCOME MEASURES

The outcomes of interest of our study were in-hospital mortality, nonhome discharges, prolonged LOS, prolonged ICU-LOS, and major complications. The study evaluated the models' performance using both graphical and numerical methods. The receiver operating characteristic (ROC) and precision-recall curves (PRC) were used to assess model performance graphically. Numerical evaluation metrics included AUROC, balanced accuracy, weighted area under PRC (AUPRC), weighted precision, and weighted recall.

METHODS

The study employed data from the American College of Surgeons (ACS) Trauma Quality Program (TQP) database to identify patients with cSCI. Four ML algorithms, namely XGBoost, LightGBM, CatBoost, and Random Forest, were utilized to develop predictive models. The most effective models were then incorporated into a publicly available web application designed to forecast the outcomes of interest.

RESULTS

There were 71,661 patients included in the analysis for the outcome mortality, 67,331 for the outcome nonhome discharges, 76,782 for the outcome prolonged LOS, 26,615 for the outcome prolonged ICU-LOS, and 72,132 for the outcome major complications. The algorithms exhibited an AUROC value range of 0.78 to 0.839 for in-hospital mortality, 0.806 to 0.815 for nonhome discharges, 0.679 to 0.742 for prolonged LOS, 0.666 to 0.682 for prolonged ICU-LOS, and 0.637 to 0.704 for major complications. An open access web application was developed as part of the study, which can generate predictions for individual patients based on their characteristics.

CONCLUSIONS

Our study suggests that ML models can be valuable in assessing risk for patients with cervical cSCI and may have considerable potential for predicting outcomes during hospitalization. ML models demonstrated good predictive ability for in-hospital mortality and nonhome discharges, fair predictive ability for prolonged LOS, but poor predictive ability for prolonged ICU-LOS and major complications. Along with these promising results, the development of a user-friendly web application that facilitates the integration of these models into clinical practice is a significant contribution of this study. The product of this study may have significant implications in clinical settings to personalize care, anticipate outcomes, facilitate shared decision making and informed consent processes for cSCI patients.