Development and internal validation of machine learning algorithms for predicting complications after primary total hip arthroplasty

The Application of Artificial Intelligence in Spine Surgery: A Scoping Review

BACKGROUND

A comprehensive review on the application of artificial intelligence (AI) within spine surgery as a specialty remains lacking.

METHODS

This scoping review was conducted upon PubMed and EMBASE databases according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Our analysis focused on publications from January 1, 2020, to March 31, 2024, with a specific focus on AI in the field of spine surgery. Review articles and articles predominantly concerning secondary validation of algorithms, medical physics, electronic devices, biomechanics, preclinical, and with a lack of clinical emphasis were excluded.

RESULTS

One hundred five studies were included after our inclusion/exclusion criteria were applied. Most studies (n = 100) were conducted through supervised learning upon prelabeled data sets. Overall, 38 studies used conventional machine learning methods upon predefined features, whereas 67 used deep learning methods, predominantly for medical image analyses. Only 25.7% of studies (27/105) collected data from more than 1,000 patients for model development and validation. Data originated from only a single center in 72 studies. The most common application was prognostication (38/105), followed by diagnosis (35/105), medical image processing (29/105), and surgical assistance (3/105).

CONCLUSION

The application of AI within the domain of spine surgery has significant potential to advance patient-specific diagnosis, management, and surgical execution.

The Application of artificial intelligence in periprosthetic joint infection

Periprosthetic joint infection (PJI) represents one of the most devastating complications following total joint arthroplasty, often necessitating additional surgeries and antimicrobial therapy, and potentially leading to disability. This significantly increases the burden on both patients and the healthcare system. Given the considerable suffering caused by PJI, its prevention and treatment have long been focal points of concern. However, challenges remain in accurately assessing individual risk, preventing the infection, improving diagnostic methods, and enhancing treatment outcomes. The development and application of artificial intelligence (AI) technologies have introduced new, more efficient possibilities for the management of many diseases. In this article, we review the applications of AI in the prevention, diagnosis, and treatment of PJI, and explore how AI methodologies might achieve individualized risk prediction, improve diagnostic algorithms through biomarkers and pathology, and enhance the efficacy of antimicrobial and surgical treatments. We hope that through multimodal AI applications, intelligent management of PJI can be realized in the future.

Comparing prediction accuracy for 30-day readmission following primary total knee arthroplasty: the ACS-NSQIP risk calculator versus a novel artificial neural network model

BACKGROUND

Unplanned readmission, a measure of surgical quality, occurs after 4.8% of primary total knee arthroplasties (TKA). Although the prediction of individualized readmission risk may inform appropriate preoperative interventions, current predictive models, such as the American College of Surgeons National Surgical Quality Improvement Program (ACS-NSQIP) surgical risk calculator (SRC), have limited utility. This study aims to compare the predictive accuracy of the SRC with a novel artificial neural network (ANN) algorithm for 30-day readmission after primary TKA, using the same set of clinical variables from a large national database.

METHODS

Patients undergoing primary TKA between 2013 and 2020 were identified from the ACS-NSQIP database and randomly stratified into training and validation cohorts. The ANN was developed using data from the training cohort with fivefold cross-validation performed five times. ANN and SRC performance were subsequently evaluated in the distinct validation cohort, and predictive performance was compared on the basis of discrimination, calibration, accuracy, and clinical utility.

RESULTS

The overall cohort consisted of 365,394 patients (trainingN = 362,559; validationN = 2835), with 11,392 (3.1%) readmitted within 30 days. While the ANN demonstrated good discrimination and calibration (area under the curve (AUC)ANN = 0.72, slope = 1.32, intercept = -0.09) in the validation cohort, the SRC demonstrated poor discrimination (AUCSRC = 0.55) and underestimated readmission risk (slope = -0.21, intercept = 0.04). Although both models possessed similar accuracy (Brier score: ANN = 0.03; SRC = 0.02), only the ANN demonstrated a higher net benefit than intervening in all or no patients on the decision curve analysis. The strongest predictors of readmission were body mass index (> 33.5 kg/m2), age (> 69 years), and male sex.

CONCLUSIONS

This study demonstrates the superior predictive ability and potential clinical utility of the ANN over the conventional SRC when constrained to the same variables. By identifying the most important predictors of readmission following TKA, our findings may assist in the development of novel clinical decision support tools, potentially improving preoperative counseling and postoperative monitoring practices in at-risk patients.

Optimal inputs for machine learning models in predicting total joint arthroplasty outcomes: a systematic review

INTRODUCTION

Machine learning (ML) models may offer a novel solution to reducing postoperative complication rates and improving post-surgical outcomes after total joint arthroplasty (TJA). However, the variety of different ML models that exist paired with the increasing number of potential inputs can make the implementation of this tool challenging. Therefore, we conducted a systematic review to assess the most optimal inputs of different ML models in predicting postoperative (1) medical outcomes, (2) orthopedic outcomes, and (3) patient-reported outcome measures (PROMs) after total joint arthroplasty.

METHODS

The PubMed, MEDLINE, EBSCOhost, and Google Scholar databases were utilized to identify all studies evaluating ML models predicting outcomes following TJA between January 1, 2000, and June 23, 2023 (PROSPERO study protocol registration: CRD42023437586). The mean risk of bias in non-randomized studies-of interventions score, was 13.8 ± 0.5. Our initial query yielded 656 articles, of which 25 articles aligned with our aims, examining over 20 machine learning models and 1,555,300 surgeries. The area under the curve (AUC), accuracy, inputs, and the importance of each input were reported.

RESULTS

Twelve studies evaluating medical complications with 13 ML models reported AUCs ranging from 0.57 to 0.997 and accuracy between 88% and 99.98%. Key predictors included age, hyper-coagulopathy, total number of diagnoses, admission month, and malnutrition. Five studies evaluating orthopedic complications with 10 ML models reported AUCs from 0.49 to 0.93 and accuracy ranging from 92 to 97%, with age, BMI, CCI, AKSS scores, and height identified as key predictors. Ten studies evaluating PROMs comprising of 12 different ML models had an AUC ranging from 0.453 to 0.97 ranked preoperative PROMs as the post-predictive input. Overall, age was the most predictive risk factor for complications post-total joint arthroplasty (TJA).

CONCLUSION

These studies demonstrate the predictive capabilities of these models for anticipating complications and outcomes. Furthermore, these studies also highlight ML models' ability to identify non-classical variables not commonly considered in addition to confirming variables known to be crucial. To advance the field, forthcoming research should adhere to established guidelines for model development and training, employ industry-standard input parameters, and subject their models to external validity assessments.

Machine learning model identifies patient gait speed throughout the episode of care, generating notifications for clinician evaluation

INTRODUCTION

The advent of digital and mobile health innovations, especially use of wearables for passive data collection, allows remote monitoring and creates an abundance of data. For this information to be interpretable, machine learning (ML) processes are necessary.

RESEARCH QUESTION

Can a machine learning model successfully identify patients expected to have low gait speed in the early recovery period following joint replacement surgery?

METHODS

A commercial database from a smartphone-based care management platform passively collecting mobility data pre- and post-lower limb arthroplasty was used. We sought to create a ML model to predict gait speed recovery curves and identify patients at risk of poor gait speed outcome, a measure associated with range of motion and patient-reported outcomes. Model performance including sensitivity, specificity, precision, and accuracy were determined. Receiver operator curve (ROC) analysis was used to compare true and false positive rates. To benchmark our model, we compared threshold-based notifications based on the patient's current gait speed.

RESULTS

The performance of the predictive model was significantly improved compared to baseline of threshold-based exceptions using current gait speed. The ML model currently provides 53 % precision, 88 % accuracy, 36 % sensitivity, and 95 % specificity on the held-out test set. The ROC analysis suggests good clinical performance (AUC=0.81).

SIGNIFICANCE

Utilization of ML to predict gait recovery following total joint replacement is feasible and provides results with excellent specificity. This model will allow inclusion of additional data for retraining as patient populations evolve. Clinician feedback regarding notifications, including resulting actions and outcomes, can be used to further inform the model and improve clinical utility.

-A machine learning model to predict surgical site infection after surgery of lower extremity fractures

PURPOSE

This study aimed to develop machine learning algorithms for identifying predictive factors associated with the risk of postoperative surgical site infection in patients with lower extremity fractures.

METHODS

A machine learning analysis was conducted on a dataset comprising 1,579 patients who underwent surgical fixation for lower extremity fractures to create a predictive model for risk stratification of postoperative surgical site infection. We evaluated different clinical and demographic variables to train four machine learning models (neural networks, boosted generalised linear model, naïve bayes, and penalised discriminant analysis). Performance was measured by the area under the curve score, Youdon's index and Brier score. A multivariate adaptive regression splines (MARS) was used to optimise predictor selection.

RESULTS

The final model consisted of five predictors. (1) Operating room time, (2) ankle region, (3) open injury, (4) body mass index, and (5) age. The best-performing machine learning algorithm demonstrated a promising predictive performance, with an area under the ROC curve, Youdon's index, and Brier score of 77.8%, 62.5%, and 5.1%-5.6%, respectively.

CONCLUSION

The proposed predictive model not only assists surgeons in determining high-risk factors for surgical site infections but also empowers patients to closely monitor these factors and take proactive measures to prevent complications. Furthermore, by considering the identified predictors, this model can serve as a reference for implementing preventive measures and reducing postoperative complications, ultimately enhancing patient outcomes. However, further investigations involving larger datasets and external validations are required to confirm the reliability and applicability of our model.

Applying models of care for total hip and knee arthroplasty: External validation of a published predictive model to identify extended stay risk prior to lower-limb arthroplasty

OBJECTIVE

This study aimed to externally validate a reported model for identifying patients requiring extended stay following lower limb arthroplasty in a new setting.

DESIGN

External validation of a previously reported prognostic model, using retrospective data.

SETTING

Medium-sized hospital orthopaedic department, Australia.

PARTICIPANTS

Electronic medical records were accessed for data collection between Sep-2019 and Feb-2020 and retrospective data extracted from 200 randomly selected total hip or knee arthroplasty patients.

INTERVENTION

Participants received total hip or knee replacement between 2-Feb-16 and 4-Apr-19. This study was a non-interventional retrospective study.

MAIN MEASURES

Model validation was assessed with discrimination, calibration on both original and adjusted forms of the candidate model. Decision curve analysis was conducted on the outputs of the adjusted model to determine net benefit at a predetermined decision threshold (0.5).

RESULTS

The original model performed poorly, grossly overestimating length of stay with mean calibration of -3.6 (95% confidence interval -3.9 to -3.2) and calibration slope of 0.52. Performance improved following adjustment of the model intercept and model coefficients (mean calibration 0.48, 95% confidence interval 0.16 to 0.80 and slope of 1.0), but remained poorly calibrated at low and medium risk threshold and net benefit was modest (three additional patients per hundred identified as at-risk) at the a-priori risk threshold.

CONCLUSIONS

External validation demonstrated poor performance when applied to a new patient population and would provide limited benefit for our institution. Implementation of predictive models for arthroplasty should include practical assessment of discrimination, calibration and net benefit at a clinically acceptable threshold.

Prediction of Complications and Prognostication in Perioperative Medicine: A Systematic Review and PROBAST Assessment of Machine Learning Tools

BACKGROUND

The utilization of artificial intelligence and machine learning as diagnostic and predictive tools in perioperative medicine holds great promise. Indeed, many studies have been performed in recent years to explore the potential. The purpose of this systematic review is to assess the current state of machine learning in perioperative medicine, its utility in prediction of complications and prognostication, and limitations related to bias and validation.

METHODS

A multidisciplinary team of clinicians and engineers conducted a systematic review using the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) protocol. Multiple databases were searched, including Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL), the Cochrane Library, PubMed, Medline, Embase, and Web of Science. The systematic review focused on study design, type of machine learning model used, validation techniques applied, and reported model performance on prediction of complications and prognostication. This review further classified outcomes and machine learning applications using an ad hoc classification system. The Prediction model Risk Of Bias Assessment Tool (PROBAST) was used to assess risk of bias and applicability of the studies.

RESULTS

A total of 103 studies were identified. The models reported in the literature were primarily based on single-center validations (75%), with only 13% being externally validated across multiple centers. Most of the mortality models demonstrated a limited ability to discriminate and classify effectively. The PROBAST assessment indicated a high risk of systematic errors in predicted outcomes and artificial intelligence or machine learning applications.

CONCLUSIONS

The findings indicate that the development of this field is still in its early stages. This systematic review indicates that application of machine learning in perioperative medicine is still at an early stage. While many studies suggest potential utility, several key challenges must be first overcome before their introduction into clinical practice.

EDITOR’S PERSPECTIVE

The risk analysis of perioperative complications of cementless hip arthroplasty in octogenarians

INTRODUCTION

Hip arthroplasty is exposed to demographic change as patients age. Analysis of risk factors for surgical treatment decisions in the group of ≥ 80-year-old patients is crucial. Healthcare systems in developed countries are being tested medically and financially by the ageing population. Therefore, this study analysed the perioperative complications of cementless primary hip arthroplasty in octogenarians and compared them with patients aged ≤ 60 years.

METHODS

A retrospective data analysis of the year 2017 was done in a maximum care hospital of General Orthopaedic Surgery. Patients aged ≥ 80 years or ≤ 60 years with primary cementless hip arthroplasty were included. The outcome of interest was surgery-related and systemic complications, the development of haemoglobin and the incidence of blood transfusion after cementless primary hip arthroplasty in octogenarians during the hospitalisation and the follow-up treatment. Chi-square tests and Fischer's exact test were used for nominal variables. The two-factorial variance analysis-mixed model was used for Hb analyses and the Welch test for group comparison for metric parameters.

RESULTS

There was a significantly increased incidence of systemic complications during hospitalisation in the ≥ 80-year-old patients (phi 0.26; Std. Ri - 0.8 (A), 2.2 (B); p = 0.007), as well as a significantly increased rate of blood transfusions (phi 0.403; Std. Ri - 1.3 (A), 3.2 (B); p = < 0.001). No clustered pre-existing conditions in the ≥ 80-year-old patients pointed out a significant association with the incidence of systemic complications. Surgery-related complications showed no significant difference during hospitalisation and follow-up treatment.

CONCLUSION

The study reveals that primary cementless hip prosthesis implantation is a safe procedure without increased incidence of surgery-related complications. Increased attention should be paid to interdisciplinary preoperative optimisation (adjustment of blood pressure, blood transfusions, if necessary, safe exclusion of urinary tract infections) and postoperative care of octogenarians (tight laboratory examinations, geriatric co-attendance).

The incidence of hip dislocation after posterior approach primary total hip arthroplasty: comparison of two different posterior repair techniques

INTRODUCTION

Total hip arthroplasty (THA) remains one of the most successful orthopedic surgical procedures. The posterior approach is associated with a higher incidence of post-operative dislocations than others. Adequate posterior soft tissue repair techniques, including capsulorrhaphy and transosseous bone sutures in the greater trochanter effectively reduce the dislocation rate. Post-operative "posterior hip precautions" were historically believed to reduce dislocation risks, although not clearly proven. The first protocol consists of capsulorrhaphy with the prescription of post-operative posterior hip precautions (TT) and the second, transosseous bone sutures without precautions (TB). This study aims to determine the optimal protocol to decrease the dislocation rate following posterior approach primary THA.

MATERIALS AND METHODS

A 10-year retrospective case-control chart review analyzed demographic, pre-, intra-, and post-surgical parameters. Primary outcomes were the difference in dislocation and revision surgery rates between protocols. Secondary outcomes included the incidence of recurrent dislocations and the identification of predictors of dislocation.

RESULTS

2,242 THAs were reviewed and 26 (1.2%) resulted in dislocation. Increased age (p = 0.04) ASA score (p = 0.03) and larger acetabular cup size (p < 0.001) were associated with heightened risk. Tendon to tendon (TT) repair saw a 1.62% dislocation rate versus 0.98% for tendon-to-bone (TB) repair, although statistically insignificant (p = 0.2). Transosseous repair resulted in recurrent dislocations for 8/16 (50%) patients compared to 6/10 (60%) in the suture group (p ≤ 0.001). No significance was found for prescription of posterior hip precautions.

CONCLUSIONS

To our knowledge, this is the first study to perform a direct comparison of TT repair with posterior precautions to TB repair without posterior precautions. Similarity in dislocation rate, decreased recurrent events and the alleviated patient burden from precautions leads the authors to recommend the TB repair without precautions for a successful THA.