Internal Validation of a Predictive Model for Satisfaction After Primary Total Knee Arthroplasty

Instruments to assess appropriateness of hip and knee arthroplasty: a systematic review

OBJECTIVE

To assess criteria and psychometric properties of instruments for assessing appropriateness of elective joint arthroplasty (JA) for adults with primary hip and knee osteoarthritis (OA).

METHODS

A systematic review guided by Cochrane methods and PRISMA guidelines. Studies were searched in five databases. Eligible articles include all study designs developing, testing, and/or using an instrument to assess JA appropriateness. Two independent reviewers screened and extracted data. Instruments were compared with Hawker et al. JA consensus criteria. Psychometric properties of instruments were described and appraised guided by Fitzpatrick's and COSMIN approaches.

RESULTS

Of 55 instruments included, none met all Hawker et al. JA consensus criteria. Criteria the most met were pain (n = 50), function (n = 49), quality of life (n = 33), and radiography (n = 24). Criteria the least met were clinical evidence of OA (n = 18), expectations (n = 15), readiness for surgery (n = 11), conservative treatments (n = 8), and patient/surgeon agree benefits outweigh risks (n = 0). Instrument by Arden et al. met the most criteria (6 of 9). The most tested psychometric properties were appropriateness (n = 55), face/content validity (n = 55), predictive validity (n = 29), construct validity and feasibility (n = 24). The least tested psychometric properties were intra-rater reliability (n = 3), internal consistency (n = 5), and inter-rater reliability (n = 13). Instruments by Gutacker et al. and Osborne et al. met the most psychometric properties (4 of 10).

CONCLUSION

Most instruments included traditional criteria for assessing JA appropriateness but did not include a trial of conservative treatments or shared decision-making elements. There was limited evidence on psychometric properties.

SMART choice (knee) tool: a patient-focused predictive model to predict improvement in health-related quality of life after total knee arthroplasty

BACKGROUND

Current predictive tools for TKA focus on clinicians rather than patients as the intended user. The purpose of this study was to develop a patient-focused model to predict health-related quality of life outcomes at 1-year post-TKA.

METHODS

Patients who underwent primary TKA for osteoarthritis from a tertiary institutional registry after January 2006 were analysed. The primary outcome was improvement after TKA defined by the minimal clinically important difference in utility score at 1-year post-surgery. Potential predictors included demographic information, comorbidities, lifestyle factors, and patient-reported outcome measures. Four models were developed, including both conventional statistics and machine learning (artificial intelligence) methods: logistic regression, classification tree, extreme gradient boosted trees, and random forest models. Models were evaluated using discrimination and calibration metrics.

RESULTS

A total of 3755 patients were included in the study. The logistic regression model performed the best with respect to both discrimination (AUC = 0.712) and calibration (intercept = -0.083, slope = 1.123, Brier score = 0.202). Less than 2% (n = 52) of the data were missing and therefore removed for complete case analysis. The final model used age (categorical), sex, baseline utility score, and baseline Veterans-RAND 12 responses as predictors.

CONCLUSION

The logistic regression model performed better than machine learning algorithms with respect to AUC and calibration plot. The logistic regression model was well calibrated enough to stratify patients into risk deciles based on their likelihood of improvement after surgery. Further research is required to evaluate the performance of predictive tools through pragmatic clinical trials.

LEVEL OF EVIDENCE

Level II, decision analysis.

A predictive score of high tibial osteotomy survivorship to help in surgical decision-making: the SKOOP score

INTRODUCTION

The high tibial osteotomy (HTO) survival rate is strongly correlated with surgical indications and predictive factors. This study aims to assess HTO survival in the long term, to determine the main predictive factors of this survival, to propose a predictive score for HTO based on those factors.

METHODS

This multicentric study included 481 HTO between 2004 and 2015. The inclusion criteria were all primary HTO in patients 70 years old and younger, without previous anterior cruciate ligament injury, and without the limitation of body mass index (BMI). The assessed data were preoperative clinical and radiological parameters, the surgical technique, the complications, the HKA (hip knee ankle angle) correction postoperatively, and the surgical revision at the last follow-up.

RESULTS

The mean follow-up was 7.8 ± 2.9 years. The HTO survival was 93.1% at 5 years and 74.1% at 10 years. Age < 55, female sex, BMI < 25 kg/m² and incomplete narrowing were preoperative factors that positively impacted HTO survival. A postoperative HKA angle greater than 180° was a positive factor for HTO survival. The SKOOP (Sfa Knee OsteOtomy Predictive) score, including age (threshold value of 55 years), BMI (threshold values of 25 and 35 kg/m²), and the presence or absence of complete joint line narrowing, have been described. If the scale was greater than 3, the survival probability was significantly lower (p < 0.001) than if the scale was less than 3.

CONCLUSION

A predictive score including age, BMI, and the presence or absence of joint line narrowing can be a helpful in making decisions about HTO, particularly in borderline cases.

LEVEL OF EVIDENCE

Retrospective cohort study.

Artificial intelligence in knee arthroplasty: current concept of the available clinical applications

Background

Artificial intelligence (AI) is defined as the study of algorithms that allow machines to reason and perform cognitive functions such as problem-solving, objects, images, word recognition, and decision-making. This study aimed to review the published articles and the comprehensive clinical relevance of AI-based tools used before, during, and after knee arthroplasty.

Methods

The search was conducted through PubMed, EMBASE, and MEDLINE databases from 2000 to 2021 using the 2009 Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol (PRISMA).

Results

A total of 731 potential articles were reviewed, and 132 were included based on the inclusion criteria and exclusion criteria. Some steps of the knee arthroplasty procedure were assisted and improved by using AI-based tools. Before surgery, machine learning was used to aid surgeons in optimizing decision-making. During surgery, the robotic-assisted systems improved the accuracy of knee alignment, implant positioning, and ligamentous balance. After surgery, remote patient monitoring platforms helped to capture patients’ functional data.

Conclusion

In knee arthroplasty, the AI-based tools improve the decision-making process, surgical planning, accuracy, and repeatability of surgical procedures.

Logistic Regression and Machine Learning Models Cannot Discriminate Between Satisfied and Dissatisfied Total Knee Arthroplasty Patients

BACKGROUND

Approximately 20% of total knee arthroplasty (TKA) patients are found to be dissatisfied or unsure of their satisfaction at 1-year post-surgery. This study attempted to predict 1-year post-surgery dissatisfied/unsure TKA patients with pre-surgery and surgical variables using logistic regression and machine learning methods.

METHODS

A retrospective analysis of patients who underwent primary TKA for osteoarthritis between 2012 and 2016 at a single institution was completed. Patients were split into satisfied and dissatisfied/unsure groups. Potential predictor variables included the following: demographic information, patella re-surfaced, posterior collateral ligament sacrificed, and subscales from the Knee Society Knee Scoring System, the Knee Society Clinical Rating System, the Western Ontario and McMaster Universities Osteoarthritis Index, and the 12-Item Short Form Health Survey version 2. Logistic regression and 6 different machine learning methods were used to create prediction models. Model performance was evaluated using discrimination (AUC [area under the receiver operating characteristic curve]) and calibration (Brier score, Cox intercept, and Cox slope) metrics.

RESULTS

There were 1432 eligible patients included in the analysis, 313 were considered to be dissatisfied/unsure. When evaluating discrimination, the logistic regression (AUC = 0.736) and extreme gradient boosted tree (AUC = 0.713) models performed best. When evaluating calibration, the logistic regression (Brier score = 0.141, Cox intercept = 0.241, and Cox slope = 1.31) and gradient boosted tree (Brier score = 0.149, Cox intercept = 0.054, and Cox slope = 1.158) models performed best.

CONCLUSION

The models developed in this study do not perform well enough as discriminatory tools to be used in a clinical setting. Further work needs to be done to improve the performance of pre-surgery TKA dissatisfaction prediction models.

Patient Satisfaction Following Primary Total Knee Arthroplasty: Contributing Factors

The reported dissatisfaction rate after primary total knee arthroplasty (TKA) ranges between 15% and 25%. The purpose of this article is to perform a narrative review of the literature with the aim of answering the following question: What are the main factors contributing to patient dissatisfaction after TKA? A review of the literature was performed on patient satisfaction after TKA. The search engines used were MedLine (PubMed) and the Cochrane Library. The keywords used were "TKA" and "satisfaction". The main reported preoperative factors positively contributing to patient satisfaction were the following: fulfilment of preoperative expectations, preoperative complete joint space collapse, increasing patellar and lateral compartment osteophyte size, and TKA communication checklist. The principal preoperative factors negatively contributing to patient satisfaction included female sex, comorbidities, and Hispanic race. The chief perioperative factor positively contributing to patient satisfaction was cosmetic closure, whereas the fundamental perioperative factors negatively contributing to patient satisfaction included joint laxity, anterior tibial component slope, and greater femoral component valgus angle. The principal postoperative factors positively contributing to patient satisfaction were the following: ameliorated walking distance, improved range of motion, and improvements in pain. The most important postoperative factors negatively contributing to patient satisfaction included poor postoperative knee stability and soft-tissue balance, functional limitation, surgical complication and reoperation, staff or quality of care issues, and increased stiffness.

Clinical Decision Support Tools for Predicting Outcomes in Patients Undergoing Total Knee Arthroplasty: A Systematic Review

BACKGROUND

Total knee arthroplasty is the standard surgical treatment for end-stage osteoarthritis. Although widely accepted as a successful procedure, approximately 30% of patients are not satisfied due to non-optimal postoperative outcomes. Clinical decision support tools that are able to accurately predict post-surgery outcomes would assist in providing individualized advice or services to help alleviate possible issues, resulting in significant benefits to both the healthcare system and individuals.

METHODS

Five databases (Ovid Medline, Ovid EMBASE, CINAHL complete, Cochrane Library, and Scopus) were searched for the key phrases "knee replacement" or "knee arthroplasty" and "decision support tool," "decision tool," "predict∗ tool," "predict∗ model," "algorithm" or "nomogram." Searches were limited to peer-reviewed journal articles published between January 2000 and June 2019. Reference lists of included articles were examined. Authors came to a consensus on the final list of included articles.

RESULTS

Eighteen articles were included for review. Most models reported low predictive success and inability to externally validate. Both candidate and final predictor variables were inconsistent between studies. Only 1 model was considered strongly predictive (AUROC >0.8), and only 2 studies were able to externally validate their developed model. In general, models that performed well used large patient numbers, were tested on similar demographics, and used either nonlinear input transformations or a completely nonlinear model.

CONCLUSION

Some models do show promise; however, there remains the question of whether the reported predictive success can continue to be replicated. Furthermore, clinical applicability and interpretation of predictive tools should be considered during development.

Predictive Models for Clinical Outcomes in Total Knee Arthroplasty: A Systematic Analysis

Background

Predictive modeling promises to improve our understanding of what variables influence patient satisfaction after total knee arthroplasty (TKA). The purpose of this article was to systematically review the relevant literature using predictive models of clinical outcomes after TKA. The aim was to identify the predictor strategies used for systematic data collection with the highest likelihood of success in predicting clinical outcomes.

Methods

A Preferred Reporting Items for Systematic Reviews and Meta-Analyses protocol systematic review was conducted using 3 databases (MEDLINE, EMBASE, and PubMed) to identify all clinical studies that had used predictive models or that assessed predictive features for outcomes after TKA between 1996 and 2020. The ROBINS-I tool was used to evaluate the quality of the studies and the risk of bias.

Results

A total of 75 studies were identified of which 48 met our inclusion criteria. Preoperative predictive factors strongly associated with postoperative clinical outcomes were knee pain, knee-specific Patient-Reported Outcome Measure (PROM) scores, and mental health scores. Demographic characteristics, pre-existing comorbidities, and knee alignment had an inconsistent association with outcomes. The outcome measures that correlated best with the predictive models were improvement of PROM scores, pain scores, and patient satisfaction.

Conclusions

Several algorithms, based on PROM improvement, patient satisfaction, or pain after TKA, have been developed to improve decision-making regarding both indications for surgery and surgical strategy. Functional features such as preoperative pain and PROM scores were highly predictive for clinical outcomes after TKA. Some variables such as demographics data or knee alignment were less strongly correlated with TKA outcomes.

Level of evidence

Systematic review – Level III.

Development of Machine Learning Algorithms to Predict Patient Dissatisfaction After Primary Total Knee Arthroplasty

BACKGROUND

Postoperative dissatisfaction after primary total knee arthroplasty (TKA) that requires additional care or readmission may impose a significant financial burden to healthcare systems. The purpose of the current study is to develop machine learning algorithms to predict dissatisfaction after TKA.

METHODS

A retrospective review of consecutive TKA patients between 2014 and 2016 from 1 large academic and 2 community hospitals was performed. Preoperative variables considered for prediction included demographics, medical history, flexion contracture, knee flexion, and outcome scores (patient-reported health state, Knee Society Score [KSS], and KSS-Function [KSS-F]). Recursive feature elimination was used to select features that optimized algorithm performance. Five supervised machine learning algorithms were developed by training with 10-fold cross-validation 3 times. These algorithms were subsequently applied to an independent testing set of patients and assessed by discrimination, calibration, Brier score, and decision curve analysis.

RESULTS

Of 430 patients, a total of 40 (9.0%) were dissatisfied with their outcome after primary TKA at a minimum of 2 years postoperatively. The random forest algorithm achieved the best performance in the independent testing set not used for algorithm development (c-statistic: 0.77, calibration intercept: 0.087, calibration slope: 0.74, Brier score: 0.082). The most important factors for predicting dissatisfaction were age, number of medical comorbidities, presence of one or more drug allergies, preoperative patient-reported health state score, and preoperative KSS.

CONCLUSION

The current study developed machine learning algorithms based on partially modifiable risk factors for predicting dissatisfaction after TKA. This model demonstrates good discriminative capacity for identifying those at greatest risk for dissatisfaction and may allow for preoperative health optimization.

Development of Machine Learning Algorithms to Predict Clinically Meaningful Improvement for the Patient-Reported Health State After Total Hip Arthroplasty

BACKGROUND

Failure to achieve clinically significant outcome (CSO) improvement after total hip arthroplasty (THA) imposes a potential cost-to-risk imbalance in the context of bundle payment models. Patient perception of their health state is one component of such risk. The purpose of the current study is to develop machine learning algorithms to predict CSO for the patient-reported health state (PRHS) and build a clinical decision-making tool based on risk factors.

METHODS

A retrospective review of primary THA patients between 2014 and 2017 was performed. Variables considered for prediction included demographics, medical history, preoperative PRHS, and modified Harris Hip Score. The minimal clinically important difference (MCID) for the PRHS was calculated using a distribution-based method. Five supervised machine learning algorithms were developed and assessed by discrimination, calibration, Brier score, and decision curve analysis.

RESULTS

Of 616 patients, a total of 407 (69.2%) achieved the MCID for the PRHS. The random forest algorithm achieved the best performance in the independent testing set not used for algorithm development (c-statistic 0.97, calibration intercept -0.05, calibration slope 1.45, Brier score 0.054). The most important factors for achieving the MCID were preoperative PRHS, preoperative opioid use, age, and body mass index. Individual patient-level explanations were provided for the algorithm predictions and the algorithms were incorporated into an open access digital application available here: https://sorg-apps.shinyapps.io/THA_PRHS_mcid/.

CONCLUSION

The current study created a clinical decision-making tool based on partially modifiable risk factors for predicting CSO after THA. The tool demonstrates excellent discriminative capacity for identifying those at greatest risk for failing to achieve CSO in their current health state and may allow for preoperative health optimization.

Improving Patient Outcomes Following Total Knee Arthroplasty: Identifying Rehabilitation Pathways Based on Modifiable Psychological Risk and Resilience Factors

Total knee arthroplasty (TKA) is a commonly implemented elective surgical treatment for end-stage osteoarthritis of the knee, demonstrating high success rates when assessed by objective medical outcomes. However, a considerable proportion of TKA patients report significant dissatisfaction postoperatively, related to enduring pain, functional limitations, and diminished quality of life. In this conceptual analysis, we highlight the importance of assessing patient-centered outcomes routinely in clinical practice, as these measures provide important information regarding whether surgery and postoperative rehabilitation interventions have effectively remediated patients’ real-world “quality of life” experiences. We propose a novel precision medicine approach to improving patient-centered TKA outcomes through the development of a multivariate machine-learning model. The primary aim of this model is to predict individual postoperative recovery trajectories. Uniquely, this model will be developed using an interdisciplinary methodology involving non-linear analysis of the unique contributions of a range of preoperative risk and resilience factors to patient-centered TKA outcomes. Of particular importance to the model’s predictive power is the inclusion of a comprehensive assessment of modifiable psychological risk and resilience factors that have demonstrated relationships with TKA and other conditions in some studies. Despite the potential for patient psychological factors to limit recovery, they are typically not routinely assessed preoperatively in this patient group, and thus can be overlooked in rehabilitative referral and intervention decision-making. This represents a research-to-practice gap that may contribute to adverse patient-centered outcomes. Incorporating psychological risk and resilience factors into a multivariate prediction model could improve the detection of patients at risk of sub-optimal outcomes following TKA. This could provide surgeons and rehabilitation providers with a simplified tool to inform postoperative referral and intervention decision-making related to a range of interdisciplinary domains outside their usual purview. The proposed approach could facilitate the development and provision of more targeted rehabilitative interventions on the basis of identified individual needs. The roles of several modifiable psychological risk and resilience factors in recovery are summarized, and intervention options are briefly presented. While focusing on rehabilitation following TKA, we advocate for the broader utilization of multivariate prediction models to inform individually tailored interventions targeting a range of health conditions.