Evaluating the fairness and accuracy of machine learning-based predictions of clinical outcomes after anatomic and reverse total shoulder arthroplasty

Artificial intelligence applications in the management of musculoskeletal disorders of the shoulder: A systematic review

Purpose

The aim of the present review is to evaluate and report on the available literature discussing artificial intelligence (AI) applications to the diagnosis of shoulder conditions, outcome prediction of shoulder interventions, and the possible application of such algorithms directly to surgical procedures.

Methods

In February 2024, a search of PubMed, Cochrane and Scopus databases was performed. Studies had to evaluate AI model effectiveness for inclusion. Research on healthcare cost predictions, deterministic algorithms, patient satisfaction, protocol studies and upper-extremity fractures not involving the shoulder were excluded. The Joanna Briggs Institute Critical Appraisal tool and the Risk of Bias in Non-randomised Studies of Interventions tools were used to assess bias.

Results

Thirty-three studies were included in the analysis. Seven studies analysed the detection of rotator cuff tears (RCTs) in magnetic resonance imaging and found area under the curve (AUC) values ranged from 0.812 to 0.94 for the detection of RCTs. One study reported Area Under the Receiver Operating Characteristics values ranging from 0.79 to 0.97 for the prediction of clinical outcomes following reverse total shoulder arthroplasty. In terms of outcomes of rotator cuff repair, an AUC value ranging from 0.58 to 0.68 was reported for prediction of patient-reported outcome measures, and an AUC range of 0.87-0.92 was found for prediction of retear rate. Five studies evaluated the identification of shoulder implant models following TSA from radiographs, with reported accuracy ranging from 89.90% to 97.20%.

Conclusion

AI application enables forecasting of clinical outcomes, permits refined diagnostic evaluation and increases surgical accuracy. While promising, the translation of these technologies into routine clinical practice requires careful consideration.

Level of Evidence

Level IV.

Initial clinical experience with a predictive clinical decision support tool for anatomic and reverse total shoulder arthroplasty

PURPOSE

Clinical decision support tools (CDSTs) are software that generate patient-specific assessments that can be used to better inform healthcare provider decision making. Machine learning (ML)-based CDSTs have recently been developed for anatomic (aTSA) and reverse (rTSA) total shoulder arthroplasty to facilitate more data-driven, evidence-based decision making. Using this shoulder CDST as an example, this external validation study provides an overview of how ML-based algorithms are developed and discusses the limitations of these tools.

METHODS

An external validation for a novel CDST was conducted on 243 patients (120F/123M) who received a personalized prediction prior to surgery and had short-term clinical follow-up from 3 months to 2 years after primary aTSA (n = 43) or rTSA (n = 200). The outcome score and active range of motion predictions were compared to each patient's actual result at each timepoint, with the accuracy quantified by the mean absolute error (MAE).

RESULTS

The results of this external validation demonstrate the CDST accuracy to be similar (within 10%) or better than the MAEs from the published internal validation. A few predictive models were observed to have substantially lower MAEs than the internal validation, specifically, Constant (31.6% better), active abduction (22.5% better), global shoulder function (20.0% better), active external rotation (19.0% better), and active forward elevation (16.2% better), which is encouraging; however, the sample size was small.

CONCLUSION

A greater understanding of the limitations of ML-based CDSTs will facilitate more responsible use and build trust and confidence, potentially leading to greater adoption. As CDSTs evolve, we anticipate greater shared decision making between the patient and surgeon with the aim of achieving even better outcomes and greater levels of patient satisfaction.

Impact of Deltoid Computer Tomography Image Data on the Accuracy of Machine Learning Predictions of Clinical Outcomes after Anatomic and Reverse Total Shoulder Arthroplasty

Background: Despite the importance of the deltoid to shoulder biomechanics, very few studies have quantified the three-dimensional shape, size, or quality of the deltoid muscle, and no studies have correlated these measurements to clinical outcomes after anatomic (aTSA) and/or reverse (rTSA) total shoulder arthroplasty in any statistically/scientifically relevant manner. Methods: Preoperative computer tomography (CT) images from 1057 patients (585 female, 469 male; 799 primary rTSA and 258 primary aTSA) of a single platform shoulder arthroplasty prosthesis (Equinoxe; Exactech, Inc., Gainesville, FL) were analyzed in this study. A machine learning (ML) framework was used to segment the deltoid muscle for 1057 patients and quantify 15 different muscle characteristics, including volumetric (size, shape, etc.) and intensity-based Hounsfield (HU) measurements. These deltoid measurements were correlated to postoperative clinical outcomes and utilized as inputs to train/test ML algorithms used to predict postoperative outcomes at multiple postoperative timepoints (1 year, 2-3 years, and 3-5 years) for aTSA and rTSA. Results: Numerous deltoid muscle measurements were demonstrated to significantly vary with age, gender, prosthesis type, and CT image kernel; notably, normalized deltoid volume and deltoid fatty infiltration were demonstrated to be relevant to preoperative and postoperative clinical outcomes after aTSA and rTSA. Incorporating deltoid image data into the ML models improved clinical outcome prediction accuracy relative to ML algorithms without image data, particularly for the prediction of abduction and forward elevation after aTSA and rTSA. Analyzing ML feature importance facilitated rank-ordering of the deltoid image measurements relevant to aTSA and rTSA clinical outcomes. Specifically, we identified that deltoid shape flatness, normalized deltoid volume, deltoid voxel skewness, and deltoid shape sphericity were the most predictive image-based features used to predict clinical outcomes after aTSA and rTSA. Many of these deltoid measurements were found to be more predictive of aTSA and rTSA postoperative outcomes than patient demographic data, comorbidity data, and diagnosis data. Conclusions: While future work is required to further refine the ML models, which include additional shoulder muscles, like the rotator cuff, our results show promise that the developed ML framework can be used to evolve traditional CT-based preoperative planning software into an evidence-based ML clinical decision support tool.