A practical guide to the development and deployment of deep learning models for the Orthopedic surgeon: part I

Diagnostic Performance of Artificial Intelligence for Detection of Scaphoid and Distal Radius Fractures: A Systematic Review

PURPOSE

To review the existing literature to (1) determine the diagnostic efficacy of artificial intelligence (AI) models for detecting scaphoid and distal radius fractures and (2) compare the efficacy to human clinical experts.

METHODS

PubMed, OVID/Medline, and Cochrane libraries were queried for studies investigating the development, validation, and analysis of AI for the detection of scaphoid or distal radius fractures. Data regarding study design, AI model development and architecture, prediction accuracy/area under the receiver operator characteristic curve (AUROC), and imaging modalities were recorded.

RESULTS

A total of 21 studies were identified, of which 12 (57.1%) used AI to detect fractures of the distal radius, and nine (42.9%) used AI to detect fractures of the scaphoid. AI models demonstrated good diagnostic performance on average, with AUROC values ranging from 0.77 to 0.96 for scaphoid fractures and from 0.90 to 0.99 for distal radius fractures. Accuracy of AI models ranged between 72.0% to 90.3% and 89.0% to 98.0% for scaphoid and distal radius fractures, respectively. When compared to clinical experts, 13 of 14 (92.9%) studies reported that AI models demonstrated comparable or better performance. The type of fracture influenced model performance, with worse overall performance on occult scaphoid fractures; however, models trained specifically on occult fractures demonstrated substantially improved performance when compared to humans.

CONCLUSIONS

AI models demonstrated excellent performance for detecting scaphoid and distal radius fractures, with the majority demonstrating comparable or better performance compared with human experts. Worse performance was demonstrated on occult fractures. However, when trained specifically on difficult fracture patterns, AI models demonstrated improved performance.

CLINICAL RELEVANCE

AI models can help detect commonly missed occult fractures while enhancing workflow efficiency for distal radius and scaphoid fracture diagnoses. As performance varies based on fracture type, future studies focused on wrist fracture detection should clearly define whether the goal is to (1) identify difficult-to-detect fractures or (2) improve workflow efficiency by assisting in routine tasks.

Deep-Learning Automation of Preoperative Radiographic Parameters Associated With Early Periprosthetic Femur Fracture After Total Hip Arthroplasty

BACKGROUND

The radiographic assessment of bone morphology impacts implant selection and fixation type in total hip arthroplasty (THA) and is important to minimize the risk of periprosthetic femur fracture (PFF). We utilized a deep-learning algorithm to automate femoral radiographic parameters and determined which automated parameters were associated with early PFF.

METHODS

Radiographs from a publicly available database and from patients undergoing primary cementless THA at a high-volume institution (2016 to 2020) were obtained. A U-Net algorithm was trained to segment femoral landmarks for bone morphology parameter automation. Automated parameters were compared against that of a fellowship-trained surgeon and compared in an independent cohort of 100 patients who underwent THA (50 with early PFF and 50 controls matched by femoral component, age, sex, body mass index, and surgical approach).

RESULTS

On the independent cohort, the algorithm generated 1,710 unique measurements for 95 images (5% lesser trochanter identification failure) in 22 minutes. Medullary canal width, femoral cortex width, canal flare index, morphological cortical index, canal bone ratio, and canal calcar ratio had good-to-excellent correlation with surgeon measurements (Pearson's correlation coefficient: 0.76 to 0.96). Canal calcar ratios (0.43 ± 0.08 versus 0.40 ± 0.07) and canal bone ratios (0.39 ± 0.06 versus 0.36 ± 0.06) were higher (P < .05) in the PFF cohort when comparing the automated parameters.

CONCLUSIONS

Deep-learning automated parameters demonstrated differences in patients who had and did not have early PFF after cementless primary THA. This algorithm has the potential to complement and improve patient-specific PFF risk-prediction tools.

Editorial Commentary: Artificial Intelligence Models Using Machine Learning Can Improve Preoperative Identification of Subscapularis Pathology

Subscapularis pathology is difficult to diagnose, in part because of decreased sensitivity and accuracy in identifying tears with magnetic resonance imaging (MRI) when compared to other cuff tendons. Artificial intelligence evaluation of patient physical examination and MRI data using a machine learning model shows that arthroscopically confirmed partial- or full-thickness subscapularis tears are highly associated with abnormal subscapularis tendon length, long head of the biceps tears, and subscapularis fatty atrophy, and on physical examination, with weakness with internal rotation and positive lift-off, belly press, and bear hug tests. Today, physicians may use machine learning as a tool, but this model may not currently be sufficient to drastically change practice. However, with continued research and development, which is occurring rapidly, similar models could aid physicians in timely identification of pathology and optimization of preoperative planning, as well as physician training and education.

A Machine Learning Model Demonstrates Excellent Performance in Predicting Subscapularis Tears Based on Pre-Operative Imaging Parameters Alone

PURPOSE

To develop a machine learning model capable of identifying subscapularis tears before surgery based on imaging and physical examination findings.

METHODS

Between 2010 and 2020, 202 consecutive shoulders underwent arthroscopic rotator cuff repair by a single surgeon. Patient demographics, physical examination findings (including range of motion, weakness with internal rotation, lift/push-off test, belly press test, and bear hug test), and imaging (including direct and indirect signs of tearing, biceps status, fatty atrophy, cystic changes, and other similar findings) were included for model creation.

RESULTS

Sixty percent of the shoulders had partial or full thickness tears of the subscapularis verified during surgery (83% of these were upper third). Using only preoperative imaging-related parameters, the XGBoost model demonstrated excellent performance at predicting subscapularis tears (c-statistic, 0.84; accuracy, 0.85; F1 score, 0.87). The top 5 features included direct signs related to the presence of tearing as evidenced on magnetic resonance imaging (MRI) (changes in tendon morphology and signal), as well as the quality of the MRI and biceps pathology.

CONCLUSIONS

In this study, machine learning was successful in predicting subscapularis tears by MRI alone in 85% of patients, and this accuracy did not decrease by isolating the model to the top features. The top five features included direct signs related to the presence of tearing as evidenced on MRI (changes in tendon morphology and signal), as well as the quality of the MRI and biceps pathology. Last, in advanced modeling, the addition of physical examination or patient characteristics did not make a significant difference in the predictive ability of this model.

LEVEL OF EVIDENCE

Level III, diagnostic case-control study.

A practical guide to the development and deployment of deep learning models for the orthopaedic surgeon: Part III, focus on registry creation, diagnosis, and data privacy

Deep learning is a subset of artificial intelligence (AI) with enormous potential to transform orthopaedic surgery. As has already become evident with the deployment of Large Language Models (LLMs) like ChatGPT (OpenAI Inc.), deep learning can rapidly enter clinical and surgical practices. As such, it is imperative that orthopaedic surgeons acquire a deeper understanding of the technical terminology, capabilities and limitations associated with deep learning models. The focus of this series thus far has been providing surgeons with an overview of the steps needed to implement a deep learning-based pipeline, emphasizing some of the important technical details for surgeons to understand as they encounter, evaluate or lead deep learning projects. However, this series would be remiss without providing practical examples of how deep learning models have begun to be deployed and highlighting the areas where the authors feel deep learning may have the most profound potential. While computer vision applications of deep learning were the focus of Parts I and II, due to the enormous impact that natural language processing (NLP) has had in recent months, NLP-based deep learning models are also discussed in this final part of the series. In this review, three applications that the authors believe can be impacted the most by deep learning but with which many surgeons may not be familiar are discussed: (1) registry construction, (2) diagnostic AI and (3) data privacy. Deep learning-based registry construction will be essential for the development of more impactful clinical applications, with diagnostic AI being one of those applications likely to augment clinical decision-making in the near future. As the applications of deep learning continue to grow, the protection of patient information will become increasingly essential; as such, applications of deep learning to enhance data privacy are likely to become more important than ever before. Level of Evidence: Level IV.

Enabling Personalized Medicine in Orthopaedic Surgery Through Artificial Intelligence: A Critical Analysis Review

» The application of artificial intelligence (AI) in the field of orthopaedic surgery holds potential for revolutionizing health care delivery across 3 crucial domains: (I) personalized prediction of clinical outcomes and adverse events, which may optimize patient selection, surgical planning, and enhance patient safety and outcomes; (II) diagnostic automated and semiautomated imaging analyses, which may reduce time burden and facilitate precise and timely diagnoses; and (III) forecasting of resource utilization, which may reduce health care costs and increase value for patients and institutions.» Computer vision is one of the most highly studied areas of AI within orthopaedics, with applications pertaining to fracture classification, identification of the manufacturer and model of prosthetic implants, and surveillance of prosthesis loosening and failure.» Prognostic applications of AI within orthopaedics include identifying patients who will likely benefit from a specified treatment, predicting prosthetic implant size, postoperative length of stay, discharge disposition, and surgical complications. Not only may these applications be beneficial to patients but also to institutions and payors because they may inform potential cost expenditure, improve overall hospital efficiency, and help anticipate resource utilization.» AI infrastructure development requires institutional financial commitment and a team of clinicians and data scientists with expertise in AI that can complement skill sets and knowledge. Once a team is established and a goal is determined, teams (1) obtain, curate, and label data; (2) establish a reference standard; (3) develop an AI model; (4) evaluate the performance of the AI model; (5) externally validate the model, and (6) reinforce, improve, and evaluate the model's performance until clinical implementation is possible.» Understanding the implications of AI in orthopaedics may eventually lead to wide-ranging improvements in patient care. However, AI, while holding tremendous promise, is not without methodological and ethical limitations that are essential to address. First, it is important to ensure external validity of programs before their use in a clinical setting. Investigators should maintain high quality data records and registry surveillance, exercise caution when evaluating others' reported AI applications, and increase transparency of the methodological conduct of current models to improve external validity and avoid propagating bias. By addressing these challenges and responsibly embracing the potential of AI, the medical field may eventually be able to harness its power to improve patient care and outcomes.

Prognostic value of the Walch classification for patients before and after shoulder arthroplasty performed for osteoarthritis with an intact rotator cuff

BACKGROUND

The Walch classification is commonly used by surgeons when determining the treatment of osteoarthritis (OA). However, its utility in prognosticating patient clinical state before and after TSA remains unproven. We assessed the prognostic value of the modified Walch glenoid classification on preoperative clinical state and postoperative clinical and radiographic outcomes in total shoulder arthroplasty (TSA).

METHODS

A prospectively collected, multicenter database for a single-platform TSA system was queried for patients with rotator cuff-intact OA and minimum 2 year follow-up after anatomic (aTSA) and reverse TSA (rTSA). Differences in patient-reported outcome scores (Simple Shoulder Test, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form, Shoulder Pain and Disability Index, visual analog scale for pain, Shoulder Function score), combined patient-reported and clinical-input scores (Constant, University of California-Los Angeles shoulder score, Shoulder Arthroplasty Smart Score), active range of motion values (forward elevation [FE], abduction, external rotation [ER], internal rotation [IR], and radiographic outcomes (humeral and glenoid radiolucency line rates, scapula notching rate) were stratified and compared by glenoid deformity type per the Walch classification for aTSA and rTSA cohorts. Comparisons were performed to assess the ability of the Walch classification to predict the preoperative, postoperative, and improved state after TSA.

RESULTS

1008 TSAs were analyzed including 576 aTSA and 432 rTSA. Comparison of outcomes between Walch glenoid types resulted in 15 pairwise comparisons of 12 clinical outcome metrics, yielding 180 total Walch glenoid pairwise comparisons for each clinical state (preoperative, postoperative, improvement). Of the 180 possible pairwise Walch glenoid type and metric comparisons studied for aTSA and rTSA cohorts, <6% and <2% significantly differed in aTSA and rTSA cohorts, respectively. Significant differences based on Walch type were seen after adjustment for multiple pairwise comparisons in the aTSA cohort for FE and ER preoperatively, the Constant score postoperatively, and for abduction, FE, ER, Constant score, and SAS score for pre- to postoperative improvement. In the rTSA cohort, significant differences were only seen in abduction and Constant score both postoperatively and for pre- to postoperative improvement. There were no statistically significant differences in humeral lucency rate, glenoid lucency rate (aTSA), scapular notching rate (rTSA), complication rates, or revision rates between Walch glenoid types after TSA.

CONCLUSION

Although useful for describing degenerative changes to the glenohumeral joint, we demonstrate a weak association between preoperative glenoid morphology according to the Walch classification and clinical state when evaluating patients undergoing TSA for rotator cuff-intact OA. Alternative glenoid classification systems or predictive models should be considered to provide more precise prognoses for patients undergoing TSA for rotator cuff-intact OA.

An Interpretable Machine Learning Model for Predicting 10-Year Total Hip Arthroplasty Risk

BACKGROUND

As the demand for total hip arthroplasty (THA) rises, a predictive model for THA risk may aid patients and clinicians in augmenting shared decision-making. We aimed to develop and validate a model predicting THA within 10 years in patients using demographic, clinical, and deep learning (DL)-automated radiographic measurements.

METHODS

Patients enrolled in the Osteoarthritis Initiative were included. DL algorithms measuring osteoarthritis- and dysplasia-relevant parameters on baseline pelvis radiographs were developed. Demographic, clinical, and radiographic measurement variables were then used to train generalized additive models to predict THA within 10 years from baseline. A total of 4,796 patients were included (9,592 hips; 58% female; 230 THAs (2.4%)). Model performance using 1) baseline demographic and clinical variables 2) radiographic variables, and 3) all variables were compared.

RESULTS

Using 110 demographic and clinical variables, the model had a baseline area under the receiver operating curve (AUROC) of 0.68 and area under the precision recall curve (AUPRC) of 0.08. Using 26 DL-automated hip measurements, the AUROC was 0.77 and AUPRC was 0.22. Combining all variables, the model improved to an AUROC of 0.81 and AUPRC of 0.28. Three of the top five predictive features in the combined model were radiographic variables including minimum joint space along with hip pain and analgesic use. Partial dependency plots revealed predictive discontinuities for radiographic measurements consistent with literature thresholds of osteoarthritis progression and hip dysplasia.

CONCLUSION

A machine learning model predicting 10-year THA performed more accurately with DL radiographic measurements. The model weighted predictive variables in concordance with clinical THA-pathology assessments.

A practical guide to the development and deployment of deep learning models for the orthopedic surgeon: part II

Deep learning has the potential to be one of the most transformative technologies to impact orthopedic surgery. Substantial innovation in this area has occurred over the past 5 years, but clinically meaningful advancements remain limited by a disconnect between clinical and technical experts. That is, it is likely that few orthopedic surgeons possess both the clinical knowledge necessary to identify orthopedic problems, and the technical knowledge needed to implement deep learning-based solutions. To maximize the utilization of rapidly advancing technologies derived from deep learning models, orthopedic surgeons should understand the steps needed to design, organize, implement, and evaluate a deep learning project and its workflow. Equipping surgeons with this knowledge is the objective of this three-part editorial review. Part I described the processes involved in defining the problem, team building, data acquisition, curation, labeling, and establishing the ground truth. Building on that, this review (Part II) provides guidance on pre-processing and augmenting the data, making use of open-source libraries/toolkits, and selecting the required hardware to implement the pipeline. Special considerations regarding model training and evaluation unique to deep learning models relative to "shallow" machine learning models are also reviewed. Finally, guidance pertaining to the clinical deployment of deep learning models in the real world is provided. As in Part I, the focus is on applications of deep learning for computer vision and imaging.