Deep Learning Assistance Closes the Accuracy Gap in Fracture Detection Across Clinician Types

Medical Imaging Applications Developed Using Artificial Intelligence Demonstrate High Internal Validity Yet Are Limited in Scope and Lack External Validation

PURPOSE

To (1) review definitions and concepts necessary to interpret applications of deep learning (DL; a domain of artificial intelligence that leverages neural networks to make predictions on media inputs such as images) and (2) identify knowledge and translational gaps in the literature to provide insight into specific areas for improvement as adoption of this technology continues.

METHODS

A comprehensive search of the literature was performed in December 2023 for articles regarding the use of DL in sports medicine. For each study, information regarding the joint of focus, specific anatomic structure/pathology to which DL was applied, imaging modality utilized, source of images used for model training and testing, data set size, model performance, and whether the DL model was externally validated was recorded. A numerical scale was used to rate each DL model's clinical impact, with 1 corresponding to proof-of-concept studies with little to no direct clinical impact and 5 corresponding to practice-changing clinical impact and ready for clinical deployment.

RESULTS

Fifty-five studies were identified, all of which were published within the past 5 years, while 82% were published within the past 3 years. Of the DL models identified, 84% were developed for classification tasks, 9% for automated measurements, and 7% for segmentation. A total of 62% of studies utilized magnetic resonance imaging as the imaging modality, 25% radiographs, and 7% ultrasound, while 1 study each used computed tomography, arthroscopic images, or arthroscopic video. Sixty-five percent of studies focused on the detection of tears (anterior cruciate ligament [ACL], rotator cuff [RC], and meniscus). The diagnostic performance of ACL tears, as determined by the area under the receiver operator curve (AUROC), ranged from 0.81 to 0.99 for ACL tears (excellent to near perfect), 0.83 to 0.94 for RC tears (excellent), and from 0.75 to 0.96 for meniscus tears (acceptable to excellent). In addition, 3 studies focused on detection of cartilage lesions had AUROC ranging from 0.90 to 0.92 (excellent performance). However, only 4 (7%) studies externally validated their models, suggesting that they may not be generalizable or may not perform well when applied to populations other than that used to develop the model. Finally, the mean clinical impact score was 2 (range, 1-3) on scale of 1 to 5, corresponding to limited clinical applicability.

CONCLUSIONS

DL models in orthopaedic sports medicine show generally excellent performance (high internal validity) but require external validation to facilitate clinical deployment. In addition, current models have low clinical applicability and fail to advance the field due to a focus on routine tasks and a narrow conceptual framework.

LEVEL OF EVIDENCE

Level IV, scoping review of Level I to IV studies.

Machine Learning Improves the Accuracy of Trauma Team Activation Level Assignments in Pediatric Patients

BACKGROUND

The assignment of trauma team activation levels can be conceptualized as a classification task. Machine learning models can be used to optimize classification predictions. Our purpose was to demonstrate proof-of-concept for a machine learning tool for predicting trauma team activation levels in pediatric patients with traumatic injuries.

METHODS

Following IRB approval, we retrospectively collected data from the institutional trauma registry and electronic medical record at our Pediatric Trauma Center for all patients (age <18 y) who triggered a trauma team activation (1/2014-12/2021), including: demographics, mechanisms of injury, comorbidities, pre-hospital interventions, numeric variables, and the six "Need for Trauma Intervention (NFTI)" criteria. Three machine learning models (Logistic Regression, Random Forest, Support Vector Machine) were tested 1000 times in separate trials using the union of the Cribari and NFTI metrics as ground-truth (Injury Severity Score >15 or positive for any of 6 NFTI criteria = full activation). Model performance was quantified and compared to emergency department (ED) staff.

RESULTS

ED staff had 75% accuracy, an area under the curve (AUC) of 0.73 ± 0.04, and an F1 score of 0.49. The best performing of all machine learning models, the support vector machine, had 80% accuracy, AUC 0.81 ± 4.1e-5, F1 Score 0.80, with less variance compared to other models and ED staff.

CONCLUSIONS

All machine learning models outperformed ED staff in all performance metrics. These results suggest that data-driven methods can optimize trauma team activations in the ED, with potential improvements in both patient safety and hospital resource utilization.

TYPE OF STUDY

Economic/Decision Analysis or Modeling Studies.

LEVEL OF EVIDENCE

II.

Commercially available artificial intelligence tools for fracture detection: the evidence

Missed fractures are a costly healthcare issue, not only negatively impacting patient lives, leading to potential long-term disability and time off work, but also responsible for high medicolegal disbursements that could otherwise be used to improve other healthcare services. When fractures are overlooked in children, they are particularly concerning as opportunities for safeguarding may be missed. Assistance from artificial intelligence (AI) in interpreting medical images may offer a possible solution for improving patient care, and several commercial AI tools are now available for radiology workflow implementation. However, information regarding their development, evidence for performance and validation as well as the intended target population is not always clear, but vital when evaluating a potential AI solution for implementation. In this article, we review the range of available products utilizing AI for fracture detection (in both adults and children) and summarize the evidence, or lack thereof, behind their performance. This will allow others to make better informed decisions when deciding which product to procure for their specific clinical requirements.

Improving traumatic fracture detection on radiographs with artificial intelligence support: a multi-reader study

Objectives

The aim of this study was to evaluate the diagnostic performance of nonspecialist readers with and without the use of an artificial intelligence (AI) support tool to detect traumatic fractures on radiographs of the appendicular skeleton.

Methods

The design was a retrospective, fully crossed multi-reader, multi-case study on a balanced dataset of patients (≥2 years of age) with an AI tool as a diagnostic intervention. Fifteen readers assessed 340 radiographic exams, with and without the AI tool in 2 different sessions and the time spent was automatically recorded. Reference standard was established by 3 consultant radiologists. Sensitivity, specificity, and false positives per patient were calculated.

Results

Patient-wise sensitivity increased from 72% to 80% (P < .05) and patient-wise specificity increased from 81% to 85% (P < .05) in exams aided by the AI tool compared to the unaided exams. The increase in sensitivity resulted in a relative reduction of missed fractures of 29%. The average rate of false positives per patient decreased from 0.16 to 0.14, corresponding to a relative reduction of 21%. There was no significant difference in average reading time spent per exam. The largest gain in fracture detection performance, with AI support, across all readers, was on nonobvious fractures with a significant increase in sensitivity of 11 percentage points (pp) (60%-71%).

Conclusions

The diagnostic performance for detection of traumatic fractures on radiographs of the appendicular skeleton improved among nonspecialist readers tested AI fracture detection support tool showed an overall reader improvement in sensitivity and specificity when supported by an AI tool. Improvement was seen in both sensitivity and specificity without negatively affecting the interpretation time.

Advances in knowledge

The division and analysis of obvious and nonobvious fractures are novel in AI reader comparison studies like this.

Robotic Technology in Foot and Ankle Surgery: A Comprehensive Review

Recent developments in robotic technologies in the field of orthopaedic surgery have largely been focused on higher volume arthroplasty procedures, with a paucity of attention paid to robotic potential for foot and ankle surgery. The aim of this paper is to summarize past and present developments foot and ankle robotics and describe outcomes associated with these interventions, with specific emphasis on the following topics: translational and preclinical utilization of robotics, deep learning and artificial intelligence modeling in foot and ankle, current applications for robotics in foot and ankle surgery, and therapeutic and orthotic-related utilizations of robotics related to the foot and ankle. Herein, we describe numerous recent robotic advancements across foot and ankle surgery, geared towards optimizing intra-operative performance, improving detection of foot and ankle pathology, understanding ankle kinematics, and rehabilitating post-surgically. Future research should work to incorporate robotics specifically into surgical procedures as other specialties within orthopaedics have done, and to further individualize machinery to patients, with the ultimate goal to improve perioperative and post-operative outcomes.