Deep learning in fracture detection: a narrative review

Automated segmentation of vertebral cortex with 3D U-Net-based deep convolutional neural network

Objectives: We developed a 3D U-Net-based deep convolutional neural network for the automatic segmentation of the vertebral cortex. The purpose of this study was to evaluate the accuracy of the 3D U-Net deep learning model.

Methods: In this study, a fully automated vertebral cortical segmentation method with 3D U-Net was developed, and ten-fold cross-validation was employed. Through data augmentation, we obtained 1,672 3D images of chest CT scans. Segmentation was performed using a conventional image processing method and manually corrected by a senior radiologist to create the gold standard. To compare the segmentation performance, 3D U-Net, Res U-Net, Ki U-Net, and Seg Net were used to segment the vertebral cortex in CT images. The segmentation performance of 3D U-Net and the other three deep learning algorithms was evaluated using DSC, mIoU, MPA, and FPS.

Results: The DSC, mIoU, and MPA of 3D U-Net are better than the other three strategies, reaching 0.71 ± 0.03, 0.74 ± 0.08, and 0.83 ± 0.02, respectively, indicating promising automated segmentation results. The FPS is slightly lower than that of Seg Net (23.09 ± 1.26 vs. 30.42 ± 3.57).

Conclusion: Cortical bone can be effectively segmented based on 3D U-net.

Artificial Intelligence in Orthopedic Radiography Analysis: A Narrative Review

Artificial intelligence (AI) in medicine is a rapidly growing field. In orthopedics, the clinical implementations of AI have not yet reached their full potential. Deep learning algorithms have shown promising results in computed radiographs for fracture detection, classification of OA, bone age, as well as automated measurements of the lower extremities. Studies investigating the performance of AI compared to trained human readers often show equal or better results, although human validation is indispensable at the current standards. The objective of this narrative review is to give an overview of AI in medicine and summarize the current applications of AI in orthopedic radiography imaging. Due to the different AI software and study design, it is difficult to find a clear structure in this field. To produce more homogeneous studies, open-source access to AI software codes and a consensus on study design should be aimed for.

Imaging of Metabolic Bone Diseases: The Spine View, Part II

Metabolic bone diseases comprise a wide spectrum. Osteoporosis, the most frequent, characteristically involves the spine, with a high impact on health care systems and on the morbidity of patients due to the occurrence of vertebral fractures (VFs).Part II of this review completes an overview of state-of-the-art techniques on the imaging of metabolic bone diseases of the spine, focusing on specific populations and future perspectives. We address the relevance of diagnosis and current status on VF assessment and quantification. We also analyze the diagnostic techniques in the pediatric population and then review the assessment of body composition around the spine and its potential application. We conclude with a discussion of the future of osteoporosis screening, through opportunistic diagnosis and the application of artificial intelligence.

Development of a Spine X-Ray-Based Fracture Prediction Model Using a Deep Learning Algorithm

BACKGRUOUND

Since image-based fracture prediction models using deep learning are lacking, we aimed to develop an X-ray-based fracture prediction model using deep learning with longitudinal data.

METHODS

This study included 1,595 participants aged 50 to 75 years with at least two lumbosacral radiographs without baseline fractures from 2010 to 2015 at Seoul National University Hospital. Positive and negative cases were defined according to whether vertebral fractures developed during follow-up. The cases were divided into training (n=1,416) and test (n=179) sets. A convolutional neural network (CNN)-based prediction algorithm, DeepSurv, was trained with images and baseline clinical information (age, sex, body mass index, glucocorticoid use, and secondary osteoporosis). The concordance index (C-index) was used to compare performance between DeepSurv and the Fracture Risk Assessment Tool (FRAX) and Cox proportional hazard (CoxPH) models.

RESULTS

Of the total participants, 1,188 (74.4%) were women, and the mean age was 60.5 years. During a mean follow-up period of 40.7 months, vertebral fractures occurred in 7.5% (120/1,595) of participants. In the test set, when DeepSurv learned with images and clinical features, it showed higher performance than FRAX and CoxPH in terms of C-index values (DeepSurv, 0.612; 95% confidence interval [CI], 0.571 to 0.653; FRAX, 0.547; CoxPH, 0.594; 95% CI, 0.552 to 0.555). Notably, the DeepSurv method without clinical features had a higher C-index (0.614; 95% CI, 0.572 to 0.656) than that of FRAX in women.

CONCLUSION

DeepSurv, a CNN-based prediction algorithm using baseline image and clinical information, outperformed the FRAX and CoxPH models in predicting osteoporotic fracture from spine radiographs in a longitudinal cohort.

Diagnostic accuracy and potential covariates of artificial intelligence for diagnosing orthopedic fractures: a systematic literature review and meta-analysis

OBJECTIVES

To systematically quantify the diagnostic accuracy and identify potential covariates affecting the performance of artificial intelligence (AI) in diagnosing orthopedic fractures.

METHODS

PubMed, Embase, Web of Science, and Cochrane Library were systematically searched for studies on AI applications in diagnosing orthopedic fractures from inception to September 29, 2021. Pooled sensitivity and specificity and the area under the receiver operating characteristic curves (AUC) were obtained. This study was registered in the PROSPERO database prior to initiation (CRD 42021254618).

RESULTS

Thirty-nine were eligible for quantitative analysis. The overall pooled AUC, sensitivity, and specificity were 0.96 (95% CI 0.94-0.98), 90% (95% CI 87-92%), and 92% (95% CI 90-94%), respectively. In subgroup analyses, multicenter designed studies yielded higher sensitivity (92% vs. 88%) and specificity (94% vs. 91%) than single-center studies. AI demonstrated higher sensitivity with transfer learning (with vs. without: 92% vs. 87%) or data augmentation (with vs. without: 92% vs. 87%), compared to those without. Utilizing plain X-rays as input images for AI achieved results comparable to CT (AUC 0.96 vs. 0.96). Moreover, AI achieved comparable results to humans (AUC 0.97 vs. 0.97) and better results than non-expert human readers (AUC 0.98 vs. 0.96; sensitivity 95% vs. 88%).

CONCLUSIONS

AI demonstrated high accuracy in diagnosing orthopedic fractures from medical images. Larger-scale studies with higher design quality are needed to validate our findings.

KEY POINTS

• Multicenter study design, application of transfer learning, and data augmentation are closely related to improving the performance of artificial intelligence models in diagnosing orthopedic fractures. • Utilizing plain X-rays as input images for AI to diagnose fractures achieved results comparable to CT (AUC 0.96 vs. 0.96). • AI achieved comparable results to humans (AUC 0.97 vs. 0.97) but was superior to non-expert human readers (AUC 0.98 vs. 0.96, sensitivity 95% vs. 88%) in diagnosing fractures.

Development of an artificial intelligence-assisted computed tomography diagnosis technology for rib fracture and evaluation of its clinical usefulness

Artificial intelligence algorithms utilizing deep learning are helpful tools for diagnostic imaging. A deep learning-based automatic detection algorithm was developed for rib fractures on computed tomography (CT) images of high-energy trauma patients. In this study, the clinical effectiveness of this algorithm was evaluated. A total of 56 cases were retrospectively examined, including 46 rib fractures and 10 control cases from our hospital, between January and June 2019. Two radiologists annotated the fracture lesions (complete or incomplete) for each CT image, which is considered the “ground truth.” Thereafter, the algorithm’s diagnostic results for all cases were compared with the ground truth, and the sensitivity and number of false positive (FP) results per case were assessed. The radiologists identified 199 images with a fracture. The sensitivity of the algorithm was 89.8%, and the number of FPs per case was 2.5. After additional learning, the sensitivity increased to 93.5%, and the number of FPs was 1.9 per case. FP results were found in the trabecular bone with the appearance of fracture, vascular grooves, and artifacts. The sensitivity of the algorithm used in this study was sufficient to aid the rapid detection of rib fractures within the evaluated validation set of CT images.

Artificial Intelligence in Fracture Detection: A Systematic Review and Meta-Analysis

Background Patients with fractures are a common emergency presentation and may be misdiagnosed at radiologic imaging. An increasing number of studies apply artificial intelligence (AI) techniques to fracture detection as an adjunct to clinician diagnosis. Purpose To perform a systematic review and meta-analysis comparing the diagnostic performance in fracture detection between AI and clinicians in peer-reviewed publications and the gray literature (ie, articles published on preprint repositories). Materials and Methods A search of multiple electronic databases between January 2018 and July 2020 (updated June 2021) was performed that included any primary research studies that developed and/or validated AI for the purposes of fracture detection at any imaging modality and excluded studies that evaluated image segmentation algorithms. Meta-analysis with a hierarchical model to calculate pooled sensitivity and specificity was used. Risk of bias was assessed by using a modified Prediction Model Study Risk of Bias Assessment Tool, or PROBAST, checklist. Results Included for analysis were 42 studies, with 115 contingency tables extracted from 32 studies (55 061 images). Thirty-seven studies identified fractures on radiographs and five studies identified fractures on CT images. For internal validation test sets, the pooled sensitivity was 92% (95% CI: 88, 93) for AI and 91% (95% CI: 85, 95) for clinicians, and the pooled specificity was 91% (95% CI: 88, 93) for AI and 92% (95% CI: 89, 92) for clinicians. For external validation test sets, the pooled sensitivity was 91% (95% CI: 84, 95) for AI and 94% (95% CI: 90, 96) for clinicians, and the pooled specificity was 91% (95% CI: 81, 95) for AI and 94% (95% CI: 91, 95) for clinicians. There were no statistically significant differences between clinician and AI performance. There were 22 of 42 (52%) studies that were judged to have high risk of bias. Meta-regression identified multiple sources of heterogeneity in the data, including risk of bias and fracture type. Conclusion Artificial intelligence (AI) and clinicians had comparable reported diagnostic performance in fracture detection, suggesting that AI technology holds promise as a diagnostic adjunct in future clinical practice. Clinical trial registration no. CRD42020186641 © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Cohen and McInnes in this issue.

In Silico Finite Element Modeling of Stress Distribution in Osteosynthesis after Pertrochanteric Fractures

A stabilization method of pertrochanteric femur fractures is a contentious issue. Here, we assess the feasibility of rapid in silico 2D finite element modeling (FEM) to predict the distribution of stresses arising during the two most often used stabilization methods: gamma nail fixation (GNF) and dynamic hip screw (DHS). The modeling was based on standard pre-surgery radiographs of hip joints of 15 patients with pertrochanteric fractures of type A1, A2, and A3 according to the AO/OTA classification. The FEM showed that the stresses were similar for both GNF and DHS, with the medians ranging between 53-60 MPa and consistently lower for A1 than A3 fractures. Stresses also appeared in the fixation materials being about two-fold higher for GNF. Given similar bone stresses caused by both GNF and DHS but shorter surgery time, less extensive dissection, and faster patient mobilization, we submit that the GNF stabilization appears to be the most optimal system for pertrochanteric fractures. In silico FEM appears a viable perioperative method that helps predict the distribution of compressive stresses after osteosynthesis of pertrochanteric fractures. The promptness of modeling fits well into the rigid time framework of hip fracture surgery and may help optimize the fixation procedure for the best outcome. The study extends the use of FEM in complex orthopedic management. However, further datasets are required to firmly position the FEM in the treatment of pertrochanteric fractures.

Diagnosis of Cubital Tunnel Syndrome Using Deep Learning on Ultrasonographic Images

Although electromyography is the routine diagnostic method for cubital tunnel syndrome (CuTS), imaging diagnosis by measuring cross-sectional area (CSA) with ultrasonography (US) has also been attempted in recent years. In this study, deep learning (DL), an artificial intelligence (AI) method, was used on US images, and its diagnostic performance for detecting CuTS was investigated. Elbow images of 30 healthy volunteers and 30 patients diagnosed with CuTS were used. Three thousand US images were prepared per each group to visualize the short axis of the ulnar nerve. Transfer learning was performed on 5000 randomly selected training images using three pre-trained models, and the remaining images were used for testing. The model was evaluated by analyzing a confusion matrix and the area under the receiver operating characteristic curve. Occlusion sensitivity and locally interpretable model-agnostic explanations were used to visualize the features deemed important by the AI. The highest score had an accuracy of 0.90, a precision of 0.86, a recall of 1.00, and an F-measure of 0.92. Visualization results show that the DL models focused on the epineurium of the ulnar nerve and the surrounding soft tissue. The proposed technique enables the accurate prediction of CuTS without the need to measure CSA.

Detection and classification of mandibular fracture on CT scan using deep convolutional neural network

OBJECTIVES

This study aimed to evaluate the accuracy and reliability of convolutional neural networks (CNNs) for the detection and classification of mandibular fracture on spiral computed tomography (CT).

MATERIALS AND METHODS

Between January 2013 and July 2020, 686 patients with mandibular fractures who underwent CT scan were classified and annotated by three experienced maxillofacial surgeons serving as the ground truth. An algorithm including two convolutional neural networks (U-Net and ResNet) was trained, validated, and tested using 222, 56, and 408 CT scans, respectively. The diagnostic performance of the algorithm was compared with the ground truth and evaluated by DICE, accuracy, sensitivity, specificity, and area under the ROC curve (AUC).

RESULTS

One thousand five hundred six mandibular fractures in nine subregions of 686 patients were diagnosed. The DICE of mandible segmentation using U-Net was 0.943. The accuracies of nine subregions were all above 90%, with a mean AUC of 0.956.

CONCLUSIONS

CNNs showed comparable reliability and accuracy in detecting and classifying mandibular fractures on CT.

CLINICAL RELEVANCE

The algorithm for automatic detection and classification of mandibular fractures will help improve diagnostic efficiency and provide expertise to areas with lower medical levels.

A Progressive and Cross-Domain Deep Transfer Learning Framework for Wrist Fracture Detection

There has been an amplified focus on and benefit from the adoption of artificial intelligence (AI) in medical imaging applications. However, deep learning approaches involve training with massive amounts of annotated data in order to guarantee generalization and achieve high accuracies. Gathering and annotating large sets of training images require expertise which is both expensive and time-consuming, especially in the medical field. Furthermore, in health care systems where mistakes can have catastrophic consequences, there is a general mistrust in the black-box aspect of AI models. In this work, we focus on improving the performance of medical imaging applications when limited data is available while focusing on the interpretability aspect of the proposed AI model. This is achieved by employing a novel transfer learning framework, progressive transfer learning, an automated annotation technique and a correlation analysis experiment on the learned representations.

Progressive transfer learning helps jump-start the training of deep neural networks while improving the performance by gradually transferring knowledge from two source tasks into the target task. It is empirically tested on the wrist fracture detection application by first training a general radiology network RadiNet and using its weights to initialize RadiNetwrist , that is trained on wrist images to detect fractures. Experiments show that RadiNetwrist achieves an accuracy of 87% and an AUC ROC of 94% as opposed to 83% and 92% when it is pre-trained on the ImageNet dataset.

This improvement in performance is investigated within an explainable AI framework. More concretely, the learned deep representations of RadiNetwrist are compared to those learned by the baseline model by conducting a correlation analysis experiment. The results show that, when transfer learning is gradually applied, some features are learned earlier in the network. Moreover, the deep layers in the progressive transfer learning framework are shown to encode features that are not encountered when traditional transfer learning techniques are applied.

In addition to the empirical results, a clinical study is conducted and the performance of RadiNetwrist is compared to that of an expert radiologist. We found that RadiNetwrist exhibited similar performance to that of radiologists with more than 20 years of experience.

This motivates follow-up research to train on more data to feasibly surpass radiologists’ performance, and investigate the interpretability of AI models in the healthcare domain where the decision-making process needs to be credible and transparent.

A Surgeon's Guide to Understanding Artificial Intelligence and Machine Learning Studies in Orthopaedic Surgery

PURPOSE OF REVIEW

In recent years, machine learning techniques have been increasingly utilized across medicine, impacting the practice and delivery of healthcare. The data-driven nature of orthopaedic surgery presents many targets for improvement through the use of artificial intelligence, which is reflected in the increasing number of publications in the medical literature. However, the unique methodologies utilized in AI studies can present a barrier to its widespread acceptance and use in orthopaedics. The purpose of our review is to provide a tool that can be used by practitioners to better understand and ultimately leverage AI studies.

RECENT FINDINGS

The increasing interest in machine learning across medicine is reflected in a greater utilization of AI in recent medical literature. The process of designing machine learning studies includes study design, model choice, data collection/handling, model development, training, testing, and interpretation. Recent studies leveraging ML in orthopaedics provide useful examples for future research endeavors. This manuscript intends to create a guide discussing the use of machine learning and artificial intelligence in orthopaedic surgery research. Our review outlines the process of creating a machine learning algorithm and discusses the different model types, utilizing examples from recent orthopaedic literature to illustrate the techniques involved.

Musculoskeletal trauma and artificial intelligence: current trends and projections

Musculoskeletal trauma accounts for a significant fraction of emergency department visits and patients seeking urgent care, with a high financial cost to society. Diagnostic imaging is indispensable in the workup and management of trauma patients. However, diagnostic imaging represents a complex multifaceted system, with many aspects of its workflow prone to inefficiencies or human error. Recent technological innovations in artificial intelligence and machine learning have shown promise to revolutionize our systems for providing medical care to patients. This review will provide a general overview of the current state of artificial intelligence and machine learning applications in different aspects of trauma imaging and provide a vision for how such applications could be leveraged to enhance our diagnostic imaging systems and optimize patient outcomes.

A Bibliometric Analysis of the Field of Computer-Assisted Orthopedic Surgery during 2002–2021

Background

This study aimed to investigate the characteristics of research articles and research trends in computer-assisted orthopedic surgery (CAOS) by conducting bibliometric analyses.

Methods

CAOS-related research articles published in international journals from 2002 to 2021 were collected using the PubMed database and analyzed using the bibliometric method. Their publication year, journal name, corresponding author's country name, and the number of citations of all collected articles were noted. Contents of the articles were analyzed to evaluate the time point and anatomical site at which the digital technique was applied. Further, the 20-year period was divided into two halves of 10 years each to analyze the research trends.

Results

A total of 639 CAOS-related articles were identified. An average of 32.0 CAOS-related articles were published annually, with an average of 20.6 and 43.3 published in the first half and second half, respectively. Of all articles, 47.6% were published in the top 10 journals, and 81.2% were written in the top 10 countries. The total numbers of citations were 11.7 and 6.3 in the first and second halves, respectively, but the average annual number of citations was higher in the second half than in the first one. Articles on application of digital techniques during surgery were 62.3% and those on pre-surgery application were 36.9%. Further, articles in the knee (39.0%), spine (28.5%), and hip and pelvis (21.5%) fields accounted for 89.0% of the total publications. But the increase in publications in the said period was highest in the fields of the hand and wrist (+1,300.0%), ankle (+466.7%), and shoulder (+366.7%).

Conclusions

Over the last 20 years, the publication of CAOS-related research articles in international journals has grown steadily. Although the knee, spine, hip, and pelvis fields account for most CAOS-related research, research in new fields is also increasing. This study analyzed the types of articles and trends in CAOS-related research and provided useful information for future research in the field of CAOS.

Artificial intelligence in musculoskeletal conditions

Artificial intelligence (AI) is an iterative process by which information is captured, transformed into knowledge and processed to produce adaptive changes in the environment. AI is a broad concept, involving virtual (computing) and physical (robotics) elements. In this narrative literature review, we focus on the aspects of AI that present major opportunities for developing health care. Within a few years, AI will be part of our daily clinical practice. Although significant advances are being made, the application of AI in musculoskeletal medicine is still in its early stages compared with its implementation in other areas of medicine. AI is increasingly being employed in fields such as musculoskeletal radiology, skeletal trauma, orthopedic surgery, physical and rehabilitation medicine and sports medicine, as well as for "big data" and AI in gastrointestinal (GI) endoscopy related injuries. Among the limitations of IA are that it analyzes information based on the data it is supplied, which must therefore be well-labeled and that some algorithms such as DL uses more time, data, and computational power than other techniques. Moreover, AI currently does not solve the problem of causality that exists in medicine with observational data; information that physicians interpret within a broad clinical context. AI should therefore be integrated in a prudent and reasonable manner into the workflows of health professionals.

An increasing number of convolutional neural networks for fracture recognition and classification in orthopaedics : are these externally validated and ready for clinical application?

Aims

The number of convolutional neural networks (CNN) available for fracture detection and classification is rapidly increasing. External validation of a CNN on a temporally separate (separated by time) or geographically separate (separated by location) dataset is crucial to assess generalizability of the CNN before application to clinical practice in other institutions. We aimed to answer the following questions: are current CNNs for fracture recognition externally valid?; which methods are applied for external validation (EV)?; and, what are reported performances of the EV sets compared to the internal validation (IV) sets of these CNNs?

Methods

The PubMed and Embase databases were systematically searched from January 2010 to October 2020 according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement. The type of EV, characteristics of the external dataset, and diagnostic performance characteristics on the IV and EV datasets were collected and compared. Quality assessment was conducted using a seven-item checklist based on a modified Methodologic Index for NOn-Randomized Studies instrument (MINORS).

Results

Out of 1,349 studies, 36 reported development of a CNN for fracture detection and/or classification. Of these, only four (11%) reported a form of EV. One study used temporal EV, one conducted both temporal and geographical EV, and two used geographical EV. When comparing the CNN’s performance on the IV set versus the EV set, the following were found: AUCs of 0.967 (IV) versus 0.975 (EV), 0.976 (IV) versus 0.985 to 0.992 (EV), 0.93 to 0.96 (IV) versus 0.80 to 0.89 (EV), and F1-scores of 0.856 to 0.863 (IV) versus 0.757 to 0.840 (EV).

Conclusion

The number of externally validated CNNs in orthopaedic trauma for fracture recognition is still scarce. This greatly limits the potential for transfer of these CNNs from the developing institute to another hospital to achieve similar diagnostic performance. We recommend the use of geographical EV and statements such as the Consolidated Standards of Reporting Trials–Artificial Intelligence (CONSORT-AI), the Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence (SPIRIT-AI) and the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis–Machine Learning (TRIPOD-ML) to critically appraise performance of CNNs and improve methodological rigor, quality of future models, and facilitate eventual implementation in clinical practice.

Cite this article: Bone Jt Open 2021;2(10):879–885.

Consecutive Serial Non-Contrast CT Scan-Based Deep Learning Model Facilitates the Prediction of Tumor Invasiveness of Ground-Glass Nodules

Introduction

Tumors are continuously evolving biological systems which can be monitored by medical imaging. Previous studies only focus on single timepoint images, whether the performance could be further improved by using serial noncontrast CT imaging obtained during nodule follow-up management remains unclear. In this study, we evaluated DL model for predicting tumor invasiveness of GGNs through analyzing time series CT images.

Methods

A total of 168 pathologically confirmed GGN cases (48 noninvasive lesions and 120 invasive lesions) were retrospectively collected and randomly assigned to the development dataset (n = 123) and independent testing dataset (n = 45). All patients underwent consecutive noncontrast CT examinations, and the baseline CT and 3-month follow-up CT images were collected. The gross region of interest (ROI) patches containing only tumor region and the full ROI patches including both tumor and peritumor regions were cropped from CT images. A baseline model was built on the image features and demographic features. Four DL models were proposed: two single-DL model using gross ROI (model 1) or full ROI patches (model 3) from baseline CT images, and two serial-DL models using gross ROI (model 2) or full ROI patches (model 4) from consecutive CT images (baseline scan and 3-month follow-up scan). In addition, a combined model integrating serial full ROI patches and clinical information was also constructed. The performance of these predictive models was assessed with respect to discrimination and clinical usefulness.

Results

The area under the curve (AUC) of the baseline model, models 1, 2, 3, and 4 were 0.562 [(95% confidence interval (C)], 0.406~0.710), 0.693 (95% CI, 0.538-0.822), 0.787 (95% CI, 0.639-0.895), 0.727 (95% CI, 0.573-0.849), and 0.811 (95% CI, 0.667-0.912) in the independent testing dataset, respectively. The results indicated that the peritumor region had potential to contribute to tumor invasiveness prediction, and the model performance was further improved by integrating imaging scans at multiple timepoints. Furthermore, the combined model showed best discrimination ability, with AUC, sensitivity, specificity, and accuracy achieving 0.831 (95% CI, 0.690-0.926), 86.7%, 73.3%, and 82.2%, respectively.

Conclusion

The DL model integrating full ROIs from serial CT images shows improved predictive performance in differentiating noninvasive from invasive GGNs than the model using only baseline CT images, which could benefit the clinical management of GGNs.

An Interpretable Approach to Automated Severity Scoring in Pelvic Trauma

Pelvic ring disruptions result from blunt injury mechanisms and are often found in patients with multi-system trauma. To grade pelvic fracture severity in trauma victims based on whole-body CT, the Tile AO/OTA classification is frequently used. Due to the high volume of whole-body trauma CTs generated in busy trauma centers, an automated approach to Tile classification would provide substantial value, e. g., to prioritize the reading queue of the attending trauma radiologist. In such scenario, an automated method should perform grading based on a transparent process and based on interpretable features to enable interaction with human readers and lower their workload by offering insights from a first automated read of the scan. This paper introduces an automated yet interpretable pelvic trauma decision support system to assist radiologists in fracture detection and Tile grade classification. The method operates similarly to human interpretation of CT scans and first detects distinct pelvic fractures on CT with high specificity using a Faster-RCNN model that are then interpreted using a structural causal model based on clinical best practices to infer an initial Tile grade. The Bayesian causal model and finally, the object detector are then queried for likely co-occurring fractures that may have been rejected initially due to the highly specific operating point of the detector, resulting in an updated list of detected fractures and corresponding final Tile grade. Our method is transparent in that it provides finding location and type using the object detector, as well as information on important counterfactuals that would invalidate the system's recommendation and achieves an AUC of 83.3%/85.1% for translational/rotational instability. Despite being designed for human-machine teaming, our approach does not compromise on performance compared to previous black-box approaches.

Knowledge-Guided Multiview Deep Curriculum Learning for Elbow Fracture Classification

Elbow fracture diagnosis often requires patients to take both frontal and lateral views of elbow X-ray radiographs. In this paper, we propose a multiview deep learning method for an elbow fracture subtype classification task. Our strategy leverages transfer learning by first training two single-view models, one for frontal view and the other for lateral view, and then transferring the weights to the corresponding layers in the proposed multiview network architecture. Meanwhile, quantitative medical knowledge was integrated into the training process through a curriculum learning framework, which enables the model to first learn from "easier" samples and then transition to "harder" samples to reach better performance. In addition, our multiview network can work both in a dual-view setting and with a single view as input. We evaluate our method through extensive experiments on a classification task of elbow fracture with a dataset of 1,964 images. Results show that our method outperforms two related methods on bone fracture study in multiple settings, and our technique is able to boost the performance of the compared methods. The code is available at https://github.com/ljaiverson/multiview-curriculum.

Deep neural networks with promising diagnostic accuracy for the classification of atypical femoral fractures

Background and purpose - A correct diagnosis is essential for the appropriate treatment of patients with atypical femoral fractures (AFFs). The diagnostic accuracy of radiographs with standard radiology reports is very poor. We derived a diagnostic algorithm that uses deep neural networks to enable clinicians to discriminate AFFs from normal femur fractures (NFFs) on conventional radiographs.Patients and methods - We entered 433 radiographs from 149 patients with complete AFF and 549 radiographs from 224 patients with NFF into a convolutional neural network (CNN) that acts as a core classifier in an automated pathway and a manual intervention pathway (manual improvement of image orientation). We tested several deep neural network structures (i.e., VGG19, InceptionV3, and ResNet) to identify the network with the highest diagnostic accuracy for distinguishing AFF from NFF. We applied a transfer learning technique and used 5-fold cross-validation and class activation mapping to evaluate the diagnostic accuracy.Results - In the automated pathway, ResNet50 had the highest diagnostic accuracy, with a mean of 91% (SD 1.3), as compared with 83% (SD 1.6) for VGG19, and 89% (SD 2.5) for InceptionV3. The corresponding accuracy levels for the intervention pathway were 94% (SD 2.0), 92% (2.7), and 93% (3.7), respectively. With regards to sensitivity and specificity, ResNet outperformed the other networks with a mean AUC (area under the curve) value of 0.94 (SD 0.01) and surpassed the accuracy of clinical diagnostics.Interpretation - Artificial intelligence systems show excellent diagnostic accuracies for the rare fracture type of AFF in an experimental setting.

Transfer Learning for an Automated Detection System of Fractures in Patients with Maxillofacial Trauma

An original maxillofacial fracture detection system (MFDS), based on convolutional neural networks and transfer learning, is proposed to detect traumatic fractures in patients. A convolutional neural network pre-trained on non-medical images was re-trained and fine-tuned using computed tomography (CT) scans to produce a model for the classification of future CTs as either “fracture” or “noFracture”. The model was trained on a total of 148 CTs (120 patients labeled with “fracture” and 28 patients labeled with “noFracture”). The validation dataset, used for statistical analysis, was characterized by 30 patients (5 with “noFracture” and 25 with “fracture”). An additional 30 CT scans, comprising 25 “fracture” and 5 “noFracture” images, were used as the test dataset for final testing. Tests were carried out both by considering the single slices and by grouping the slices for patients. A patient was categorized as fractured if two consecutive slices were classified with a fracture probability higher than 0.99. The patients’ results show that the model accuracy in classifying the maxillofacial fractures is 80%. Even if the MFDS model cannot replace the radiologist’s work, it can provide valuable assistive support, reducing the risk of human error, preventing patient harm by minimizing diagnostic delays, and reducing the incongruous burden of hospitalization.

How Can a Deep Learning Algorithm Improve Fracture Detection on X-rays in the Emergency Room?

The growing need for emergency imaging has greatly increased the number of conventional X-rays, particularly for traumatic injury. Deep learning (DL) algorithms could improve fracture screening by radiologists and emergency room (ER) physicians. We used an algorithm developed for the detection of appendicular skeleton fractures and evaluated its performance for detecting traumatic fractures on conventional X-rays in the ER, without the need for training on local data. This algorithm was tested on all patients (N = 125) consulting at the Louis Mourier ER in May 2019 for limb trauma. Patients were selected by two emergency physicians from the clinical database used in the ER. Their X-rays were exported and analyzed by a radiologist. The prediction made by the algorithm and the annotation made by the radiologist were compared. For the 125 patients included, 25 patients with a fracture were identified by the clinicians, 24 of whom were identified by the algorithm (sensitivity of 96%). The algorithm incorrectly predicted a fracture in 14 of the 100 patients without fractures (specificity of 86%). The negative predictive value was 98.85%. This study shows that DL algorithms are potentially valuable diagnostic tools for detecting fractures in the ER and could be used in the training of junior radiologists.

Differential diagnosis of benign and malignant vertebral fracture on CT using deep learning

OBJECTIVES

To evaluate the performance of deep learning using ResNet50 in differentiation of benign and malignant vertebral fracture on CT.

METHODS

A dataset of 433 patients confirmed with 296 malignant and 137 benign fractures was retrospectively selected from our spinal CT image database. A senior radiologist performed visual reading to evaluate six imaging features, and three junior radiologists gave diagnostic prediction. A ROI was placed on the most abnormal vertebrae, and the smallest square bounding box was generated. The input channel into ResNet50 network was 3, including the slice with its two neighboring slices. The diagnostic performance was evaluated using 10-fold cross-validation. After obtaining the malignancy probability from all slices in a patient, the highest probability was assigned to that patient to give the final diagnosis, using the threshold of 0.5.

RESULTS

Visual features such as soft tissue mass and bone destruction were highly suggestive of malignancy; the presence of a transverse fracture line was highly suggestive of a benign fracture. The reading by three radiologists with 5, 3, and 1 year of experience achieved an accuracy of 99%, 95.2%, and 92.8%, respectively. In ResNet50 analysis, the per-slice diagnostic sensitivity, specificity, and accuracy were 0.90, 0.79, and 85%. When the slices were combined to ve per-patient diagnosis, the sensitivity, specificity, and accuracy were 0.95, 0.80, and 88%.

CONCLUSION

Deep learning has become an important tool for the detection of fractures on CT. In this study, ResNet50 achieved good accuracy, which can be further improved with more cases and optimized methods for future clinical implementation.

KEY POINTS

• Deep learning using ResNet50 can yield a high accuracy for differential diagnosis of benign and malignant vertebral fracture on CT. • The per-slice diagnostic sensitivity, specificity, and accuracy were 0.90, 0.79, and 85% in deep learning using ResNet50 analysis. • The slices combined with per-patient diagnostic sensitivity, specificity, and accuracy were 0.95, 0.80, and 88% in deep learning using ResNet50 analysis.

Artificial intelligence improves the accuracy of residents in the diagnosis of hip fractures: a multicenter study

BACKGROUND

Less experienced clinicians sometimes make misdiagnosis of hip fractures. We developed computer-aided diagnosis (CAD) system for hip fractures on plain X-rays using a deep learning model trained on a large dataset. In this study, we examined whether the accuracy of the diagnosis of hip fracture of the residents could be improved by using this system.

METHODS

A deep convolutional neural network approach was used for machine learning. Pytorch 1.3 and Fast.ai 1.0 were applied as frameworks, and an EfficientNet-B4 model (a pre-trained ImageNet model) was used. We handled the 5295 X-rays from the patients with femoral neck fracture or femoral trochanteric fracture from 2009 to 2019. We excluded cases in which the bilateral hips were not included within an image range, and cases of femoral shaft fracture and periprosthetic fracture. Finally, we included 5242 AP pelvic X-rays from 4851 cases. We divided these 5242 images into two images per image, and prepared 5242 images including fracture site and 5242 images without fracture site. Thus, a total of 10,484 images were used for machine learning. The accuracy, sensitivity, specificity, F-value, and area under the curve (AUC) were assessed. Gradient-weighted class activation mapping (Grad-CAM) was used to conceptualize the basis for the diagnosis of the fracture by the deep learning algorithm. Secondly, we conducted a controlled experiment with clinicians. Thirty-one residents;young doctors within 2 years of graduation from medical school who rotate through various specialties, were tested using 300 hip fracture images that were randomly extracted from the dataset. We evaluated the diagnostic accuracy with and without the use of the CAD system for each of the 300 images.

RESULTS

The accuracy, sensitivity, specificity, F-value, and AUC were 96.1, 95.2, 96.9%, 0.961, and 0.99, respectively, with the correct diagnostic basis generated by Grad-CAM. In the controlled experiment, the diagnostic accuracy of the residents significantly improved when they used the CAD system.

CONCLUSIONS

We developed a newly CAD system with a deep learning algorithm from a relatively large dataset from multiple institutions. Our system achieved high diagnostic performance. Our system improved the diagnostic accuracy of residents for hip fractures.

LEVEL OF EVIDENCE

Level III, Foundational evidence, before-after study.

CLINICAL RELEVANCE

high.

Artificial Intelligence in the Healthcare System: An Overview

This chapter aims to present insights into the influence of artificial intelligence (AI) on medicine, public health, and the economy. PubMed and Google Scholar databases were used for the identification and collection of articles with search commands of "artificial intelligence" AND "public health" and "artificial intelligence" AND "medicine". A total of 273 articles specifically handling the issue of artificial intelligence, dating ten years back, in three major medical journals: Science, The Lancet, and The New England Journal of Medicine, were analyzed. Computational power gets stronger by the day, giving us new solutions and possibilities. Current medicine problems like personalized medicine, storage of data, and documentation overload will likely be replaced by AI shortly. The application of AI may also bring substantial benefits to other areas of medicine like the diagnostic and therapeutic processes. The development and spread of AI are inescapable as it lowers healthcare and administrative costs, improves medical efficiency, and predicts and prevents major disease complications. The use of AI in medicine seems destined to carry the day.

Investigation of bone fracture diagnosis system using transverse vibration response

In this study, a new diagnostic system is developed to easily identify bone fractures in non-medical environments. It is difficult to determine the extent of cracks, fractures, and the healing process inside humans owing to the differences among people and limitations of state-of-the-art medical devices. Thus, various medical techniques, such as X-ray, computed tomography, or fork tuning systems have been developed, and more advanced technologies are emerging in the medical engineering field. In hazardous circumstances, medical devices to detect bone fracture are not available or cannot be easily applied. Thus, there is a need for the rapid detection of bone fractures without medical devices. To this end, this study analyzes the transverse vibration responses of bones because bone fractures cause different mechanical vibration reactions. By comparing the transverse vibration responses of both healthy and fractured bones, the modal assurance criterion can be calculated and applied to detect the existence of bone fractures. The transverse vibration responses at low and high frequencies are different and exhibit different modal assurance criteria depending on whether or not they are abnormal. Then, the virtual spectrogram of the differences between the signals from non-fractured and fractured bones is calculated. With the help of the present criterion with transverse vibration data, this difference can be analyzed quantitatively and effectively. To validate the proposed system, experiments with artificial specimens, animal legs, and a cadaver are performed.

Retinal Imaging Techniques Based on Machine Learning Models in Recognition and Prediction of Mild Cognitive Impairment

BACKGROUND AND PURPOSE

Mild Cognitive Impairment (MCI) is thought to be the signal of many progressive diseases but is easily ignored. Therefore, a simple and easy screening method for recognizing and predicting MCI is urgently needed. The study aimed to establish machine learning models of retinal vascular features to categorize and predict MCI.

PATIENTS AND METHODS

Subjects enrolled underwent cognitive function assessment and were divided into a normal group, an MCI group, and a dementia group, and fundus photography was performed. MATLAB 2019b was used for fundus image preprocessing and vascular segmentation. Via the Green channel, adaptive histogram equalization (AHE), image binarization, and median filtering, we obtained the original and segmentation retinal vessel images. Afterwards, the histogram of oriented gradient (HOG) was used for image feature extraction. Support vector machine (SVM) and extreme learning machine (ELM) were selected for training models in the fundus original images and fundus vascular segmentation images, respectively. Among the three cognitive groups, sensitivity, specificity, the receiver operating characteristic (ROC) curves, and the area under the curve (AUC) were used to evaluate and compare the predictive performance of the two models in the fundus original and vascular segmentation images, respectively.

RESULTS

A total of 86 eligible subjects were enrolled in the study. After a clinical cognitive assessment, the participants were divided into the normal group (N = 38), the MCI group (N = 26), and the dementia group (N = 22). A total of 332 qualified fundus images were adopted after screening. Comparing the models among the three groups showed that the SVM model had more advantages than the ELM model in the fundus original images and vascular segmentation images. Meanwhile, we found that the original images performed better than the segmentation images in the same prediction model. Among the three groups, the SVM model of the fundus original images had the best performance.

CONCLUSION

The establishment of a predictive model based on vascular-related feature extraction from fundus images has high recognition and prediction abilities for cognitive function and can be used as a screening method for MCI.

CLINICAL TRIAL REGISTRATION

ChiCTR.org.cn (ChiCTR1900027404), Registered on Nov 12, 2019.

A Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography: System Development and Validation Study

Background

Hip fracture is the most common type of fracture in elderly individuals. Numerous deep learning (DL) algorithms for plain pelvic radiographs (PXRs) have been applied to improve the accuracy of hip fracture diagnosis. However, their efficacy is still undetermined.

Objective

The objective of this study is to develop and validate a human-algorithm integration (HAI) system to improve the accuracy of hip fracture diagnosis in a real clinical environment.

Methods

The HAI system with hip fracture detection ability was developed using a deep learning algorithm trained on trauma registry data and 3605 PXRs from August 2008 to December 2016. To compare their diagnostic performance before and after HAI system assistance using an independent testing dataset, 34 physicians were recruited. We analyzed the physicians’ accuracy, sensitivity, specificity, and agreement with the algorithm; we also performed subgroup analyses according to physician specialty and experience. Furthermore, we applied the HAI system in the emergency departments of different hospitals to validate its value in the real world.

Results

With the support of the algorithm, which achieved 91% accuracy, the diagnostic performance of physicians was significantly improved in the independent testing dataset, as was revealed by the sensitivity (physician alone, median 95%; HAI, median 99%; P<.001), specificity (physician alone, median 90%; HAI, median 95%; P<.001), accuracy (physician alone, median 90%; HAI, median 96%; P<.001), and human-algorithm agreement [physician alone κ, median 0.69 (IQR 0.63-0.74); HAI κ, median 0.80 (IQR 0.76-0.82); P<.001. With the help of the HAI system, the primary physicians showed significant improvement in their diagnostic performance to levels comparable to those of consulting physicians, and both the experienced and less-experienced physicians benefited from the HAI system. After the HAI system had been applied in 3 departments for 5 months, 587 images were examined. The sensitivity, specificity, and accuracy of the HAI system for detecting hip fractures were 97%, 95.7%, and 96.08%, respectively.

Conclusions

HAI currently impacts health care, and integrating this technology into emergency departments is feasible. The developed HAI system can enhance physicians’ hip fracture diagnostic performance.