Automatic detection and segmentation of lumbar vertebrae from X-ray images for compression fracture evaluation

Deep learning-based automatic measurement system for patellar height: a multicenter retrospective study

BACKGROUND

The patellar height index is important; however, the measurement procedures are time-consuming and prone to significant variability among and within observers. We developed a deep learning-based automatic measurement system for the patellar height and evaluated its performance and generalization ability to accurately measure the patellar height index.

METHODS

We developed a dataset containing 3,923 lateral knee X-ray images. Notably, all X-ray images were from three tertiary level A hospitals, and 2,341 cases were included in the analysis after screening. By manually labeling key points, the model was trained using the residual network (ResNet) and high-resolution network (HRNet) for human pose estimation architectures to measure the patellar height index. Various data enhancement techniques were used to enhance the robustness of the model. The root mean square error (RMSE), object keypoint similarity (OKS), and percentage of correct keypoint (PCK) metrics were used to evaluate the training results. In addition, we used the intraclass correlation coefficient (ICC) to assess the consistency between manual and automatic measurements.

RESULTS

The HRNet model performed excellently in keypoint detection tasks by comparing different deep learning models. Furthermore, the pose_hrnet_w48 model was particularly outstanding in the RMSE, OKS, and PCK metrics, and the Insall-Salvati index (ISI) automatically calculated by this model was also highly consistent with the manual measurements (intraclass correlation coefficient [ICC], 0.809-0.885). This evidence demonstrates the accuracy and generalizability of this deep learning system in practical applications.

CONCLUSION

We successfully developed a deep learning-based automatic measurement system for the patellar height. The system demonstrated accuracy comparable to that of experienced radiologists and a strong generalizability across different datasets. It provides an essential tool for assessing and treating knee diseases early and monitoring and rehabilitation after knee surgery. Due to the potential bias in the selection of datasets in this study, different datasets should be examined in the future to optimize the model so that it can be reliably applied in clinical practice.

TRIAL REGISTRATION

The study was registered at the Medical Research Registration and Filing Information System (medicalresearch.org.cn) MR-61-23-013065. Date of registration: May 04, 2023 (retrospectively registered).

Deep learning-based automated high-accuracy location and identification of fresh vertebral compression fractures from spinal radiographs: a multicenter cohort study

Background

Digital radiography (DR) is a common and widely available examination. However, spinal DR cannot detect bone marrow edema, therefore, determining vertebral compression fractures (VCFs), especially fresh VCFs, remains challenging for clinicians.

Methods

We trained, validated, and externally tested the deep residual network (DRN) model that automated the detection and identification of fresh VCFs from spinal DR images. A total of 1,747 participants from five institutions were enrolled in this study and divided into the training cohort, validation cohort and external test cohorts (YHDH and BMUH cohorts). We evaluated the performance of DRN model based on the area under the receiver operating characteristic curve (AUC), feature attention maps, sensitivity, specificity, and accuracy. We compared it with five other deep learning models and validated and tested the model internally and externally and explored whether it remains highly accurate for an external test cohort. In addition, the influence of old VCFs on the performance of the DRN model was assessed.

Results

The AUC was 0.99, 0.89, and 0.88 in the validation, YHDH, and BMUH cohorts, respectively, for the DRN model for detecting and discriminating fresh VCFs. The accuracies were 81.45% and 72.90%, sensitivities were 84.75% and 91.43%, and specificities were 80.25% and 63.89% in the YHDH and BMUH cohorts, respectively. The DRN model generated correct activation on the fresh VCFs and accurate peak responses on the area of the target vertebral body parts and demonstrated better feature representation learning and classification performance. The AUC was 0.90 (95% confidence interval [CI] 0.84-0.95) and 0.84 (95% CI 0.72-0.93) in the non-old VCFs and old VCFs groups, respectively, in the YHDH cohort (p = 0.067). The AUC was 0.89 (95% CI 0.84-0.94) and 0.85 (95% CI 0.72-0.95) in the non-old VCFs and old VCFs groups, respectively, in the BMUH cohort (p = 0.051).

Conclusion

In present study, we developed the DRN model for automated diagnosis and identification of fresh VCFs from spinal DR images. The DRN model can provide interpretable attention maps to support the excellent prediction results, which is the key that most clinicians care about when using the model to assist decision-making.

Deep learning-based spinal canal segmentation of computed tomography image for disease diagnosis: A proposed system for spinal stenosis diagnosis

BACKGROUND

Lumbar disc herniation was regarded as an age-related degenerative disease. Nevertheless, emerging reports highlight a discernible shift, illustrating the prevalence of these conditions among younger individuals.

METHODS

This study introduces a novel deep learning methodology tailored for spinal canal segmentation and disease diagnosis, emphasizing image processing techniques that delve into essential image attributes such as gray levels, texture, and statistical structures to refine segmentation accuracy.

RESULTS

Analysis reveals a progressive increase in the size of vertebrae and intervertebral discs from the cervical to lumbar regions. Vertebrae, bearing weight and safeguarding the spinal cord and nerves, are interconnected by intervertebral discs, resilient structures that counteract spinal pressure. Experimental findings demonstrate a lack of pronounced anteroposterior bending during flexion and extension, maintaining displacement and rotation angles consistently approximating zero. This consistency maintains uniform anterior and posterior vertebrae heights, coupled with parallel intervertebral disc heights, aligning with theoretical expectations.

CONCLUSIONS

Accuracy assessment employs 2 methods: IoU and Dice, and the average accuracy of IoU is 88% and that of Dice is 96.4%. The proposed deep learning-based system showcases promising results in spinal canal segmentation, laying a foundation for precise stenosis diagnosis in computed tomography images. This contributes significantly to advancements in spinal pathology understanding and treatment.

Deep Learning Model for Grading and Localization of Lumbar Disc Herniation on Magnetic Resonance Imaging

BACKGROUND

Methods for grading and localization of lumbar disc herniation (LDH) on MRI are complex, time-consuming, and subjective. Utilizing deep learning (DL) models as assistance would mitigate such complexities.

PURPOSE

To develop an interpretable DL model capable of grading and localizing LDH.

STUDY TYPE

Retrospective.

SUBJECTS

1496 patients (M/F: 783/713) were evaluated, and randomly divided into training (70%), validation (10%), and test (20%) sets.

FIELD STRENGTH/SEQUENCE

1.5T MRI for axial T2-weighted sequences (spin echo).

ASSESSMENT

The training set was annotated by three spinal surgeons using the Michigan State University classification to train the DL model. The test set was annotated by a spinal surgery expert (as ground truth labels), and two spinal surgeons (comparison with the trained model). An external test set was employed to evaluate the generalizability of the DL model.

STATISTICAL TESTS

Calculated intersection over union (IoU) for detection consistency, utilized Gwet's AC1 to assess interobserver agreement, and evaluated model performance based on sensitivity and specificity, with statistical significance set at P < 0.05.

RESULTS

The DL model achieved high detection consistency in both the internal test dataset (grading: mean IoU 0.84, recall 99.6%; localization: IoU 0.82, recall 99.5%) and external test dataset (grading: 0.72, 98.0%; localization: 0.71, 97.6%). For internal testing, the DL model (grading: 0.81; localization: 0.76), Rater 1 (0.88; 0.82), and Rater 2 (0.86; 0.83) demonstrated results highly consistent with the ground truth labels. The overall sensitivity of the DL model was 87.0% for grading and 84.0% for localization, while the specificity was 95.5% and 94.4%. For external testing, the DL model showed an appreciable decrease in consistency (grading: 0.69; localization: 0.66), sensitivity (77.2%; 76.7%), and specificity (92.3%; 91.8%).

DATA CONCLUSION

The classification capabilities of the DL model closely resemble those of spinal surgeons. For future improvement, enriching the diversity of cases could enhance the model's generalization.

LEVEL OF EVIDENCE: 4

TECHNICAL EFFICACY

Stage 2.

An Automated Multi-scale Feature Fusion Network for Spine Fracture Segmentation Using Computed Tomography Images

Spine fractures represent a critical health concern with far-reaching implications for patient care and clinical decision-making. Accurate segmentation of spine fractures from medical images is a crucial task due to its location, shape, type, and severity. Addressing these challenges often requires the use of advanced machine learning and deep learning techniques. In this research, a novel multi-scale feature fusion deep learning model is proposed for the automated spine fracture segmentation using Computed Tomography (CT) to these challenges. The proposed model consists of six modules; Feature Fusion Module (FFM), Squeeze and Excitation (SEM), Atrous Spatial Pyramid Pooling (ASPP), Residual Convolution Block Attention Module (RCBAM), Residual Border Refinement Attention Block (RBRAB), and Local Position Residual Attention Block (LPRAB). These modules are used to apply multi-scale feature fusion, spatial feature extraction, channel-wise feature improvement, segmentation border results border refinement, and positional focus on the region of interest. After that, a decoder network is used to predict the fractured spine. The experimental results show that the proposed approach achieves better accuracy results in solving the above challenges and also performs well compared to the existing segmentation methods.

Deep Learning-Assisted Quantitative Measurement of Thoracolumbar Fracture Features on Lateral Radiographs

OBJECTIVE

This study aimed to develop and validate a deep learning (DL) algorithm for the quantitative measurement of thoracolumbar (TL) fracture features, and to evaluate its efficacy across varying levels of clinical expertise.

METHODS

Using the pretrained Mask Region-Based Convolutional Neural Networks model, originally developed for vertebral body segmentation and fracture detection, we fine-tuned the model and added a new module for measuring fracture metrics-compression rate (CR), Cobb angle (CA), Gardner angle (GA), and sagittal index (SI)-from lumbar spine lateral radiographs. These metrics were derived from six-point labeling by 3 radiologists, forming the ground truth (GT). Training utilized 1,000 nonfractured and 318 fractured radiographs, while validations employed 213 internal and 200 external fractured radiographs. The accuracy of the DL algorithm in quantifying fracture features was evaluated against GT using the intraclass correlation coefficient. Additionally, 4 readers with varying expertise levels, including trainees and an attending spine surgeon, performed measurements with and without DL assistance, and their results were compared to GT and the DL model.

RESULTS

The DL algorithm demonstrated good to excellent agreement with GT for CR, CA, GA, and SI in both internal (0.860, 0.944, 0.932, and 0.779, respectively) and external (0.836, 0.940, 0.916, and 0.815, respectively) validations. DL-assisted measurements significantly improved most measurement values, particularly for trainees.

CONCLUSION

The DL algorithm was validated as an accurate tool for quantifying TL fracture features using radiographs. DL-assisted measurement is expected to expedite the diagnostic process and enhance reliability, particularly benefiting less experienced clinicians.

Development of a shape-based algorithm for identification of asymptomatic vertebral compression fractures: A proof-of-principle study

Objectives

Vertebral fracture is both common and serious among adults, yet it often goes undiagnosed. This study aimed to develop a shape-based algorithm (SBA) for the automatic identification of vertebral fractures.

Methods

The study included 144 participants (50 individuals with a fracture and 94 without a fracture) whose plain thoracolumbar spine X-rays were taken. Clinical diagnosis of vertebral fracture (grade 0 to 3) was made by rheumatologists using Genant's semiquantitative method. The SBA algorithm was developed to determine the ratio of vertebral body height loss. Based on the ratio, SBA classifies a vertebra into 4 classes: 0 = normal, 1 = mild fracture, 2 = moderate fracture, 3 = severe fracture). The concordance between clinical diagnosis and SBA-based classification was assessed at both person and vertebra levels.

Results

At the person level, the SBA achieved a sensitivity of 100% and specificity of 62% (95% CI, 51%-72%). At the vertebra level, the SBA achieved a sensitivity of 84% (95% CI, 72%-93%), and a specificity of 88% (95% CI, 85%-90%). On average, the SBA took 0.3 s to assess each X-ray.

Conclusions

The SBA developed here is a fast and efficient tool that can be used to systematically screen for asymptomatic vertebral fractures and reduce the workload of healthcare professionals.

Commercially available artificial intelligence tools for fracture detection: the evidence

Missed fractures are a costly healthcare issue, not only negatively impacting patient lives, leading to potential long-term disability and time off work, but also responsible for high medicolegal disbursements that could otherwise be used to improve other healthcare services. When fractures are overlooked in children, they are particularly concerning as opportunities for safeguarding may be missed. Assistance from artificial intelligence (AI) in interpreting medical images may offer a possible solution for improving patient care, and several commercial AI tools are now available for radiology workflow implementation. However, information regarding their development, evidence for performance and validation as well as the intended target population is not always clear, but vital when evaluating a potential AI solution for implementation. In this article, we review the range of available products utilizing AI for fracture detection (in both adults and children) and summarize the evidence, or lack thereof, behind their performance. This will allow others to make better informed decisions when deciding which product to procure for their specific clinical requirements.

Artificial Intelligence-Assisted Detection of Osteoporotic Vertebral Fractures on Lateral Chest Radiographs in Post-Menopausal Women

Osteoporotic vertebral fractures (OVFs) are often not reported by radiologists on routine chest radiographs. This study aims to investigate the clinical value of a newly developed artificial intelligence (AI) tool, Ofeye 1.0, for automated detection of OVFs on lateral chest radiographs in post-menopausal women (>60 years) who were referred to undergo chest x-rays for other reasons. A total of 510 de-identified lateral chest radiographs from three clinical sites were retrieved and analysed using the Ofeye 1.0 tool. These images were then reviewed by a consultant radiologist with findings serving as the reference standard for determining the diagnostic performance of the AI tool for the detection of OVFs. Of all the original radiologist reports, missed OVFs were found in 28.8% of images but were detected using the AI tool. The AI tool demonstrated high specificity of 92.8% (95% CI: 89.6, 95.2%), moderate accuracy of 80.3% (95% CI: 76.3, 80.4%), positive predictive value (PPV) of 73.7% (95% CI: 65.2, 80.8%), and negative predictive value (NPV) of 81.5% (95% CI: 79, 83.8%), but low sensitivity of 49% (95% CI: 40.7, 57.3%). The AI tool showed improved sensitivity compared with the original radiologist reports, which was 20.8% (95% CI: 14.5, 28.4). The new AI tool can be used as a complementary tool in routine diagnostic reports for the reduction in missed OVFs in elderly women.

DC-AAE: Dual channel adversarial autoencoder with multitask learning for KL-grade classification in knee radiographs

Knee osteoarthritis (OA) is a frequent musculoskeletal disorder that leads to physical disability in older adults. Manual OA assessment is performed via visual inspection, which is highly subjective as it suffers from moderate to high inter-observer variability. Many deep learning-based techniques have been proposed to address this issue. However, owing to the limited amount of labelled data, all existing solutions have limitations in terms of performance or the number of classes. This paper proposes a novel fully automatic Kellgren and Lawrence (KL) grade classification scheme in knee radiographs. We developed a semi-supervised multi-task learning-based approach that enables the exploitation of additional unlabelled data in an unsupervised as well as supervised manner. Specifically, we propose a dual-channel adversarial autoencoder, which is first trained in an unsupervised manner for reconstruction tasks only. To exploit the additional data in a supervised way, we propose a multi-task learning framework by introducing an auxiliary task. In particular, we use leg side identification as an auxiliary task, which allows the use of more datasets, e.g., CHECK dataset. The work demonstrates that the utilization of additional data can improve the primary task of KL-grade classification for which only limited labelled data is available. This semi-supervised learning essentially helps to improve the feature learning ability of our framework, which leads to improved performance for KL-grade classification. We rigorously evaluated our proposed model on the two largest publicly available datasets for various aspects, i.e., overall performance, the effect of additional unlabelled samples and auxiliary tasks, robustness analysis, and ablation study. The proposed model achieved the accuracy, precision, recall, and F1 score of 75.53%, 74.1%, 78.51%, and 75.34%, respectively. Furthermore, the experimental results show that the suggested model not only achieves state-of-the-art performance on two publicly available datasets but also exhibits remarkable robustness.

Deep learning system for automated detection of posterior ligamentous complex injury in patients with thoracolumbar fracture on MRI

This study aimed to develop a deep learning (DL) algorithm for automated detection and localization of posterior ligamentous complex (PLC) injury in patients with acute thoracolumbar (TL) fracture on magnetic resonance imaging (MRI) and evaluate its diagnostic performance. In this retrospective multicenter study, using midline sagittal T2-weighted image with fracture (± PLC injury), a training dataset and internal and external validation sets of 300, 100, and 100 patients, were constructed with equal numbers of injured and normal PLCs. The DL algorithm was developed through two steps (Attention U-net and Inception-ResNet-V2). We evaluate the diagnostic performance for PLC injury between the DL algorithm and radiologists with different levels of experience. The area under the curves (AUCs) generated by the DL algorithm were 0.928, 0.916 for internal and external validations, and by two radiologists for observer performance test were 0.930, 0.830, respectively. Although no significant difference was found in diagnosing PLC injury between the DL algorithm and radiologists, the DL algorithm exhibited a trend of higher AUC than the radiology trainee. Notably, the radiology trainee's diagnostic performance significantly improved with DL algorithm assistance. Therefore, the DL algorithm exhibited high diagnostic performance in detecting PLC injuries in acute TL fractures.

A Deep Learning-Based Model for Classifying Osteoporotic Lumbar Vertebral Fractures on Radiographs: A Retrospective Model Development and Validation Study

Early diagnosis and initiation of treatment for fresh osteoporotic lumbar vertebral fractures (OLVF) are crucial. Magnetic resonance imaging (MRI) is generally performed to differentiate between fresh and old OLVF. However, MRIs can be intolerable for patients with severe back pain. Furthermore, it is difficult to perform in an emergency. MRI should therefore only be performed in appropriately selected patients with a high suspicion of fresh fractures. As radiography is the first-choice imaging examination for the diagnosis of OLVF, improving screening accuracy with radiographs will optimize the decision of whether an MRI is necessary. This study aimed to develop a method to automatically classify lumbar vertebrae (LV) conditions such as normal, old, or fresh OLVF using deep learning methods with radiography. A total of 3481 LV images for training, validation, and testing and 662 LV images for external validation were collected. Visual evaluation by two radiologists determined the ground truth of LV diagnoses. Three convolutional neural networks were ensembled. The accuracy, sensitivity, and specificity were 0.89, 0.83, and 0.92 in the test and 0.84, 0.76, and 0.89 in the external validation, respectively. The results suggest that the proposed method can contribute to the accurate automatic classification of LV conditions on radiography.

ALiGN: Attention based Line Guided Network for Vertebral Comprssion Fracture Detection

Vertebral Compression Fracture (VCF) is one of the common fractures, especially for elderlies. As it affects postural deformation that may cause secondary disorders in the respiratory or digestive system if not treated in time, diagnosis of VCF is crucial. Using deep learning model based detection technology in diagnosis can reduce the workload of healthcare workers and misdiagnosis. Hence in this work, we propose ALiGN, a compression fracture detection model in the lumbar vertebra based on a deep convolutional neural network (CNN). Specifically, we take the location of each vertebral body into account via a feature pyramid network with an attention mechanism. Our proposed model outperforms the earlier works with a sensitivity 0.9729, specificity 0.9914, and mAP 0.7882.

Diagnosis of osteoporotic vertebral compression fractures and fracture level detection using multitask learning with U-Net in lumbar spine lateral radiographs

Recent studies of automatic diagnosis of vertebral compression fractures (VCFs) using deep learning mainly focus on segmentation and vertebral level detection in lumbar spine lateral radiographs (LSLRs). Herein, we developed a model for simultaneous VCF diagnosis and vertebral level detection without using adjacent vertebral bodies. In total, 1102 patients with VCF, 1171 controls were enrolled. The 1865, 208, and 198 LSLRS were divided into training, validation, and test dataset. A ground truth label with a 4-point trapezoidal shape was made based on radiological reports showing normal or VCF at some vertebral level. We applied a modified U-Net architecture, in which decoders were trained to detect VCF and vertebral levels, sharing the same encoder. The multi-task model was significantly better than the single-task model in sensitivity and area under the receiver operating characteristic curve. In the internal dataset, the accuracy, sensitivity, and specificity of fracture detection per patient or vertebral body were 0.929, 0.944, and 0.917 or 0.947, 0.628, and 0.977, respectively. In external validation, those of fracture detection per patient or vertebral body were 0.713, 0.979, and 0.447 or 0.828, 0.936, and 0.820, respectively. The success rates were 96 % and 94 % for vertebral level detection in internal and external validation, respectively. The multi-task-shared encoder was significantly better than the single-task encoder. Furthermore, both fracture and vertebral level detection was good in internal and external validation. Our deep learning model may help radiologists perform real-life medical examinations.

Initial study on an expert system for spine diseases screening using inertial measurement unit

In recent times, widely understood spine diseases have advanced to one of the most urgetn problems where quick diagnosis and treatment are needed. To diagnose its specifics (e.g. to decide whether this is a scoliosis or sagittal imbalance) and assess its extend, various kind of imaging diagnostic methods (such as X-Ray, CT, MRI scan or ST) are used. However, despite their common use, some may be regarded as (to a level) invasive methods and there are cases where there are contraindications to using them. Besides, which is even more of a problem, these are very expensive methods and whilst their use for pure diagnostic purposes is absolutely valid, then due to their cost, they cannot rather be considered as tools which would be equally valid for bad posture screening programs purposes. This paper provides an initial evaluation of the alternative approach to the spine diseases diagnostic/screening using inertial measurement unit and we propose policy-based computing as the core for the inference systems. Although the methodology presented herein is potentially applicable to a variety of spine diseases, in the nearest future we will focus specifically on sagittal imbalance detection.

Vertebrae localization, segmentation and identification using a graph optimization and an anatomic consistency cycle

Vertebrae localization, segmentation and identification in CT images is key to numerous clinical applications. While deep learning strategies have brought to this field significant improvements over recent years, transitional and pathological vertebrae are still plaguing most existing approaches as a consequence of their poor representation in training datasets. Alternatively, proposed non-learning based methods take benefit of prior knowledge to handle such particular cases. In this work we propose to combine both strategies. To this purpose we introduce an iterative cycle in which individual vertebrae are recurrently localized, segmented and identified using deep-networks, while anatomic consistency is enforced using statistical priors. In this strategy, the transitional vertebrae identification is handled by encoding their configurations in a graphical model that aggregates local deep-network predictions into an anatomically consistent final result. Our approach achieves the state-of-the-art results on the VerSe20 challenge benchmark, and outperforms all methods on transitional vertebrae as well as the generalization to the VerSe19 challenge benchmark. Furthermore, our method can detect and report inconsistent spine regions that do not satisfy the anatomic consistency priors. Our code and model are openly available for research purposes.¹.

Deep Learning-Based Medical Images Segmentation of Musculoskeletal Anatomical Structures: A Survey of Bottlenecks and Strategies

By leveraging the recent development of artificial intelligence algorithms, several medical sectors have benefited from using automatic segmentation tools from bioimaging to segment anatomical structures. Segmentation of the musculoskeletal system is key for studying alterations in anatomical tissue and supporting medical interventions. The clinical use of such tools requires an understanding of the proper method for interpreting data and evaluating their performance. The current systematic review aims to present the common bottlenecks for musculoskeletal structures analysis (e.g., small sample size, data inhomogeneity) and the related strategies utilized by different authors. A search was performed using the PUBMED database with the following keywords: deep learning, musculoskeletal system, segmentation. A total of 140 articles published up until February 2022 were obtained and analyzed according to the PRISMA framework in terms of anatomical structures, bioimaging techniques, pre/post-processing operations, training/validation/testing subset creation, network architecture, loss functions, performance indicators and so on. Several common trends emerged from this survey; however, the different methods need to be compared and discussed based on each specific case study (anatomical region, medical imaging acquisition setting, study population, etc.). These findings can be used to guide clinicians (as end users) to better understand the potential benefits and limitations of these tools.

A review of advances in image-guided orthopedic surgery

Orthopedic surgery remains technically demanding due to the complex anatomical structures and cumbersome surgical procedures. The introduction of image-guided orthopedic surgery (IGOS) has significantly decreased the surgical risk and improved the operation results. This review focuses on the application of recent advances in artificial intelligence (AI), deep learning (DL), augmented reality (AR) and robotics in image-guided spine surgery, joint arthroplasty, fracture reduction and bone tumor resection. For the pre-operative stage, key technologies of AI and DL based medical image segmentation, 3D visualization and surgical planning procedures are systematically reviewed. For the intra-operative stage, the development of novel image registration, surgical tool calibration and real-time navigation are reviewed. Furthermore, the combination of the surgical navigation system with AR and robotic technology is also discussed. Finally, the current issues and prospects of the IGOS system are discussed, with the goal of establishing a reference and providing guidance for surgeons, engineers, and researchers involved in the research and development of this area.

A Minority Class Balanced Approach Using the DCNN-LSTM Method to Detect Human Wrist Fracture

The emergency department of hospitals receives a massive number of patients with wrist fracture. For the clinical diagnosis of a suspected fracture, X-ray imaging is the major screening tool. A wrist fracture is a significant global health concern for children, adolescents, and the elderly. A missed diagnosis of wrist fracture on medical imaging can have significant consequences for patients, resulting in delayed treatment and poor functional recovery. Therefore, an intelligent method is needed in the medical department to precisely diagnose wrist fracture via an automated diagnosing tool by considering it a second option for doctors. In this research, a fused model of the deep learning method, a convolutional neural network (CNN), and long short-term memory (LSTM) is proposed to detect wrist fractures from X-ray images. It gives a second option to doctors to diagnose wrist facture using the computer vision method to lessen the number of missed fractures. The dataset acquired from Mendeley comprises 192 wrist X-ray images. In this framework, image pre-processing is applied, then the data augmentation approach is used to solve the class imbalance problem by generating rotated oversamples of images for minority classes during the training process, and pre-processed images and augmented normalized images are fed into a 28-layer dilated CNN (DCNN) to extract deep valuable features. Deep features are then fed to the proposed LSTM network to distinguish wrist fractures from normal ones. The experimental results of the DCNN-LSTM with and without augmentation is compared with other deep learning models. The proposed work is also compared to existing algorithms in terms of accuracy, sensitivity, specificity, precision, the F1-score, and kappa. The results show that the DCNN-LSTM fusion achieves higher accuracy and has high potential for medical applications to use as a second option.

Multi-perspective region-based CNNs for vertebrae labeling in intraoperative long-length images

PURPOSE

Effective aggregation of intraoperative x-ray images that capture the patient anatomy from multiple view-angles has the potential to enable and improve automated image analysis that can be readily performed during surgery. We present multi-perspective region-based neural networks that leverage knowledge of the imaging geometry for automatic vertebrae labeling in Long-Film images - a novel tomographic imaging modality with an extended field-of-view for spine imaging.

METHOD

A multi-perspective network architecture was designed to exploit small view-angle disparities produced by a multi-slot collimator and consolidate information from overlapping image regions. A second network incorporates large view-angle disparities to jointly perform labeling on images from multiple views (viz., AP and lateral). A recurrent module incorporates contextual information and enforce anatomical order for the detected vertebrae. The three modules are combined to form the multi-view multi-slot (MVMS) network for labeling vertebrae using images from all available perspectives. The network was trained on images synthesized from 297 CT images and tested on 50 AP and 50 lateral Long-Film images acquired from 13 cadaveric specimens. Labeling performance of the multi-perspective networks was evaluated with respect to the number of vertebrae appearances and presence of surgical instrumentation.

RESULTS

The MVMS network achieved an F1 score of >96% and an average vertebral localization error of 3.3 mm, with 88.3% labeling accuracy on both AP and lateral images - (15.5% and 35.0% higher than conventional Faster R-CNN on AP and lateral views, respectively). Aggregation of multiple appearances of the same vertebra using the multi-slot network significantly improved the labeling accuracy (p < 0.05). Using the multi-view network, labeling accuracy on the more challenging lateral views was improved to the same level as that of the AP views. The approach demonstrated robustness to the presence of surgical instrumentation, commonly encountered in intraoperative images, and achieved comparable performance in images with and without instrumentation (88.9% vs. 91.2% labeling accuracy).

CONCLUSION

The MVMS network demonstrated effective multi-perspective aggregation, providing means for accurate, automated vertebrae labeling during spine surgery. The algorithms may be generalized to other imaging tasks and modalities that involve multiple views with view-angle disparities (e.g., bi-plane radiography). Predicted labels can help avoid adverse events during surgery (e.g., wrong-level surgery), establish correspondence with labels in preoperative modalities to facilitate image registration, and enable automated measurement of spinal alignment metrics for intraoperative assessment of spinal curvature.

Part Affinity Fields and CoordConv for Detecting Landmarks of Lumbar Vertebrae and Sacrum in X-ray Images

With the prevalence of degenerative diseases due to the increase in the aging population, we have encountered many spine-related disorders. Since the spine is a crucial part of the body, fast and accurate diagnosis is critically important. Generally, clinicians use X-ray images to diagnose the spine, but X-ray images are commonly occluded by the shadows of some bones, making it hard to identify the whole spine. Therefore, recently, various deep-learning-based spinal X-ray image analysis approaches have been proposed to help diagnose the spine. However, these approaches did not consider the characteristics of frequent occlusion in the X-ray image and the properties of the vertebra shape. Therefore, based on the X-ray image properties and vertebra shape, we present a novel landmark detection network specialized in lumbar X-ray images. The proposed network consists of two stages: The first step detects the centers of the lumbar vertebrae and the upper end plate of the first sacral vertebra (S1), and the second step detects the four corner points of each lumbar vertebra and two corner points of S1 from the image obtained in the first step. We used random spine cutout augmentation in the first step to robustify the network against the commonly obscured X-ray images. Furthermore, in the second step, we used CoordConv to make the network recognize the location distribution of landmarks and part affinity fields to understand the morphological features of the vertebrae, resulting in more accurate landmark detection. The proposed network was evaluated using 304 X-ray images, and it achieved 98.02% accuracy in center detection and 8.34% relative distance error in corner detection. This indicates that our network can detect spinal landmarks reliably enough to support radiologists in analyzing the lumbar X-ray images.

X23D-Intraoperative 3D Lumbar Spine Shape Reconstruction Based on Sparse Multi-View X-ray Data

Visual assessment based on intraoperative 2D X-rays remains the predominant aid for intraoperative decision-making, surgical guidance, and error prevention. However, correctly assessing the 3D shape of complex anatomies, such as the spine, based on planar fluoroscopic images remains a challenge even for experienced surgeons. This work proposes a novel deep learning-based method to intraoperatively estimate the 3D shape of patients' lumbar vertebrae directly from sparse, multi-view X-ray data. High-quality and accurate 3D reconstructions were achieved with a learned multi-view stereo machine approach capable of incorporating the X-ray calibration parameters in the neural network. This strategy allowed a priori knowledge of the spinal shape to be acquired while preserving patient specificity and achieving a higher accuracy compared to the state of the art. Our method was trained and evaluated on 17,420 fluoroscopy images that were digitally reconstructed from the public CTSpine1K dataset. As evaluated by unseen data, we achieved an 88% average F1 score and a 71% surface score. Furthermore, by utilizing the calibration parameters of the input X-rays, our method outperformed a counterpart method in the state of the art by 22% in terms of surface score. This increase in accuracy opens new possibilities for surgical navigation and intraoperative decision-making solely based on intraoperative data, especially in surgical applications where the acquisition of 3D image data is not part of the standard clinical workflow.

Realistic C-arm to pCT registration for vertebral localization in spine surgery

Abstract

Spine surgeries are vulnerable to wrong-level surgeries and postoperative complications because of their complex structure. Unavailability of the 3D intraoperative imaging device, low-contrast intraoperative X-ray images, variable clinical and patient conditions, manual analyses, lack of skilled technicians, and human errors increase the chances of wrong-site or wrong-level surgeries. State of the art work refers 3D-2D image registration systems and other medical image processing techniques to address the complications associated with spine surgeries. Intensity-based 3D-2D image registration systems had been widely practiced across various clinical applications. However, these frameworks are limited to specific clinical conditions such as anatomy, dimension of image correspondence, and imaging modalities. Moreover, there are certain prerequisites for these frameworks to function in clinical application, such as dataset requirement, speed of computation, requirement of high-end system configuration, limited capture range, and multiple local maxima. A simple and effective registration framework was designed with a study objective of vertebral level identification and its pose estimation from intraoperative fluoroscopic images by combining intensity-based and iterative control point (ICP)–based 3D-2D registration. A hierarchical multi-stage registration framework was designed that comprises coarse and finer registration. The coarse registration was performed in two stages, i.e., intensity similarity-based spatial localization and source-to-detector localization based on the intervertebral distance correspondence between vertebral centroids in projected and intraoperative X-ray images. Finally, to speed up target localization in the intraoperative application, based on 3D-2D vertebral centroid correspondence, a rigid ICP-based finer registration was performed. The mean projection distance error (mPDE) measurement and visual similarity between projection image at finer registration point and intraoperative X-ray image and surgeons’ feedback were held accountable for the quality assurance of the designed registration framework. The average mPDE after peak signal to noise ratio (PSNR)–based coarse registration was 20.41mm. After the coarse registration in spatial region and source to detector direction, the average mPDE reduced to 12.18mm. On finer ICP-based registration, the mean mPDE was finally reduced to 0.36 mm. The approximate mean time required for the coarse registration, finer registration, and DRR image generation at the final registration point were 10 s, 15 s, and 1.5 min, respectively. The designed registration framework can act as a supporting tool for vertebral level localization and its pose estimation in an intraoperative environment. The framework was designed with the future perspective of intraoperative target localization and its pose estimation irrespective of the target anatomy.

Graphical abstract

A software program for automated compressive vertebral fracture detection on elderly women's lateral chest radiograph: Ofeye 1.0

Background

Because osteoporotic vertebral fracture (OVF) on chest radiographs is commonly missed in radiological reports, we aimed to develop a software program which offers automated detection of compressive vertebral fracture (CVF) on lateral chest radiographs, and which emphasizes CVF detection specificity with a low false positivity rate.

Methods

For model training, we retrieved 3,991 spine radiograph cases and 1,979 chest radiograph cases from 16 sources, with among them in total 1,404 cases had OVF. For model testing, we retrieved 542 chest radiograph cases and 162 spine radiograph cases from four independent clinics, with among them 215 cases had OVF. All cases were female subjects, and except for 31 training data cases which were spine trauma cases, all the remaining cases were post-menopausal women. Image data included DICOM (Digital Imaging and Communications in Medicine) format, hard film scanned PNG (Portable Network Graphics) format, DICOM exported PNG format, and PACS (Picture Archiving and Communication System) downloaded resolution reduced DICOM format. OVF classification included: minimal and mild grades with <20% or ≥20–25% vertebral height loss respectively, moderate grade with ≥25–40% vertebral height loss, severe grade with ≥40%–2/3 vertebral height loss, and collapsed grade with ≥2/3 vertebral height loss. The CVF detection base model was mainly composed of convolution layers that include convolution kernels of different sizes, pooling layers, up-sampling layers, feature merging layers, and residual modules. When the model loss function could not be further decreased with additional training, the model was considered to be optimal and termed ‘base-model 1.0’. A user-friendly interface was also developed, with the synthesized software termed ‘Ofeye 1.0’.

Results

Counting cases and with minimal and mild OVFs included, base-model 1.0 demonstrated a specificity of 97.1%, a sensitivity of 86%, and an accuracy of 93.9% for the 704 testing cases. In total, 33 OVFs in 30 cases had a false negative reading, which constituted a false negative rate of 14.0% (30/215) by counting all OVF cases. Eighteen OVFs in 15 cases had OVFs of ≥ moderate grades missed, which constituted a false negative rate of 7.0% (15/215, i.e., sensitivity 93%) if only counting cases with ≥ moderate grade OVFs missed. False positive reading was recorded in 13 vertebrae in 13 cases (one vertebra in each case), which constituted a false positivity rate of 2.7% (13/489). These vertebrae with false positivity labeling could be readily differentiated from a true OVF by a human reader. The software Ofeye 1.0 allows ‘batch processing’, for example, 100 radiographs can be processed in a single operation. This software can be integrated into hospital PACS, or installed in a standalone personal computer.

Conclusions

A user-friendly software program was developed for CVF detection on elderly women’s lateral chest radiographs. It has an overall low false positivity rate, and for moderate and severe CVFs an acceptably low false negativity rate. The integration of this software into radiological practice is expected to improve osteoporosis management for elderly women.

Identification of osteoporosis based on gene biomarkers using support vector machine

Osteoporosis is a major health concern worldwide. The present study aimed to identify effective biomarkers for osteoporosis detection. In osteoporosis, 559 differentially expressed genes (DEGs) were enriched in PI3K-Akt signaling pathway and Foxo signaling pathway. Weighted gene co-expression network analysis showed that green, pink, and tan modules were clinically significant modules, and that six genes (VEGFA, DDX5, SOD2, HNRNPD, EIF5B, and HSP90B1) were identified as “real” hub genes in the protein–protein interaction network, co-expression network, and 559 DEGs. The sensitivity and specificity of the support vector machine (SVM) for identifying patients with osteoporosis was 100%, with an area under curve of 1 in both training and validation datasets. Our results indicated that the current system using the SVM method could identify patients with osteoporosis.

Biomechanical Morphing for Personalized Fitting of Scoliotic Torso Skeleton Models

The use of patient-specific biomechanical models offers many opportunities in the treatment of adolescent idiopathic scoliosis, such as the design of personalized braces. The first step in the development of these patient-specific models is to fit the geometry of the torso skeleton to the patient’s anatomy. However, existing methods rely on high-quality imaging data. The exposure to radiation of these methods limits their applicability for regular monitoring of patients. We present a method to fit personalized models of the torso skeleton that takes as input biplanar low-dose radiographs. The method morphs a template to fit annotated points on visible portions of the spine, and it relies on a default biomechanical model of the torso for regularization and robust fitting of hardly visible parts of the torso skeleton, such as the rib cage. The proposed method provides an accurate and robust solution to obtain personalized models of the torso skeleton, which can be adopted as part of regular management of scoliosis patients. We have evaluated the method on ten young patients who participated in our study. We have analyzed and compared clinical metrics on the spine and the full torso skeleton, and we have found that the accuracy of the method is at least comparable to other methods that require more demanding imaging methods, while it offers superior robustness to artifacts such as interpenetration of ribs. Normal-dose X-rays were available for one of the patients, and for the other nine we acquired low-dose X-rays, allowing us to validate that the accuracy of the method persisted under less invasive imaging modalities.

Treatment of Fracture of the Calcaneus via Bone Axial X-Ray Image-Based Minimally Invasive Approach

To discuss the values of two bone axial X-ray image-based minimally invasive approach surgeries in the diagnosis and treatment of fracture of the calcaneus, 80 patients diagnosed with fracture of the calcaneus by bone axial X-ray examination were selected and divided equally into the minimally invasive longitudinal approach (MILA) group (40 cases) and the sinus tarsal approach (STA) group (40 cases). Besides, the duration of operation, the incidence of complications, the time-to-start weight training, and the American Orthopaedic Foot and Ankle Society (AOFAS) foot function scoring system between the patients in the two groups were compared. The results showed that the duration of operation and incidence of complications among the patients in the MILA group (42.87 ± 5.12 minutes, 20%) were both superior to those among the patients in the STA group (60.43 ± 7.31 minutes, 32.5%). The time-to-start weight training in the MILA group was 5.2 weeks, which was obviously shorter than that in the STA group (5.7 weeks). The difference in AOFAS scores between the two groups was not significant. The walking pavement score in the MILA group (4.2 ± 0.37 points) was slightly higher than that in the STA group (3.3 ± 0.45 points), and the differences demonstrated statistical meaning (P < 0.05). To sum up, the bone axial X-ray image is an essential examination method of diagnosing fracture of the calcaneus. The two minimally invasive methods both showed good clinical therapeutic effects. The operation of MILA was relatively shorter with fewer complications and is worthy of being promoted as an effective treatment method of fracture of the calcaneus.

Attention Gate based dual-pathway Network for Vertebra Segmentation of X-ray Spine images

Automatic spine and vertebra segmentation from X-ray spine images is a critical and challenging problem in many computer-aid spinal image analysis and disease diagnosis applications. In this paper, a two-stage automatic segmentation framework for spine X-ray images is proposed, which can firstly locate the spine regions (including backbone, sacrum and illum) in the coarse stage and then identify eighteen vertebrae (i.e., cervical vertebra 1, thoracic vertebra 1-12 and lumbar vertebra 1-5) with isolate and clear boundary in the fine stage. A novel Attention Gate based dual-pathway Network (AGNet) composed of context and edge pathways is designed to extract semantic and boundary information for segmentation of both spine and vertebra regions. Multi-scale supervision mechanism is applied to explore comprehensive features and an Edge aware Fusion Mechanism (EFM) is proposed to fuse features extracted from the two pathways. Some other image processing skills, such as centralized backbone clipping, patch cropping and convex hull detection are introduced to further refine the vertebra segmentation results. Experimental validations on spine X-ray images dataset and vertebrae dataset suggest that the proposed AGNet achieves superior performance compared with state-of-the-art segmentation methods, and the coarse-to-fine framework can be implemented in real spinal diagnosis systems.

Hanging protocol optimization of lumbar spine radiographs with machine learning

OBJECTIVES

The purpose of this study was to determine whether machine learning algorithms can be utilized to optimize the hanging protocol of lumbar spine radiographs. Specifically, we explored whether machine learning models can accurately label lumbar spine views/positions, detect hardware, and rotate the lateral views to straighten the image.

METHODS

We identified 1727 patients with 6988 lumbar spine radiographs. The view (anterior-posterior, right oblique, left oblique, left lateral, right lateral, left lumbosacral or right lumbosacral), hardware (present or not present), dynamic position (neutral, flexion, or extension), and correctional rotation of each radiograph were manually documented by a board-certified radiologist. Various output metrics were calculated, including area under the curve (AUC) for the categorical output models (view, hardware, and dynamic position). For non-binary categories, an all-versus-other technique was utilized designating one category as true and all others as false, allowing for a binary evaluation (e.g., AP vs. non-AP or extension vs. non-extension). For correctional rotation, the degree of rotation required to straighten the lateral spine radiograph was documented. The mean absolute difference was calculated between the ground truth and model-predicted value reported in degrees of rotation. Ensembles of the rotation models were created. We evaluated the rotation models on 3 test dataset splits: only 0 rotation, only non-0 rotation, and all cases.

RESULTS

The AUC values for the categorical models ranged from 0.985 to 1.000. For the only 0 rotation data, the ensemble combining the absolute minimum value between the 20- and 60-degree models performed best (mean absolute difference of 0.610). For the non-0 rotation data, the ensemble merging the absolute maximum value between the 40- and 160-degree models performed best (mean absolute difference of 4.801). For the all cases split, the ensemble combining the minimum value of the 20- and 40-degree models performed best (mean absolute difference of 3.083).

CONCLUSION

Machine learning techniques can be successfully implemented to optimize lumbar spine x-ray hanging protocols by accounting for views, hardware, dynamic position, and rotation correction.