Attention LinkNet-152: a novel encoder-decoder based deep learning network for automated spine segmentation

Segmenting the spine from CT images is crucial for diagnosing and treating spine-related conditions but remains challenging due to the spine's complex anatomy and imaging artifacts. This study introduces a novel encoder-decoder-based deep learning approach, named LinkNet-152, tailored for automated spine segmentation. The model integrates a modified EfficientNetB7 encoder with attention modules to enhance feature extraction by focusing on regions of interest. The decoder leverages a modified LinkNet architecture, replacing ResNet34 with the deeper ResNet152 to improve feature extraction and segmentation accuracy. Additionally, gradient sensitivity-based pruning is applied to optimize the model's complexity and computational efficiency. Evaluated on the VerSe 2019 and VerSe 2020 datasets, the proposed model achieves superior performance, with a Dice coefficient of 96.85% and a Jaccard index of 95.37%, outperforming state-of-the-art methods. These results highlight the model's effectiveness in addressing the challenges of spine segmentation and its potential to advance clinical applications.

Automated 3D Cobb Angle Measurement Using U-Net in CT Images of Preoperative Scoliosis Patients

To propose a deep learning framework "SpineCurve-net" for automated measuring the 3D Cobb angles from computed tomography (CT) images of presurgical scoliosis patients. A total of 116 scoliosis patients were analyzed, divided into a training set of 89 patients (average age 32.4 ± 24.5 years) and a validation set of 27 patients (average age 17.3 ± 5.8 years). Vertebral identification and curve fitting were achieved through U-net and NURBS-net and resulted in a Non-Uniform Rational B-Spline (NURBS) curve of the spine. The 3D Cobb angles were measured in two ways: the predicted 3D Cobb angle (PRED-3D-CA), which is the maximum value in the smoothed angle map derived from the NURBS curve, and the 2D mapping Cobb angle (MAP-2D-CA), which is the maximal angle formed by the tangent vectors along the projected 2D spinal curve. The model segmented spinal masks effectively, capturing easily missed vertebral bodies. Spoke kernel filtering distinguished vertebral regions, centralizing spinal curves. The SpineCurve Network method's Cobb angle (PRED-3D-CA and MAP-2D-CA) measurements correlated strongly with the surgeons' annotated Cobb angle (ground truth, GT) based on 2D radiographs, revealing high Pearson correlation coefficients of 0.983 and 0.934, respectively. This paper proposed an automated technique for calculating the 3D Cobb angle in preoperative scoliosis patients, yielding results that are highly correlated with traditional 2D Cobb angle measurements. Given its capacity to accurately represent the three-dimensional nature of spinal deformities, this method shows potential in aiding physicians to develop more precise surgical strategies in upcoming cases.

Knowledge distillation on individual vertebrae segmentation exploiting 3D U-Net

Recent advances in medical imaging have highlighted the critical development of algorithms for individual vertebral segmentation on computed tomography (CT) scans. Essential for diagnostic accuracy and treatment planning in orthopaedics, neurosurgery and oncology, these algorithms face challenges in clinical implementation, including integration into healthcare systems. Consequently, our focus lies in exploring the application of knowledge distillation (KD) methods to train shallower networks capable of efficiently segmenting vertebrae in CT scans. This approach aims to reduce segmentation time, enhance suitability for emergency cases, and optimize computational and memory resource efficiency. Building upon prior research in the field, a two-step segmentation approach was employed. Firstly, the spine's location was determined by predicting a heatmap, indicating the probability of each voxel belonging to the spine. Subsequently, an iterative segmentation of vertebrae was performed from the top to the bottom of the CT volume over the located spine, using a memory instance to record the already segmented vertebrae. KD methods were implemented by training a teacher network with performance similar to that found in the literature, and this knowledge was distilled to a shallower network (student). Two KD methods were applied: (1) using the soft outputs of both networks and (2) matching logits. Two publicly available datasets, comprising 319 CT scans from 300 patients and a total of 611 cervical, 2387 thoracic, and 1507 lumbar vertebrae, were used. To ensure dataset balance and robustness, effective data augmentation methods were applied, including cleaning the memory instance to replicate the first vertebra segmentation. The teacher network achieved an average Dice similarity coefficient (DSC) of 88.22% and a Hausdorff distance (HD) of 7.71 mm, showcasing performance similar to other approaches in the literature. Through knowledge distillation from the teacher network, the student network's performance improved, with an average DSC increasing from 75.78% to 84.70% and an HD decreasing from 15.17 mm to 8.08 mm. Compared to other methods, our teacher network exhibited up to 99.09% fewer parameters, 90.02% faster inference time, 88.46% shorter total segmentation time, and 89.36% less associated carbon (CO2) emission rate. Regarding our student network, it featured 75.00% fewer parameters than our teacher, resulting in a 36.15% reduction in inference time, a 33.33% decrease in total segmentation time, and a 42.96% reduction in CO2 emissions. This study marks the first exploration of applying KD to the problem of individual vertebrae segmentation in CT, demonstrating the feasibility of achieving comparable performance to existing methods using smaller neural networks.

VertXNet: an ensemble method for vertebral body segmentation and identification from cervical and lumbar spinal X-rays

Accurate annotation of vertebral bodies is crucial for automating the analysis of spinal X-ray images. However, manual annotation of these structures is a laborious and costly process due to their complex nature, including small sizes and varying shapes. To address this challenge and expedite the annotation process, we propose an ensemble pipeline called VertXNet. This pipeline currently combines two segmentation mechanisms, semantic segmentation using U-Net, and instance segmentation using Mask R-CNN, to automatically segment and label vertebral bodies in lateral cervical and lumbar spinal X-ray images. VertXNet enhances its effectiveness by adopting a rule-based strategy (termed the ensemble rule) for effectively combining segmentation outcomes from U-Net and Mask R-CNN. It determines vertebral body labels by recognizing specific reference vertebral instances, such as cervical vertebra 2 ('C2') in cervical spine X-rays and sacral vertebra 1 ('S1') in lumbar spine X-rays. Those references are commonly relatively easy to identify at the edge of the spine. To assess the performance of our proposed pipeline, we conducted evaluations on three spinal X-ray datasets, including two in-house datasets and one publicly available dataset. The ground truth annotations were provided by radiologists for comparison. Our experimental results have shown that the proposed pipeline outperformed two state-of-the-art (SOTA) segmentation models on our test dataset with a mean Dice of 0.90, vs. a mean Dice of 0.73 for Mask R-CNN and 0.72 for U-Net. We also demonstrated that VertXNet is a modular pipeline that enables using other SOTA model, like nnU-Net to further improve its performance. Furthermore, to evaluate the generalization ability of VertXNet on spinal X-rays, we directly tested the pre-trained pipeline on two additional datasets. A consistently strong performance was observed, with mean Dice coefficients of 0.89 and 0.88, respectively. In summary, VertXNet demonstrated significantly improved performance in vertebral body segmentation and labeling for spinal X-ray imaging. Its robustness and generalization were presented through the evaluation of both in-house clinical trial data and publicly available datasets.

A spinal cord neuroprosthesis for locomotor deficits due to Parkinson's disease

People with late-stage Parkinson's disease (PD) often suffer from debilitating locomotor deficits that are resistant to currently available therapies. To alleviate these deficits, we developed a neuroprosthesis operating in closed loop that targets the dorsal root entry zones innervating lumbosacral segments to reproduce the natural spatiotemporal activation of the lumbosacral spinal cord during walking. We first developed this neuroprosthesis in a non-human primate model that replicates locomotor deficits due to PD. This neuroprosthesis not only alleviated locomotor deficits but also restored skilled walking in this model. We then implanted the neuroprosthesis in a 62-year-old male with a 30-year history of PD who presented with severe gait impairments and frequent falls that were medically refractory to currently available therapies. We found that the neuroprosthesis interacted synergistically with deep brain stimulation of the subthalamic nucleus and dopaminergic replacement therapies to alleviate asymmetry and promote longer steps, improve balance and reduce freezing of gait. This neuroprosthesis opens new perspectives to reduce the severity of locomotor deficits in people with PD.

Artificial Intelligence in Neuroradiology: A Review of Current Topics and Competition Challenges

There is an expanding body of literature that describes the application of deep learning and other machine learning and artificial intelligence methods with potential relevance to neuroradiology practice. In this article, we performed a literature review to identify recent developments on the topics of artificial intelligence in neuroradiology, with particular emphasis on large datasets and large-scale algorithm assessments, such as those used in imaging AI competition challenges. Numerous applications relevant to ischemic stroke, intracranial hemorrhage, brain tumors, demyelinating disease, and neurodegenerative/neurocognitive disorders were discussed. The potential applications of these methods to spinal fractures, scoliosis grading, head and neck oncology, and vascular imaging were also reviewed. The AI applications examined perform a variety of tasks, including localization, segmentation, longitudinal monitoring, diagnostic classification, and prognostication. While research on this topic is ongoing, several applications have been cleared for clinical use and have the potential to augment the accuracy or efficiency of neuroradiologists.

An Automated Deep Learning Approach for Spine Segmentation and Vertebrae Recognition Using Computed Tomography Images

Spine image analysis is based on the accurate segmentation and vertebrae recognition of the spine. Several deep learning models have been proposed for spine segmentation and vertebrae recognition, but they are very computationally demanding. In this research, a novel deep learning model is introduced for spine segmentation and vertebrae recognition using CT images. The proposed model works in two steps: (1) A cascaded hierarchical atrous spatial pyramid pooling residual attention U-Net (CHASPPRAU-Net), which is a modified version of U-Net, is used for the segmentation of the spine. Cascaded spatial pyramid pooling layers, along with residual blocks, are used for feature extraction, while the attention module is used for focusing on regions of interest. (2) A 3D mobile residual U-Net (MRU-Net) is used for vertebrae recognition. MobileNetv2 includes residual and attention modules to accurately extract features from the axial, sagittal, and coronal views of 3D spine images. The features from these three views are concatenated to form a 3D feature map. After that, a 3D deep learning model is used for vertebrae recognition. The VerSe 20 and VerSe 19 datasets were used to validate the proposed model. The model achieved more accurate results in spine segmentation and vertebrae recognition than the state-of-the-art methods.

A kinematics-based method for creating deformed patient-derived head and neck CT scans. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38083025 DOI: 10.1109/embc40787.2023.10340930] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

CT scans of the head and neck have multiple clinical uses, and simulating deformation of these CT scans allows for predicting patient motion and data augmentation for machine-learning methods. Current methods for creating patient-derived deformed CT scans require multiple scans or use unrealistic head and neck motion. This paper describes the CTHeadDeformation software package which allows for realistic synthetic deformation of head and neck CT scans for small amounts of motion. CTHeadDeformation is a python-based package that uses a kinematics-based approach using anatomical landmarks, and rigid/non-rigid registration to create a realistic patient-derived deformed CT scan. CTHeadDeformation is also designed for simple clinical implementation. The CTHeadDeformation software package was demonstrated on a head and neck CT scan of one patient. The CT scan was deformed in the anterior-posterior, superior-inferior, and left-right directions. Internal organ motion and more complex combination motions were also simulated. The results showed the patient's CT scan was able to be deformed in a way that preserved the shape and location of the anatomy.Clinical Relevance- This method allows for the realistic simulation of head and neck motion in CT scans. Clinical applications including simulating how patient motion affects radiation therapy treatment effectiveness. The CTHeadDeformation software can also be used to train machine-learning networks that are robust to patient motion, or to generate ground truth images for imaging or segmentation grand challenges.

An intelligent non-invasive system for automated diagnosis of anemia exploiting a novel dataset

Anemia is a condition in which the oxygen-carrying capacity of red blood cells is insufficient to meet the body's physiological needs. It affects billions of people worldwide. An early diagnosis of this disease could prevent the advancement of other disorders. Traditional methods used to detect anemia consist of venipuncture, which requires a patient to frequently undergo laboratory tests. Therefore, anemia diagnosis using noninvasive and cost-effective methods is an open challenge. The pallor of the fingertips, palms, nail beds, and eye conjunctiva can be observed to establish whether a patient suffers from anemia. This article addresses the above challenges by presenting a novel intelligent system, based on machine learning, that supports the automated diagnosis of anemia. This system is innovative from different points of view. Specifically, it has been trained on a dataset that contains eye conjunctiva photos of Indian and Italian patients. This dataset, which was created using a very strict experimental set, is now made available to the Scientific Community. Moreover, compared to previous systems in the literature, the proposed system uses a low-cost device, which makes it suitable for widespread use. The performance of the learning algorithms utilizing two different areas of the mucous membrane of the eye is discussed. In particular, the RUSBoost algorithm, when appropriately trained on palpebral conjunctiva images, shows good performance in classifying anemic and nonanemic patients. The results are very robust, even when considering different ethnicities.

Augmenting the Cobb angle: Three-dimensional analysis of whole spine shapes using Bézier curves

BACKGROUND AND OBJECTIVE

The identification and classification of pathological spinal deformities poses a major challenge to any diagnostician. First, available medical images are usually two-dimensional projections, obscuring elaborated spatial information. Second, several measurement techniques with different thresholds for certain clinical syndromes make it difficult to classify measured results. Here, a method is presented to augment and standardize the analysis of spinal shapes in three dimensions.

METHODS

Regarding the first limitation, (semi-)automatic, three-dimensional segmentation techniques of medical images have already been developed. To overcome the second, we propose here a representation of the whole spine by a Bézier curve using the vertebral centers as control points. After normalization, a differential-geometric approach yields information on curvature and torsion at each spinal level as well as in between.

RESULTS

Based on literature data and multi-body simulations, we show how these quantities alter with individual posture and during motion. Robustness with respect to missing data is investigated. Approaches towards the identification of spinal disorders are motivated.

CONCLUSION

Our results emphasize the need for individualizable identification and classification of spinal deformities and give an outlook on how it might be achieved. The presented methodology constitutes the first fully three-dimensional analysis of spinal shapes, i.e. without the requirement of certain physiological planes (e.g. the sagittal plane) or landmarks (e.g. the apex vertebra).

Unsupervised Domain Adaptation for Vertebrae Detection and Identification in 3D CT Volumes Using a Domain Sanity Loss

A variety of medical computer vision applications analyze 2D slices of computed tomography (CT) scans, whereas axial slices from the body trunk region are usually identified based on their relative position to the spine. A limitation of such systems is that either the correct slices must be extracted manually or labels of the vertebrae are required for each CT scan to develop an automated extraction system. In this paper, we propose an unsupervised domain adaptation (UDA) approach for vertebrae detection and identification based on a novel Domain Sanity Loss (DSL) function. With UDA the model’s knowledge learned on a publicly available (source) data set can be transferred to the target domain without using target labels, where the target domain is defined by the specific setup (CT modality, study protocols, applied pre- and processing) at the point of use (e.g., a specific clinic with its specific CT study protocols). With our approach, a model is trained on the source and target data set in parallel. The model optimizes a supervised loss for labeled samples from the source domain and the DSL loss function based on domain-specific “sanity checks” for samples from the unlabeled target domain. Without using labels from the target domain, we are able to identify vertebra centroids with an accuracy of 72.8%. By adding only ten target labels during training the accuracy increases to 89.2%, which is on par with the current state-of-the-art for full supervised learning, while using about 20 times less labels. Thus, our model can be used to extract 2D slices from 3D CT scans on arbitrary data sets fully automatically without requiring an extensive labeling effort, contributing to the clinical adoption of medical imaging by hospitals.

A Fusion Biopsy Framework for Prostate Cancer Based on Deformable Superellipses and nnU-Net

In prostate cancer, fusion biopsy, which couples magnetic resonance imaging (MRI) with transrectal ultrasound (TRUS), poses the basis for targeted biopsy by allowing the comparison of information coming from both imaging modalities at the same time. Compared with the standard clinical procedure, it provides a less invasive option for the patients and increases the likelihood of sampling cancerous tissue regions for the subsequent pathology analyses. As a prerequisite to image fusion, segmentation must be achieved from both MRI and TRUS domains. The automatic contour delineation of the prostate gland from TRUS images is a challenging task due to several factors including unclear boundaries, speckle noise, and the variety of prostate anatomical shapes. Automatic methodologies, such as those based on deep learning, require a huge quantity of training data to achieve satisfactory results. In this paper, the authors propose a novel optimization formulation to find the best superellipse, a deformable model that can accurately represent the prostate shape. The advantage of the proposed approach is that it does not require extensive annotations, and can be used independently of the specific transducer employed during prostate biopsies. Moreover, in order to show the clinical applicability of the method, this study also presents a module for the automatic segmentation of the prostate gland from MRI, exploiting the nnU-Net framework. Lastly, segmented contours from both imaging domains are fused with a customized registration algorithm in order to create a tool that can help the physician to perform a targeted prostate biopsy by interacting with the graphical user interface.

Focal Dice Loss-Based V-Net for Liver Segments Classification

Liver segmentation is a crucial step in surgical planning from computed tomography scans. The possibility to obtain a precise delineation of the liver boundaries with the exploitation of automatic techniques can help the radiologists, reducing the annotation time and providing more objective and repeatable results. Subsequent phases typically involve liver vessels’ segmentation and liver segments’ classification. It is especially important to recognize different segments, since each has its own vascularization, and so, hepatic segmentectomies can be performed during surgery, avoiding the unnecessary removal of healthy liver parenchyma. In this work, we focused on the liver segments’ classification task. We exploited a 2.5D Convolutional Neural Network (CNN), namely V-Net, trained with the multi-class focal Dice loss. The idea of focal loss was originally thought as the cross-entropy loss function, aiming at focusing on “hard” samples, avoiding the gradient being overwhelmed by a large number of falsenegatives. In this paper, we introduce two novel focal Dice formulations, one based on the concept of individual voxel’s probability and another related to the Dice formulation for sets. By applying multi-class focal Dice loss to the aforementioned task, we were able to obtain respectable results, with an average Dice coefficient among classes of 82.91%. Moreover, the knowledge of anatomic segments’ configurations allowed the application of a set of rules during the post-processing phase, slightly improving the final segmentation results, obtaining an average Dice coefficient of 83.38%. The average accuracy was close to 99%. The best model turned out to be the one with the focal Dice formulation based on sets. We conducted the Wilcoxon signed-rank test to check if these results were statistically significant, confirming their relevance.

Evaluation of Vision-Based Hand Tool Tracking Methods for Quality Assessment and Training in Human-Centered Industry 4.0

Smart industrial workstations for the training and evaluation of workers are an innovative approach to face the problems of manufacturing quality assessment and fast training. However, such products do not implement algorithms that are able to accurately track the pose of a hand tool that might also be partially occluded by the operator’s hands. In the best case, the already proposed systems roughly track the position of the operator’s hand center assuming that a certain task has been performed if the hand center position is close enough to a specified area. The problem of the pose estimation of 3D objects, including the hand tool, is an open and debated problem. The methods that lead to high accuracies are time consuming and require a 3D model of the object to detect, which is why they cannot be adopted for a real-time training system. The rise in deep learning has stimulated the search for better-performing vision-based solutions. Nevertheless, the problem of hand tool pose estimation for assembly and training procedures appears to not have been extensively investigated. In this study, four different vision-based methods based, respectively, on ArUco markers, OpenPose, Azure Kinect Body Tracking and the YOLO network have been proposed in order to estimate the position of a specific point of interest of the tool that has to be tracked in real-time during an assembly or maintenance procedure. The proposed approaches have been tested on a real scenario with four users handling a power drill simulating three different conditions during an assembly procedure. The performance of the methods has been evaluated and compared with the HTC Vive tracking system as a benchmark. Then, the advantages and drawbacks in terms of the accuracy and invasiveness of the method have been discussed. The authors can state that OpenPose is the most robust proposal arising from the study. The authors will investigate the OpenPose performance in more depth in further studies. The framework appears to be very interesting regarding its integration into a smart workstation for quality assessment and training.

Computer Vision and Machine Learning for Tuna and Salmon Meat Classification

Aquatic products are popular among consumers, and their visual quality used to be detected manually for freshness assessment. This paper presents a solution to inspect tuna and salmon meat from digital images. The solution proposes hardware and a protocol for preprocessing images and extracting parameters from the RGB, HSV, HSI, and L*a*b* spaces of the collected images to generate the datasets. Experiments are performed using machine learning classification methods. We evaluated the AutoML models to classify the freshness levels of tuna and salmon samples through the metrics of: accuracy, receiver operating characteristic curve, precision, recall, f1-score, and confusion matrix (CM). The ensembles generated by AutoML, for both tuna and salmon, reached 100% in all metrics, noting that the method of inspection of fish freshness from image collection, through preprocessing and extraction/fitting of features showed exceptional results when datasets were subjected to the machine learning models. We emphasize how easy it is to use the proposed solution in different contexts. Computer vision and machine learning, as a nondestructive method, were viable for external quality detection of tuna and salmon meat products through its efficiency, objectiveness, consistency, and reliability due to the experiments’ high accuracy.

Lung Segmentation and Characterization in COVID-19 Patients for Assessing Pulmonary Thromboembolism: An Approach Based on Deep Learning and Radiomics

The COVID-19 pandemic is inevitably changing the world in a dramatic way, and the role of computed tomography (CT) scans can be pivotal for the prognosis of COVID-19 patients. Since the start of the pandemic, great care has been given to the relationship between interstitial pneumonia caused by the infection and the onset of thromboembolic phenomena. In this preliminary study, we collected n = 20 CT scans from the Polyclinic of Bari, all from patients positive with COVID-19, nine of which developed pulmonary thromboembolism (PTE). For eight CT scans, we obtained masks of the lesions caused by the infection, annotated by expert radiologists; whereas for the other four CT scans, we obtained masks of the lungs (including both healthy parenchyma and lesions). We developed a deep learning-based segmentation model that utilizes convolutional neural networks (CNNs) in order to accurately segment the lung and lesions. By considering the images from publicly available datasets, we also realized a training set composed of 32 CT scans and a validation set of 10 CT scans. The results obtained from the segmentation task are promising, allowing to reach a Dice coefficient higher than 97%, posing the basis for analysis concerning the assessment of PTE onset. We characterized the segmented region in order to individuate radiomic features that can be useful for the prognosis of PTE. Out of 919 extracted radiomic features, we found that 109 present different distributions according to the Mann–Whitney U test with corrected p-values less than 0.01. Lastly, nine uncorrelated features were retained that can be exploited to realize a prognostic signature.