RACE-Net: A Recurrent Neural Network for Biomedical Image Segmentation

Multiresolution cascaded attention U-Net for localization and segmentation of optic disc and fovea in fundus images

Identification of retinal diseases in automated screening methods, such as those used in clinical settings or computer-aided diagnosis, usually depends on the localization and segmentation of the Optic Disc (OD) and fovea. However, this task is difficult since these anatomical features have irregular spatial, texture, and shape characteristics, limited sample sizes, and domain shifts due to different data distributions across datasets. This study proposes a novel Multiresolution Cascaded Attention U-Net (MCAU-Net) model that addresses these problems by optimally balancing receptive field size and computational efficiency. The MCAU-Net utilizes two skip connections to accurately localize and segment the OD and fovea in fundus images. We incorporated a Multiresolution Wavelet Pooling Module (MWPM) into the CNN at each stage of U-Net input to compensate for spatial information loss. Additionally, we integrated a cascaded connection of the spatial and channel attentions as a skip connection in MCAU-Net to concentrate precisely on the target object and improve model convergence for segmenting and localizing OD and fovea centers. The proposed model has a low parameter count of 0.8 million, improving computational efficiency and reducing the risk of overfitting. For OD segmentation, the MCAU-Net achieves high IoU values of 0.9771, 0.945, and 0.946 for the DRISHTI-GS, DRIONS-DB, and IDRiD datasets, respectively, outperforming previous results for all three datasets. For the IDRiD dataset, the MCAU-Net locates the OD center with an Euclidean Distance (ED) of 16.90 pixels and the fovea center with an ED of 33.45 pixels, demonstrating its effectiveness in overcoming the common limitations of state-of-the-art methods.

Towards more precise automatic analysis: a systematic review of deep learning-based multi-organ segmentation

Accurate segmentation of multiple organs in the head, neck, chest, and abdomen from medical images is an essential step in computer-aided diagnosis, surgical navigation, and radiation therapy. In the past few years, with a data-driven feature extraction approach and end-to-end training, automatic deep learning-based multi-organ segmentation methods have far outperformed traditional methods and become a new research topic. This review systematically summarizes the latest research in this field. We searched Google Scholar for papers published from January 1, 2016 to December 31, 2023, using keywords "multi-organ segmentation" and "deep learning", resulting in 327 papers. We followed the PRISMA guidelines for paper selection, and 195 studies were deemed to be within the scope of this review. We summarized the two main aspects involved in multi-organ segmentation: datasets and methods. Regarding datasets, we provided an overview of existing public datasets and conducted an in-depth analysis. Concerning methods, we categorized existing approaches into three major classes: fully supervised, weakly supervised and semi-supervised, based on whether they require complete label information. We summarized the achievements of these methods in terms of segmentation accuracy. In the discussion and conclusion section, we outlined and summarized the current trends in multi-organ segmentation.

Compensation of small data with large filters for accurate liver vessel segmentation from contrast-enhanced CT images

BACKGROUND

Segmenting liver vessels from contrast-enhanced computed tomography images is essential for diagnosing liver diseases, planning surgeries and delivering radiotherapy. Nevertheless, identifying vessels is a challenging task due to the tiny cross-sectional areas occupied by vessels, which has posed great challenges for vessel segmentation, such as limited features to be learned and difficult to construct high-quality as well as large-volume data.

METHODS

We present an approach that only requires a few labeled vessels but delivers significantly improved results. Our model starts with vessel enhancement by fading out liver intensity and generates candidate vessels by a classifier fed with a large number of image filters. Afterwards, the initial segmentation is refined using Markov random fields.

RESULTS

In experiments on the well-known dataset 3D-IRCADb, the averaged Dice coefficient is lifted to 0.63, and the mean sensitivity is increased to 0.71. These results are significantly better than those obtained from existing machine-learning approaches and comparable to those generated from deep-learning models.

CONCLUSION

Sophisticated integration of a large number of filters is able to pinpoint effective features from liver images that are sufficient to distinguish vessels from other liver tissues under a scarcity of large-volume labeled data. The study can shed light on medical image segmentation, especially for those without sufficient data.

Light-M: An efficient lightweight medical image segmentation framework for resource-constrained IoMT

The Internet of Medical Things (IoMT) is being incorporated into current healthcare systems. This technology intends to connect patients, IoMT devices, and hospitals over mobile networks, allowing for more secure, quick, and convenient health monitoring and intelligent healthcare services. However, existing intelligent healthcare applications typically rely on large-scale AI models, and standard IoMT devices have significant resource constraints. To alleviate this paradox, in this paper, we propose a Knowledge Distillation (KD)-based IoMT end-edge-cloud orchestrated architecture for medical image segmentation tasks, called Light-M, aiming to deploy a lightweight medical model in resource-constrained IoMT devices. Specifically, Light-M trains a large teacher model in the cloud server and employs computation in local nodes through imitation of the performance of the teacher model using knowledge distillation. Light-M contains two KD strategies: (1) active exploration and passive transfer (AEPT) and (2) self-attention-based inter-class feature variation (AIFV) distillation for the medical image segmentation task. The AEPT encourages the student model to learn undiscovered knowledge/features of the teacher model without additional feature layers, aiming to explore new features and outperform the teacher. To improve the distinguishability of the student for different classes, the student learns the self-attention-based feature variation (AIFV) between classes. Since the proposed AEPT and AIFV only appear in the training process, our framework does not involve any additional computation burden for a student model during the segmentation task deployment. Extensive experiments on cardiac images and public real-scene datasets demonstrate that our approach improves student model learning representations and outperforms state-of-the-art methods by combining two knowledge distillation strategies. Moreover, when deployed on the IoT device, the distilled student model takes only 29.6 ms for one sample at the inference step.

Automated segmentation of 3D cine cardiovascular magnetic resonance imaging

Introduction

As the life expectancy of children with congenital heart disease (CHD) is rapidly increasing and the adult population with CHD is growing, there is an unmet need to improve clinical workflow and efficiency of analysis. Cardiovascular magnetic resonance (CMR) is a noninvasive imaging modality for monitoring patients with CHD. CMR exam is based on multiple breath-hold 2-dimensional (2D) cine acquisitions that should be precisely prescribed and is expert and institution dependent. Moreover, 2D cine images have relatively thick slices, which does not allow for isotropic delineation of ventricular structures. Thus, development of an isotropic 3D cine acquisition and automatic segmentation method is worthwhile to make CMR workflow straightforward and efficient, as the present work aims to establish.

Methods

Ninety-nine patients with many types of CHD were imaged using a non-angulated 3D cine CMR sequence covering the whole-heart and great vessels. Automatic supervised and semi-supervised deep-learning-based methods were developed for whole-heart segmentation of 3D cine images to separately delineate the cardiac structures, including both atria, both ventricles, aorta, pulmonary arteries, and superior and inferior vena cavae. The segmentation results derived from the two methods were compared with the manual segmentation in terms of Dice score, a degree of overlap agreement, and atrial and ventricular volume measurements.

Results

The semi-supervised method resulted in a better overlap agreement with the manual segmentation than the supervised method for all 8 structures (Dice score 83.23 ± 16.76% vs. 77.98 ± 19.64%; P-value ≤0.001). The mean difference error in atrial and ventricular volumetric measurements between manual segmentation and semi-supervised method was lower (bias ≤ 5.2 ml) than the supervised method (bias ≤ 10.1 ml).

Discussion

The proposed semi-supervised method is capable of cardiac segmentation and chamber volume quantification in a CHD population with wide anatomical variability. It accurately delineates the heart chambers and great vessels and can be used to accurately calculate ventricular and atrial volumes throughout the cardiac cycle. Such a segmentation method can reduce inter- and intra- observer variability and make CMR exams more standardized and efficient.

A residual connection enabled deep neural network model for optic disk and optic cup segmentation for glaucoma diagnosis

Glaucoma diagnosis at an early stage is vital for the timely initiation of its treatment for and preventing possible vision loss. For glaucoma diagnosis, an accurate estimation of the cup-to-disk ratio (CDR) is required. The current automatic CDR computation techniques attribute lower accuracy and higher complexity, which are important considerations for diagnostics system design to be used for such critical diagnoses. The current methods involve a deeper deep learning model, comprising a large number of parameters, which results in higher system complexity and training/testing time. To address these challenges, this paper proposes a Residual Connection (non-identity)-based Deep Neural Network (RC-DNN), which is based on non-identity residual connectivity for joint optic disk (OD) and optic cup (OC) detection. The proposed model is emboldened by efficient residual connectivity, which is beneficial in several ways. First, the model is efficient and can perform simultaneous segmentation of the OC and OD. Second, the efficient residual information flow permeates the vanishing gradient problem which results in faster converges of the model. Third, feature inspiration empowers the network to perform the segmentation with only a few network layers. We performed a comprehensive performance evaluation of the developed model based on its training in RIM-ONE and DRISHTIGS databases. For OC segmentation, for the images (test set) from {DRISHTI-GS and RIM-ONE} datasets, our proposed model achieves the dice coefficient, Jaccard coefficient, sensitivity, specificity, and balanced accuracy of {92.62, 86.52}, {86.87, 77.54}, {94.21, 95.36}, {99.83, 99.639}, and {94.2, 98.9}, respectively. These experimental results indicate that the developed model provides significant performance enhancement for joint OC and OD segmentation. Additionally, the reduced computational complexity based on reduced model parameters and higher segmentation accuracy provides the additional features of efficacy, robustness, and reliability of the developed model. These attributes of the developed model advocate for its deployment of population-scale glaucoma screening programs.

Lens structure segmentation from AS-OCT images via shape-based learning

BACKGROUND AND OBJECTIVES

The lens is one of the important refractive media in the eyeball. Abnormality of the nucleus or cortex in the lens can lead to ocular disorders such as cataracts and presbyopia. To achieve an accurate diagnosis, segmentation of these ocular structures from anterior segment optical coherence tomography (AS-OCT) is essential. However, weak-contrast boundaries of the object in the images present a challenge for accurate segmentation. The state-of-the-art (SOTA) methods, such as U-Net, treat segmentation as a binary classification of pixels, which cannot handle pixels on weak-contrast boundaries well.

METHODS

In this paper, we propose to incorporate shape prior into a deep learning framework for accurate nucleus and cortex segmentation. Specifically, we propose to learn a level set function, whose zero-level set represents the object boundary, through a convolutional neural network. Moreover, we design a novel shape-based loss function, where the shape prior knowledge can be naturally embedded into the learning procedure, leading to improvement in performance. We collect a high-quality AS-OCT image dataset with precise annotations to train our model.

RESULTS

Abundant experiments are conducted to verify the effectiveness of the proposed framework and the novel shape-based loss. The mean Intersection over Unions (MIoUs) of the proposed method for lens nucleus and cortex segmentation are 0.946 and 0.957, and the mean Euclidean Distance (MED) measure, which can reflect the accuracy of the segmentation boundary, are 6.746 and 2.045 pixels. In addition, the proposed shape-based loss improves the SOTA models on the nucleus and cortex segmentation tasks by an average of 0.0156 and 0.0078 in the MIoU metric and 1.394 and 0.134 pixels in the MED metric.

CONCLUSION

We transform the segmentation from a classification task to a regression task by making the model learn the level set function, and embed shape information in deep learning by designing loss functions. This allows the proposed method to be more efficient in the segmentation of the object with weak-contrast boundaries.

CONCISE ABSTRACT

We propose to incorporate shape priors into a deep learning framework for accurate nucleus and cortex segmentation from AS-OCT images. Specifically, we propose to learn a level set function, where the zero-level set represents the boundary of the target. Meanwhile, we design a novel shape-based loss function in which additional convex shape prior can be embedded in the learning process, leading to an improvement in performance. The IOUs for nucleus and cortex segmentation are 0.946 and 0.957, while the MED that reflects the accuracy of the boundary are 6.746 and 2.045 pixels. The proposed shape-based loss improves the SOTA model for nucleus and cortex segmentation by an average of 0.0156 and 0.0078 in IOU, and 1.394 and 0.134 pixels in MED. We transform segmentation from classification to regression by making the model learn a level set function, resulting in improved performance at the boundary with weak contrast.

DMFF-Net: A dual encoding multiscale feature fusion network for ovarian tumor segmentation

Ovarian cancer is a serious threat to the female reproductive system. Precise segmentation of the tumor area helps the doctors to further diagnose the disease. Automatic segmentation techniques for abstracting high-quality features from images through autonomous learning of model have become a hot research topic nowadays. However, the existing methods still have the problem of poor segmentation of ovarian tumor details. To cope with this problem, a dual encoding based multiscale feature fusion network (DMFF-Net) is proposed for ovarian tumor segmentation. Firstly, a dual encoding method is proposed to extract diverse features. These two encoding paths are composed of residual blocks and single dense aggregation blocks, respectively. Secondly, a multiscale feature fusion block is proposed to generate more advanced features. This block constructs feature fusion between two encoding paths to alleviate the feature loss during deep extraction and further increase the information content of the features. Finally, coordinate attention is added to the decoding stage after the feature concatenation, which enables the decoding stage to capture the valid information accurately. The test results show that the proposed method outperforms existing medical image segmentation algorithms for segmenting lesion details. Moreover, the proposed method also performs well in two other segmentation tasks.

Variational Autoencoders-BasedSelf-Learning Model for Tumor Identification and Impact Analysis from 2-D MRI Images

Over the past few years, a tremendous change has occurred in computer-aided diagnosis (CAD) technology. The evolution of numerous medical imaging techniques has enhanced the accuracy of the preliminary analysis of several diseases. Magnetic resonance imaging (MRI) is a prevalent technology extensively used in evaluating the progress of the spread of malignant tissues or abnormalities in the human body. This article aims to automate a computationally efficient mechanism that can accurately identify the tumor from MRI images and can analyze the impact of the tumor. The proposed model is robust enough to classify the tumors with minimal training data. The generative variational autoencoder models are efficient in reconstructing the images identical to the original images, which are used in adequately training the model. The proposed self-learning algorithm can learn from the insights from the autogenerated images and the original images. Incorporating long short-term memory (LSTM) is faster processing of the high dimensional imaging data, making the radiologist's task and the practitioners more comfortable assessing the tumor's progress. Self-learning models need comparatively less data for the training, and the models are more resource efficient than the various state-of-art models. The efficiency of the proposed model has been assessed using various benchmark metrics, and the obtained results have exhibited an accuracy of 89.7%. The analysis of the progress of tumor growth is presented in the current study. The obtained accuracy is not pleasing in the healthcare domain, yet the model is reasonably fair in dealing with a smaller size dataset by making use of an image generation mechanism. The study would outline the role of an autoencoder in self-learning models. Future technologies may include sturdy feature engineering models and optimized activation functions that would yield a better result.

Learned iterative segmentation of highly variable anatomy from limited data: Applications to whole heart segmentation for congenital heart disease

Training deep learning models that segment an image in one step typically requires a large collection of manually annotated images that captures the anatomical variability in a cohort. This poses challenges when anatomical variability is extreme but training data is limited, as when segmenting cardiac structures in patients with congenital heart disease (CHD). In this paper, we propose an iterative segmentation model and show that it can be accurately learned from a small dataset. Implemented as a recurrent neural network, the model evolves a segmentation over multiple steps, from a single user click until reaching an automatically determined stopping point. We develop a novel loss function that evaluates the entire sequence of output segmentations, and use it to learn model parameters. Segmentations evolve predictably according to growth dynamics encapsulated by training data, which consists of images, partially completed segmentations, and the recommended next step. The user can easily refine the final segmentation by examining those that are earlier or later in the output sequence. Using a dataset of 3D cardiac MR scans from patients with a wide range of CHD types, we show that our iterative model offers better generalization to patients with the most severe heart malformations.

Which Color Channel Is Better for Diagnosing Retinal Diseases Automatically in Color Fundus Photographs?

Color fundus photographs are the most common type of image used for automatic diagnosis of retinal diseases and abnormalities. As all color photographs, these images contain information about three primary colors, i.e., red, green, and blue, in three separate color channels. This work aims to understand the impact of each channel in the automatic diagnosis of retinal diseases and abnormalities. To this end, the existing works are surveyed extensively to explore which color channel is used most commonly for automatically detecting four leading causes of blindness and one retinal abnormality along with segmenting three retinal landmarks. From this survey, it is clear that all channels together are typically used for neural network-based systems, whereas for non-neural network-based systems, the green channel is most commonly used. However, from the previous works, no conclusion can be drawn regarding the importance of the different channels. Therefore, systematic experiments are conducted to analyse this. A well-known U-shaped deep neural network (U-Net) is used to investigate which color channel is best for segmenting one retinal abnormality and three retinal landmarks.

Graph deep network for optic disc and optic cup segmentation for glaucoma disease using retinal imaging

The fundus imaging method of eye screening detects eye diseases by segmenting the optic disc (OD) and optic cup (OC). OD and OC are still challenging to segment accurately. This work proposes three-layer graph-based deep architecture with an enhanced fusion method for OD and OC segmentation. CNN encoder-decoder architecture, extended graph network, and approximation via fusion-based rule are explored for connecting local and global information. A graph-based model is developed for combining local and overall knowledge. By extending feature masking, regularization of repetitive features with fusion for combining channels has been done. The performance of the proposed network is evaluated through the analysis of different metric parameters such as dice similarity coefficient (DSC), intersection of union (IOU), accuracy, specificity, sensitivity. Experimental verification of this methodology has been done using the four benchmarks publicly available datasets DRISHTI-GS, RIM-ONE for OD, and OC segmentation. In addition, DRIONS-DB and HRF fundus imaging datasets were analyzed for optimizing the model's performance based on OD segmentation. DSC metric of methodology achieved 0.97 and 0.96 for DRISHTI-GS and RIM-ONE, respectively. Similarly, IOU measures for DRISHTI-GS and RIM-ONE datasets were 0.96 and 0.93, respectively, for OD measurement. For OC segmentation, DSC and IOU were measured as 0.93 and 0.90 respectively for DRISHTI-GS and 0.83 and 0.82 for RIM-ONE data. The proposed technique improved value of metrics with most of the existing methods in terms of DSC and IOU of the results metric of the experiments for OD and OC segmentation.

State-of-the-art Deep Learning Methods on Electrocardiogram Data: A Systematic Review (Preprint)

Background

Electrocardiogram (ECG) is one of the most common noninvasive diagnostic tools that can provide useful information regarding a patient’s health status. Deep learning (DL) is an area of intense exploration that leads the way in most attempts to create powerful diagnostic models based on physiological signals.

Objective

This study aimed to provide a systematic review of DL methods applied to ECG data for various clinical applications.

Methods

The PubMed search engine was systematically searched by combining “deep learning” and keywords such as “ecg,” “ekg,” “electrocardiogram,” “electrocardiography,” and “electrocardiology.” Irrelevant articles were excluded from the study after screening titles and abstracts, and the remaining articles were further reviewed. The reasons for article exclusion were manuscripts written in any language other than English, absence of ECG data or DL methods involved in the study, and absence of a quantitative evaluation of the proposed approaches.

Results

We identified 230 relevant articles published between January 2020 and December 2021 and grouped them into 6 distinct medical applications, namely, blood pressure estimation, cardiovascular disease diagnosis, ECG analysis, biometric recognition, sleep analysis, and other clinical analyses. We provide a complete account of the state-of-the-art DL strategies per the field of application, as well as major ECG data sources. We also present open research problems, such as the lack of attempts to address the issue of blood pressure variability in training data sets, and point out potential gaps in the design and implementation of DL models.

Conclusions

We expect that this review will provide insights into state-of-the-art DL methods applied to ECG data and point to future directions for research on DL to create robust models that can assist medical experts in clinical decision-making.

Tomographic reconstruction from planar thermal imaging using convolutional neural network

In this study, we investigate perspectives for thermal tomography based on planar infrared thermal images. Volumetric reconstruction of temperature distribution inside an object is hardly applicable in a way similar to ionizing-radiation-based modalities due to its non-penetrating character. Here, we aim at employing the autoencoder deep neural network to collect knowledge on the single-source heat transfer model. For that purpose, we prepare a series of synthetic 3D models of a cylindrical phantom with assumed thermal properties with various heat source locations, captured at different times. A set of planar thermal images taken around the model is subjected to initial backprojection reconstruction, then passed to the deep model. This paper reports the training and testing results in terms of five metrics assessing spatial similarity between volumetric models, signal-to-noise ratio, or heat source location accuracy. We also evaluate the assumptions of the synthetic model with an experiment involving thermal imaging of a real object (pork) and a single heat source. For validation, we investigate objects with multiple heat sources of a random location and temperature. Our results show the capability of a deep model to reconstruct the temperature distribution inside the object.

A Novel Context Aware Joint Segmentation and Classification Framework for Glaucoma Detection

Glaucoma is a chronic ocular disease characterized by damage to the optic nerve resulting in progressive and irreversible visual loss. Early detection and timely clinical interventions are critical in improving glaucoma-related outcomes. As a typical and complicated ocular disease, glaucoma detection presents a unique challenge due to its insidious onset and high intra- and interpatient variabilities. Recent studies have demonstrated that robust glaucoma detection systems can be realized with deep learning approaches. The optic disc (OD) is the most commonly studied retinal structure for screening and diagnosing glaucoma. This paper proposes a novel context aware deep learning framework called GD-YNet, for OD segmentation and glaucoma detection. It leverages the potential of aggregated transformations and the simplicity of the YNet architecture in context aware OD segmentation and binary classification for glaucoma detection. Trained with the RIGA and RIMOne-V2 datasets, this model achieves glaucoma detection accuracies of 99.72%, 98.02%, 99.50%, and 99.41% with the ACRIMA, Drishti-gs, REFUGE, and RIMOne-V1 datasets. Further, the proposed model can be extended to a multiclass segmentation and classification model for glaucoma staging and severity assessment.

Joint optic disc and cup segmentation based on multi-scale feature analysis and attention pyramid architecture for glaucoma screening

Automatic segmentation of optic disc (OD) and optic cup (OC) is an essential task for analysing colour fundus images. In clinical practice, accurate OD and OC segmentation assist ophthalmologists in diagnosing glaucoma. In this paper, we propose a unified convolutional neural network, named ResFPN-Net, which learns the boundary feature and the inner relation between OD and OC for automatic segmentation. The proposed ResFPN-Net is mainly composed of multi-scale feature extractor, multi-scale segmentation transition and attention pyramid architecture. The multi-scale feature extractor achieved the feature encoding of fundus images and captured the boundary representations. The multi-scale segmentation transition is employed to retain the features of different scales. Moreover, an attention pyramid architecture is proposed to learn rich representations and the mutual connection in the OD and OC. To verify the effectiveness of the proposed method, we conducted extensive experiments on two public datasets. On the Drishti-GS database, we achieved a Dice coefficient of 97.59%, 89.87%, the accuracy of 99.21%, 98.77%, and the Averaged Hausdorff distance of 0.099, 0.882 on the OD and OC segmentation, respectively. We achieved a Dice coefficient of 96.41%, 83.91%, the accuracy of 99.30%, 99.24%, and the Averaged Hausdorff distance of 0.166, 1.210 on the RIM-ONE database for OD and OC segmentation, respectively. Comprehensive results show that the proposed method outperforms other competitive OD and OC segmentation methods and appears more adaptable in cross-dataset scenarios. The introduced multi-scale loss function achieved significantly lower training loss and higher accuracy compared with other loss functions. Furthermore, the proposed method is further validated in OC to OD ratio calculation task and achieved the best MAE of 0.0499 and 0.0630 on the Drishti-GS and RIM-ONE datasets, respectively. Finally, we evaluated the effectiveness of the glaucoma screening on Drishti-GS and RIM-ONE datasets, achieving the AUC of 0.8947 and 0.7964. These results proved that the proposed ResFPN-Net is effective in analysing fundus images for glaucoma screening and can be applied in other relative biomedical image segmentation applications.

Deep neural network for automated simultaneous intervertebral disc (IVDs) identification and segmentation of multi-modal MR images

BACKGROUND AND OBJECTIVE

Lower back pain in humans has become a major risk. Classical approaches follow a non-invasive imaging technique for the assessment of spinal intervertebral disc (IVDs) abnormalities, where identification and segmentation of discs are done separately, making it a time-consuming phenomenon. This necessitates designing a robust automated and simultaneous IVDs identification and segmentation of multi-modality MRI images.

METHODS

We introduced a novel deep neural network architecture coined as 'RIMNet', a Region-to-Image Matching Network model, capable of performing an automated and simultaneous IVDs identification and segmentation of MRI images. The multi-modal input data is being fed to the network with a dropout strategy, by randomly disabling modalities in mini-batches. The performance accuracy as a function of the testing dataset was determined. The execution of the deep neural network model was evaluated by computing the IVDs Identification Accuracy, Dice coefficient, MDOC, Average Symmetric Surface Distance, Jaccard Coefficient, Hausdorff Distance and F1 Score.

RESULTS

Proposed model has attained 94% identification accuracy, dice coefficient value of 91.7±1% in segmentation and MDOC 90.2±1%. Our model also achieved 0.87±0.02 for Jaccard Coefficient, 0.54±0.04 for ASD and 0.62±0.02 mm Hausdorff Distance. The results have been validated and compared with other methodologies on dataset of MICCAI IVD 2018 challenge.

CONCLUSIONS

Our proposed deep-learning methodology is capable of performing simultaneous identification and segmentation on IVDs MRI images of the human spine with high accuracy.

Anatomy-aided deep learning for medical image segmentation: a review

Deep learning (DL) has become widely used for medical image segmentation in recent years. However, despite these advances, there are still problems for which DL-based segmentation fails. Recently, some DL approaches had a breakthrough by using anatomical information which is the crucial cue for manual segmentation. In this paper, we provide a review of anatomy-aided DL for medical image segmentation which covers systematically summarized anatomical information categories and corresponding representation methods. We address known and potentially solvable challenges in anatomy-aided DL and present a categorized methodology overview on using anatomical information with DL from over 70 papers. Finally, we discuss the strengths and limitations of the current anatomy-aided DL approaches and suggest potential future work.

Generative Adversarial Network Based Automatic Segmentation of Corneal Subbasal Nerves on In Vivo Confocal Microscopy Images

Purpose

In vivo confocal microscopy (IVCM) is a noninvasive, reproducible, and inexpensive diagnostic tool for corneal diseases. However, widespread and effortless image acquisition in IVCM creates serious image analysis workloads on ophthalmologists, and neural networks could solve this problem quickly. We have produced a novel deep learning algorithm based on generative adversarial networks (GANs), and we compare its accuracy for automatic segmentation of subbasal nerves in IVCM images with a fully convolutional neural network (U-Net) based method.

Methods

We have collected IVCM images from 85 subjects. U-Net and GAN-based image segmentation methods were trained and tested under the supervision of three clinicians for the segmentation of corneal subbasal nerves. Nerve segmentation results for GAN and U-Net-based methods were compared with the clinicians by using Pearson's R correlation, Bland-Altman analysis, and receiver operating characteristics (ROC) statistics. Additionally, different noises were applied on IVCM images to evaluate the performances of the algorithms with noises of biomedical imaging.

Results

The GAN-based algorithm demonstrated similar correlation and Bland-Altman analysis results with U-Net. The GAN-based method showed significantly higher accuracy compared to U-Net in ROC curves. Additionally, the performance of the U-Net deteriorated significantly with different noises, especially in speckle noise, compared to GAN.

Conclusions

This study is the first application of GAN-based algorithms on IVCM images. The GAN-based algorithms demonstrated higher accuracy than U-Net for automatic corneal nerve segmentation in IVCM images, in patient-acquired images and noise applied images. This GAN-based segmentation method can be used as a facilitating diagnostic tool in ophthalmology clinics.

Translational Relevance

Generative adversarial networks are emerging deep learning models for medical image processing, which could be important clinical tools for rapid segmentation and analysis of corneal subbasal nerves in IVCM images.

Deep learning based automatic diagnosis of first-episode psychosis, bipolar disorder and healthy controls

Neuroimaging data driven machine learning based predictive modeling and pattern recognition has been attracted strongly attention in biomedical sciences. Machine learning based diagnosis techniques are widely applied in diagnosis of neurological diseases. However, machine learning techniques are difficult to effectively extract deep information in neuroimaging data, resulting in low classification accuracy of mental illnesses. To address this problem, we propose a deep learning based automatic diagnosis first-episode psychosis (FEP), bipolar disorder (BD) and healthy controls (HC) method. Specifically, we design a convolutional neural network (CNN) framework to automatically diagnosis based on structural magnetic functional imaging (sMRI). Our dataset consists of 89 FEP patients, 40 BD patients and 83 HC. A three-way classifier (FEP vs. BD vs. HC) and three binary classifiers (FEP vs. BD, FEP vs. HC, BD vs. HC) are trained based on their gray matter volume images. Experiment results show that the performance of CNN-based method outperforms the classic classifiers both in two and three categories classification task. Our research reveals that abnormal gray matter volume is one of the main characteristics for discriminating FEP, BD and HC.

Multi-Scale and Multi-Branch Convolutional Neural Network for Retinal Image Segmentation

The accurate segmentation of retinal images is a basic step in screening for retinopathy and glaucoma. Most existing retinal image segmentation methods have insufficient feature information extraction. They are susceptible to the impact of the lesion area and poor image quality, resulting in the poor recovery of contextual information. This also causes the segmentation results of the model to be noisy and low in accuracy. Therefore, this paper proposes a multi-scale and multi-branch convolutional neural network model (multi-scale and multi-branch network (MSMB-Net)) for retinal image segmentation. The model uses atrous convolution with different expansion rates and skip connections to reduce the loss of feature information. Receiving domains of different sizes captures global context information. The model fully integrates shallow and deep semantic information and retains rich spatial information. The network embeds an improved attention mechanism to obtain more detailed information, which can improve the accuracy of segmentation. Finally, the method of this paper was validated on the fundus vascular datasets, DRIVE, STARE and CHASE datasets, with accuracies/F1 of 0.9708/0.8320, 0.9753/0.8469 and 0.9767/0.8190, respectively. The effectiveness of the method in this paper was further validated on the optic disc visual cup DRISHTI-GS1 dataset with an accuracy/F1 of 0.9985/0.9770. Experimental results show that, compared with existing retinal image segmentation methods, our proposed method has good segmentation performance in all four benchmark tests.

Semi-automatic sigmoid colon segmentation in CT for radiation therapy treatment planning via an iterative 2.5-D deep learning approach

Automatic sigmoid colon segmentation in CT for radiotherapy treatment planning is challenging due to complex organ shape, close distances to other organs, and large variations in size, shape, and filling status. The patient bowel is often not evacuated, and CT contrast enhancement is not used, which further increase problem difficulty. Deep learning (DL) has demonstrated its power in many segmentation problems. However, standard 2-D approaches cannot handle the sigmoid segmentation problem due to incomplete geometry information and 3-D approaches often encounters the challenge of a limited training data size. Motivated by human's behavior that segments the sigmoid slice by slice while considering connectivity between adjacent slices, we proposed an iterative 2.5-D DL approach to solve this problem. We constructed a network that took an axial CT slice, the sigmoid mask in this slice, and an adjacent CT slice to segment as input and output the predicted mask on the adjacent slice. We also considered other organ masks as prior information. We trained the iterative network with 50 patient cases using five-fold cross validation. The trained network was repeatedly applied to generate masks slice by slice. The method achieved average Dice similarity coefficients of 0.82 0.06 and 0.88 0.02 in 10 test cases without and with using prior information.

Machine vision-driven automatic recognition of particle size and morphology in SEM images

Scanning Electron Microscopy (SEM) images provide a variety of structural and morphological information of nanomaterials. In the material informatics domain, automatic recognition and quantitative analysis of SEM images in a high-throughput manner are critical, but challenges still remain due to the complexity and the diversity of image configurations in both shape and size. In this paper, we present a generally applicable approach using computer vision and machine learning techniques to quantitatively extract particle size, size distribution and morphology information in SEM images. The proposed pipeline offers automatic, high-throughput measurements even when overlapping nanoparticles, rod shapes, and core-shell nanostructures are present. We demonstrate effectiveness of the proposed approach by performing experiments on SEM images of nanoscale materials and structures with different shapes and sizes. The proposed approach shows promising results (Spearman coefficients of 0.91 and 0.99 using fully automated and semi-automated processes, respectively) when compared with manually measured sizes. The code is made available as open source software at https://github.com/LLNL/LIST.

Joint disc and cup segmentation based on recurrent fully convolutional network

The optic disc(OD) and the optic cup(OC) segmentation is an key step in fundus medical image analysis. Previously, FCN-based methods have been proposed for medical image segmentation tasks. However, the consecutive convolution and pooling operations usually hinder dense prediction tasks which require detailed spatial information, such as image segmentation. In this paper, we propose a network called Recurrent Fully Convolution Network(RFC-Net) for automatic joint segmentation of the OD and the OC, which can captures more high-level information and subtle edge information. The RFC-Net can minimize the loss of spatial information. It is mainly composed of multi-scale input layer, recurrent fully convolutional network, multiple output layer and polar transformation. In RFC-Net, the multi-scale input layer constructs an image pyramid. We propose four recurrent units, which are respectively applied to RFC-Net. Recurrent convolution layer effectively ensures feature representation for OD and OC segmentation tasks through feature accumulation. For each multiple output image, the multiple output cross entropy loss function is applied. To better balance the cup ratio of the segmented image, the polar transformation is used to transform the fundus image from the cartesian coordinate system to the polar coordinate system. We evaluate the effectiveness and generalization of the proposed method on the DRISHTI-GS1 dataset. Compared with the original FCN method and other state-of-the-art methods, the proposed method achieves better segmentation performance.

Iterative Segmentation from Limited Training Data: Applications to Congenital Heart Disease

We propose a new iterative segmentation model which can be accurately learned from a small dataset. A common approach is to train a model to directly segment an image, requiring a large collection of manually annotated images to capture the anatomical variability in a cohort. In contrast, we develop a segmentation model that recursively evolves a segmentation in several steps, and implement it as a recurrent neural network. We learn model parameters by optimizing the intermediate steps of the evolution in addition to the final segmentation. To this end, we train our segmentation propagation model by presenting incomplete and/or inaccurate input segmentations paired with a recommended next step. Our work aims to alleviate challenges in segmenting heart structures from cardiac MRI for patients with congenital heart disease (CHD), which encompasses a range of morphological deformations and topological changes. We demonstrate the advantages of this approach on a dataset of 20 images from CHD patients, learning a model that accurately segments individual heart chambers and great vessels. Compared to direct segmentation, the iterative method yields more accurate segmentation for patients with the most severe CHD malformations.