Segmenting Retinal Blood Vessels With Deep Neural Networks

n D Variational Restoration of Curvilinear Structures With Prior-Based Directional Regularization

Curvilinear structure restoration in image processing procedures is a difficult task, which can be compounded when these structures are thin, i.e., when their smallest dimension is close to the resolution of the sensor. Many recent restoration methods involve considering a local gradient-based regularization term as prior, assuming gradient sparsity. An isotropic gradient operator is typically not suitable for thin curvilinear structures, since gradients are not sparse for these. In this paper, we propose a mixed gradient operator that combines a standard gradient in the isotropic image regions, and a directional gradient in the regions where specific orientations are likely. In particular, such information can be provided by curvilinear structure detectors (e.g., RORPO or Frangi filters). Our proposed mixed gradient operator, that can be viewed as a companion tool of such detectors, is proposed in a discrete framework and its formulation/computation holds in any dimension; in other words, it is valid in [Formula: see text], n ≥ 1 . We show how this mixed gradient can be used to construct image priors that take edge orientation, as well as intensity, into account, and then involved in various image processing tasks while preserving curvilinear structures. The experiments carried out on 2D, 3D, real, and synthetic images illustrate the relevance of the proposed gradient, and its use in variational frameworks for both denoising and segmentation tasks.

Deep Q Learning Driven CT Pancreas Segmentation With Geometry-Aware U-Net

The segmentation of pancreas is important for medical image analysis, yet it faces great challenges of class imbalance, background distractions, and non-rigid geometrical features. To address these difficulties, we introduce a deep Q network (DQN) driven approach with deformable U-Net to accurately segment the pancreas by explicitly interacting with contextual information and extract anisotropic features from pancreas. The DQN-based model learns a context-adaptive localization policy to produce a visually tightened and precise localization bounding box of the pancreas. Furthermore, deformable U-Net captures geometry-aware information of pancreas by learning geometrically deformable filters for feature extraction. The experiments on NIH dataset validate the effectiveness of the proposed framework in pancreas segmentation.

Multilevel and Multiscale Deep Neural Network for Retinal Blood Vessel Segmentation

Retinal blood vessel segmentation influences a lot of blood vessel-related disorders such as diabetic retinopathy, hypertension, cardiovascular and cerebrovascular disorders, etc. It is found that vessel segmentation using a convolutional neural network (CNN) showed increased accuracy in feature extraction and vessel segmentation compared to the classical segmentation algorithms. CNN does not need any artificial handcrafted features to train the network. In the proposed deep neural network (DNN), a better pre-processing technique and multilevel/multiscale deep supervision (DS) layers are being incorporated for proper segmentation of retinal blood vessels. From the first four layers of the VGG-16 model, multilevel/multiscale deep supervision layers are formed by convolving vessel-specific Gaussian convolutions with two different scale initializations. These layers output the activation maps that are capable to learn vessel-specific features at multiple scales, levels, and depth. Furthermore, the receptive field of these maps is increased to obtain the symmetric feature maps that provide the refined blood vessel probability map. This map is completely free from the optic disc, boundaries, and non-vessel background. The segmented results are tested on Digital Retinal Images for Vessel Extraction (DRIVE), STructured Analysis of the Retina (STARE), High-Resolution Fundus (HRF), and real-world retinal datasets to evaluate its performance. This proposed model achieves better sensitivity values of 0.8282, 0.8979 and 0.8655 in DRIVE, STARE and HRF datasets with acceptable specificity and accuracy performance metrics.

CycleGAN for style transfer in X-ray angiography

PURPOSE

We aim to perform generation of angiograms for various vascular structures as a mean of data augmentation in learning tasks. The task is to enhance the realism of vessels images generated from an anatomically realistic cardiorespiratory simulator to make them look like real angiographies.

METHODS

The enhancement is performed by applying the CycleGAN deep network for transferring the style of real angiograms acquired during percutaneous interventions into a data set composed of realistically simulated arteries.

RESULTS

The cycle consistency was evaluated by comparing an input simulated image with the one obtained after two cycles of image translation. An average structural similarity (SSIM) of 0.948 on our data sets has been obtained. The vessel preservation was measured by comparing segmentations of an input image and its corresponding enhanced image using Dice coefficient.

CONCLUSIONS

We proposed an application of the CycleGAN deep network for enhancing the artificial data as an alternative to classical data augmentation techniques for medical applications, particularly focused on angiogram generation. We discussed success and failure cases, explaining conditions for the realistic data augmentation which respects both the complex physiology of arteries and the various patterns and textures generated by X-ray angiography.

Segmenting brain tumors from FLAIR MRI using fully convolutional neural networks

BACKGROUND AND OBJECTIVE

Magnetic resonance imaging (MRI) is an indispensable tool in diagnosing brain-tumor patients. Automated tumor segmentation is being widely researched to accelerate the MRI analysis and allow clinicians to precisely plan treatment-accurate delineation of brain tumors is a critical step in assessing their volume, shape, boundaries, and other characteristics. However, it is still a very challenging task due to inherent MR data characteristics and high variability, e.g., in tumor sizes or shapes. We present a new deep learning approach for accurate brain tumor segmentation which can be trained from small and heterogeneous datasets annotated by a human reader (providing high-quality ground-truth segmentation is very costly in practice).

METHODS

In this paper, we present a new deep learning technique for segmenting brain tumors from fluid attenuation inversion recovery MRI. Our technique exploits fully convolutional neural networks, and it is equipped with a battery of augmentation techniques that make the algorithm robust against low data quality, and heterogeneity of small training sets. We train our models using only positive (tumorous) examples, due to the limited amount of available data.

RESULTS

Our algorithm was tested on a set of stage II-IV brain-tumor patients (image data collected using MAGNETOM Prisma 3T, Siemens). Rigorous experiments, backed up with statistical tests, revealed that our approach outperforms the state-of-the-art approach (utilizing hand-crafted features) in terms of segmentation accuracy, offers very fast training and instant segmentation (analysis of an image takes less than a second). Building our deep model is 1.3 times faster compared with extracting features for extremely randomized trees, and this training time can be controlled. Finally, we showed that too aggressive data augmentation may lead to deteriorated performance of the model, especially in the fixed-budget training (with maximum numbers of training epochs).

CONCLUSIONS

Our method yields the better performance when compared with the state of the art method which utilizes hand-crafted features. In addition, our deep network can be effectively applied to difficult (small, imbalanced, and heterogeneous) datasets, offers controllable training time, and infers in real-time.

Segmentation and suppression of pulmonary vessels in low-dose chest CT scans

PURPOSE

The suppression of pulmonary vessels in chest computed tomography (CT) images can enhance the conspicuity of lung nodules, thereby improving the detection rate of early lung cancer. This study aimed to develop two key techniques in vessel suppression, that is, segmentation and removal of pulmonary vessels while preserving the nodules.

METHODS

Pulmonary vessel segmentation and removal methods in CT images were developed. The vessel segmentation method used a framework of two cascaded convolutional neural networks (CNNs). A bi-class segmentation network was utilized in the first step to extract high-intensity structures, including both vessels and nonvascular tissues such as nodules. A tri-class segmentation network was employed in the second step to distinguish the vessels from nonvascular tissues (mainly nodules) and the lung parenchyma. In the vessel removal method, the voxels in the segmented vessels were replaced with randomly selected voxels from the surrounding lung parenchyma. The dataset in this study comprised 50 three-dimensional (3D) low-dose chest CT images. The labels for vessel and nodule segmentation were annotated with a semi automatic approach. The two cascaded networks for pulmonary vessel segmentation were trained with CT images of 40 cases and tested with CT images of ten cases. Pulmonary vessels were removed from the ten testing scans based on the predicted segmentation results. In addition to qualitative evaluation to the effects of segmentation and removal, the segmentation results were quantitatively evaluated using Dice coefficient (DICE), Jaccard index (JAC), and volumetric similarity (VS) and the removal results were evaluated using contrast-to-noise ratio (CNR).

RESULTS

In the first step of vessel segmentation, the mean DICE, JAC, and VS for high-intensity tissues, including both vessels and nodules, were 0.943, 0.893, and 0.991, respectively. In the second step, all the nodules were separated from the vessels, and the mean DICE, JAC, and VS for the vessels were 0.941, 0.890, and 0.991, respectively. After vessel removal, the mean CNR for nodules was improved from 4.23 (6.26 dB) to 6.95 (8.42 dB).

CONCLUSIONS

Quantitative and qualitative evaluations demonstrated that the proposed method achieved a high accuracy for pulmonary vessel segmentation and a good effect on pulmonary vessel suppression.

Development of a deep neural network for generating synthetic dual-energy chest x-ray images with single x-ray exposure

Dual-energy chest radiography (DECR) is a medical imaging technology that can improve diagnostic accuracy. This technique can decompose single-energy chest radiography (SECR) images into separate bone- and soft tissue-only images. This can, however, double the radiation exposure to the patient. To address this limitation, we developed an algorithm for the synthesis of DECR from a SECR through deep learning. To predict high resolution images, we developed a novel deep learning architecture by modifying a conventional U-net to take advantage of the high frequency-dominant information that propagates from the encoding part to the decoding part. In addition, we used the anticorrelated relationship (ACR) of DECR for improving the quality of the predicted images. For training data, 300 pairs of SECR and their corresponding DECR images were used. To test the trained model, 50 DECR images from Yonsei University Severance Hospital and 662 publicly accessible SECRs were used. To evaluate the performance of the proposed method, we compared DECR and predicted images using a structural similarity approach (SSIM). In addition, we quantitatively evaluated image quality calculating the modulation transfer function and coefficient of variation. The proposed model selectively predicted the bone- and soft tissue-only CR images from an SECR image. The strategy for improving the spatial resolution by ACR was effective. Quantitative evaluation showed that the proposed method with ACR showed relatively high SSIM (over 0.85). In addition, predicted images with the proposed ACR model achieved better image quality measures than those of U-net. In conclusion, the proposed method can obtain high-quality bone- and soft tissue-only CR images without the need for additional hardware for double x-ray exposures in clinical practice.

SDFN: Segmentation-based deep fusion network for thoracic disease classification in chest X-ray images

This study aims to automatically diagnose thoracic diseases depicted on the chest x-ray (CXR) images using deep convolutional neural networks. The existing methods generally used the entire CXR images for training purposes, but this strategy may suffer from two drawbacks. First, potential misalignment or the existence of irrelevant objects in the entire CXR images may cause unnecessary noise and thus limit the network performance. Second, the relatively low image resolution caused by the resizing operation, which is a common pre-processing procedure for training neural networks, may lead to the loss of image details, making it difficult to detect pathologies with small lesion regions. To address these issues, we present a novel method termed as segmentation-based deep fusion network (SDFN), which leverages the domain knowledge and the higher-resolution information of local lung regions. Specifically, the local lung regions were identified and cropped by the Lung Region Generator (LRG). Two CNN-based classification models were then used as feature extractors to obtain the discriminative features of the entire CXR images and the cropped lung region images. Lastly, the obtained features were fused by the feature fusion module for disease classification. Evaluated by the NIH benchmark split on the Chest X-ray 14 Dataset, our experimental result demonstrated that the developed method achieved more accurate disease classification compared with the available approaches via the receiver operating characteristic (ROC) analyses. It was also found that the SDFN could localize the lesion regions more precisely as compared to the traditional method.

Patch-Based Semantic Segmentation for Detecting Arterioles and Venules in Epifluorescence Imagery

Segmentation and quantification of microvasculature structures are the main steps toward studying microvasculature remodeling. The proposed patch based semantic architecture enables accurate segmentation for the challenging epifluorescence microscopy images. Our pixel-based fast semantic network trained on random patches from different epifluorescence images to learn how to discriminate between vessels versus nonvessels pixels. The proposed semantic vessel network (SVNet) relies on understanding the morphological structure of the thin vessels in the patches rather than considering the whole image as input to speed up the training process and to maintain the clarity of thin structures. Experimental results on our ovariectomized - ovary removed (OVX) - mice dura mater epifluorescence microscopy images shows promising results in both arteriole and venule part. We compared our results with different segmentation methods such as local, global thresholding, matched based filter approaches and related state of the art deep learning networks. Our overall accuracy (> 98%) outperforms all the methods including our previous work (VNet). [1].

Improved cGAN based linear lesion segmentation in high myopia ICGA images

The increasing prevalence of myopia has attracted global attention recently. Linear lesions including lacquer cracks and myopic stretch lines are the main signs in high myopia retinas, and can be revealed by indocyanine green angiography (ICGA). Automatic linear lesion segmentation in ICGA images can help doctors diagnose and analyze high myopia quantitatively. To achieve accurate segmentation of linear lesions, an improved conditional generative adversarial network (cGAN) based method is proposed. A new partial densely connected network is adopted as the generator of cGAN to encourage the reuse of features and make the network time-saving. Dice loss and weighted binary cross-entropy loss are added to solve the data imbalance problem. Experiments on our data set indicated that the proposed network achieved better performance compared to other networks.

Finger-Vein Verification Based on LSTM Recurrent Neural Networks

Finger-vein biometrics has been extensively investigated for personal verification. A challenge is that the finger-vein acquisition is affected by many factors, which results in many ambiguous regions in the finger-vein image. Generally, the separability between vein and background is poor in such regions. Despite recent advances in finger-vein pattern segmentation, current solutions still lack the robustness to extract finger-vein features from raw images because they do not take into account the complex spatial dependencies of vein pattern. This paper proposes a deep learning model to extract vein features by combining the Convolutional Neural Networks (CNN) model and Long Short-Term Memory (LSTM) model. Firstly, we automatically assign the label based on a combination of known state of the art handcrafted finger-vein image segmentation techniques, and generate various sequences for each labeled pixel along different directions. Secondly, several Stacked Convolutional Neural Networks and Long Short-Term Memory (SCNN-LSTM) models are independently trained on the resulting sequences. The outputs of various SCNN-LSTMs form a complementary and over-complete representation and are conjointly put into Probabilistic Support Vector Machine (P-SVM) to predict the probability of each pixel of being foreground (i.e., vein pixel) given several sequences centered on it. Thirdly, we propose a supervised encoding scheme to extract the binary vein texture. A threshold is automatically computed by taking into account the maximal separation between the inter-class distance and the intra-class distance. In our approach, the CNN learns robust features for vein texture pattern representation and LSTM stores the complex spatial dependencies of vein patterns. So, the pixels in any region of a test image can then be classified effectively. In addition, the supervised information is employed to encode the vein patterns, so the resulting encoding images contain more discriminating features. The experimental results on one public finger-vein database show that the proposed approach significantly improves the finger-vein verification accuracy.

Classification of Metaphase Chromosomes Using Deep Convolutional Neural Network

Karyotype analysis has important clinical significance in the diagnosis, treatment, and prognosis of diseases such as birth defects and hematological tumors. Identifying chromosomes and their structure variations from G-banded metaphase images is an important process in karyotyping, and also is the most difficult one. Automatic chromosome classification becomes urgent in recent years since more and more samples of patients are subject to medical test such as bone marrow biopsy. With the development of artificial intelligence, convolutional neural networks (CNNs) have shown good performance in image recognition. In this study, a CNN with 6 convolutional layers, 3 pooling layers, 4 dropout layers, and 2 fully connected layers was trained using the labeled data set to classify chromosomes into 24 classes through softmax activation function mapping. The classifier gave an accuracy of 93.79% for chromosome identification. The result demonstrated that the CNN has potential application value in chromosome classification and will contribute to the construction of an automatic karyotyping platform.

BTS-DSN: Deeply supervised neural network with short connections for retinal vessel segmentation

BACKGROUND AND OBJECTIVE

The condition of vessel of the human eye is an important factor for the diagnosis of ophthalmological diseases. Vessel segmentation in fundus images is a challenging task due to complex vessel structure, the presence of similar structures such as microaneurysms and hemorrhages, micro-vessel with only one to several pixels wide, and requirements for finer results.

METHODS

In this paper, we present a multi-scale deeply supervised network with short connections (BTS-DSN) for vessel segmentation. We used short connections to transfer semantic information between side-output layers. Bottom-top short connections pass low level semantic information to high level for refining results in high-level side-outputs, and top-bottom short connection passes much structural information to low level for reducing noises in low-level side-outputs. In addition, we employ cross-training to show that our model is suitable for real world fundus images.

RESULTS

The proposed BTS-DSN has been verified on DRIVE, STARE and CHASE_DB1 datasets, and showed competitive performance over other state-of-the-art methods. Specially, with patch level input, the network achieved 0.7891/0.8212 sensitivity, 0.9804/0.9843 specificity, 0.9806/0.9859 AUC, and 0.8249/0.8421 F1-score on DRIVE and STARE, respectively. Moreover, our model behaves better than other methods in cross-training experiments.

CONCLUSIONS

BTS-DSN achieves competitive performance in vessel segmentation task on three public datasets. It is suitable for vessel segmentation. The source code of our method is available at: https://github.com/guomugong/BTS-DSN.

Automatic Needle Segmentation and Localization in MRI With 3-D Convolutional Neural Networks: Application to MRI-Targeted Prostate Biopsy

Image guidance improves tissue sampling during biopsy by allowing the physician to visualize the tip and trajectory of the biopsy needle relative to the target in MRI, CT, ultrasound, or other relevant imagery. This paper reports a system for fast automatic needle tip and trajectory localization and visualization in MRI that has been developed and tested in the context of an active clinical research program in prostate biopsy. To the best of our knowledge, this is the first reported system for this clinical application and also the first reported system that leverages deep neural networks for segmentation and localization of needles in MRI across biomedical applications. Needle tip and trajectory were annotated on 583 T2-weighted intra-procedural MRI scans acquired after needle insertion for 71 patients who underwent transperineal MRI-targeted biopsy procedure at our institution. The images were divided into two independent training-validation and test sets at the patient level. A deep 3-D fully convolutional neural network model was developed, trained, and deployed on these samples. The accuracy of the proposed method, as tested on previously unseen data, was 2.80-mm average in needle tip detection and 0.98° in needle trajectory angle. An observer study was designed in which independent annotations by a second observer, blinded to the original observer, were compared with the output of the proposed method. The resultant error was comparable to the measured inter-observer concordance, reinforcing the clinical acceptability of the proposed method. The proposed system has the potential for deployment in clinical routine.

Distributed deep learning for robust multi-site segmentation of CT imaging after traumatic brain injury

Machine learning models are becoming commonplace in the domain of medical imaging, and with these methods comes an ever-increasing need for more data. However, to preserve patient anonymity it is frequently impractical or prohibited to transfer protected health information (PHI) between institutions. Additionally, due to the nature of some studies, there may not be a large public dataset available on which to train models. To address this conundrum, we analyze the efficacy of transferring the model itself in lieu of data between different sites. By doing so we accomplish two goals: 1) the model gains access to training on a larger dataset that it could not normally obtain and 2) the model better generalizes, having trained on data from separate locations. In this paper, we implement multi-site learning with disparate datasets from the National Institutes of Health (NIH) and Vanderbilt University Medical Center (VUMC) without compromising PHI. Three neural networks are trained to convergence on a computed tomography (CT) brain hematoma segmentation task: one only with NIH data, one only with VUMC data, and one multi-site model alternating between NIH and VUMC data. Resultant lesion masks with the multi-site model attain an average Dice similarity coefficient of 0.64 and the automatically segmented hematoma volumes correlate to those done manually with a Pearson correlation coefficient of 0.87, corresponding to an 8% and 5% improvement, respectively, over the single-site model counterparts.

Accurate vessel extraction via tensor completion of background layer in X-ray coronary angiograms

This paper proposes an effective method for accurately recovering vessel structures and intensity information from the X-ray coronary angiography (XCA) images of moving organs or tissues. Specifically, a global logarithm transformation of XCA images is implemented to fit the X-ray attenuation sum model of vessel/background layers into a low-rank, sparse decomposition model for vessel/background separation. The contrast-filled vessel structures are extracted by distinguishing the vessels from the low-rank backgrounds by using a robust principal component analysis and by constructing a vessel mask via Radon-like feature filtering plus spatially adaptive thresholding. Subsequently, the low-rankness and inter-frame spatio-temporal connectivity in the complex and noisy backgrounds are used to recover the vessel-masked background regions using tensor completion of all other background regions, while the twist tensor nuclear norm is minimized to complete the background layers. Finally, the method is able to accurately extract vessels' intensities from the noisy XCA data by subtracting the completed background layers from the overall XCA images. We evaluated the vessel visibility of resulting images on real X-ray angiography data and evaluated the accuracy of vessel intensity recovery on synthetic data. Experiment results show the superiority of the proposed method over the state-of-the-art methods.

Automatic Choroid Layer Segmentation from Optical Coherence Tomography Images Using Deep Learning

The choroid layer is a vascular layer in human retina and its main function is to provide oxygen and support to the retina. Various studies have shown that the thickness of the choroid layer is correlated with the diagnosis of several ophthalmic diseases. For example, diabetic macular edema (DME) is a leading cause of vision loss in patients with diabetes. Despite contemporary advances, automatic segmentation of the choroid layer remains a challenging task due to low contrast, inhomogeneous intensity, inconsistent texture and ambiguous boundaries between the choroid and sclera in Optical Coherence Tomography (OCT) images. The majority of currently implemented methods manually or semi-automatically segment out the region of interest. While many fully automatic methods exist in the context of choroid layer segmentation, more effective and accurate automatic methods are required in order to employ these methods in the clinical sector. This paper proposed and implemented an automatic method for choroid layer segmentation in OCT images using deep learning and a series of morphological operations. The aim of this research was to segment out Bruch's Membrane (BM) and choroid layer to calculate the thickness map. BM was segmented using a series of morphological operations, whereas the choroid layer was segmented using a deep learning approach as more image statistics were required to segment accurately. Several evaluation metrics were used to test and compare the proposed method against other existing methodologies. Experimental results showed that the proposed method greatly reduced the error rate when compared with the other state-of-the-art methods.

PixelBNN: Augmenting the PixelCNN with Batch Normalization and the Presentation of a Fast Architecture for Retinal Vessel Segmentation

Analysis of retinal fundus images is essential for eye-care physicians in the diagnosis, care and treatment of patients. Accurate fundus and/or retinal vessel maps give rise to longitudinal studies able to utilize multimedia image registration and disease/condition status measurements, as well as applications in surgery preparation and biometrics. The segmentation of retinal morphology has numerous applications in assessing ophthalmologic and cardiovascular disease pathologies. Computer-aided segmentation of the vasculature has proven to be a challenge, mainly due to inconsistencies such as noise and variations in hue and brightness that can greatly reduce the quality of fundus images. The goal of this work is to collate different key performance indicators (KPIs) and state-of-the-art methods applied to this task, frame computational efficiency–performance trade-offs under varying degrees of information loss using common datasets, and introduce PixelBNN, a highly efficient deep method for automating the segmentation of fundus morphologies. The model was trained, tested and cross tested on the DRIVE, STARE and CHASE_DB1 retinal vessel segmentation datasets. Performance was evaluated using G-mean, Mathews Correlation Coefficient and F1-score, with the main success measure being computation speed. The network was 8.5× faster than the current state-of-the-art at test time and performed comparatively well, considering a 5× to 19× reduction in information from resizing images during preprocessing.

Recurrent residual U-Net for medical image segmentation

Deep learning (DL)-based semantic segmentation methods have been providing state-of-the-art performance in the past few years. More specifically, these techniques have been successfully applied in medical image classification, segmentation, and detection tasks. One DL technique, U-Net, has become one of the most popular for these applications. We propose a recurrent U-Net model and a recurrent residual U-Net model, which are named RU-Net and R2U-Net, respectively. The proposed models utilize the power of U-Net, residual networks, and recurrent convolutional neural networks. There are several advantages to using these proposed architectures for segmentation tasks. First, a residual unit helps when training deep architectures. Second, feature accumulation with recurrent residual convolutional layers ensures better feature representation for segmentation tasks. Third, it allows us to design better U-Net architectures with the same number of network parameters with better performance for medical image segmentation. The proposed models are tested on three benchmark datasets, such as blood vessel segmentation in retinal images, skin cancer segmentation, and lung lesion segmentation. The experimental results show superior performance on segmentation tasks compared to equivalent models, including a variant of a fully connected convolutional neural network called SegNet, U-Net, and residual U-Net.

A Hierarchical Image Matting Model for Blood Vessel Segmentation in Fundus Images

In this paper, a hierarchical image matting model is proposed to extract blood vessels from fundus images. More specifically, a hierarchical strategy is integrated into the image matting model for blood vessel segmentation. Normally the matting models require a user specified trimap, which separates the input image into three regions: the foreground, background and unknown regions. However, creating a user specified trimap is laborious for vessel segmentation tasks. In this paper, we propose a method that first generates trimap automatically by utilizing region features of blood vessels, then applies a hierarchical image matting model to extract the vessel pixels from the unknown regions. The proposed method has low calculation time and outperforms many other state-of-art supervised and unsupervised methods. It achieves a vessel segmentation accuracy of 96.0%, 95.7% and 95.1% in an average time of 10.72s, 15.74s and 50.71s on images from three publicly available fundus image datasets DRIVE, STARE, and CHASE DB1, respectively.

A novel retinal vessel detection approach based on multiple deep convolution neural networks

BACKGROUND AND OBJECTIVE

Computer aided detection (CAD) offers an efficient way to assist doctors to interpret fundus images. In a CAD system, retinal vessel (RV) detection is a crucial step to identify the retinal disease regions. However, RV detection is still a challenging problem due to variations in morphology of the vessels on noisy and low contrast fundus images.

METHODS

In this paper, we formulate the detection task as a classification problem and solve it using a multiple classifier framework based on deep convolutional neural networks. The multiple deep convolutional neural network (MDCNN) is constructed and trained on fundus images with limited image quantity. The MDCNN is trained using an incremental learning strategy to improve the networks' performance. The final classification results are obtained from the voting procedure on the results of MDCNN.

RESULTS

The MDCNN achieves better performance and significantly outperforms the state-of-the-art for automatic retinal vessel segmentation on the DRIVE dataset with 95.97% and 96.13% accuracy and 0.9726 and 0.9737 AUC (area below the operator receiver character curve) score on training and testing sets, respectively. Another public dataset, STARE, is also used to evaluate the proposed network. The experimental results demonstrate that the proposed MDCNN network achieves 95.39% accuracy and 0.9539 AUC score in STARE dataset. We further compare our result with several state-of-the-art methods based on AUC values. The comparison is shown that our proposal yields the third best AUC value.

CONCLUSIONS

Our method yields the better performance in the compared the state of the art methods. In addition, our proposal has no preprocessing stage, and the input color fundus images are fed into the CNN directly.

A Coarse-to-Fine Fully Convolutional Neural Network for Fundus Vessel Segmentation

Fundus vessel analysis is a significant tool for evaluating the development of retinal diseases such as diabetic retinopathy and hypertension in clinical practice. Hence, automatic fundus vessel segmentation is essential and valuable for medical diagnosis in ophthalmopathy and will allow identification and extraction of relevant symmetric and asymmetric patterns. Further, due to the uniqueness of fundus vessel, it can be applied in the field of biometric identification. In this paper, we remold fundus vessel segmentation as a task of pixel-wise classification task, and propose a novel coarse-to-fine fully convolutional neural network (CF-FCN) to extract vessels from fundus images. Our CF-FCN is aimed at making full use of the original data information and making up for the coarse output of the neural network by harnessing the space relationship between pixels in fundus images. Accompanying with necessary pre-processing and post-processing operations, the efficacy and efficiency of our CF-FCN is corroborated through our experiments on DRIVE, STARE, HRF and CHASE DB1 datasets. It achieves sensitivity of 0.7941, specificity of 0.9870, accuracy of 0.9634 and Area Under Receiver Operating Characteristic Curve (AUC) of 0.9787 on DRIVE datasets, which surpasses the state-of-the-art approaches.

Efficient automated detection of mitotic cells from breast histological images using deep convolution neutral network with wavelet decomposed patches

In medical practice, the mitotic cell count from histological images acts as a proliferative marker for cancer diagnosis. Therefore, an accurate method for detecting mitotic cells in histological images is essential for cancer screening. Manual evaluation of clinically relevant image features that might reflect mitotic cells in histological images is time-consuming and error prone, due to the heterogeneous physical characteristics of mitotic cells. Computer-assisted automated detection of mitotic cells could overcome these limitations of manual analysis and act as a useful tool for pathologists to make cancer diagnoses efficiently and accurately. Here, we propose a new approach for mitotic cell detection in breast histological images that uses a deep convolution neural network (CNN) with wavelet decomposed image patches. In this approach, raw image patches of 81 × 81 pixels are decomposed to patches of 21 × 21 pixels using Haar wavelet and subsequently used in developing a deep CNN model for automated detection of mitotic cells. The decomposition step reduces convolution time for mitotic cell detection relative to the use of raw image patches in conventional CNN models. The proposed deep network was tested using the MITOS (ICPR2012) and MITOS-ATYPIA-14 breast cancer histological datasets and shown to outperform existing algorithms for mitotic cell detection. Overall, our method improves the performance and reduces the computational burden of conventional deep CNN approaches for mitotic cell detection.

Automated identification of cone photoreceptors in adaptive optics optical coherence tomography images using transfer learning

Automated measurements of the human cone mosaic requires the identification of individual cone photoreceptors. The current gold standard, manual labeling, is a tedious process and can not be done in a clinically useful timeframe. As such, we present an automated algorithm for identifying cone photoreceptors in adaptive optics optical coherence tomography (AO-OCT) images. Our approach fine-tunes a pre-trained convolutional neural network originally trained on AO scanning laser ophthalmoscope (AO-SLO) images, to work on previously unseen data from a different imaging modality. On average, the automated method correctly identified 94% of manually labeled cones when compared to manual raters, from twenty different AO-OCT images acquired from five normal subjects. Voronoi analysis confirmed the general hexagonal-packing structure of the cone mosaic as well as the general cone density variability across portions of the retina. The consistency of our measurements demonstrates the high reliability and practical utility of having an automated solution to this problem.

Deep convolutional neural network-based patch classification for retinal nerve fiber layer defect detection in early glaucoma

Glaucoma is a progressive optic neuropathy characterized by peripheral visual field loss, which is caused by degeneration of retinal nerve fibers. The peripheral vision loss due to glaucoma is asymptomatic. If not detected and treated at an early stage, it leads to complete blindness, which is irreversible in nature. The retinal nerve fiber layer defect (RNFLD) provides an earliest objective evidence of glaucoma. In this regard, we explore cost-effective redfree fundus imaging for RNFLD detection to be practically useful for computer-assisted early glaucoma risk assessment. RNFLD appears as a wedge shaped arcuate structure radiating from the optic disc. The very low contrast between RNFLD and background makes its visual detection quite challenging even by medical experts. In our study, we formulate a deep convolutional neural network (CNN) based patch classification strategy for RNFLD boundary localization. A large number of RNFLD and background image patches train the deep CNN model, which extracts sufficient discriminative information from the patches and results in accurate RNFLD boundary pixel classification. The proposed approach is found to achieve enhanced RNFLD detection performance with sensitivity of 0.8205 and false positive per image of 0.2000 on a newly created early glaucomatic fundus image database.

Toward Improving Safety in Neurosurgery with an Active Handheld Instrument

Microsurgical procedures, such as petroclival meningioma resection, require careful surgical actions in order to remove tumor tissue, while avoiding brain and vessel damaging. Such procedures are currently performed under microscope magnification. Robotic tools are emerging in order to filter surgeons' unintended movements and prevent tools from entering forbidden regions such as vascular structures. The present work investigates the use of a handheld robotic tool (Micron) to automate vessel avoidance in microsurgery. In particular, we focused on vessel segmentation, implementing a deep-learning-based segmentation strategy in microscopy images, and its integration with a feature-based passive 3D reconstruction algorithm to obtain accurate and robust vessel position. We then implemented a virtual-fixture-based strategy to control the handheld robotic tool and perform vessel avoidance. Clay vascular phantoms, lying on a background obtained from microscopy images recorded during petroclival meningioma surgery, were used for testing the segmentation and control algorithms. When testing the segmentation algorithm on 100 different phantom images, a median Dice similarity coefficient equal to 0.96 was achieved. A set of 25 Micron trials of 80 s in duration, each involving the interaction of Micron with a different vascular phantom, were recorded, with a safety distance equal to 2 mm, which was comparable to the median vessel diameter. Micron's tip entered the forbidden region 24% of the time when the control algorithm was active. However, the median penetration depth was 16.9 μm, which was two orders of magnitude lower than median vessel diameter. Results suggest the system can assist surgeons in performing safe vessel avoidance during neurosurgical procedures.

Extraction of Blood Vessels in Retina

This article presents a novel method to extract retinal vascular tree automatically. The proposed method consists of four steps; smoothing image using low pass spatial filter to reduce spurious noise in the image; extracting candidate borders of the vessels based on a local window property; tracking process, starting with a candidate pixel and following in the optimum direction with monitoring the connectivity of the vessel twin border; constructing the whole tree of retinal blood vessels by connecting the vessel segments based on their spatial locations, widths and directions. The algorithm was trained with 20 images from the DRIVE dataset, and tested using the remaining 20 images.

A Three-Stage Deep Learning Model for Accurate Retinal Vessel Segmentation

Automatic retinal vessel segmentation is a fundamental step in the diagnosis of eye-related diseases, in which both thick vessels and thin vessels are important features for symptom detection. All existing deep learning models attempt to segment both types of vessels simultaneously by using a unified pixel-wise loss that treats all vessel pixels with equal importance. Due to the highly imbalanced ratio between thick vessels and thin vessels (namely the majority of vessel pixels belong to thick vessels), the pixel-wise loss would be dominantly guided by thick vessels and relatively little influence comes from thin vessels, often leading to low segmentation accuracy for thin vessels. To address the imbalance problem, in this paper, we explore to segment thick vessels and thin vessels separately by proposing a three-stage deep learning model. The vessel segmentation task is divided into three stages, namely thick vessel segmentation, thin vessel segmentation, and vessel fusion. As better discriminative features could be learned for separate segmentation of thick vessels and thin vessels, this process minimizes the negative influence caused by their highly imbalanced ratio. The final vessel fusion stage refines the results by further identifying nonvessel pixels and improving the overall vessel thickness consistency. The experiments on public datasets DRIVE, STARE, and CHASE_DB1 clearly demonstrate that the proposed three-stage deep learning model outperforms the current state-of-the-art vessel segmentation methods.

Deep learning based detection of cone photoreceptors with multimodal adaptive optics scanning light ophthalmoscope images of achromatopsia

Fast and reliable quantification of cone photoreceptors is a bottleneck in the clinical utilization of adaptive optics scanning light ophthalmoscope (AOSLO) systems for the study, diagnosis, and prognosis of retinal diseases. To-date, manual grading has been the sole reliable source of AOSLO quantification, as no automatic method has been reliably utilized for cone detection in real-world low-quality images of diseased retina. We present a novel deep learning based approach that combines information from both the confocal and non-confocal split detector AOSLO modalities to detect cones in subjects with achromatopsia. Our dual-mode deep learning based approach outperforms the state-of-the-art automated techniques and is on a par with human grading.

Retinal Lesion Detection With Deep Learning Using Image Patches

Purpose

To develop an automated method of localizing and discerning multiple types of findings in retinal images using a limited set of training data without hard-coded feature extraction as a step toward generalizing these methods to rare disease detection in which a limited number of training data are available.

Methods

Two ophthalmologists verified 243 retinal images, labeling important subsections of the image to generate 1324 image patches containing either hemorrhages, microaneurysms, exudates, retinal neovascularization, or normal-appearing structures from the Kaggle dataset. These image patches were used to train one standard convolutional neural network to predict the presence of these five classes. A sliding window method was used to generate probability maps across the entire image.

Results

The method was validated on the eOphta dataset of 148 whole retinal images for microaneurysms and 47 for exudates. A pixel-wise classification of the area under the curve of the receiver operating characteristic of 0.94 and 0.95, as well as a lesion-wise area under the precision recall curve of 0.86 and 0.64, was achieved for microaneurysms and exudates, respectively.

Conclusions

Regionally trained convolutional neural networks can generate lesion-specific probability maps able to detect and distinguish between subtle pathologic lesions with only a few hundred training examples per lesion.

Automated nasopharyngeal carcinoma segmentation in magnetic resonance images by combination of convolutional neural networks and graph cut

Accurate and reliable segmentation of nasopharyngeal carcinoma (NPC) in medical images is an import task for clinical applications, including radiotherapy. However, NPC features large variations in lesion size and shape, as well as inhomogeneous intensities within the tumor and similar intensity to that of nearby tissues, making its segmentation a challenging task. The present study proposes a novel automated NPC segmentation method in magnetic resonance (MR) images by combining a deep convolutional neural network (CNN) model and a 3-dimensional (3D) graph cut-based method in a two-stage manner. First, a multi-view deep CNN-based segmentation method is performed. A voxel-wise initial segmentation is generated by integrating the inferential classification information of three trained single-view CNNs. Instead of directly using the CNN classification results to achieve a final segmentation, the proposed method uses a 3D graph cut-based method to refine the initial segmentation. Specifically, the probability response map obtained using the multi-view CNN method is utilized to calculate the region cost, which represents the likelihood of a voxel being assigned to the tumor or non-tumor. Structure information in 3D from the original MR images is used to calculate the boundary cost, which measures the difference between the two voxels in the 3D neighborhood. The proposed method was evaluated on T1-weighted images from 30 NPC patients using the leave-one-out method. The experimental results demonstrated that the proposed method is effective and accurate for NPC segmentation.

Segmentation of histological images and fibrosis identification with a convolutional neural network

Segmentation of histological images is one of the most crucial tasks for many biomedical analyses involving quantification of certain tissue types, such as fibrosis via Masson's trichrome staining. However, challenges are posed by the high variability and complexity of structural features in such images, in addition to imaging artifacts. Further, the conventional approach of manual thresholding is labor-intensive, and highly sensitive to inter- and intra-image intensity variations. An accurate and robust automated segmentation method is of high interest. We propose and evaluate an elegant convolutional neural network (CNN) designed for segmentation of histological images, particularly those with Masson's trichrome stain. The network comprises 11 successive convolutional - rectified linear unit - batch normalization layers. It outperformed state-of-the-art CNNs on a dataset of cardiac histological images (labeling fibrosis, myocytes, and background) with a Dice similarity coefficient of 0.947. With 100 times fewer (only 300,000) trainable parameters than the state-of-the-art, our CNN is less susceptible to overfitting, and is efficient. Additionally, it retains image resolution from input to output, captures fine-grained details, and can be trained end-to-end smoothly. To the best of our knowledge, this is the first deep CNN tailored to the problem of concern, and may potentially be extended to solve similar segmentation tasks to facilitate investigations into pathology and clinical treatment.

Retinal artery/vein classification using genetic-search feature selection

BACKGROUND AND OBJECTIVES

The automatic classification of retinal blood vessels into artery and vein (A/V) is still a challenging task in retinal image analysis. Recent works on A/V classification mainly focus on the graph analysis of the retinal vasculature, which exploits the connectivity of vessels to improve the classification performance. While they have overlooked the importance of pixel-wise classification to the final classification results. This paper shows that a complicated feature set is efficient for vessel centerline pixels classification.

METHODS

We extract enormous amount of features for vessel centerline pixels, and apply a genetic-search based feature selection technique to obtain the optimal feature subset for A/V classification.

RESULTS

The proposed method achieves an accuracy of 90.2%, the sensitivity of 89.6%, the specificity of 91.3% on the INSPIRE dataset. It shows that our method, using only the information of centerline pixels, gives a comparable performance as the techniques which use complicated graph analysis. In addition, the results on the images acquired by different fundus cameras show that our framework is capable for discriminating vessels independent of the imaging device characteristics, image resolution and image quality.

CONCLUSION

The complicated feature set is essential for A/V classification, especially on the individual vessels where graph-based methods receive limitations. And it could provide a higher entry to the graph-analysis to achieve a better A/V labeling.