Deep Learning Segmentation of Optical Microscopy Images Improves 3-D Neuron Reconstruction

Artificial neural network filters for enhancing 3D optical microscopy images of neurites

The ability to extract accurate morphology of labeled neurons from microscopy images is crucial for mapping brain connectivity and for understanding changes in connectivity that underlie learning and neurological disorders. There are, however, two problems, specific to optical microscopy imaging of neurons, which make accurate neuron tracing exceedingly challenging: (i) neurites can appear broken due to inhomogeneous labeling and (ii) neurites can appear fused in 3D due to limited resolution. Here, we propose and evaluate several artificial neural network (ANN) architectures and conventional image enhancement filters with the aim of alleviating both problems. We developed four image quality metrics to evaluate the effects of the proposed filters: normalized intensity in the cross-over regions between neurites, effective radius of neurites, coefficient of variation of intensity along neurites, and local background to neurite intensity ratio. Our results show that ANN-based filters, trained on optimized semi-manual traces of neurites, can significantly outperform conventional filters. In particular, U-Net based filtering can virtually eliminate background intensity, while also reducing the effective radius of neurites to nearly 1 voxel. In addition, this filter significantly decreases intensity in the cross-over regions between neurites and reduces fluctuations of intensity on neurites' centerlines. These results suggest that including an ANN-based filtering step, which does not require substantial extra time or computing power, can be beneficial for automated neuron tracing projects.

Segmentation of Heavily Clustered Nuclei from Histopathological Images

Automated cell nucleus segmentation is the key to gain further insight into cell features and functionality which support computer-aided pathology in early diagnosis of diseases such as breast cancer and brain tumour. Despite considerable advances in automated segmentation, it still remains a challenging task to split heavily clustered nuclei due to intensity variations caused by noise and uneven absorption of stains. To address this problem, we propose a novel method applicable to variety of histopathological images stained for different proteins, with high speed, accuracy and level of automation. Our algorithm is initiated by applying a new locally adaptive thresholding method on watershed regions. Followed by a new splitting technique based on multilevel thresholding and the watershed algorithm to separate clustered nuclei. Finalized by a model-based merging step to eliminate oversegmentation and a model-based correction step to improve segmentation results and eliminate small objects. We have applied our method to three image datasets: breast cancer stained for hematoxylin and eosin (H&E), Drosophila Kc167 cells stained for DNA to label nuclei, and mature neurons stained for NeuN. Evaluated results show our method outperforms the state-of-the-art methods in terms of accuracy, precision, F1-measure, and computational time.

Large-scale Exploration of Neuronal Morphologies Using Deep Learning and Augmented Reality

Recently released large-scale neuron morphological data has greatly facilitated the research in neuroinformatics. However, the sheer volume and complexity of these data pose significant challenges for efficient and accurate neuron exploration. In this paper, we propose an effective retrieval framework to address these problems, based on frontier techniques of deep learning and binary coding. For the first time, we develop a deep learning based feature representation method for the neuron morphological data, where the 3D neurons are first projected into binary images and then learned features using an unsupervised deep neural network, i.e., stacked convolutional autoencoders (SCAEs). The deep features are subsequently fused with the hand-crafted features for more accurate representation. Considering the exhaustive search is usually very time-consuming in large-scale databases, we employ a novel binary coding method to compress feature vectors into short binary codes. Our framework is validated on a public data set including 58,000 neurons, showing promising retrieval precision and efficiency compared with state-of-the-art methods. In addition, we develop a novel neuron visualization program based on the techniques of augmented reality (AR), which can help users take a deep exploration of neuron morphologies in an interactive and immersive manner.

Investigation of Polymer Coatings Formed by Polyvinyl Alcohol and Silver Nanoparticles on Copper Surface in Acid Medium by Means of Deep Convolutional Neural Networks

In order to assemble effective protective coatings against corrosion, electrochemical techniques such as linear potentiometry and cyclic voltammetry were performed on a copper surface in 0.1 mol·L−1 HCl solution containing 0.1% polyvinyl alcohol (PVA) in the absence and presence of silver nanoparticles (nAg/PVA). A recent paradigm was used to distinguish the features of the coatings, that is, a deep convolutional neural network (CNN) was implemented to automatically and hierarchically extract the discriminative characteristics from the information given by optical microscopy images. In our study, the material surface morphology, controlled by the CNN without the interference of the human factor, was successfully conducted to extract the similarities/differences between unprotected and protected surfaces in order to establish the PVA and nAg/PVA performance to retard copper corrosion. The CNN results were confirmed by the classical investigation of copper behavior in hydrochloric acid solution in the absence and presence of polyvinyl alcohol and silver nanoparticles. The electrochemical measurements showed that the corrosion current density (icorr) decreased and polarization resistance (Rp) increased, with both PVA and nAg/PVA being effective inhibitors for copper corrosion in an acid environment, forming polymer protective coatings by adsorption on the metal surface. Furthermore, scanning electron microscopy (SEM) certifies the formation of polymer coatings, revealing a specific morphology of the copper surface in the presence of PVA and nAg/PVA, very different from that of corroded copper in uninhibited solutions. Finally, the correlation of the CNN information with experimental data was reported.

Identifying Weak Signals in Inhomogeneous Neuronal Images for Large-Scale Tracing of Sparsely Distributed Neurites

Tracing neurites constitutes the core of neuronal morphology reconstruction, a key step toward neuronal circuit mapping. Modern optical-imaging techniques allow observation of nearly complete mouse neuron morphologies across brain regions or even the whole brain. However, high-level automation reconstruction of neurons, i.e., the reconstruction with a few of manual edits requires discrimination of weak foreground points from the inhomogeneous background. We constructed an identification model, where empirical observations made from neuronal images were summarized into rules for designing feature vectors that to classify foreground and background, and a support vector machine (SVM) was used to learn these feature vectors. We embedded this constructed SVM classifier into a previously developed tool, SparseTracer, to obtain SparseTracer-Learned Feature Vector (ST-LFV). ST-LFV can trace sparsely distributed neurites with weak signals (contrast-to-noise ratio < 1.5) against an inhomogeneous background in datasets imaged by widely used light-microscopy techniques like confocal microscopy and two-photon microscopy. Moreover, 12 sub-blocks were extracted from different brain regions. The average recall and precision rates were 99% and 97%, respectively. These results indicated that ST-LFV is well suited for weak signal identification with varying image characteristics. We also applied ST-LFV to trace long-range neurites from images where neurites are sparsely distributed but their image intensities are weak in some cases. When tracing this long-range neurites, manual edit was required once to obtain results equivalent to the ground truth, compared with 20 times of manual edits required by SparseTracer. This improvement in the level of automatic reconstruction indicates that ST-LFV has the potential to rapidly reconstruct sparsely distributed neurons at the large scale.

PAT-Probabilistic Axon Tracking for Densely Labeled Neurons in Large 3-D Micrographs

A major goal of contemporary neuroscience research is to map the structural connectivity of mammalian brain using microscopy imaging data. In this context, the reconstruction of densely labeled axons from two-photon microscopy images is a challenging and important task. The visually overlapping, crossing, and often strongly distorted images of the axons allow many ambiguous interpretations to be made. We address the problem of tracking axons in densely labeled samples of neurons in large image data sets acquired from marmoset brains. Our high-resolution images were acquired using two-photon microscopy and they provided whole brain coverage, occupying terabytes of memory. Both the image distortions and the large data set size frequently make it impractical to apply present-day neuron tracing algorithms to such data due to the optimization of such algorithms to the precise tracing of either single or sparse sets of neurons. Thus, new tracking techniques are needed. We propose a probabilistic axon tracking algorithm (PAT). PAT tackles the tracking of axons in two steps: locally (L-PAT) and globally (G-PAT). L-PAT is a probabilistic tracking algorithm that can tackle distorted, cluttered images of densely labeled axons. L-PAT divides a large micrograph into smaller image stacks. It then processes each image stack independently before mapping the axons in each image to a sparse model of axon trajectories. G-PAT merges the sparse L-PAT models into a single global model of axon trajectories by minimizing a global objective function using a probabilistic optimization method. We demonstrate the superior performance of PAT over standard approaches on synthetic data. Furthermore, we successfully apply PAT to densely labeled axons in large images acquired from marmoset brains.

Automated 3-D Neuron Tracing With Precise Branch Erasing and Confidence Controlled Back Tracking

The automatic reconstruction of single neurons from microscopic images is essential to enable large-scale data-driven investigations in neuron morphology research. However, few previous methods were able to generate satisfactory results automatically from 3-D microscopic images without human intervention. In this paper, we developed a new algorithm for automatic 3-D neuron reconstruction. The main idea of the proposed algorithm is to iteratively track backward from the potential neuronal termini to the soma centre. An online confidence score is computed to decide if a tracing iteration should be stopped and discarded from the final reconstruction. The performance improvements comparing with the previous methods are mainly introduced by a more accurate estimation of the traced area and the confidence controlled back-tracking algorithm. The proposed algorithm supports large-scale batch-processing by requiring only one user specified parameter for background segmentation. We bench tested the proposed algorithm on the images obtained from both the DIADEM challenge and the BigNeuron challenge. Our proposed algorithm achieved the state-of-the-art results.

A Segmentation Scheme for Complex Neuronal Arbors and Application to Vibration Sensitive Neurons in the Honeybee Brain

The morphology of a neuron is strongly related to its physiological properties, application of logical product and thus to information processing functions. Optical microscope images are widely used for extracting the structure of neurons. Although several approaches have been proposed to trace and extract complex neuronal structures from microscopy images, available methods remain prone to errors. In this study, we present a practical scheme for processing confocal microscope images and reconstructing neuronal structures. We evaluated this scheme using image data samples and associated “gold standard” reconstructions from the BigNeuron Project. In these samples, dendritic arbors belonging to multiple projection branches of the same neuron overlapped in space, making it difficult to automatically and accurately trace their structural connectivity. Our proposed scheme, which combines several software tools for image masking and filtering with an existing tool for dendritic segmentation and tracing, outperformed state-of-the-art automatic methods in reconstructing such neuron structures. For evaluating our scheme, we applied it to a honeybee local interneuron, DL-Int-1, which has complex arbors and is considered to be a critical neuron for encoding the distance information indicated in the waggle dance of the honeybee.

DeepNeuron: an open deep learning toolbox for neuron tracing

Reconstructing three-dimensional (3D) morphology of neurons is essential for understanding brain structures and functions. Over the past decades, a number of neuron tracing tools including manual, semiautomatic, and fully automatic approaches have been developed to extract and analyze 3D neuronal structures. Nevertheless, most of them were developed based on coding certain rules to extract and connect structural components of a neuron, showing limited performance on complicated neuron morphology. Recently, deep learning outperforms many other machine learning methods in a wide range of image analysis and computer vision tasks. Here we developed a new Open Source toolbox, DeepNeuron, which uses deep learning networks to learn features and rules from data and trace neuron morphology in light microscopy images. DeepNeuron provides a family of modules to solve basic yet challenging problems in neuron tracing. These problems include but not limited to: (1) detecting neuron signal under different image conditions, (2) connecting neuronal signals into tree(s), (3) pruning and refining tree morphology, (4) quantifying the quality of morphology, and (5) classifying dendrites and axons in real time. We have tested DeepNeuron using light microscopy images including bright-field and confocal images of human and mouse brain, on which DeepNeuron demonstrates robustness and accuracy in neuron tracing.