An automated method for cell detection in zebrafish

NeuroSeg-III: efficient neuron segmentation in two-photon Ca2+ imaging data using self-supervised learning

Two-photon Ca2+ imaging technology increasingly plays an essential role in neuroscience research. However, the requirement for extensive professional annotation poses a significant challenge to improving the performance of neuron segmentation models. Here, we present NeuroSeg-III, an innovative self-supervised learning approach specifically designed to achieve fast and precise segmentation of neurons in imaging data. This approach consists of two modules: a self-supervised pre-training network and a segmentation network. After pre-training the encoder of the segmentation network via a self-supervised learning method without any annotated data, we only need to fine-tune the segmentation network with a small amount of annotated data. The segmentation network is designed with YOLOv8s, FasterNet, efficient multi-scale attention mechanism (EMA), and bi-directional feature pyramid network (BiFPN), which enhanced the model's segmentation accuracy while reducing the computational cost and parameters. The generalization of our approach was validated across different Ca2+ indicators and scales of imaging data. Significantly, the proposed neuron segmentation approach exhibits exceptional speed and accuracy, surpassing the current state-of-the-art benchmarks when evaluated using a publicly available dataset. The results underscore the effectiveness of NeuroSeg-III, with employing an efficient training strategy tailored for two-photon Ca2+ imaging data and delivering remarkable precision in neuron segmentation.

Advanced Techniques for the Intelligent Diagnosis of Fish Diseases: A Review

Aquatic products, as essential sources of protein, have attracted considerable concern by producers and consumers. Precise fish disease prevention and treatment may provide not only healthy fish protein but also ecological and economic benefits. However, unlike intelligent two-dimensional diagnoses of plants and crops, one of the most serious challenges confronted in intelligent aquaculture diagnosis is its three-dimensional space. Expert systems have been applied to diagnose fish diseases in recent decades, allowing for restricted diagnosis of certain aquaculture. However, this method needs aquaculture professionals and specialists. In addition, diagnosis speed and efficiency are limited. Therefore, developing a new quick, automatic, and real-time diagnosis approach is very critical. The integration of image-processing and computer vision technology intelligently allows the diagnosis of fish diseases. This study comprehensively reviews image-processing technology and image-based fish disease detection methods, and analyzes the benefits and drawbacks of each diagnostic approach in different environments. Although it is widely acknowledged that there are many approaches for disease diagnosis and pathogen identification, some improvements in detection accuracy and speed are still needed. Constructing AR 3D images of fish diseases, standard and shared datasets, deep learning, and data fusion techniques will be helpful in improving the accuracy and speed of fish disease diagnosis.

Computational Methods for Neuron Segmentation in Two-Photon Calcium Imaging Data: A Survey

Calcium imaging has rapidly become a methodology of choice for real-time in vivo neuron analysis. Its application to large sets of data requires automated tools to annotate and segment cells, allowing scalable image segmentation under reproducible criteria. In this paper, we review and summarize the most recent methods for computational segmentation of calcium imaging. The contributions of the paper are three-fold: we provide an overview of the main algorithms taxonomized in three categories (signal processing, matrix factorization and machine learning-based approaches), we highlight the main advantages and disadvantages of each category and we provide a summary of the performance of the methods that have been tested on public benchmarks (with links to the public code when available).

Recent Advances of Deep Learning for Computational Histopathology: Principles and Applications

Simple Summary

The histopathological image is widely considered as the gold standard for the diagnosis and prognosis of human cancers. Recently, deep learning technology has been extremely successful in the field of computer vision, which has also boosted considerable interest in digital pathology analysis. The aim of our paper is to provide a comprehensive and up-to-date review of the deep learning methods for digital H&E-stained pathology image analysis, including color normalization, nuclei/tissue segmentation, and cancer diagnosis and prognosis. The experimental results of the existing studies demonstrated that deep learning is a promising tool to assist clinicians in the clinical management of human cancers.

Abstract

With the remarkable success of digital histopathology, we have witnessed a rapid expansion of the use of computational methods for the analysis of digital pathology and biopsy image patches. However, the unprecedented scale and heterogeneous patterns of histopathological images have presented critical computational bottlenecks requiring new computational histopathology tools. Recently, deep learning technology has been extremely successful in the field of computer vision, which has also boosted considerable interest in digital pathology applications. Deep learning and its extensions have opened several avenues to tackle many challenging histopathological image analysis problems including color normalization, image segmentation, and the diagnosis/prognosis of human cancers. In this paper, we provide a comprehensive up-to-date review of the deep learning methods for digital H&E-stained pathology image analysis. Specifically, we first describe recent literature that uses deep learning for color normalization, which is one essential research direction for H&E-stained histopathological image analysis. Followed by the discussion of color normalization, we review applications of the deep learning method for various H&E-stained image analysis tasks such as nuclei and tissue segmentation. We also summarize several key clinical studies that use deep learning for the diagnosis and prognosis of human cancers from H&E-stained histopathological images. Finally, online resources and open research problems on pathological image analysis are also provided in this review for the convenience of researchers who are interested in this exciting field.

A Histogram-Based Low-Complexity Approach for the Effective Detection of COVID-19 Disease from CT and X-ray Images

The global COVID-19 pandemic certainly has posed one of the more difficult challenges for researchers in the current century. The development of an automatic diagnostic tool, able to detect the disease in its early stage, could undoubtedly offer a great advantage to the battle against the pandemic. In this regard, most of the research efforts have been focused on the application of Deep Learning (DL) techniques to chest images, including traditional chest X-rays (CXRs) and Computed Tomography (CT) scans. Although these approaches have demonstrated their effectiveness in detecting the COVID-19 disease, they are of huge computational complexity and require large datasets for training. In addition, there may not exist a large amount of COVID-19 CXRs and CT scans available to researchers. To this end, in this paper, we propose an approach based on the evaluation of the histogram from a common class of images that is considered as the target. A suitable inter-histogram distance measures how this target histogram is far from the histogram evaluated on a test image: if this distance is greater than a threshold, the test image is labeled as anomaly, i.e., the scan belongs to a patient affected by COVID-19 disease. Extensive experimental results and comparisons with some benchmark state-of-the-art methods support the effectiveness of the developed approach, as well as demonstrate that, at least when the images of the considered datasets are homogeneous enough (i.e., a few outliers are present), it is not really needed to resort to complex-to-implement DL techniques, in order to attain an effective detection of the COVID-19 disease. Despite the simplicity of the proposed approach, all the considered metrics (i.e., accuracy, precision, recall, and F-measure) attain a value of 1.0 under the selected datasets, a result comparable to the corresponding state-of-the-art DNN approaches, but with a remarkable computational simplicity.

Automatic Identification of Mixed Retinal Cells in Time-Lapse Fluorescent Microscopy Images using High-Dimensional DBSCAN

Despite providing high spatial resolution, functional imaging remains largely unsuitable for high-throughput experiments because current practices require cells to be manually identified in a time-consuming procedure. Against this backdrop, we seek to integrate such high-resolution technique in high-throughput workflow by automating the process of cell identification. As a step forward, we attempt to identify mixed retinal cells in time-lapse fluorescent microscopy images. Unfortunately, usual 2D image segmentation as well as other existing methods do not adequately distinguish between time courses of different spatial locations. Here, the task gets further complicated due to the inherent heterogeneity of cell morphology. To overcome such challenge, we propose to use a high-dimensional (HiD) version of DBSCAN (density based spatial clustering of applications with noise) algorithm, where difference in such time courses are appropriately accounted. Significantly, outcome of the proposed method matches manually identified cells with over 80% accuracy, marking more than 50% improvement compared to a reference 2D method.

Quantification of surviving neurons after contusion, dislocation, and distraction spinal cord injuries using automated methods

This study proposes and validates an automated method for counting neurons in spinal cord injury (SCI) and then uses it to examine and compare the surviving cells in common types of SCI mechanisms. Moderate contusion, dislocation, and distraction SCIs were surgically induced in Sprague Dawley male rats (n = 6 for each type of injury). Their spinal cords were harvested 8 weeks post injury with 5 normal weight-matched rats. The spinal cords were cut, stained with anti-NeuN antibody and fluorescent Nissl, and imaged in the dorsal and ventral horns at various distances to the epicenter. Neurons in the images were automatically counted using an algorithm that was designed to filter non-soma-like objects based on morphological characteristics (size, solidity, circular pattern) and check the remaining objects for the double-stained nucleus/cell body features (brightness variation, brightness distribution, color). To validate the automated method, some of the images were randomly selected for manual counting. The number of surviving cells that were automatically measured by the algorithm was found to be correlated with the values that were manually measured by 2 observers (P < .001) with similar differences (P > .05). Neurons in the dorsal and ventral horns were reduced after the SCIs (P < .05). Dislocation and distraction, respectively, caused the most severe damage to the ventral horn neurons especially near the epicenter and the most extensive and uniform damage to the dorsal horn neurons (P < .05). Our method was proved to be reliable, which is suitable for studying different types of SCI.

Touching Soma Segmentation Based on the Rayburst Sampling Algorithm

Neuronal soma segmentation is essential for morphology quantification analysis. Rapid advances in light microscope imaging techniques have generated such massive amounts of data that time-consuming manual methods cannot meet requirements for high throughput. However, touching soma segmentation is still a challenge for automatic segmentation methods. In this paper, we propose a soma segmentation method that combines the Rayburst sampling algorithm and ellipsoid fitting. The improved Rayburst sampling algorithm is used to detect the soma surface; the ellipsoid fitting method then refines jagged sampled soma surface to generate smooth ellipsoidal shapes for efficient analysis. In experiments, we validated the proposed method by applying it to datasets from the fluorescence micro-optical sectioning tomography (fMOST) system. The results indicate that the proposed method is comparable to the manual segmented gold standard with accurate soma segmentation at a relatively high speed. The proposed method can be extended to large-scale image stacks in the future.

Multi-Pass Adaptive Voting for Nuclei Detection in Histopathological Images

Nuclei detection is often a critical initial step in the development of computer aided diagnosis and prognosis schemes in the context of digital pathology images. While over the last few years, a number of nuclei detection methods have been proposed, most of these approaches make idealistic assumptions about the staining quality of the tissue. In this paper, we present a new Multi-Pass Adaptive Voting (MPAV) for nuclei detection which is specifically geared towards images with poor quality staining and noise on account of tissue preparation artifacts. The MPAV utilizes the symmetric property of nuclear boundary and adaptively selects gradient from edge fragments to perform voting for a potential nucleus location. The MPAV was evaluated in three cohorts with different staining methods: Hematoxylin &Eosin, CD31 &Hematoxylin, and Ki-67 and where most of the nuclei were unevenly and imprecisely stained. Across a total of 47 images and nearly 17,700 manually labeled nuclei serving as the ground truth, MPAV was able to achieve a superior performance, with an area under the precision-recall curve (AUC) of 0.73. Additionally, MPAV also outperformed three state-of-the-art nuclei detection methods, a single pass voting method, a multi-pass voting method, and a deep learning based method.

Robust Nucleus/Cell Detection and Segmentation in Digital Pathology and Microscopy Images: A Comprehensive Review

Digital pathology and microscopy image analysis is widely used for comprehensive studies of cell morphology or tissue structure. Manual assessment is labor intensive and prone to interobserver variations. Computer-aided methods, which can significantly improve the objectivity and reproducibility, have attracted a great deal of interest in recent literature. Among the pipeline of building a computer-aided diagnosis system, nucleus or cell detection and segmentation play a very important role to describe the molecular morphological information. In the past few decades, many efforts have been devoted to automated nucleus/cell detection and segmentation. In this review, we provide a comprehensive summary of the recent state-of-the-art nucleus/cell segmentation approaches on different types of microscopy images including bright-field, phase-contrast, differential interference contrast, fluorescence, and electron microscopies. In addition, we discuss the challenges for the current methods and the potential future work of nucleus/cell detection and segmentation.

Automation of Technology for Cancer Research

Zebrafish embryos can be obtained for research purposes in large numbers at low cost and embryos develop externally in limited space, making them highly suitable for high-throughput cancer studies and drug screens. Non-invasive live imaging of various processes within the larvae is possible due to their transparency during development, and a multitude of available fluorescent transgenic reporter lines.To perform high-throughput studies, handling large amounts of embryos and larvae is required. With such high number of individuals, even minute tasks may become time-consuming and arduous. In this chapter, an overview is given of the developments in the automation of various steps of large scale zebrafish cancer research for discovering important cancer pathways and drugs for the treatment of human disease. The focus lies on various tools developed for cancer cell implantation, embryo handling and sorting, microfluidic systems for imaging and drug treatment, and image acquisition and analysis. Examples will be given of employment of these technologies within the fields of toxicology research and cancer research.

Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images

Automated nuclear detection is a critical step for a number of computer assisted pathology related image analysis algorithms such as for automated grading of breast cancer tissue specimens. The Nottingham Histologic Score system is highly correlated with the shape and appearance of breast cancer nuclei in histopathological images. However, automated nucleus detection is complicated by 1) the large number of nuclei and the size of high resolution digitized pathology images, and 2) the variability in size, shape, appearance, and texture of the individual nuclei. Recently there has been interest in the application of "Deep Learning" strategies for classification and analysis of big image data. Histopathology, given its size and complexity, represents an excellent use case for application of deep learning strategies. In this paper, a Stacked Sparse Autoencoder (SSAE), an instance of a deep learning strategy, is presented for efficient nuclei detection on high-resolution histopathological images of breast cancer. The SSAE learns high-level features from just pixel intensities alone in order to identify distinguishing features of nuclei. A sliding window operation is applied to each image in order to represent image patches via high-level features obtained via the auto-encoder, which are then subsequently fed to a classifier which categorizes each image patch as nuclear or non-nuclear. Across a cohort of 500 histopathological images (2200 × 2200) and approximately 3500 manually segmented individual nuclei serving as the groundtruth, SSAE was shown to have an improved F-measure 84.49% and an average area under Precision-Recall curve (AveP) 78.83%. The SSAE approach also out-performed nine other state of the art nuclear detection strategies.

Detection of Dendritic Spines Using Wavelet-Based Conditional Symmetric Analysis and Regularized Morphological Shared-Weight Neural Networks

Identification and detection of dendritic spines in neuron images are of high interest in diagnosis and treatment of neurological and psychiatric disorders (e.g., Alzheimer's disease, Parkinson's diseases, and autism). In this paper, we have proposed a novel automatic approach using wavelet-based conditional symmetric analysis and regularized morphological shared-weight neural networks (RMSNN) for dendritic spine identification involving the following steps: backbone extraction, localization of dendritic spines, and classification. First, a new algorithm based on wavelet transform and conditional symmetric analysis has been developed to extract backbone and locate the dendrite boundary. Then, the RMSNN has been proposed to classify the spines into three predefined categories (mushroom, thin, and stubby). We have compared our proposed approach against the existing methods. The experimental result demonstrates that the proposed approach can accurately locate the dendrite and accurately classify the spines into three categories with the accuracy of 99.1% for “mushroom” spines, 97.6% for “stubby” spines, and 98.6% for “thin” spines.

An automated three-dimensional detection and segmentation method for touching cells by integrating concave points clustering and random walker algorithm

Characterizing cytoarchitecture is crucial for understanding brain functions and neural diseases. In neuroanatomy, it is an important task to accurately extract cell populations' centroids and contours. Recent advances have permitted imaging at single cell resolution for an entire mouse brain using the Nissl staining method. However, it is difficult to precisely segment numerous cells, especially those cells touching each other. As presented herein, we have developed an automated three-dimensional detection and segmentation method applied to the Nissl staining data, with the following two key steps: 1) concave points clustering to determine the seed points of touching cells; and 2) random walker segmentation to obtain cell contours. Also, we have evaluated the performance of our proposed method with several mouse brain datasets, which were captured with the micro-optical sectioning tomography imaging system, and the datasets include closely touching cells. Comparing with traditional detection and segmentation methods, our approach shows promising detection accuracy and high robustness.

Automated prior knowledge-based quantification of neuronal patterns in the spinal cord of zebrafish

MOTIVATION

To reliably assess the effects of unknown chemicals on the development of fluorescently labeled sensory-, moto- and interneuron populations in the spinal cord of zebrafish, automated data analysis is essential.

RESULTS

For the evaluation of a high-throughput screen of a large chemical library, we developed a new method for the automated extraction of quantitative information from green fluorescent protein (eGFP) and red fluorescent protein (RFP) labeled spinal cord neurons in double-transgenic zebrafish embryos. The methodology comprises region of interest detection, intensity profiling with reference comparison and neuron distribution histograms. All methods were validated on a manually evaluated pilot study using a Notch inhibitor dose-response experiment. The automated evaluation showed superior performance to manual investigation regarding time consumption, information detail and reproducibility.

AVAILABILITY AND IMPLEMENTATION

Being part of GNU General Public Licence (GNU-GPL) licensed open-source MATLAB toolbox Gait-CAD, an implementation of the presented methods is publicly available for download at http://sourceforge.net/projects/zebrafishimage/.

NeuroGPS: automated localization of neurons for brain circuits using L1 minimization model

Drawing the map of neuronal circuits at microscopic resolution is important to explain how brain works. Recent progresses in fluorescence labeling and imaging techniques have enabled measuring the whole brain of a rodent like a mouse at submicron-resolution. Considering the huge volume of such datasets, automatic tracing and reconstruct the neuronal connections from the image stacks is essential to form the large scale circuits. However, the first step among which, automated location the soma across different brain areas remains a challenge. Here, we addressed this problem by introducing L1 minimization model. We developed a fully automated system, NeuronGlobalPositionSystem (NeuroGPS) that is robust to the broad diversity of shape, size and density of the neurons in a mouse brain. This method allows locating the neurons across different brain areas without human intervention. We believe this method would facilitate the analysis of the neuronal circuits for brain function and disease studies.

Automated processing of zebrafish imaging data: a survey

Due to the relative transparency of its embryos and larvae, the zebrafish is an ideal model organism for bioimaging approaches in vertebrates. Novel microscope technologies allow the imaging of developmental processes in unprecedented detail, and they enable the use of complex image-based read-outs for high-throughput/high-content screening. Such applications can easily generate Terabytes of image data, the handling and analysis of which becomes a major bottleneck in extracting the targeted information. Here, we describe the current state of the art in computational image analysis in the zebrafish system. We discuss the challenges encountered when handling high-content image data, especially with regard to data quality, annotation, and storage. We survey methods for preprocessing image data for further analysis, and describe selected examples of automated image analysis, including the tracking of cells during embryogenesis, heartbeat detection, identification of dead embryos, recognition of tissues and anatomical landmarks, and quantification of behavioral patterns of adult fish. We review recent examples for applications using such methods, such as the comprehensive analysis of cell lineages during early development, the generation of a three-dimensional brain atlas of zebrafish larvae, and high-throughput drug screens based on movement patterns. Finally, we identify future challenges for the zebrafish image analysis community, notably those concerning the compatibility of algorithms and data formats for the assembly of modular analysis pipelines.

Automated and accurate detection of soma location and surface morphology in large-scale 3D neuron images

Automated and accurate localization and morphometry of somas in 3D neuron images is essential for quantitative studies of neural networks in the brain. However, previous methods are limited in obtaining the location and surface morphology of somas with variable size and uneven staining in large-scale 3D neuron images. In this work, we proposed a method for automated soma locating in large-scale 3D neuron images that contain relatively sparse soma distributions. This method involves three steps: (i) deblocking the image with overlap between adjacent sub-stacks; (ii) locating the somas in each small sub-stack using multi-scale morphological close and adaptive thresholds; and (iii) fusion of the repeatedly located somas in all sub-stacks. We also describe a new method for the accurate detection of the surface morphology of somas containing hollowness; this was achieved by improving the classical Rayburst Sampling with a new gradient-based criteria. Three 3D neuron image stacks of different sizes were used to quantitatively validate our methods. For the soma localization algorithm, the average recall and precision were greater than 93% and 96%, respectively. For the soma surface detection algorithm, the overlap of the volumes created by automatic detection of soma surfaces and manually segmenting soma volumes was more than 84% for 89% of all correctly detected somas. Our method for locating somas can reveal the soma distributions in large-scale neural networks more efficiently. The method for soma surface detection will serve as a valuable tool for systematic studies of neuron types based on neuron structure.

Advances in zebrafish chemical screening technologies

Due to several inherent advantages, zebrafish are being utilized in increasingly sophisticated screens to assess the physiological effects of chemical compounds directly in living vertebrate organisms. Diverse screening platforms showcase these advantages. Morphological assays encompassing basic qualitative observations to automated imaging, manipulation, and data-processing systems provide whole organism to subcellular levels of detail. Behavioral screens extend chemical screening to the level of complex systems. In addition, zebrafish-based disease models provide a means of identifying new potential therapeutic strategies. Automated systems for handling/sorting, high-resolution imaging and quantitative data collection have significantly increased throughput in recent years. These advances will make it easier to capture multiple streams of information from a given sample and facilitate integration of zebrafish at the earliest stages of the drug-discovery process, providing potential solutions to current drug-development bottlenecks. Here we outline advances that have been made within the growing field of zebrafish chemical screening.

A high-throughput analysis method to detect regions of interest and quantify zebrafish embryo images

Zebrafish is widely used to understand neural development and model various neurodegenerative diseases. Zebrafish embryos are optically transparent, have a short development period, and can be kept alive in microplates for days, making them amenable to high-throughput microscopic imaging. As a result of high-throughput experiments, a large number of images can be generated in a single experiment, posing a challenge to researchers to analyze them efficiently and quantitatively. In this work, we develop an image processing focused on detecting and quantifying pigments in zebrafish embryos. The algorithm automatically detects a region of interest (ROI) enclosing an area around the pigments and then segment the pigments for quantification. In this process, the algorithm identifies the head and torso at first, and then finds the boundaries corresponding to the back and abdomen by taking advantage of a priori information about the anatomy of zebrafish embryos. The method is robust in terms that it can detect and quantify pigments even when the embryos have different orientations and curvatures. We used real data to demonstrate the performance of the method to extract phenotypic information from zebrafish embryo images and compared its results with manual analysis for verification.

Automatic identification of fluorescently labeled brain cells for rapid functional imaging

The on-line identification of labeled cells and vessels is a rate-limiting step in scanning microscopy. We use supervised learning to formulate an algorithm that rapidly and automatically tags fluorescently labeled somata in full-field images of cortex and constructs an optimized scan path through these cells. A single classifier works across multiple subjects, regions of the cortex of similar depth, and different magnification and contrast levels without the need to retrain the algorithm. Retraining only has to be performed when the morphological properties of the cells change significantly. In conjunction with two-photon laser scanning microscopy and bulk-labeling of cells in layers 2/3 of rat parietal cortex with a calcium indicator, we can automatically identify ∼ 50 cells within 1 min and sample them at ∼ 100 Hz with a signal-to-noise ratio of ∼ 10.

Automated three-dimensional detection and counting of neuron somata

We present a novel approach for automated detection of neuron somata. A three-step processing pipeline is described on the example of confocal image stacks of NeuN-stained neurons from rat somato-sensory cortex. It results in a set of position landmarks, representing the midpoints of all neuron somata. In the first step, foreground and background pixels are identified, resulting in a binary image. It is based on local thresholding and compensates for imaging and staining artifacts. Once this pre-processing guarantees a standard image quality, clusters of touching neurons are separated in the second step, using a marker-based watershed approach. A model-based algorithm completes the pipeline. It assumes a dominant neuron population with Gaussian distributed volumes within one microscopic field of view. Remaining larger objects are hence split or treated as a second neuron type. A variation of the processing pipeline is presented, showing that our method can also be used for co-localization of neurons in multi-channel images. As an example, we process 2-channel stacks of NeuN-stained somata, labeling all neurons, counterstained with GAD67, labeling GABAergic interneurons, using an adapted pre-processing step for the second channel. The automatically generated landmark sets are compared to manually placed counterparts. A comparison yields that the deviation in landmark position is negligible and that the difference between the numbers of manually and automatically counted neurons is less than 4%. In consequence, this novel approach for neuron counting is a reliable and objective alternative to manual detection.