A deep-learning approach for online cell identification and trace extraction in functional two-photon calcium imaging

Aberration correction in long GRIN lens-based microendoscopes for extended field-of-view two-photon imaging in deep brain regions

Two-photon (2P) fluorescence imaging through gradient index (GRIN) lens-based endoscopes is fundamental to investigate the functional properties of neural populations in deep brain circuits. However, GRIN lenses have intrinsic optical aberrations, which severely degrade their imaging performance. GRIN aberrations decrease the signal-to-noise ratio (SNR) and spatial resolution of fluorescence signals, especially in lateral portions of the field-of-view (FOV), leading to restricted FOV and smaller number of recorded neurons. This is especially relevant for GRIN lenses of several millimeters in length, which are needed to reach the deeper regions of the rodent brain. We have previously demonstrated a novel method to enlarge the FOV and improve the spatial resolution of 2P microendoscopes based on GRIN lenses of length <4.1 mm (Antonini et al., 2020). However, previously developed microendoscopes were too short to reach the most ventral regions of the mouse brain. In this study, we combined optical simulations with fabrication of aspherical polymer microlenses through three-dimensional (3D) microprinting to correct for optical aberrations in long (length >6 mm) GRIN lens-based microendoscopes (diameter, 500 µm). Long corrected microendoscopes had improved spatial resolution, enabling imaging in significantly enlarged FOVs. Moreover, using synthetic calcium data we showed that aberration correction enabled detection of cells with higher SNR of fluorescent signals and decreased cross-contamination between neurons. Finally, we applied long corrected microendoscopes to perform large-scale and high-precision recordings of calcium signals in populations of neurons in the olfactory cortex, a brain region laying approximately 5 mm from the brain surface, of awake head-fixed mice. Long corrected microendoscopes are powerful new tools enabling population imaging with unprecedented large FOV and high spatial resolution in the most ventral regions of the mouse brain.

A deep learning framework for automated and generalized synaptic event analysis

Quantitative information about synaptic transmission is key to our understanding of neural function. Spontaneously occurring synaptic events carry fundamental information about synaptic function and plasticity. However, their stochastic nature and low signal-to-noise ratio present major challenges for the reliable and consistent analysis. Here, we introduce miniML, a supervised deep learning-based method for accurate classification and automated detection of spontaneous synaptic events. Comparative analysis using simulated ground-truth data shows that miniML outperforms existing event analysis methods in terms of both precision and recall. miniML enables precise detection and quantification of synaptic events in electrophysiological recordings. We demonstrate that the deep learning approach generalizes easily to diverse synaptic preparations, different electrophysiological and optical recording techniques, and across animal species. miniML provides not only a comprehensive and robust framework for automated, reliable, and standardized analysis of synaptic events, but also opens new avenues for high-throughput investigations of neural function and dysfunction.

Simultaneous, real-time tracking of many neuromodulatory signals with Multiplexed Optical Recording of Sensors on a micro-Endoscope

Dozens of extracellular molecules jointly impact a given neuron, yet we lack methods to simultaneously record many such signals in real time. We developed a probe to track ten or more neuropeptides and neuromodulators using spatial multiplexing of genetically encoded fluorescent sensors. Cultured cells expressing one sensor at a time are immobilized at the front of a gradient refractive index (GRIN) lens for 3D two-photon imaging in vitro and in vivo . The sensor identity and detection sensitivity of each cell are determined via robotic dipping of the probe into wells containing various ligands and concentrations. Using this probe, we detected stimulation-evoked release of multiple neuromodulators in acute brain slices. We also tracked endogenous and drug-evoked changes in cerebrospinal fluid composition in the awake mouse lateral ventricle, which triggered downstream activation of the choroid plexus epithelium. Our approach offers a first step towards quantitative, real-time, high-dimensional tracking of brain fluid composition.

Window into the Brain: In Vivo Multiphoton Imaging

Decoding the principles underlying neuronal information processing necessitates the emergence of techniques and methodologies to monitor multiscale brain networks in behaving animals over long periods of time. Novel advances in biophotonics, specifically progress in multiphoton microscopy, combined with the development of optical indicators for neuronal activity have provided the possibility to concurrently track brain functions at scales ranging from individual neurons to thousands of neurons across connected brain regions. This Review presents state-of-the-art multiphoton imaging modalities and optical indicators for in vivo brain imaging, highlighting recent advancements and current challenges in the field.

A generative benchmark for evaluating the performance of fluorescent cell image segmentation

Fluorescent cell imaging technology is fundamental in life science research, offering a rich source of image data crucial for understanding cell spatial positioning, differentiation, and decision-making mechanisms. As the volume of this data expands, precise image analysis becomes increasingly critical. Cell segmentation, a key analysis step, significantly influences quantitative analysis outcomes. However, selecting the most effective segmentation method is challenging, hindered by existing evaluation methods' inaccuracies, lack of graded evaluation, and narrow assessment scope. Addressing this, we developed a novel framework with two modules: StyleGAN2-based contour generation and Pix2PixHD-based image rendering, producing diverse, graded-density cell images. Using this dataset, we evaluated three leading cell segmentation methods: DeepCell, CellProfiler, and CellPose. Our comprehensive comparison revealed CellProfiler's superior accuracy in segmenting cytoplasm and nuclei. Our framework diversifies cell image data generation and systematically addresses evaluation challenges in cell segmentation technologies, establishing a solid foundation for advancing research and applications in cell image analysis.

CalTrig: A GUI-based Machine Learning Approach for Decoding Neuronal Calcium Transients in Freely Moving Rodents

Advances in in vivoC a 2 + imaging using miniature microscopes have enabled researchers to study single-neuron activity in freely-moving animals. Tools such as MiniAN and CalmAn have been developed to convertCa 2 + visual signals to numerical data, collectively referred to as CalV2N. However, substantial challenges remain in analyzing the large datasets generated by CalV2N, particularly in integrating data streams, evaluating CalV2N output quality, and reliably and efficiently identifyingC a 2 + transients. In this study, we introduce CalTrig, an open-source graphical user interface (GUI) tool designed to address these challenges at the post-CalV2N stage of data processing. CalTrig integrates multiple data streams, includingC a 2 + imaging, neuronal footprints,C a 2 + traces, and behavioral tracking, and offers capabilities for evaluating the quality of CalV2N outputs. It enables synchronized visualization and efficientC a 2 + transient identification. We evaluated four machine learning models (i.e., GRU, LSTM, Transformer, and Local Transformer) forC a 2 + transient detection. Our results indicate that the GRU model offers the highest predictability and computational efficiency, achieving stable performance across training sessions, different animals and even among different brain regions. The integration of manual, parameter-based, and machine learning-based detection methods in CalTrig provides flexibility and accuracy for various research applications. The user-friendly interface and low computing demands of CalTrig make it accessible to neuroscientists without programming expertise. We further conclude that CalTrig enables deeper exploration of brain function, supports hypothesis generation about neuronal mechanisms, and opens new avenues for understanding neurological disorders and developing treatments.

NeuroART: Real-Time Analysis and Targeting of Neuronal Population Activity during Calcium Imaging for Informed Closed-Loop Experiments

Two-photon calcium imaging allows for the activity readout of large populations of neurons at single cell resolution in living organisms, yielding new insights into how the brain processes information. Holographic optogenetics allows us to trigger activity of this population directly, raising the possibility of injecting information into a living brain. Optogenetic triggering of activity that mimics "natural" information, however, requires identification of stimulation targets based on real-time analysis of the functional network. We have developed NeuroART (Neuronal Analysis in Real Time), software that provides real-time readout of neuronal activity integrated with downstream analysis of correlations and synchrony and of sensory metadata. On the example of auditory stimuli, we demonstrate real-time inference of the contribution of each neuron in the field of view to sensory information processing. To avoid the limitations of microscope hardware and enable collaboration of multiple research groups, NeuroART taps into microscope data streams without the need for modification of microscope control software and is compatible with a wide range of microscope platforms. NeuroART also integrates the capability to drive a spatial light modulator (SLM) for holographic photostimulation of optimal stimulation targets, enabling real-time modification of functional networks. Neurons used for photostimulation experiments were extracted from Sprague Dawley rat embryos of both sexes.

Combinatorial responsiveness of single chemosensory neurons to external stimulation of mouse explants revealed by DynamicNeuronTracker

Calcium fluorescence imaging enables us to investigate how individual neurons of live animals encode sensory input or drive specific behaviors. Extracting and interpreting large-scale neuronal activity from imaging data are crucial steps in harnessing this information. A significant challenge arises from uncorrectable tissue deformation, which disrupts the effectiveness of existing neuron segmentation methods. Here, we propose an open-source software, DynamicNeuronTracker (DyNT), which generates dynamic neuron masks for deforming and/or incompletely registered 3D calcium imaging data using patch-matching iterations. We demonstrate that DyNT accurately tracks densely populated neurons, whereas a widely used static segmentation method often produces erroneous masks. DyNT also includes automated statistical analyses for interpreting neuronal responses to multiple sequential stimuli. We applied DyNT to analyze the responses of pheromone-sensing neurons in mice to controlled stimulation. We found that four bile acids and four sulfated steroids activated 15 subpopulations of sensory neurons with distinct combinatorial response profiles, revealing a strong bias toward detecting sulfated estrogen and pregnanolone.

CalciSeg: A versatile approach for unsupervised segmentation of calcium imaging data

Recent advances in calcium imaging, including the development of fast and sensitive genetically encoded indicators, high-resolution camera chips for wide-field imaging, and resonant scanning mirrors in laser scanning microscopy, have notably improved the temporal and spatial resolution of functional imaging analysis. Nonetheless, the variability of imaging approaches and brain structures challenges the development of versatile and reliable segmentation methods. Standard techniques, such as manual selection of regions of interest or machine learning solutions, often fall short due to either user bias, non-transferability among systems, or computational demand. To overcome these issues, we developed CalciSeg, a data-driven and reproducible approach for unsupervised functional calcium imaging data segmentation. CalciSeg addresses the challenges associated with brain structure variability and user bias by offering a computationally efficient solution for automatic image segmentation based on two parameters: regions' size limits and number of refinement iterations. We evaluated CalciSeg efficacy on datasets of varied complexity, different insect species (locusts, bees, and cockroaches), and imaging systems (wide-field, confocal, and multiphoton), showing the robustness and generality of our approach. Finally, the user-friendly nature and open-source availability of CalciSeg facilitate the integration of this algorithm into existing analysis pipelines.

Closed-loop experiments and brain machine interfaces with multiphoton microscopy

In the field of neuroscience, the importance of constructing closed-loop experimental systems has increased in conjunction with technological advances in measuring and controlling neural activity in live animals. We provide an overview of recent technological advances in the field, focusing on closed-loop experimental systems where multiphoton microscopy-the only method capable of recording and controlling targeted population activity of neurons at a single-cell resolution in vivo-works through real-time feedback. Specifically, we present some examples of brain machine interfaces (BMIs) using in vivo two-photon calcium imaging and discuss applications of two-photon optogenetic stimulation and adaptive optics to real-time BMIs. We also consider conditions for realizing future optical BMIs at the synaptic level, and their possible roles in understanding the computational principles of the brain.

NeuroSeg-III: efficient neuron segmentation in two-photon Ca2+ imaging data using self-supervised learning

Two-photon Ca2+ imaging technology increasingly plays an essential role in neuroscience research. However, the requirement for extensive professional annotation poses a significant challenge to improving the performance of neuron segmentation models. Here, we present NeuroSeg-III, an innovative self-supervised learning approach specifically designed to achieve fast and precise segmentation of neurons in imaging data. This approach consists of two modules: a self-supervised pre-training network and a segmentation network. After pre-training the encoder of the segmentation network via a self-supervised learning method without any annotated data, we only need to fine-tune the segmentation network with a small amount of annotated data. The segmentation network is designed with YOLOv8s, FasterNet, efficient multi-scale attention mechanism (EMA), and bi-directional feature pyramid network (BiFPN), which enhanced the model's segmentation accuracy while reducing the computational cost and parameters. The generalization of our approach was validated across different Ca2+ indicators and scales of imaging data. Significantly, the proposed neuron segmentation approach exhibits exceptional speed and accuracy, surpassing the current state-of-the-art benchmarks when evaluated using a publicly available dataset. The results underscore the effectiveness of NeuroSeg-III, with employing an efficient training strategy tailored for two-photon Ca2+ imaging data and delivering remarkable precision in neuron segmentation.

ViNe-Seg: deep-learning-assisted segmentation of visible neurons and subsequent analysis embedded in a graphical user interface

SUMMARY

Segmentation of neural somata is a crucial and usually the most time-consuming step in the analysis of optical functional imaging of neuronal microcircuits. In recent years, multiple auto-segmentation tools have been developed to improve the speed and consistency of the segmentation process, mostly, using deep learning approaches. Current segmentation tools, while advanced, still encounter challenges in producing accurate segmentation results, especially in datasets with a low signal-to-noise ratio. This has led to a reliance on manual segmentation techniques. However, manual methods, while customized to specific laboratory protocols, can introduce variability due to individual differences in interpretation, potentially affecting dataset consistency across studies. In response to this challenge, we present ViNe-Seg: a deep-learning-based semi-automatic segmentation tool that offers (i) detection of visible neurons, irrespective of their activity status; (ii) the ability to perform segmentation during an ongoing experiment; (iii) a user-friendly graphical interface that facilitates expert supervision, ensuring precise identification of Regions of Interest; (iv) an array of segmentation models with the option of training custom models and sharing them with the community; and (v) seamless integration of subsequent analysis steps.

AVAILABILITY AND IMPLEMENTATION

ViNe-Seg code and documentation are publicly available at https://github.com/NiRuff/ViNe-Seg and can be installed from https://pypi.org/project/ViNeSeg/.

Wireless closed-loop deep brain stimulation using microelectrode array probes

Deep brain stimulation (DBS), including optical stimulation and electrical stimulation, has been demonstrated considerable value in exploring pathological brain activity and developing treatments for neural disorders. Advances in DBS microsystems based on implantable microelectrode array (MEA) probes have opened up new opportunities for closed-loop DBS (CL-DBS) in situ. This technology can be used to detect damaged brain circuits and test the therapeutic potential for modulating the output of these circuits in a variety of diseases simultaneously. Despite the success and rapid utilization of MEA probe-based CL-DBS microsystems, key challenges, including excessive wired communication, need to be urgently resolved. In this review, we considered recent advances in MEA probe-based wireless CL-DBS microsystems and outlined the major issues and promising prospects in this field. This technology has the potential to offer novel therapeutic options for psychiatric disorders in the future.

Optogenetics 2.0: challenges and solutions towards a quantitative probing of neural circuits

To exploit the full potential of optogenetics, we need to titrate and tailor optogenetic methods to emulate naturalistic circuit function. For that, the following prerequisites need to be met: first, we need to target opsin expression not only to genetically defined neurons per se, but to specifically target a functional node. Second, we need to assess the scope of optogenetic modulation, i.e. the fraction of optogenetically modulated neurons. Third, we need to integrate optogenetic control in a closed loop setting. Fourth, we need to further safe and stable gene expression and light delivery to bring optogenetics to the clinics. Here, we review these concepts for the human and rodent brain.

A Deep Learning Approach for Improving Two-Photon Vascular Imaging Speeds

A potential method for tracking neurovascular disease progression over time in preclinical models is multiphoton fluorescence microscopy (MPM), which can image cerebral vasculature with capillary-level resolution. However, obtaining high-quality, three-dimensional images with traditional point scanning MPM is time-consuming and limits sample sizes for chronic studies. Here, we present a convolutional neural network-based (PSSR Res-U-Net architecture) algorithm for fast upscaling of low-resolution or sparsely sampled images and combine it with a segmentation-less vectorization process for 3D reconstruction and statistical analysis of vascular network structure. In doing so, we also demonstrate that the use of semi-synthetic training data can replace the expensive and arduous process of acquiring low- and high-resolution training pairs without compromising vectorization outcomes, and thus open the possibility of utilizing such approaches for other MPM tasks where collecting training data is challenging. We applied our approach to images with large fields of view from a mouse model and show that our method generalizes across imaging depths, disease states and other differences in neurovasculature. Our pretrained models and lightweight architecture can be used to reduce MPM imaging time by up to fourfold without any changes in underlying hardware, thereby enabling deployability across a range of settings.

Deep-brain optical recording of neural dynamics during behavior

In vivo fluorescence recording techniques have produced landmark discoveries in neuroscience, providing insight into how single cell and circuit-level computations mediate sensory processing and generate complex behaviors. While much attention has been given to recording from cortical brain regions, deep-brain fluorescence recording is more complex because it requires additional measures to gain optical access to harder to reach brain nuclei. Here we discuss detailed considerations and tradeoffs regarding deep-brain fluorescence recording techniques and provide a comprehensive guide for all major steps involved, from project planning to data analysis. The goal is to impart guidance for new and experienced investigators seeking to use in vivo deep fluorescence optical recordings in awake, behaving rodent models.

Deep learning in mesoscale brain image analysis: A review

Mesoscale microscopy images of the brain contain a wealth of information which can help us understand the working mechanisms of the brain. However, it is a challenging task to process and analyze these data because of the large size of the images, their high noise levels, the complex morphology of the brain from the cellular to the regional and anatomical levels, the inhomogeneous distribution of fluorescent labels in the cells and tissues, and imaging artifacts. Due to their impressive ability to extract relevant information from images, deep learning algorithms are widely applied to microscopy images of the brain to address these challenges and they perform superiorly in a wide range of microscopy image processing and analysis tasks. This article reviews the applications of deep learning algorithms in brain mesoscale microscopy image processing and analysis, including image synthesis, image segmentation, object detection, and neuron reconstruction and analysis. We also discuss the difficulties of each task and possible directions for further research.

Bridging live-cell imaging and next-generation cancer treatment

By providing spatial, molecular and morphological data over time, live-cell imaging can provide a deeper understanding of the cellular and signalling events that determine cancer response to treatment. Understanding this dynamic response has the potential to enhance clinical outcome by identifying biomarkers or actionable targets to improve therapeutic efficacy. Here, we review recent applications of live-cell imaging for uncovering both tumour heterogeneity in treatment response and the mode of action of cancer-targeting drugs. Given the increasing uses of T cell therapies, we discuss the unique opportunity of time-lapse imaging for capturing the interactivity and motility of immunotherapies. Although traditionally limited in the number of molecular features captured, novel developments in multidimensional imaging and multi-omics data integration offer strategies to connect single-cell dynamics to molecular phenotypes. We review the effect of these recent technological advances on our understanding of the cellular dynamics of tumour targeting and discuss their implication for next-generation precision medicine.

Pupil engineering for extended depth-of-field imaging in a fluorescence miniscope

Significance

Fluorescence head-mounted microscopes, i.e., miniscopes, have emerged as powerful tools to analyze in-vivo neural populations but exhibit a limited depth-of-field (DoF) due to the use of high numerical aperture (NA) gradient refractive index (GRIN) objective lenses.

Aim

We present extended depth-of-field (EDoF) miniscope, which integrates an optimized thin and lightweight binary diffractive optical element (DOE) onto the GRIN lens of a miniscope to extend the DoF by 2.8× between twin foci in fixed scattering samples.

Approach

We use a genetic algorithm that considers the GRIN lens' aberration and intensity loss from scattering in a Fourier optics-forward model to optimize a DOE and manufacture the DOE through single-step photolithography. We integrate the DOE into EDoF-Miniscope with a lateral accuracy of 70  μm to produce high-contrast signals without compromising the speed, spatial resolution, size, or weight.

Results

We characterize the performance of EDoF-Miniscope across 5- and 10-μm fluorescent beads embedded in scattering phantoms and demonstrate that EDoF-Miniscope facilitates deeper interrogations of neuronal populations in a 100-μm-thick mouse brain sample and vessels in a whole mouse brain sample.

Conclusions

Built from off-the-shelf components and augmented by a customizable DOE, we expect that this low-cost EDoF-Miniscope may find utility in a wide range of neural recording applications.

Sensitive multicolor indicators for monitoring norepinephrine in vivo

Genetically encoded indicators engineered from G-protein-coupled receptors are important tools that enable high-resolution in vivo neuromodulator imaging. Here, we introduce a family of sensitive multicolor norepinephrine (NE) indicators, which includes nLightG (green) and nLightR (red). These tools report endogenous NE release in vitro, ex vivo and in vivo with improved sensitivity, ligand selectivity and kinetics, as well as a distinct pharmacological profile compared with previous state-of-the-art GRABNE indicators. Using in vivo multisite fiber photometry recordings of nLightG, we could simultaneously monitor optogenetically evoked NE release in the mouse locus coeruleus and hippocampus. Two-photon imaging of nLightG revealed locomotion and reward-related NE transients in the dorsal CA1 area of the hippocampus. Thus, the sensitive NE indicators introduced here represent an important addition to the current repertoire of indicators and provide the means for a thorough investigation of the NE system.

Improved Workflow for Analysis of Vascular Myocyte Time-Series and Line-Scan Ca2+ Imaging Datasets

Intracellular Ca2+ signals are key for the regulation of cellular processes ranging from myocyte contraction, hormonal secretion, neural transmission, cellular metabolism, transcriptional regulation, and cell proliferation. Measurement of cellular Ca2+ is routinely performed using fluorescence microscopy with biological indicators. Analysis of deterministic signals is reasonably straightforward as relevant data can be discriminated based on the timing of cellular responses. However, analysis of stochastic, slower oscillatory events, as well as rapid subcellular Ca2+ responses, takes considerable time and effort which often includes visual analysis by trained investigators, especially when studying signals arising from cells embedded in complex tissues. The purpose of the current study was to determine if full-frame time-series and line-scan image analysis workflow of Fluo-4 generated Ca2+ fluorescence data from vascular myocytes could be automated without introducing errors. This evaluation was addressed by re-analyzing a published "gold standard" full-frame time-series dataset through visual analysis of Ca2+ signals from recordings made in pulmonary arterial myocytes of en face arterial preparations. We applied a combination of data driven and statistical approaches with comparisons to our published data to assess the fidelity of the various approaches. Regions of interest with Ca2+ oscillations were detected automatically post hoc using the LCPro plug-in for ImageJ. Oscillatory signals were separated based on event durations between 4 and 40 s. These data were filtered based on cutoffs obtained from multiple methods and compared to the published manually curated "gold standard" dataset. Subcellular focal and rapid Ca2+ "spark" events from line-scan recordings were examined using SparkLab 5.8, which is a custom automated detection and analysis program. After filtering, the number of true positives, false positives, and false negatives were calculated through comparisons to visually derived "gold standard" datasets. Positive predictive value, sensitivity, and false discovery rates were calculated. There were very few significant differences between the automated and manually curated results with respect to quality of the oscillatory and Ca2+ spark events, and there were no systematic biases in the data curation or filtering techniques. The lack of statistical difference in event quality between manual data curation and statistically derived critical cutoff techniques leads us to believe that automated analysis techniques can be reliably used to analyze spatial and temporal aspects to Ca2+ imaging data, which will improve experiment workflow.

ASTRA: a deep learning algorithm for fast semantic segmentation of large-scale astrocytic networks

Changes in the intracellular calcium concentration are a fundamental fingerprint of astrocytes, the main type of glial cell. Astrocyte calcium signals can be measured with two-photon microscopy, occur in anatomically restricted subcellular regions, and are coordinated across astrocytic networks. However, current analytical tools to identify the astrocytic subcellular regions where calcium signals occur are time-consuming and extensively rely on user-defined parameters. These limitations limit reproducibility and prevent scalability to large datasets and fields-of-view. Here, we present Astrocytic calcium Spatio-Temporal Rapid Analysis (ASTRA), a novel software combining deep learning with image feature engineering for fast and fully automated semantic segmentation of two-photon calcium imaging recordings of astrocytes. We applied ASTRA to several two-photon microscopy datasets and found that ASTRA performed rapid detection and segmentation of astrocytic cell somata and processes with performance close to that of human experts, outperformed state-of-the-art algorithms for the analysis of astrocytic and neuronal calcium data, and generalized across indicators and acquisition parameters. We also applied ASTRA to the first report of two-photon mesoscopic imaging of hundreds of astrocytes in awake mice, documenting large-scale redundant and synergistic interactions in extended astrocytic networks. ASTRA is a powerful tool enabling closed-loop and large-scale reproducible investigation of astrocytic morphology and function.

NeuroWRAP: integrating, validating, and sharing neurodata analysis workflows

Multiphoton calcium imaging is one of the most powerful tools in modern neuroscience. However, multiphoton data require significant pre-processing of images and post-processing of extracted signals. As a result, many algorithms and pipelines have been developed for the analysis of multiphoton data, particularly two-photon imaging data. Most current studies use one of several algorithms and pipelines that are published and publicly available, and add customized upstream and downstream analysis elements to fit the needs of individual researchers. The vast differences in algorithm choices, parameter settings, pipeline composition, and data sources combine to make collaboration difficult, and raise questions about the reproducibility and robustness of experimental results. We present our solution, called NeuroWRAP (www.neurowrap.org), which is a tool that wraps multiple published algorithms together, and enables integration of custom algorithms. It enables development of collaborative, shareable custom workflows and reproducible data analysis for multiphoton calcium imaging data enabling easy collaboration between researchers. NeuroWRAP implements an approach to evaluate the sensitivity and robustness of the configured pipelines. When this sensitivity analysis is applied to a crucial step of image analysis, cell segmentation, we find a substantial difference between two popular workflows, CaImAn and Suite2p. NeuroWRAP harnesses this difference by introducing consensus analysis, utilizing two workflows in conjunction to significantly increase the trustworthiness and robustness of cell segmentation results.

COMBINe enables automated detection and classification of neurons and astrocytes in tissue-cleared mouse brains

Tissue clearing renders entire organs transparent to accelerate whole-tissue imaging; for example, with light-sheet fluorescence microscopy. Yet, challenges remain in analyzing the large resulting 3D datasets that consist of terabytes of images and information on millions of labeled cells. Previous work has established pipelines for automated analysis of tissue-cleared mouse brains, but the focus there was on single-color channels and/or detection of nuclear localized signals in relatively low-resolution images. Here, we present an automated workflow (COMBINe, Cell detectiOn in Mouse BraIN) to map sparsely labeled neurons and astrocytes in genetically distinct mouse forebrains using mosaic analysis with double markers (MADM). COMBINe blends modules from multiple pipelines with RetinaNet at its core. We quantitatively analyzed the regional and subregional effects of MADM-based deletion of the epidermal growth factor receptor (EGFR) on neuronal and astrocyte populations in the mouse forebrain.

A parameter-optimization framework for neural decoding systems

Real-time neuron detection and neural activity extraction are critical components of real-time neural decoding. They are modeled effectively in dataflow graphs. However, these graphs and the components within them in general have many parameters, including hyper-parameters associated with machine learning sub-systems. The dataflow graph parameters induce a complex design space, where alternative configurations (design points) provide different trade-offs involving key operational metrics including accuracy and time-efficiency. In this paper, we propose a novel optimization framework that automatically configures the parameters in different neural decoders. The proposed optimization framework is evaluated in depth through two case studies. Significant performance improvement in terms of accuracy and efficiency is observed in both case studies compared to the manual parameter optimization that was associated with the published results of those case studies. Additionally, we investigate the application of efficient multi-threading strategies to speed-up the running time of our parameter optimization framework. Our proposed optimization framework enables efficient and effective estimation of parameters, which leads to more powerful neural decoding capabilities and allows researchers to experiment more easily with alternative decoding models.

NeuroSeg-II: A deep learning approach for generalized neuron segmentation in two-photon Ca2+ imaging

The development of two-photon microscopy and Ca²⁺ indicators has enabled the recording of multiscale neuronal activities in vivo and thus advanced the understanding of brain functions. However, it is challenging to perform automatic, accurate, and generalized neuron segmentation when processing a large amount of imaging data. Here, we propose a novel deep-learning-based neural network, termed as NeuroSeg-II, to conduct automatic neuron segmentation for in vivo two-photon Ca²⁺ imaging data. This network architecture is based on Mask region-based convolutional neural network (R-CNN) but has enhancements of an attention mechanism and modified feature hierarchy modules. We added an attention mechanism module to focus the computation on neuron regions in imaging data. We also enhanced the feature hierarchy to extract feature information at diverse levels. To incorporate both spatial and temporal information in our data processing, we fused the images from average projection and correlation map extracting the temporal information of active neurons, and the integrated information was expressed as two-dimensional (2D) images. To achieve a generalized neuron segmentation, we conducted a hybrid learning strategy by training our model with imaging data from different labs, including multiscale data with different Ca²⁺ indicators. The results showed that our approach achieved promising segmentation performance across different imaging scales and Ca²⁺ indicators, even including the challenging data of large field-of-view mesoscopic images. By comparing state-of-the-art neuron segmentation methods for two-photon Ca²⁺ imaging data, we showed that our approach achieved the highest accuracy with a publicly available dataset. Thus, NeuroSeg-II enables good segmentation accuracy and a convenient training and testing process.

Review on data analysis methods for mesoscale neural imaging in vivo

Significance: Mesoscale neural imaging in vivo has gained extreme popularity in neuroscience for its capacity of recording large-scale neurons in action. Optical imaging with single-cell resolution and millimeter-level field of view in vivo has been providing an accumulated database of neuron-behavior correspondence. Meanwhile, optical detection of neuron signals is easily contaminated by noises, background, crosstalk, and motion artifacts, while neural-level signal processing and network-level coordinate are extremely complicated, leading to laborious and challenging signal processing demands. The existing data analysis procedure remains unstandardized, which could be daunting to neophytes or neuroscientists without computational background. Aim: We hope to provide a general data analysis pipeline of mesoscale neural imaging shared between imaging modalities and systems. Approach: We divide the pipeline into two main stages. The first stage focuses on extracting high-fidelity neural responses at single-cell level from raw images, including motion registration, image denoising, neuron segmentation, and signal extraction. The second stage focuses on data mining, including neural functional mapping, clustering, and brain-wide network deduction. Results: Here, we introduce the general pipeline of processing the mesoscale neural images. We explain the principles of these procedures and compare different approaches and their application scopes with detailed discussions about the shortcomings and remaining challenges. Conclusions: There are great challenges and opportunities brought by the large-scale mesoscale data, such as the balance between fidelity and efficiency, increasing computational load, and neural network interpretability. We believe that global circuits on single-neuron level will be more extensively explored in the future.

YOLO Algorithm for Long-Term Tracking and Detection of Escherichia Coli at Different Depths of Microchannels Based on Microsphere Positioning Assistance

The effect evaluation of the antibiotic susceptibility test based on bacterial solution is of great significance for clinical diagnosis and prevention of antibiotic abuse. Applying a microfluidic chip as the detection platform, the detection method of using microscopic images to observe bacteria under antibiotic can greatly speed up the detection time, which is more suitable for high-throughput detection. However, due to the influence of the depth of the microchannel, there are multiple layers of bacteria under the focal depth of the microscope, which greatly affects the counting and recognition accuracy and increases the difficulty of relocation of the target bacteria, as well as extracting the characteristics of bacterial liquid changes under the action of antibiotics. After the focal depth of the target bacteria is determined, although the z-axis can be controlled with the help of a three-dimensional micro-operator, the equipment is difficult to operate and the long-term changes of the target bacteria cannot be tracked quickly and accurately. In this paper, the YOLOv5 algorithm is adopted to accurately identify bacteria with different focusing states of multi-layer bacteria at the z-axis with any focal depth. In the meantime, a certain amount of microspheres were mixed into bacteria to assist in locating bacteria, which was convenient for tracking the growth state of bacteria over a long period, and the recognition rates of both bacteria and microspheres were high. The recognition accuracy and counting accuracy of bacteria are 0.734 and 0.714, and the two recognition rates of microspheres are 0.910 and 0.927, respectively, which are much higher than the counting accuracy of 0.142 for bacteria and 0.781 for microspheres with the method of enhanced depth of field (EDF method). Moreover, during long-term bacterial tracking and detection, target bacteria at multiple z-axis focal depth positions can be recorded by the aid of microspheres as a positioning aid for 3D reconstruction, and the focal depth positions can be repositioned within 3-10 h. The structural similarity (SSIM) of microscopic image structure differences at the same focal depth fluctuates between 0.960 and 0.975 at different times, and the root-mean-square error (RMSE) fluctuates between 8 and 12, which indicates that the method also has good relocation accuracy. Thus, this method provides the basis for rapid, high-throughput, and long-term analysis of microscopic changes (e.g., morphology, size) of bacteria detection under the addition of antibiotics with different concentrations based on microfluidic channels in the future.

SmaRT2P: a software for generating and processing smart line recording trajectories for population two-photon calcium imaging

Two-photon fluorescence calcium imaging allows recording the activity of large neural populations with subcellular spatial resolution, but it is typically characterized by low signal-to-noise ratio (SNR) and poor accuracy in detecting single or few action potentials when large number of neurons are imaged. We recently showed that implementing a smart line scanning approach using trajectories that optimally sample the regions of interest increases both the SNR fluorescence signals and the accuracy of single spike detection in population imaging in vivo. However, smart line scanning requires highly specialised software to design recording trajectories, interface with acquisition hardware, and efficiently process acquired data. Furthermore, smart line scanning needs optimized strategies to cope with movement artefacts and neuropil contamination. Here, we develop and validate SmaRT2P, an open-source, user-friendly and easy-to-interface Matlab-based software environment to perform optimized smart line scanning in two-photon calcium imaging experiments. SmaRT2P is designed to interface with popular acquisition software (e.g., ScanImage) and implements novel strategies to detect motion artefacts, estimate neuropil contamination, and minimize their impact on functional signals extracted from neuronal population imaging. SmaRT2P is structured in a modular way to allow flexibility in the processing pipeline, requiring minimal user intervention in parameter setting. The use of SmaRT2P for smart line scanning has the potential to facilitate the functional investigation of large neuronal populations with increased SNR and accuracy in detecting the discharge of single and few action potentials.

Computational Methods for Neuron Segmentation in Two-Photon Calcium Imaging Data: A Survey

Calcium imaging has rapidly become a methodology of choice for real-time in vivo neuron analysis. Its application to large sets of data requires automated tools to annotate and segment cells, allowing scalable image segmentation under reproducible criteria. In this paper, we review and summarize the most recent methods for computational segmentation of calcium imaging. The contributions of the paper are three-fold: we provide an overview of the main algorithms taxonomized in three categories (signal processing, matrix factorization and machine learning-based approaches), we highlight the main advantages and disadvantages of each category and we provide a summary of the performance of the methods that have been tested on public benchmarks (with links to the public code when available).