Statistically unbiased prediction enables accurate denoising of voltage imaging data

Imaging neuronal voltage beyond the scattering limit

Voltage imaging is a promising technique for high-speed recording of neuronal population activity. However, tissue scattering severely limits its application in dense neuronal populations. Here we adopt the principle of localization microscopy, a technique that enables super-resolution imaging of single molecules, to resolve dense neuronal activities in vivo. Leveraging the sparse activation of neurons during action potentials (APs), we precisely localize the fluorescence changes associated with each AP, creating a super-resolution image of neuronal activity. This approach, termed activity localization imaging (ALI), identifies overlapping neurons and separates their activities with over tenfold greater precision than what tissue scattering permits. We applied ALI to widefield, targeted illumination and light sheet microscopy data, resolving neurons that cannot be distinguished by existing signal extraction pipelines. In the mouse hippocampus, ALI generates a cellular resolution map of theta oscillations, revealing the diversity of neuronal phase locking within a dense local network.

A sensitive soma-localized red fluorescent calcium indicator for in vivo imaging of neuronal populations at single-cell resolution

Recent advancements in genetically encoded calcium indicators, particularly those based on green fluorescent proteins, have optimized their performance for monitoring neuronal activities in a variety of model organisms. However, progress in developing red-shifted GECIs, despite their advantages over green indicators, has been slower, resulting in fewer options for end users. In this study, we explored topological inversion and soma-targeting strategies, which are complementary to conventional mutagenesis, to re-engineer a red genetically encoded calcium indicator, FRCaMP, for enhanced in vivo performance. The resulting sensors, FRCaMPi and soma-targeted FRCaMPi (SomaFRCaMPi), exhibit up to 2-fold higher dynamic range and peak ΔF/F0 per single AP compared to widely used jRGECO1a in neurons both in culture and in vivo. Compared to jRGECO1a and FRCaMPi, SomaFRCaMPi reduces erroneous correlation of neuronal activity in the brains of mice and zebrafish by two- to 4-fold due to diminished neuropil contamination without compromising the signal-to-noise ratio. Under wide-field imaging in primary somatosensory and visual cortices in mice with high labeling density (80-90%), SomaFRCaMPi exhibits up to 40% higher SNR and decreased artifactual correlation across neurons. Altogether, SomaFRCaMPi improves the accuracy and scale of neuronal activity imaging at single-neuron resolution in densely labeled brain tissues due to a 2-3-fold enhanced automated neuronal segmentation, 50% higher fraction of responsive cells, up to 2-fold higher SNR compared to jRGECO1a. Our findings highlight the potential of SomaFRCaMPi, comparable to the most sensitive soma-targeted GCaMP, for precise spatial recording of neuronal populations using popular imaging modalities in model organisms such as zebrafish and mice.

Functional and structural profiling of circulation via genetically encoded modular fluorescent probes

Sustained labeling of fluids is crucial for their investigation in animal models. Here, we introduce a mouse line (Alb-mSc-ST), where blood and interstitial fluid are labeled with the red fluorescent protein mScarlet and SpyTag. The SpyTag-SpyCatcher technology is exploited to monitor circulating fluid properties by biosensors or detect blood-brain barrier disruption. This approach represents a valuable tool for studying vascular structure, permeability and microenvironment in body organs in vivo.

Nanoscale Resolution Imaging of Whole Mouse Embryos Using Expansion Microscopy

Nanoscale imaging of whole vertebrates is essential for the systematic understanding of human diseases, yet this goal has not yet been achieved. Expansion microscopy (ExM) is an attractive option for accomplishing this aim; however, the expansion of even mouse embryos at mid- and late-developmental stages, which have fewer calcified body parts than adult mice, is yet to be demonstrated due to the challenges of expanding calcified tissues. Here, we introduce a state-of-the-art ExM technique, termed whole-body ExM, that utilizes cyclic digestion. This technique allows for the super-resolution, volumetric imaging of anatomical structures, proteins, and endogenous fluorescent proteins (FPs) within embryonic and neonatal mice by expanding them 4-fold. The key feature of whole-body ExM is the alternating application of two enzyme compositions repeated multiple times. Through the simple repetition of this digestion process with an increasing number of cycles, mouse embryos of various stages up to E18.5, and even neonatal mice, which display a dramatic difference in the content of calcified tissues compared to embryos, are expanded without further laborious optimization. Furthermore, the whole-body ExM's ability to retain FP signals allows the visualization of various neuronal structures in transgenic mice. Whole-body ExM could facilitate studies of molecular changes in various vertebrates.

Self-organization of conducting pathways explains complex wave trajectories in procedurally interpolated fibrotic cardiac tissue: A virtual replica study

In precision cardiology, virtual replicas (VRs) hold promise for predicting arrhythmias by leveraging patient-specific data and biophysics knowledge. A crucial first step is creating VRs of cardiac tissue based on retrospective patient data. However, VRs aim to replicate biopotential conduction directly, whereas only non-invasive methods are feasible for clinical use on real organs and tissues. This discrepancy challenges our understanding of VR applicability limits. This study aims to enhance the mathematical template of VR by developing an in vitro validation complement. We performed a frame-by-frame comparison of in vitro optical mapping of biopotential conduction with VR predictions. Patient-specific self-organized tissue samples from human induced pluripotent stem cell-derived cardiomyocytes (CMs) with diffuse fibrosis were utilized as VR prototypes. High-resolution optical mapping recordings (Δx = 117 ± 4 μm, Δt = 7.69 ms) and immunostaining were used to reproduce fibrotic samples of linear size 7.5 mm. We applied data-driven Bayesian optimization of the Cellular Potts model (CPM) to study wave propagation at the subcellular level. The modified CPM accurately reflected the "perinatal window" until the 20th day of differentiation, affecting CMs' self-organization. The percolation threshold of virtual conductive pathways reached 0.26 (0.27 ± 0.03 of CMs in vitro), yielding a spatial correlation of amplitude maps with Pearson's coefficients of 0.83 ± 0.02. As a proof-of-concept, we demonstrated that CPM-enhanced VR could predict wavefront trajectories in optical mapping recordings, showing that approximating fibrosis distribution is crucial for improving VR prediction accuracy.

A Sensitive Soma-localized Red Fluorescent Calcium Indicator for Multi-Modality Imaging of Neuronal Populations In Vivo

Recent advancements in genetically encoded calcium indicators, particularly those based on green fluorescent proteins, have optimized their performance for monitoring neuronal activities in a variety of model organisms. However, progress in developing red-shifted GECIs, despite their advantages over green indicators, has been slower, resulting in fewer options for end-users. In this study, we explored topological inversion and soma-targeting strategies, which are complementary to conventional mutagenesis, to re-engineer a red genetically encoded calcium indicator, FRCaMP, for enhanced in vivo performance. The resulting sensors, FRCaMPi and soma-targeted FRCaMPi (SomaFRCaMPi), exhibit up to 2-fold higher dynamic range and peak ΔF/F0 per single AP compared to widely used jRGECO1a in neurons in culture and in vivo. Compared to jRGECO1a and FRCaMPi, SomaFRCaMPi reduces erroneous correlation of neuronal activity in the brains of mice and zebrafish by two- to four-fold due to diminished neuropil contamination without compromising the signal-to-noise ratio. Under wide-field imaging in primary somatosensory and visual cortex in mice with high labeling density (80-90%), SomaFRCaMPi exhibits up to 40% higher SNR and decreased artifactual correlation across neurons. Altogether, SomaFRCaMPi improves the accuracy and scale of neuronal activity imaging at single-neuron resolution in densely labeled brain tissues due to a 2-3-fold enhanced automated neuronal segmentation, 50% higher fraction of responsive cells, up to 2-fold higher SNR compared to jRGECO1a. Our findings highlight the potential of SomaFRCaMPi, comparable to the most sensitive soma-targeted GCaMP, for precise spatial recording of neuronal populations using popular imaging modalities in model organisms such as zebrafish and mice.

Ultrafast optical imaging techniques for exploring rapid neuronal dynamics

Optical neuroimaging has significantly advanced our understanding of brain function, particularly through techniques such as two-photon microscopy, which captures three-dimensional brain structures with sub-cellular resolution. However, traditional methods struggle to record fast, complex neuronal interactions in real time, which are crucial for understanding brain networks and developing treatments for neurological diseases such as Alzheimer's, Parkinson's, and chronic pain. Recent advancements in ultrafast imaging technologies, including kilohertz two-photon microscopy, light field microscopy, and event-based imaging, are pushing the boundaries of temporal resolution in neuroimaging. These techniques enable the capture of rapid neural events with unprecedented speed and detail. This review examines the principles, applications, and limitations of these technologies, highlighting their potential to revolutionize neuroimaging and improve the diagnose and treatment of neurological disorders. Despite challenges such as photodamage risks and spatial resolution trade-offs, integrating these approaches promises to enhance our understanding of brain function and drive future breakthroughs in neuroscience and medicine. Continued interdisciplinary collaboration is essential to fully leverage these innovations for advancements in both basic and clinical neuroscience.

Robust self-supervised denoising of voltage imaging data using CellMincer

Voltage imaging is a powerful technique for studying neuronal activity, but its effectiveness is often constrained by low signal-to-noise ratios (SNR). Traditional denoising methods, such as matrix factorization, impose rigid assumptions about noise and signal structures, while existing deep learning approaches fail to fully capture the rapid dynamics and complex dependencies inherent in voltage imaging data. Here, we introduce CellMincer, a novel self-supervised deep learning method specifically developed for denoising voltage imaging datasets. CellMincer operates by masking and predicting sparse pixel sets across short temporal windows and conditions the denoiser on precomputed spatiotemporal auto-correlations to effectively model long-range dependencies without large temporal contexts. We developed and utilized a physics-based simulation framework to generate realistic synthetic datasets, enabling rigorous hyperparameter optimization and ablation studies. This approach highlighted the critical role of conditioning on spatiotemporal auto-correlations, resulting in an additional 3-fold SNR gain. Comprehensive benchmarking on both simulated and real datasets, including those validated with patch-clamp electrophysiology (EP), demonstrates CellMincer's state-of-the-art performance, with substantial noise reduction across the frequency spectrum, enhanced subthreshold event detection, and high-fidelity recovery of EP signals. CellMincer consistently outperforms existing methods in SNR gain (0.5-2.9 dB) and reduces SNR variability by 17-55%. Incorporating CellMincer into standard workflows significantly improves neuronal segmentation, peak detection, and functional phenotype identification, consistently surpassing current methods in both SNR gain and consistency.

TAG-SPARK: Empowering High-Speed Volumetric Imaging With Deep Learning and Spatial Redundancy

Two-photon high-speed fluorescence calcium imaging stands as a mainstream technique in neuroscience for capturing neural activities with high spatiotemporal resolution. However, challenges arise from the inherent tradeoff between acquisition speed and image quality, grappling with a low signal-to-noise ratio (SNR) due to limited signal photon flux. Here, a contrast-enhanced video-rate volumetric system, integrating a tunable acoustic gradient (TAG) lens-based high-speed microscopy with a TAG-SPARK denoising algorithm is demonstrated. The former facilitates high-speed dense z-sampling at sub-micrometer-scale intervals, allowing the latter to exploit the spatial redundancy of z-slices for self-supervised model training. This spatial redundancy-based approach, tailored for 4D (xyzt) dataset, not only achieves >700% SNR enhancement but also retains fast-spiking functional profiles of neuronal activities. High-speed plus high-quality images are exemplified by in vivo Purkinje cells calcium observation, revealing intriguing dendritic-to-somatic signal convolution, i.e., similar dendritic signals lead to reverse somatic responses. This tailored technique allows for capturing neuronal activities with high SNR, thus advancing the fundamental comprehension of neuronal transduction pathways within 3D neuronal architecture.

Deep-prior ODEs augment fluorescence imaging with chemical sensors

To study biological signalling, great effort goes into designing sensors whose fluorescence follows the concentration of chemical messengers as closely as possible. However, the binding kinetics of the sensors are often overlooked when interpreting cell signals from the resulting fluorescence measurements. We propose a method to reconstruct the spatiotemporal concentration of the underlying chemical messengers in consideration of the binding process. Our method fits fluorescence data under the constraint of the corresponding chemical reactions and with the help of a deep-neural-network prior. We test it on several GCaMP calcium sensors. The recovered concentrations concur in a common temporal waveform regardless of the sensor kinetics, whereas assuming equilibrium introduces artifacts. We also show that our method can reveal distinct spatiotemporal events in the calcium distribution of single neurons. Our work augments current chemical sensors and highlights the importance of incorporating physical constraints in computational imaging.

Self-inspired learning for denoising live-cell super-resolution microscopy

Every collected photon is precious in live-cell super-resolution (SR) microscopy. Here, we describe a data-efficient, deep learning-based denoising solution to improve diverse SR imaging modalities. The method, SN2N, is a Self-inspired Noise2Noise module with self-supervised data generation and self-constrained learning process. SN2N is fully competitive with supervised learning methods and circumvents the need for large training set and clean ground truth, requiring only a single noisy frame for training. We show that SN2N improves photon efficiency by one-to-two orders of magnitude and is compatible with multiple imaging modalities for volumetric, multicolor, time-lapse SR microscopy. We further integrated SN2N into different SR reconstruction algorithms to effectively mitigate image artifacts. We anticipate SN2N will enable improved live-SR imaging and inspire further advances.

DeepVID v2: self-supervised denoising with decoupled spatiotemporal enhancement for low-photon voltage imaging

Significance

Voltage imaging is a powerful tool for studying the dynamics of neuronal activities in the brain. However, voltage imaging data are fundamentally corrupted by severe Poisson noise in the low-photon regime, which hinders the accurate extraction of neuronal activities. Self-supervised deep learning denoising methods have shown great potential in addressing the challenges in low-photon voltage imaging without the need for ground-truth but usually suffer from the trade-off between spatial and temporal performances.

Aim

We present DeepVID v2, a self-supervised denoising framework with decoupled spatial and temporal enhancement capability to significantly augment low-photon voltage imaging.

Approach

DeepVID v2 is built on our original DeepVID framework, which performs frame-based denoising by utilizing a sequence of frames around the central frame targeted for denoising to leverage temporal information and ensure consistency. Similar to DeepVID, the network further integrates multiple blind pixels in the central frame to enrich the learning of local spatial information. In addition, DeepVID v2 introduces a new spatial prior extraction branch to capture fine structural details to learn high spatial resolution information. Two variants of DeepVID v2 are introduced to meet specific denoising needs: an online version tailored for real-time inference with a limited number of frames and an offline version designed to leverage the full dataset, achieving optimal temporal and spatial performances.

Results

We demonstrate that DeepVID v2 is able to overcome the trade-off between spatial and temporal performances and achieve superior denoising capability in resolving both high-resolution spatial structures and rapid temporal neuronal activities. We further show that DeepVID v2 can generalize to different imaging conditions, including time-series measurements with various signal-to-noise ratios and extreme low-photon conditions.

Conclusions

Our results underscore DeepVID v2 as a promising tool for enhancing voltage imaging. This framework has the potential to generalize to other low-photon imaging modalities and greatly facilitate the study of neuronal activities in the brain.

DENOISING: Dynamic enhancement and noise overcoming in multimodal neural observations via high-density CMOS-based biosensors

Large-scale multimodal neural recordings on high-density biosensing microelectrode arrays (HD-MEAs) offer unprecedented insights into the dynamic interactions and connectivity across various brain networks. However, the fidelity of these recordings is frequently compromised by pervasive noise, which obscures meaningful neural information and complicates data analysis. To address this challenge, we introduce DENOISING, a versatile data-derived computational engine engineered to adjust thresholds adaptively based on large-scale extracellular signal characteristics and noise levels. This facilitates the separation of signal and noise components without reliance on specific data transformations. Uniquely capable of handling a diverse array of noise types (electrical, mechanical, and environmental) and multidimensional neural signals, including stationary and non-stationary oscillatory local field potential (LFP) and spiking activity, DENOISING presents an adaptable solution applicable across different recording modalities and brain networks. Applying DENOISING to large-scale neural recordings from mice hippocampal and olfactory bulb networks yielded enhanced signal-to-noise ratio (SNR) of LFP and spike firing patterns compared to those computed from raw data. Comparative analysis with existing state-of-the-art denoising methods, employing SNR and root mean square noise (RMS), underscores DENOISING's performance in improving data quality and reliability. Through experimental and computational approaches, we validate that DENOISING improves signal clarity and data interpretation by effectively mitigating independent noise in spatiotemporally structured multimodal datasets, thus unlocking new dimensions in understanding neural connectivity and functional dynamics.

Convolutional neural network transformer (CNNT) for fluorescence microscopy image denoising with improved generalization and fast adaptation

Deep neural networks can improve the quality of fluorescence microscopy images. Previous methods, based on Convolutional Neural Networks (CNNs), require time-consuming training of individual models for each experiment, impairing their applicability and generalization. In this study, we propose a novel imaging-transformer based model, Convolutional Neural Network Transformer (CNNT), that outperforms CNN based networks for image denoising. We train a general CNNT based backbone model from pairwise high-low Signal-to-Noise Ratio (SNR) image volumes, gathered from a single type of fluorescence microscope, an instant Structured Illumination Microscope. Fast adaptation to new microscopes is achieved by fine-tuning the backbone on only 5-10 image volume pairs per new experiment. Results show that the CNNT backbone and fine-tuning scheme significantly reduces training time and improves image quality, outperforming models trained using only CNNs such as 3D-RCAN and Noise2Fast. We show three examples of efficacy of this approach in wide-field, two-photon, and confocal fluorescence microscopy.

Optical constraints on two-photon voltage imaging

Significance

Genetically encoded voltage indicators (GEVIs) are a valuable tool for studying neural circuits in vivo, but the relative merits and limitations of one-photon (1P) versus two-photon (2P) voltage imaging are not well characterized.

Aim

We consider the optical and biophysical constraints particular to 1P and 2P voltage imaging and compare the imaging properties of commonly used GEVIs under 1P and 2P excitation.

Approach

We measure the brightness and voltage sensitivity of voltage indicators from commonly used classes under 1P and 2P illumination. We also measure the decrease in fluorescence as a function of depth in the mouse brain. We develop a simple model of the number of measurable cells as a function of reporter properties, imaging parameters, and desired signal-to-noise ratio (SNR). We then discuss how the performance of voltage imaging would be affected by sensor improvements and by recently introduced advanced imaging modalities.

Results

Compared with 1P excitation, 2P excitation requires ∼ 10 4 -fold more illumination power per cell to produce similar photon count rates. For voltage imaging with JEDI-2P in the mouse cortex with a target SNR of 10 (spike height to baseline shot noise), a measurement bandwidth of 1 kHz, a thermally limited laser power of 200 mW, and an imaging depth of > 300    μ m , 2P voltage imaging using an 80-MHz source can record from no more than ∼ 12 neurons simultaneously.

Conclusions

Due to the stringent photon-count requirements of voltage imaging and the modest voltage sensitivity of existing reporters, 2P voltage imaging in vivo faces a stringent tradeoff between shot noise and tissue photodamage. 2P imaging of hundreds of neurons with high SNR at a depth of > 300    μ m will require either major improvements in 2P GEVIs or qualitatively new approaches to imaging.

Pixel-wise programmability enables dynamic high-SNR cameras for high-speed microscopy

High-speed wide-field fluorescence microscopy has the potential to capture biological processes with exceptional spatiotemporal resolution. However, conventional cameras suffer from low signal-to-noise ratio at high frame rates, limiting their ability to detect faint fluorescent events. Here, we introduce an image sensor where each pixel has individually programmable sampling speed and phase, so that pixels can be arranged to simultaneously sample at high speed with a high signal-to-noise ratio. In high-speed voltage imaging experiments, our image sensor significantly increases the output signal-to-noise ratio compared to a low-noise scientific CMOS camera (~2-3 folds). This signal-to-noise ratio gain enables the detection of weak neuronal action potentials and subthreshold activities missed by the standard scientific CMOS cameras. Our camera with flexible pixel exposure configurations offers versatile sampling strategies to improve signal quality in various experimental conditions.

DeepVID v2: Self-Supervised Denoising with Decoupled Spatiotemporal Enhancement for Low-Photon Voltage Imaging

Significance

Voltage imaging is a powerful tool for studying the dynamics of neuronal activities in the brain. However, voltage imaging data are fundamentally corrupted by severe Poisson noise in the low-photon regime, which hinders the accurate extraction of neuronal activities. Self-supervised deep learning denoising methods have shown great potential in addressing the challenges in low-photon voltage imaging without the need for ground truth, but usually suffer from the tradeoff between spatial and temporal performance.

Aim

We present DeepVID v2, a novel self-supervised denoising framework with decoupled spatial and temporal enhancement capability to significantly augment low-photon voltage imaging.

Approach

DeepVID v2 is built on our original DeepVID framework,1,2 which performs frame-based denoising by utilizing a sequence of frames around the central frame targeted for denoising to leverage temporal information and ensure consistency. The network further integrates multiple blind pixels in the central frame to enrich the learning of local spatial information. Additionally, DeepVID v2 introduces a new edge extraction branch to capture fine structural details in order to learn high spatial resolution information.

Results

We demonstrate that DeepVID v2 is able to overcome the tradeoff between spatial and temporal performance, and achieve superior denoising capability in resolving both high-resolution spatial structures and rapid temporal neuronal activities. We further show that DeepVID v2 is able to generalize to different imaging conditions, including time-series measurements with various signal-to-noise ratios (SNRs) and in extreme low-photon conditions.

Conclusions

Our results underscore DeepVID v2 as a promising tool for enhancing voltage imaging. This framework has the potential to generalize to other low-photon imaging modalities and greatly facilitate the study of neuronal activities in the brain.

Imaging the voltage of neurons distributed across entire brains of larval zebrafish

Neurons interact in networks distributed throughout the brain. Although much effort has focused on whole-brain calcium imaging, recent advances in genetically encoded voltage indicators (GEVIs) raise the possibility of imaging voltage of neurons distributed across brains. To achieve this, a microscope must image at high volumetric rate and signal-to-noise ratio. We present a remote scanning light-sheet microscope capable of imaging GEVI-expressing neurons distributed throughout entire brains of larval zebrafish at a volumetric rate of 200.8 Hz. We measured voltage of ∼1/3 of the neurons of the brain, distributed throughout. We observed that neurons firing at different times during a sequence were located at different brain locations, for sequences elicited by a visual stimulus, which mapped onto locations throughout the optic tectum, as well as during stimulus-independent bursts, which mapped onto locations in the cerebellum and medulla. Whole-brain voltage imaging may open up frontiers in the fundamental operation of neural systems.