Wide-field multiphoton imaging through scattering media without correction

Deep compressed multichannel adaptive optics scanning light ophthalmoscope

Adaptive optics scanning light ophthalmoscopy (AOSLO) reveals individual retinal cells and their function, microvasculature, and micropathologies in vivo. As compared to the single-channel offset pinhole and two-channel split-detector nonconfocal AOSLO designs, by providing multidirectional imaging capabilities, a recent generation of multidetector and (multi-)offset aperture AOSLO modalities has been demonstrated to provide critical information about retinal microstructures. However, increasing detection channels requires expensive optical components and/or critically increases imaging time. To address this issue, we present an innovative combination of machine learning and optics as an integrated technology to compressively capture 12 nonconfocal channel AOSLO images simultaneously. Imaging of healthy participants and diseased subjects using the proposed deep compressed multichannel AOSLO showed enhanced visualization of rods, cones, and mural cells with over an order-of-magnitude improvement in imaging speed as compared to conventional offset aperture imaging. To facilitate the adaptation and integration with other in vivo microscopy systems, we made optical design, acquisition, and computational reconstruction codes open source.

Widefield ultra-high-density optical breast tomography system supplementing x-ray mammography

We report a wide-field compressive diffuse optical tomography (DOT) system - optical mammography co-imager (OMCI) - which aims to augment tens of thousands of existing x-ray mammography or tomosynthesis systems worldwide by adding functional assessment of breast tissue and improve cancer diagnosis. The OMCI system utilizes large field-of-view structured light illumination and single-pixel-camera based detection techniques to produce ultra-high spatial sampling density while ensuring that the inverse problem remains compact via the development of a unique target-adaptive pattern optimization technique to achieve compressive-sensing based measurements. The reconstructed images can be further enhanced by applying a compositional-prior-guided DOT reconstruction algorithm with tissue structural priors derived from a separately acquired x-ray mammography scans. In this report, we describe the design details and performance characterization of the imaging hardware as well as DOT image reconstruction pipelines. To validate this multi-modal breast DOT system, we include reconstruction results from both tissue-mimicking optical phantoms as well as clinical measurements from normal breasts obtained from a clinical study. Sample reconstructions from a breast containing a malignant tumor are also included, showing the potential of localizing and characterizing breast lesions using multi-modal measurements combining x-ray and DOT.

Scattering Correction through Fourier-Domain Intensity Coupling in Two-Photon Microscopy (2P-FOCUS)

Light penetration depth in biological tissue is limited by tissue scattering. Correcting scattering becomes particularly challenging in scenarios with limited photon availability and when access to the transmission side of the scattering tissue is not possible. Here, we introduce a new two-photon microscopy system with Fourier-domain intensity coupling for scattering correction (2P-FOCUS). 2P-FOCUS corrects scattering by intensity modulation in the Fourier domain, leveraging the nonlinearity of multiple-beam interference and two-photon excitation, eliminating the need for a guide star, iterative optimization, or measuring transmission or reflection matrices. 2P-FOCUS uses random patterns to probe scattering properties, combined with a single-shot algorithm to rapidly generate the correction mask. 2P-FOCUS can also correct scattering beyond the limitation of the memory effect by automatically customizing correction masks for each subregion in a large field-of-view. We provide several proof-of-principle demonstrations here, including focusing and imaging through a bone sample, and imaging neurons and cerebral blood vessels in the mouse brain ex vivo. 2P-FOCUS significantly enhances two-photon fluorescence signals by several tens of folds compared to cases without scattering correction at the same excitation power. 2P-FOCUS can also correct tissue scattering over a 230×230×510 μm3 volume, which is beyond the memory effect range. 2P-FOCUS is able to measure, calculate, and correct scattering within a few seconds, effectively delivering more light deep into the scattering tissue. 2P-FOCUS could be broadly adopted for deep tissue imaging owing to its powerful combination of effectiveness, speed, and cost.

Large-scale scattering-augmented optical encryption

Data proliferation in the digital age necessitates robust encryption techniques to protect information privacy. Optical encryption leverages the multiple degrees of freedom inherent in light waves to encode information with parallel processing and enhanced security features. However, implementations of large-scale, high-security optical encryption have largely remained theoretical or limited to digital simulations due to hardware constraints, signal-to-noise ratio challenges, and precision fabrication of encoding elements. Here, we present an optical encryption platform utilizing scattering multiplexing ptychography, simultaneously enhancing security and throughput. Unlike optical encoders which rely on computer-generated randomness, our approach leverages the inherent complexity of light scattering as a natural unclonable function. This enables multi-dimensional encoding with superior randomness. Furthermore, the ptychographic configuration expands encryption throughput beyond hardware limitations through spatial multiplexing of different scatterer regions. We propose a hybrid decryption algorithm integrating model- and data-driven strategies, ensuring robust decryption against various sources of measurement noise and communication interference. We achieved optical encryption at a scale of ten-megapixel pixels with 1.23 µm resolution. Communication experiments validate the resilience of our decryption algorithm, yielding high-fidelity results even under extreme transmission conditions characterized by a 20% bit error rate. Our encryption platform offers a holistic solution for large-scale, high-security, and cost-effective cryptography.

3D super-resolution optical fluctuation imaging with temporal focusing two-photon excitation

3D super-resolution fluorescence microscopy typically requires sophisticated setups, sample preparation, or long measurements. A notable exception, SOFI, only requires recording a sequence of frames and no hardware modifications whatsoever but being a wide-field method, it faces problems in thick, dense samples. We combine SOFI with temporal focusing two-photon excitation - the wide-field method that is capable of exciting a thin slice in 3D volume. Temporal focusing is simple to implement whenever the excitation path of the microscope can be accessed. The implementation of SOFI is straightforward. By merging these two methods, we obtain super-resolved 3D images of neurons stained with quantum dots. Our approach offers reduced bleaching of out-of-focus fluorescent probes and an improved signal-to-background ratio that can be used when robust resolution improvement is required in thick, dense samples.

Multiline orthogonal scanning temporal focusing (mosTF) microscopy for scattering reduction in in vivo brain imaging

Temporal focusing two-photon microscopy has been utilized for high-resolution imaging of neuronal and synaptic structures across volumes spanning hundreds of microns in vivo. However, a limitation of temporal focusing is the rapid degradation of the signal-to-background ratio and resolution with increasing imaging depth. This degradation is due to scattered emission photons being widely distributed, resulting in a strong background. To overcome this challenge, we have developed multiline orthogonal scanning temporal focusing (mosTF) microscopy. mosTF captures a sequence of images at each scan location of the excitation line. A reconstruction algorithm then reassigns scattered photons back to their correct scan positions. We demonstrate the effectiveness of mosTF by acquiring neuronal images of mice in vivo. Our results show remarkable improvements in in vivo brain imaging with mosTF, while maintaining its speed advantage.

Multiline Orthogonal Scanning Temporal Focusing (mosTF) Microscopy for Scattering Reduction in High-speed in vivo Brain Imaging

Temporal focusing two-photon microscopy enables high resolution imaging of fine structures in vivo over a large volume. A limitation of temporal focusing is that signal-to-background ratio and resolution degrade rapidly with increasing imaging depth. This degradation originates from the scattered emission photons are widely distributed resulting in a strong background. We have developed Multiline Orthogonal Scanning Temporal Focusing (mosTF) microscopy that overcomes this problem. mosTF captures a sequence of images at each scan location of the excitation line, followed by a reconstruction algorithm reassigns scattered photons back to the correct scan position. We demonstrate mosTF by acquiring mice neuronal images in vivo. Our results show remarkably improvements with mosTF for in vivo brain imaging while maintaining its speed advantage.

DEEP-squared: deep learning powered De-scattering with Excitation Patterning

Limited throughput is a key challenge in in vivo deep tissue imaging using nonlinear optical microscopy. Point scanning multiphoton microscopy, the current gold standard, is slow especially compared to the widefield imaging modalities used for optically cleared or thin specimens. We recently introduced "De-scattering with Excitation Patterning" or "DEEP" as a widefield alternative to point-scanning geometries. Using patterned multiphoton excitation, DEEP encodes spatial information inside tissue before scattering. However, to de-scatter at typical depths, hundreds of such patterned excitations were needed. In this work, we present DEEP2, a deep learning-based model that can de-scatter images from just tens of patterned excitations instead of hundreds. Consequently, we improve DEEP's throughput by almost an order of magnitude. We demonstrate our method in multiple numerical and experimental imaging studies, including in vivo cortical vasculature imaging up to 4 scattering lengths deep in live mice.

Optical single-pixel volumetric imaging by three-dimensional light-field illumination

Three-dimensional single-pixel imaging (3D SPI) has become an attractive imaging modality for both biomedical research and optical sensing. 3D-SPI techniques generally depend on time-of-flight or stereovision principle to extract depth information from backscattered light. However, existing implementations for these two optical schemes are limited to surface mapping of 3D objects at depth resolutions, at best, at the millimeter level. Here, we report 3D light-field illumination single-pixel microscopy (3D-LFI-SPM) that enables volumetric imaging of microscopic objects with a near-diffraction-limit 3D optical resolution. Aimed at 3D space reconstruction, 3D-LFI-SPM optically samples the 3D Fourier spectrum by combining 3D structured light-field illumination with single-element intensity detection. We build a 3D-LFI-SPM prototype that provides an imaging volume of ∼390 × 390 × 3,800 μm3 and achieves 2.7-μm lateral resolution and better than 37-μm axial resolution. Its capability of 3D visualization of label-free optical absorption contrast is demonstrated by imaging single algal cells in vivo. Our approach opens broad perspectives for 3D SPI with potential applications in various fields, such as biomedical functional imaging.

De-scattering Deep Neural Network Enables Fast Imaging of Spines through Scattering Media by Temporal Focusing Microscopy

Today the gold standard for in vivo imaging through scattering tissue is point-scanning two-photon microscopy (PSTPM). Especially in neuroscience, PSTPM is widely used for deep-tissue imaging in the brain. However, due to sequential scanning, PSTPM is slow. Temporal focusing microscopy (TFM), on the other hand, focuses femtosecond pulsed laser light temporally while keeping wide-field illumination, and is consequently much faster. However, due to the use of a camera detector, TFM suffers from the scattering of emission photons. As a result, TFM produces images of poor quality, obscuring fluorescent signals from small structures such as dendritic spines. In this work, we present a de-scattering deep neural network (DeScatterNet) to improve the quality of TFM images. Using a 3D convolutional neural network (CNN) we build a map from TFM to PSTPM modalities, to enable fast TFM imaging while maintaining high image quality through scattering media. We demonstrate this approach for in vivo imaging of dendritic spines on pyramidal neurons in the mouse visual cortex. We quantitatively show that our trained network rapidly outputs images that recover biologically relevant features previously buried in the scattered fluorescence in the TFM images. In vivo imaging that combines TFM and the proposed neural network is one to two orders of magnitude faster than PSTPM but retains the high quality necessary to analyze small fluorescent structures. The proposed approach could also be beneficial for improving the performance of many speed-demanding deep-tissue imaging applications, such as in vivo voltage imaging.

Single photon single pixel imaging into thick scattering medium

Imaging into thick scattering medium is a long-standing challenge. Beyond the quasi-ballistic regime, multiple scattering scrambles the spatiotemporal information of incident/emitted light, making canonical imaging based on light focusing nearly impossible. Diffusion optical tomography (DOT) is one of the most popular approach to look inside scattering medium, but quantitatively inverting the diffusion equation is ill-posed, and prior information of the medium is typically necessary, which is nontrivial to obtain. Here, we show theoretically and experimentally that, by synergizing the one-way light scattering characteristic of single pixel imaging with ultrasensitive single photon detection and a metric-guided image reconstruction, single photon single pixel imaging can serve as a simple and powerful alternative to DOT for imaging into thick scattering medium without prior knowledge or inverting the diffusion equation. We demonstrated an image resolution of 12 mm inside a 60 mm thick (∼ 78 mean free paths) scattering medium.

Three-dimensional bi-functional refractive index and fluorescence microscopy (BRIEF)

Fluorescence microscopy is a powerful tool for imaging biological samples with molecular specificity. In contrast, phase microscopy provides label-free measurement of the sample's refractive index (RI), which is an intrinsic optical property that quantitatively relates to cell morphology, mass, and stiffness. Conventional imaging techniques measure either the labeled fluorescence (functional) information or the label-free RI (structural) information, though it may be valuable to have both. For example, biological tissues have heterogeneous RI distributions, causing sample-induced scattering that degrades the fluorescence image quality. When both fluorescence and 3D RI are measured, one can use the RI information to digitally correct multiple-scattering effects in the fluorescence image. Here, we develop a new computational multi-modal imaging method based on epi-mode microscopy that reconstructs both 3D fluorescence and 3D RI from a single dataset. We acquire dozens of fluorescence images, each 'illuminated' by a single fluorophore, then solve an inverse problem with a multiple-scattering forward model. We experimentally demonstrate our method for epi-mode 3D RI imaging and digital correction of multiple-scattering effects in fluorescence images.

Image improvement of temporal focusing multiphoton microscopy via superior spatial modulation excitation and Hilbert-Huang transform decomposition

Temporal focusing-based multiphoton excitation microscopy (TFMPEM) just provides the advantage of widefield optical sectioning ability with axial resolution of several micrometers. However, under the plane excitation, the photons emitted from the molecules in turbid tissues undergo scattering, resulting in complicated background noise and an impaired widefield image quality. Accordingly, this study constructs a general and comprehensive numerical model of TFMPEM utilizing Fourier optics and performs simulations to determine the superior spatial frequency and orientation of the structured pattern which maximize the axial excitation confinement. It is shown experimentally that the optimized pattern minimizes the intensity of the out-of-focus signal, and hence improves the quality of the image reconstructed using the Hilbert transform (HT). However, the square-like reflection components on digital micromirror device leads to pattern residuals in the demodulated image when applying high spatial frequency of structured pattern. Accordingly, the HT is replaced with Hilbert-Huang transform (HHT) in order to sift out the low-frequency background noise and pattern residuals in the demodulation process. The experimental results obtained using a kidney tissue sample show that the HHT yields a significant improvement in the TFMPEM image quality.

Imaging approaches for monitoring three-dimensional cell and tissue culture systems

The past decade has seen an increasing demand for more complex, reproducible and physiologically relevant tissue cultures that can mimic the structural and biological features of living tissues. Monitoring the viability, development and responses of such tissues in real-time are challenging due to the complexities of cell culture physical characteristics and the environments in which these cultures need to be maintained in. Significant developments in optics, such as optical manipulation, improved detection and data analysis, have made optical imaging a preferred choice for many three-dimensional (3D) cell culture monitoring applications. The aim of this review is to discuss the challenges associated with imaging and monitoring 3D tissues and cell culture, and highlight topical label-free imaging tools that enable bioengineers and biophysicists to non-invasively characterise engineered living tissues.

Wavefront shaping with a Hadamard basis for scattering soil imaging

Soil is a scattering medium that inhibits imaging of plant-microbial-mineral interactions that are essential to plant health and soil carbon sequestration. However, optical imaging in the complex medium of soil has been stymied by the seemingly intractable problems of scattering and contrast. Here, we develop a wavefront shaping method based on adaptive stochastic parallel gradient descent optimization with a Hadamard basis to focus light through soil mineral samples. Our approach allows a sparse representation of the wavefront with reduced dimensionality for the optimization. We further divide the used Hadamard basis set into subsets and optimize a certain subset at once. Simulation and experimental optimization results demonstrate our method has an approximately seven times higher convergence rate and overall better performance compared to that with optimizing all pixels at once. The proposed method can benefit other high-dimensional optimization problems in adaptive optics and wavefront shaping.

Compressed sensing in fluorescence microscopy

Compressed sensing (CS) is a signal processing approach that solves ill-posed inverse problems, from under-sampled data with respect to the Nyquist criterium. CS exploits sparsity constraints based on the knowledge of prior information, relative to the structure of the object in the spatial or other domains. It is commonly used in image and video compression as well as in scientific and medical applications, including computed tomography and magnetic resonance imaging. In the field of fluorescence microscopy, it has been demonstrated to be valuable for fast and high-resolution imaging, from single-molecule localization, super-resolution to light-sheet microscopy. Furthermore, CS has found remarkable applications in the field of mesoscopic imaging, facilitating the study of small animals' organs and entire organisms. This review article illustrates the working principles of CS, its implementations in optical imaging and discusses several relevant uses of CS in the field of fluorescence imaging from super-resolution microscopy to mesoscopy.

Remote Focusing in a Temporal Focusing Microscope

In a temporal focusing microscope, dispersion can remotely shift the temporal focal plane axially, but only a single depth can be in focus at a time on a fixed camera. In this paper, we demonstrate remote focusing in a temporal focusing microscope. Dispersion tuning with an electrically tunable lens (ETL) in a 4 f pulse shaper scans the excitation plane axially, and another ETL in the detection path keeps the shifted excitation plane in focus on the camera. Image stacks formed using two ETLs versus a traditional stage scan are equivalent.

Ghost panorama using a convex mirror

Computational ghost imaging or single-pixel imaging enables the image formation of an unknown scene using a lens-free photodetector. In this Letter, we present a computational panoramic ghost imaging system that can achieve a full-color panorama using a single-pixel photodetector, where a convex mirror performs the optical transformation of the engineered Hadamard-based circular illumination pattern from unidirectionally to omnidirectionally. To our best knowledge, it is the first time to propose the concept of ghost panoramas and realize preliminary experimentations. It is foreseeable that ghost panoramas will have more advantages in imaging and detection in many extreme conditions (e.g., scattering/turbulence and unconventional spectra), as well as broad application prospects.

Single pixel imaging at megahertz switching rates via cyclic Hadamard masks

Optical imaging is commonly performed with either a camera and wide-field illumination or with a single detector and a scanning collimated beam; unfortunately, these options do not exist at all wavelengths. Single-pixel imaging offers an alternative that can be performed with a single detector and wide-field illumination, potentially enabling imaging applications in which the detection and illumination technologies are immature. However, single-pixel imaging currently suffers from low imaging rates owing to its reliance on configurable spatial light modulators, generally limited to 22 kHz rates. We develop an approach for rapid single-pixel imaging which relies on cyclic patterns coded onto a spinning mask and demonstrate it for in vivo imaging of C. elegans worms. Spatial modulation rates of up to 2.4 MHz, imaging rates of up to 72 fps, and image-reconstruction times of down to 1.5 ms are reported, enabling real-time visualization of dynamic objects.

De-scattering with Excitation Patterning enables rapid wide-field imaging through scattering media

Nonlinear optical microscopy has enabled in vivo deep tissue imaging on the millimeter scale. A key unmet challenge is its limited throughput especially compared to rapid wide-field modalities that are used ubiquitously in thin specimens. Wide-field imaging methods in tissue specimens have found successes in optically cleared tissues and at shallower depths, but the scattering of emission photons in thick turbid samples severely degrades image quality at the camera. To address this challenge, we introduce a novel technique called De-scattering with Excitation Patterning or "DEEP," which uses patterned nonlinear excitation followed by computational imaging-assisted wide-field detection. Multiphoton temporal focusing allows high-resolution excitation patterns to be projected deep inside specimen at multiple scattering lengths due to the use of long wavelength light. Computational reconstruction allows high-resolution structural features to be reconstructed from tens to hundreds of DEEP images instead of millions of point-scanning measurements.

Holography-based structured light illumination for temporal focusing microscopy

In this Letter, we present a holography-based structured light illumination (SLI) method to enhance the resolution of widefield temporal focusing microscopy (TFM). In the system, a digital micromirror device is employed to simultaneously disperse the incoming femtosecond laser to induce temporal focusing at the focal plane and generate designed structured patterns via a Lee hologram. As the generated structured patterns do not contain the zeroth order beam, it improves the contrast and modulation frequency. Mathematical models have been derived to calculate the electric fields at the focal plane and to explain the effects of improved optical cross-sectioning capability. Imaging experiments have been devised and performed on fluorescent beads and mouse kidney sections; the results demonstrate enhanced axial confinement and improved suppression of out-of-focus fluorescence. The new SLI method realizes high-resolution TFM and can be readily applied to other microscopy platforms for biophotonics applications.

Single-pixel imaging of dynamic objects using multi-frame motion estimation

Single-pixel imaging (SPI) enables the visualization of objects with a single detector by using a sequence of spatially modulated illumination patterns. For natural images, the number of illumination patterns may be smaller than the number of pixels when compressed-sensing algorithms are used. Nonetheless, the sequential nature of the SPI measurement requires that the object remains static until the signals from all the required patterns have been collected. In this paper, we present a new approach to SPI that enables imaging scenarios in which the imaged object, or parts thereof, moves within the imaging plane during data acquisition. Our algorithms estimate the motion direction from inter-frame cross-correlations and incorporate it in the reconstruction model. Moreover, when the illumination pattern is cyclic, the motion may be estimated directly from the raw data, further increasing the numerical efficiency of the algorithm. A demonstration of our approach is presented for both numerically simulated and measured data.

In situ measurement of the isoplanatic patch for imaging through intact bone

Wavefront-shaping (WS) enables imaging through scattering tissues like bone, which is important for neuroscience and bone-regeneration research. WS corrects for the optical aberrations at a given depth and field-of-view (FOV) within the sample; the extent of the validity of which is limited to a region known as the isoplanatic patch (IP). Knowing this parameter helps to estimate the number of corrections needed for WS imaging over a given FOV. In this paper, we first present direct transmissive measurement of murine skull IP using digital optical phase conjugation based focusing. Second, we extend our previously reported phase accumulation ray tracing (PART) method to provide in-situ in-silico estimation of IP, called correlative PART (cPART). Our results show an IP range of 1 to 3 μm for mice within an age range of 8 to 14 days old and 1.00 ± 0.25 μm in a 12-week old adult skull. Consistency between the two measurement approaches indicates that cPART can be used to approximate the IP before a WS experiment, which can be used to calculate the number of corrections required within a given field of view.

Scanless two-photon excitation with temporal focusing

Temporal focusing, with its ability to focus light in time, enables scanless illumination of large surface areas at the sample with micrometer axial confinement and robust propagation through scattering tissue. In conventional two-photon microscopy, widely used for the investigation of intact tissue in live animals, images are formed by point scanning of a spatially focused pulsed laser beam, resulting in limited temporal resolution of the excitation. Replacing point scanning with temporally focused widefield illumination removes this limitation and represents an important milestone in two-photon microscopy. Temporal focusing uses a diffusive or dispersive optical element placed in a plane conjugate to the objective focal plane to generate position-dependent temporal pulse broadening that enables axially confined multiphoton absorption, without the need for tight spatial focusing. Many techniques have benefitted from temporal focusing, including scanless imaging, super-resolution imaging, photolithography, uncaging of caged neurotransmitters and control of neuronal activity via optogenetics.

Control of the temporal and polarization response of a multimode fiber

Control of the spatial and temporal properties of light propagating in disordered media have been demonstrated over the last decade using spatial light modulators. Most of the previous studies demonstrated spatial focusing to the speckle grain size, and manipulation of the temporal properties of the achieved focus. In this work, we demonstrate an approach to control the total temporal impulse response, not only at a single speckle grain but over all spatial degrees of freedom (spatial and polarization modes) at any arbitrary delay time through a multimode fiber. Global enhancement or suppression of the total light intensity exiting a multimode fibre is shown for arbitrary delays and polarization states. This work could benefit to applications that require pulse delivery in disordered media.

Optimal compressive multiphoton imaging at depth using single-pixel detection

Compressive sensing can overcome the Nyquist criterion and record images with a fraction of the usual number of measurements required. However, conventional measurement bases are susceptible to diffraction and scattering, prevalent in high-resolution microscopy. In this Letter, we explore the random Morlet basis as an optimal set for compressive multiphoton imaging, based on its ability to minimize the space-frequency uncertainty. We implement this approach for wide-field multiphoton microscopy with single-pixel detection, which allows imaging through turbid media without correction. The Morlet basis promises a route for rapid acquisition with lower photodamage.

Volumetric Ca2+ Imaging in the Mouse Brain Using Hybrid Multiplexed Sculpted Light Microscopy

Calcium imaging using two-photon scanning microscopy has become an essential tool in neuroscience. However, in its typical implementation, the tradeoffs between fields of view, acquisition speeds, and depth restrictions in scattering brain tissue pose severe limitations. Here, using an integrated systems-wide optimization approach combined with multiple technical innovations, we introduce a new design paradigm for optical microscopy based on maximizing biological information while maintaining the fidelity of obtained neuron signals. Our modular design utilizes hybrid multi-photon acquisition and allows volumetric recording of neuroactivity at single-cell resolution within up to 1 × 1 × 1.22 mm volumes at up to 17 Hz in awake behaving mice. We establish the capabilities and potential of the different configurations of our imaging system at depth and across brain regions by applying it to in vivo recording of up to 12,000 neurons in mouse auditory cortex, posterior parietal cortex, and hippocampus.