Development of a beam propagation method to simulate the point spread function degradation in scattering media

Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network

Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacerbates the condition of the inverse problem. Here, we develop a scattering simulator that models low-contrast target signals buried in heterogeneous strong background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). We apply this network to our previously developed computational miniature mesoscope and demonstrate the robustness of our deep learning algorithm on scattering phantoms with different scattering conditions. The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model's generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking.

Single-shot digital optical fluorescence phase conjugation through forward multiple-scattering samples

Aberrations and multiple scattering in biological tissues critically distort light beams into highly complex speckle patterns. In this regard, digital optical phase conjugation (DOPC) is a promising technique enabling in-depth focusing. However, DOPC becomes challenging when using fluorescent guide stars for four main reasons: the low photon budget available, the large spectral bandwidth of the fluorescent signal, the Stokes shift between the emission and the excitation wavelength, and the absence of reference beam preventing holographic measurement. Here, we demonstrate the possibility to focus a laser beam through multiple-scattering samples by measuring speckle fields in a single acquisition step with a reference-free, high-resolution wavefront sensor. By taking advantage of the large spectral bandwidth of forward multiply scattering samples, digital fluorescence phase conjugation is achieved to focus a laser beam at the excitation wavelength while measuring the broadband speckle field arising from a micrometer-sized fluorescent bead.

Robust single-shot 3D fluorescence imaging in scattering media with a simulator-trained neural network

Imaging through scattering is a pervasive and difficult problem in many biological applications. The high background and the exponentially attenuated target signals due to scattering fundamentally limits the imaging depth of fluorescence microscopy. Light-field systems are favorable for high-speed volumetric imaging, but the 2D-to-3D reconstruction is fundamentally ill-posed, and scattering exacerbates the condition of the inverse problem. Here, we develop a scattering simulator that models low-contrast target signals buried in heterogeneous strong background. We then train a deep neural network solely on synthetic data to descatter and reconstruct a 3D volume from a single-shot light-field measurement with low signal-to-background ratio (SBR). We apply this network to our previously developed Computational Miniature Mesoscope and demonstrate the robustness of our deep learning algorithm on scattering phantoms with different scattering conditions. The network can robustly reconstruct emitters in 3D with a 2D measurement of SBR as low as 1.05 and as deep as a scattering length. We analyze fundamental tradeoffs based on network design factors and out-of-distribution data that affect the deep learning model's generalizability to real experimental data. Broadly, we believe that our simulator-based deep learning approach can be applied to a wide range of imaging through scattering techniques where experimental paired training data is lacking.

Pupil engineering for extended depth-of-field imaging in a fluorescence miniscope

Significance

Fluorescence head-mounted microscopes, i.e., miniscopes, have emerged as powerful tools to analyze in-vivo neural populations but exhibit a limited depth-of-field (DoF) due to the use of high numerical aperture (NA) gradient refractive index (GRIN) objective lenses.

Aim

We present extended depth-of-field (EDoF) miniscope, which integrates an optimized thin and lightweight binary diffractive optical element (DOE) onto the GRIN lens of a miniscope to extend the DoF by 2.8× between twin foci in fixed scattering samples.

Approach

We use a genetic algorithm that considers the GRIN lens' aberration and intensity loss from scattering in a Fourier optics-forward model to optimize a DOE and manufacture the DOE through single-step photolithography. We integrate the DOE into EDoF-Miniscope with a lateral accuracy of 70  μm to produce high-contrast signals without compromising the speed, spatial resolution, size, or weight.

Results

We characterize the performance of EDoF-Miniscope across 5- and 10-μm fluorescent beads embedded in scattering phantoms and demonstrate that EDoF-Miniscope facilitates deeper interrogations of neuronal populations in a 100-μm-thick mouse brain sample and vessels in a whole mouse brain sample.

Conclusions

Built from off-the-shelf components and augmented by a customizable DOE, we expect that this low-cost EDoF-Miniscope may find utility in a wide range of neural recording applications.

Extended angular-spectrum modeling (EASM) of light energy transport in scattering media

The exact modeling of light transport in scattering media is critical in biological imaging, free-space communication, and phosphor-converted lighting. Angular spectrum is proved to be a fast and effective approach to reconstructing the wavefront dynamics during the propagation in scattering media, however, finding it difficult in acquiring the wavefront and energy change simultaneously. Besides, conventional methods for energy tracing, such as the Monte Carlo method, are inefficient in speed and hard to simulate the wavefront change. Here, we propose an extended angular-spectrum modeling (EASM) approach using tenuous scattering approximate solutions to obtain a time-efficient and accurate method for reconstruction of energy and wavefront dynamics in various scattering media. The generality of our method is numerically simulated and experimentally verified with a set of scattering media with different properties. EASM has a time advantage under the guarantee of calculation accuracy, especially when calculating several thickness changes after the calculation model is established. Furthermore, multi-layered media can also be simulated by EASM with a good precision. The results suggest that EASM performs certain computations more efficiently than the conventional method and thus provides an effective and flexible calculation tool for scattering media.

Using beam-offset optical coherence tomography to reconstruct backscattered photon profiles in scattering media

Raster scanning imaging technologies capture least scattered photons (LSPs) and reject multiple scattered photons (MSPs) in backscattered photons to image the underlying structures of a scattering medium. However, MSPs can still squeeze into the images, resulting in limited imaging depth, degraded contrast, and significantly reduced lateral resolution. Great efforts have been made to understand how MSPs affect imaging performance through modeling, but the techniques for visualizing the backscattered photon profile (BSPP) in scattering media during imaging are unavailable. Here, a method of reconstructing BSPP is demonstrated using beam-offset optical coherence tomography (OCT), in which OCT images are acquired at offset positions from the illumination beam. The separation of LSPs and MSPs based on the BSPP enables quantification of imaging depth, contrast, and lateral resolution, as well as access to the depth-resolved modulated transfer function (MTF). This approach presents great opportunities for better retrieving tissue optical properties, correctly interpreting images, or directly using MTF as the feedback for adaptive optical imaging.

Simulating scan formation in multimodal optical coherence tomography: angular-spectrum formulation based on ballistic scattering of arbitrary-form beams

We present a computationally highly efficient full-wave spectral model of OCT-scan formation with the following features: allowance of arbitrary phase-amplitude profile of illuminating beams; absence of paraxial approximation; utilization of broadly used approximation of ballistic scattering by discrete scatterers without limitations on their density/location and scattering strength. The model can easily incorporate the wave decay, dispersion, measurement noises with given signal-to-noise ratios and arbitrary inter-scan displacements of scatterers. We illustrate several of such abilities, including comparative simulations of OCT-scans for Bessel versus Gaussian beams, presence of arbitrary aberrations at the tissue boundary and various scatterer motions. The model flexibility and computational efficiency allow one to accurately study various properties of OCT-scans for developing new methods of their processing in various biomedical applications.

A Simplified Model for Optical Systems with Random Phase Screens

A first-order optical system with arbitrary multiple masks placed at arbitrary positions is the basic scheme of various optical systems. Generally, masks in optical systems have a non-shift invariant (SI) effect; thus, the individual effect of each mask on the output cannot be entirely separated. The goal of this paper is to develop a technique where complete separation might be achieved in the common case of random phase screens (RPSs) as masks. RPSs are commonly used to model light propagation through the atmosphere or through biological tissues. We demonstrate the utility of the technique on an optical system with multiple RPSs that model random scattering media.

Image transmission using Airy array beam

An approach for image transmission based on the Airy array beam is proposed and demonstrated. The Airy array beam is generated by employing the product of a special cubic phase and a shift function at its Fourier plane. The image can be modulated into this Airy array beam by overlapping it at the Fourier plane of this beam. After passing through a certain distance, the image information can be recovered from the modulated Airy array beam by Fourier transform. Compared to the existing Airy array beam, higher integrity and image information quality can be achieved by increasing the width of the obstacle that blocks the propagation of these beams. The capability mentioned above is experimentally verified. Moreover, to research the diffraction of this Airy array beam in the scattering environment, the propagation process of this Airy array beam in a scattering medium is theoretically derived and numerically studied. The corresponding experiment demonstrates that the propagation process matches well with the numerical study and simulation.

Approximate image synthesis in optical coherence tomography

Full-wave models of OCT image formation, which are based on Maxwell's equations, are highly realistic. However, such models incur a high computational cost, particularly when modelling sample volumes consistent with those encountered in practice. Here, we present an approximate means of synthesizing volumetric image formation to reduce this computational burden. Instead of performing a full-wave scattered light calculation for each A-scan, we perform a full-wave scattered light calculation for a normally incident plane wave only. We use the angular spectrum field representation to implement beam focussing and scanning, making use of an assumption similar to the tilt optical memory effect, to approximately synthesize volumetric data sets. Our approach leads to an order of magnitude reduction in the computation time required to simulate typical B-scans. We evaluate this method by comparing rigorously and approximately evaluated point spread functions and images of highly scattering structured samples for a typical OCT system. Our approach also reveals new insights into image formation in OCT.

Three-dimensional broadband light beam manipulation in forward scattering samples

Focusing light into highly disordered biological tissue is a major challenge in optical microscopy and biomedical imaging due to scattering. However, correlations in the scattering matrix, known as "memory effects", can be used to improve imaging capabilities. Here we discuss theoretically and numerically the possibility to achieve three-dimensional ultrashort laser focusing and scanning inside forward scattering media, beyond the scattering mean free path, by simultaneously taking advantage of the angular and the chromato-axial memory effects. The numerical model is presented in details, is validated within the state of the art theoretical and experimental framework and is finally used to propose a scheme for focusing ultra-short laser pulses in depth through forward scattering media.

Ultra-thin boundary layer for high-accuracy simulations of light propagation

The modified Born series method is currently one of the most efficient methods available for simulating light scattering in large inhomogeneous media. However, to achieve high accuracy, the method requires thick gradually absorbing layers around the simulation domain. Here, we introduce new boundary conditions, combining a padding-free acyclic convolution with an ultra-thin boundary layer. Our new boundary conditions minimize the wrap-around and reflection artefacts originating from the edges of the simulation domain, while also greatly reducing the computational costs and the memory requirements of the method. Our GPU-accelerated Matlab implementation is available on GitHub.

Single-shot 3D wide-field fluorescence imaging with a Computational Miniature Mesoscope

Fluorescence microscopes are indispensable to biology and neuroscience. The need for recording in freely behaving animals has further driven the development in miniaturized microscopes (miniscopes). However, conventional microscopes/miniscopes are inherently constrained by their limited space-bandwidth product, shallow depth of field (DOF), and inability to resolve three-dimensional (3D) distributed emitters. Here, we present a Computational Miniature Mesoscope (CM2) that overcomes these bottlenecks and enables single-shot 3D imaging across an 8 mm by 7 mm field of view and 2.5-mm DOF, achieving 7-μm lateral resolution and better than 200-μm axial resolution. The CM2 features a compact lightweight design that integrates a microlens array for imaging and a light-emitting diode array for excitation. Its expanded imaging capability is enabled by computational imaging that augments the optics by algorithms. We experimentally validate the mesoscopic imaging capability on 3D fluorescent samples. We further quantify the effects of scattering and background fluorescence on phantom experiments.

Comparing the fundamental imaging depth limit of two-photon, three-photon, and non-degenerate two-photon microscopy

We have systematically characterized the degradation of imaging quality with depth in deep brain multi-photon microscopy, utilizing our recently developed numerical model that computes wave propagation in scattering media. The signal-to-background ratio (SBR) and the resolution determined by the width of the point spread function are obtained as functions of depth. We compare the imaging quality of two-photon (2PM), three-photon (3PM), and non-degenerate two-photon microscopy (ND-2PM) for mouse brain imaging. We show that the imaging depth of 2PM and ND-2PM are fundamentally limited by the SBR, while the SBR remains approximately invariant with imaging depth for 3PM. Instead, the imaging depth of 3PM is limited by the degradation of the resolution, if there is sufficient laser power to maintain the signal level at large depth. The roles of the concentration of dye molecules, the numerical aperture of the input light, the anisotropy factor g, noise level, input laser power, and the effect of temporal broadening are also discussed.