Two-photon lensless micro-endoscopy with in-situ wavefront correction

Image scanning lensless fiber-bundle endomicroscopy

Fiber-based confocal endomicroscopy has shown great promise for minimally-invasive deep-tissue imaging. Despite its advantages, confocal fiber-bundle endoscopy inherently suffers from undersampling due to the spacing between fiber cores, and low collection efficiency when the target is not in proximity to the distal fiber facet. Here, we demonstrate an adaptation of image-scanning microscopy (ISM) to lensless fiber bundle endoscopy, doubling the spatial sampling frequency and significantly improving collection efficiency. Our approach only requires replacing the confocal detector with a camera. It improves the spatial resolution for targets placed at a distance from the fiber tip, and addresses the fundamental challenge of aliasing/pixelization artifacts.

Unsupervised full-color cellular image reconstruction through disordered optical fiber

Recent years have witnessed the tremendous development of fusing fiber-optic imaging with supervised deep learning to enable high-quality imaging of hard-to-reach areas. Nevertheless, the supervised deep learning method imposes strict constraints on fiber-optic imaging systems, where the input objects and the fiber outputs have to be collected in pairs. To unleash the full potential of fiber-optic imaging, unsupervised image reconstruction is in demand. Unfortunately, neither optical fiber bundles nor multimode fibers can achieve a point-to-point transmission of the object with a high sampling density, as is a prerequisite for unsupervised image reconstruction. The recently proposed disordered fibers offer a new solution based on the transverse Anderson localization. Here, we demonstrate unsupervised full-color imaging with a cellular resolution through a meter-long disordered fiber in both transmission and reflection modes. The unsupervised image reconstruction consists of two stages. In the first stage, we perform a pixel-wise standardization on the fiber outputs using the statistics of the objects. In the second stage, we recover the fine details of the reconstructions through a generative adversarial network. Unsupervised image reconstruction does not need paired images, enabling a much more flexible calibration under various conditions. Our new solution achieves full-color high-fidelity cell imaging within a working distance of at least 4 mm by only collecting the fiber outputs after an initial calibration. High imaging robustness is also demonstrated when the disordered fiber is bent with a central angle of 60°. Moreover, the cross-domain generality on unseen objects is shown to be enhanced with a diversified object set.

Fourier holographic endoscopy for imaging continuously moving objects

Coherent fiber bundles are widely used for endoscopy, but conventional approaches require distal optics to form an object image and acquire pixelated information owing to the geometry of the fiber cores. Recently, holographic recording of a reflection matrix enables a bare fiber bundle to perform pixelation-free microscopic imaging as well as allows a flexible mode operation, because the random core-to-core phase retardations due to any fiber bending and twisting could be removed in situ from the recorded matrix. Despite its flexibility, the method is not suitable for a moving object because the fiber probe should remain stationary during the matrix recording to avoid the alteration of the phase retardations. Here, we acquire a reflection matrix of a Fourier holographic endoscope equipped with a fiber bundle and explore the effect of fiber bending on the recorded matrix. By removing the motion effect, we develop a method that can resolve the perturbation of the reflection matrix caused by a continuously moving fiber bundle. Thus, we demonstrate high-resolution endoscopic imaging through a fiber bundle, even when the fiber probe changes its shape along with the moving objects. The proposed method can be used for minimally invasive monitoring of behaving animals.

Novel Design and Application of High-NA Fiber Imaging Bundles for In Vivo Brain Imaging with Two-Photon Scanning Fluorescence Microscopy

Here, we provide experimental verification supporting the use of short-section imaging bundles for two-photon microscopy imaging of the mouse brain. The 8 mm long bundle is made of a pair of heavy-metal oxide glasses with a refractive index contrast of 0.38 to ensure a high numerical aperture NA = 1.15. The bundle is composed of 825 multimode cores, ordered in a hexagonal lattice with a pixel size of 14 μm and a total diameter of 914 μm. We demonstrate successful imaging through custom-made bundles with 14 μm resolution. As the input, we used a 910 nm Ti-sapphire laser with 140 fs pulse and a peak power of 9 × 10⁴ W. The excitation beam and fluorescent image were transferred through the fiber imaging bundle. As test samples, we used 1 μm green fluorescent latex beads, ex vivo hippocampal neurons expressing green fluorescent protein and cortical neurons in vivo expressing the fluorescent reporter GCaMP6s or immediate early gene Fos fluorescent reporter. This system can be used for minimal-invasive in vivo imaging of the cerebral cortex, hippocampus, or deep brain areas as a part of a tabletop system or an implantable setup. It is a low-cost solution, easy to integrate and operate for high-throughput experiments.

Two-Photon Endoscopy: State of the Art and Perspectives

In recent years, the demand for non-destructive deep-tissue imaging modalities has led to interest in multiphoton endoscopy. In contrast to bench top systems, multiphoton endoscopy enables subcellular resolution imaging in areas not reachable before. Several groups have recently presented their development towards the goal of producing user friendly plug and play system, which could be used in biological research and, potentially, clinical applications. We first present the technological challenges, prerequisites, and solutions in two-photon endoscopic systems. Secondly, we focus on the applications already found in literature. These applications mostly serve as a quality check of the built system, but do not answer a specific biomedical research question. Therefore, in the last part, we will describe our vision on the enormous potential applicability of adult two-photon endoscopic systems in biological and clinical research. We will thus bring forward the concept that two-photon endoscopy is a sine qua non in bringing this technique to the forefront in clinical applications.

Comparison between Hadamard and canonical bases for in situ wavefront correction and the effect of ordering in compressive sensing

In this work we compare the canonical and Hadamard bases for in situ wavefront correction of a focused Gaussian beam using a spatial light modulator (SLM). The beam is perturbed with a transparent optical element (sparse) or a random scatterer (both prevent focusing at a single spot). The phase corrections are implemented with different basis sizes (N=64,256,1024,4096) and the phase contribution of each basis element is measured with three-step interferometry. The field is reconstructed from the complete 3N measurements, and the correction is implemented by projecting the conjugate phase at the SLM. Our experiments show that in general, the Hadamard basis measurements yield better corrections because every element spans the relevant area of the SLM, thus reducing the noise in the interferograms. In contrast, the canonical basis has the fundamental limitation that the area of the elements is proportional to 1/N, and it requires dimensions that are compatible with the spatial period of the grating. In the case of the random scatterer, we only obtain reasonable corrections with the Hadamard basis and the intensity of the corrected spot increases monotonically with N, which is consistent with fast random changes in phase over small spatial scales. We also explore compressive sensing with the Hadamard basis and find that the minimum compression ratio needed to achieve corrections with similar quality to those that use the complete measurements depends on the basis ordering. The best results are achieved in the case of the Hadamard-Walsh and cake-cutting orderings. Surprisingly, in the case of the random scatterer we find that moderate compression ratios on the order of 10%-20% (N=4096) allow us to recover focused spots, although as expected, the maximum intensities increase monotonically with the number of measurements due to the non-sparsity of the signal.

Real-time holographic lensless micro-endoscopy through flexible fibers via fiber bundle distal holography

Fiber-based micro-endoscopes are a critically important tool for minimally-invasive deep-tissue imaging. However, current micro-endoscopes cannot perform three-dimensional imaging through dynamically-bent fibers without the use of bulky optical elements such as lenses and scanners at the distal end, increasing the footprint and tissue-damage. Great efforts have been invested in developing approaches that avoid distal bulky optical elements. However, the fundamental barrier of dynamic optical wavefront-distortions in propagation through flexible fibers limits current approaches to nearly-static or non-flexible fibers. Here, we present an approach that allows holographic, bend-insensitive, coherence-gated, micro-endoscopic imaging using commercially available multi-core fibers (MCFs). We achieve this by adding a partially-reflecting mirror to the distal fiber-tip, allowing to perform low-coherence full-field phase-shifting holography. We demonstrate widefield diffraction-limited reflection imaging of amplitude and phase targets through dynamically bent fibers at video-rate. Our approach holds potential for label-free investigations of dynamic samples.

Flexible-type ultrathin holographic endoscope for microscopic imaging of unstained biological tissues

Ultrathin lensless fibre endoscopes offer minimally invasive investigation, but they mostly operate as a rigid type due to the need for prior calibration of a fibre probe. Furthermore, most implementations work in fluorescence mode rather than label-free imaging mode, making them unsuitable for general medical diagnosis. Herein, we report a fully flexible ultrathin fibre endoscope taking 3D holographic images of unstained tissues with 0.85-μm spatial resolution. Using a bare fibre bundle as thin as 200-μm diameter, we design a lensless Fourier holographic imaging configuration to selectively detect weak reflections from biological tissues, a critical step for label-free endoscopic reflectance imaging. A unique algorithm is developed for calibration-free holographic image reconstruction, allowing us to image through a narrow and curved passage regardless of fibre bending. We demonstrate endoscopic reflectance imaging of unstained rat intestine tissues that are completely invisible to conventional endoscopes. The proposed endoscope will expedite a more accurate and earlier diagnosis than before with minimal complications.

Quantitative phase imaging through an ultra-thin lensless fiber endoscope

Quantitative phase imaging (QPI) is a label-free technique providing both morphology and quantitative biophysical information in biomedicine. However, applying such a powerful technique to in vivo pathological diagnosis remains challenging. Multi-core fiber bundles (MCFs) enable ultra-thin probes for in vivo imaging, but current MCF imaging techniques are limited to amplitude imaging modalities. We demonstrate a computational lensless microendoscope that uses an ultra-thin bare MCF to perform quantitative phase imaging with microscale lateral resolution and nanoscale axial sensitivity of the optical path length. The incident complex light field at the measurement side is precisely reconstructed from the far-field speckle pattern at the detection side, enabling digital refocusing in a multi-layer sample without any mechanical movement. The accuracy of the quantitative phase reconstruction is validated by imaging the phase target and hydrogel beads through the MCF. With the proposed imaging modality, three-dimensional imaging of human cancer cells is achieved through the ultra-thin fiber endoscope, promising widespread clinical applications.

Adaptive Wave-Front Shaping and Beam Focusing through Fiber Bundles for High-Resolution Bioimaging

We demonstrate an adaptive wave-front shaping of optical beams transmitted through fiber bundles as a powerful resource for multisite, high-resolution bioimaging. With the phases of all the beamlets delivered through up to 6000 different fibers within the fiber bundle controlled individually, by means of a high-definition spatial light modulator, the overall beam transmitted through the fiber bundle can be focused into a beam waist with a diameter less than 1 μm within a targeted area in a biotissue, providing a diffraction-limited spatial resolution adequate for single-cell or even subcellular bioimaging. The field intensity in the adaptively-focused continuous-wave laser beam in our fiber-bundle-imaging setting is more than two orders of magnitude higher than the intensity of the speckle background. Once robust beam focusing was achieved with a suitable phase profile across the input face of the fiber bundle, the beam focus can be scanned over a targeted area with no need for a further adaptive search, by applying a physically intuitive, wave-front-tilting phase mask on the field of input beamlets. This method of beam-focus scanning promises imaging speeds compatible with the requirements of in vivo calcium imaging.

Guidestar-free image-guided wavefront shaping

Optical imaging through scattering media is a fundamental challenge in many applications. Recently, breakthroughs such as imaging through biological tissues and looking around corners have been obtained via wavefront-shaping approaches. However, these require an implanted guidestar for determining the wavefront correction, controlled coherent illumination, and most often raster scanning of the shaped focus. Alternative novel computational approaches that exploit speckle correlations avoid guidestars and wavefront control but are limited to small two-dimensional objects contained within the "memory-effect" correlation range. Here, we present a new concept, image-guided wavefront shaping, allowing widefield noninvasive, guidestar-free, incoherent imaging through highly scattering layers, without illumination control. The wavefront correction is found even for objects that are larger than the memory-effect range, by blindly optimizing image quality metrics. We demonstrate imaging of extended objects through highly scattering layers and multicore fibers, paving the way for noninvasive imaging in various applications, from microscopy to endoscopy.

Photoacoustic endoscopy: A progress review

Endoscopy has been widely used in biomedical imaging and integrated with various optical and acoustic imaging modalities. Photoacoustic imaging (PAI), one of the fastest growing biomedical imaging modalities, is a noninvasive and nonionizing method that owns rich optical contrast, deep acoustic penetration depth, multiscale and multiparametric imaging capability. Hence, it is preferred to miniaturize the volume of PAI and develop an emerged endoscopic imaging modality referred to as photoacoustic endoscopy (PAE). It has been developed for more than one decade since the first report of PAE. Unfortunately, until now, there is no mature photoacoustic endoscopic technique recognized in clinic due to various technical limitations. To address this concern, recent development of new scanning mechanisms, adoption of novel optical/acoustic devices, utilization of superior computation methods and exploration of multimodality strategies have significantly promoted the progress of PAE toward clinic. In this review, we comprehensively reviewed recent progresses in single- and multimodality PAE with new physics, mechanisms and strategies to achieve practical devices for potential applicable scenarios including esophageal, gastrointestinal, urogenital and intravascular imaging. We ended this review with challenges and prospects for future development of PAE.

Using fiber-bending-generated speckles for improved working distance and background rejection in lensless micro-endoscopy

Lensless flexible fiber-bundle-based endoscopes allow imaging at depths beyond the reach of conventional microscopes with a minimal footprint. These multicore fibers provide a simple solution for wide-field fluorescent imaging when the target is adjacent to the fiber facet. However, they suffer from a very limited working distance and out-of-focus background. Here, we carefully study the dynamic speckle illumination patterns generated by bending a commercial fiber bundle and show that they can be exploited to allow extended working distance and background rejection, using a super-resolution fluctuations imaging analysis of multiple frames, without the addition of any optical elements.

Seeing through multimode fibers with real-valued intensity transmission matrices

Image transmission through multimode optical fibers has been an area of immense interests driven by the demand for miniature endoscopes in biomedicine and higher speed and capacity in telecommunications. Conventionally, a complex-valued transmission matrix is obtained experimentally to link the input and output light fields of a multimode fiber for image retrieval, which complicates the experimental setup and increases the computational complexity. Here, we report a simple and high-speed method for image retrieval based on our demonstration of a pseudo-linearity between the input and output light intensity distributions of multimode fibers. We studied the impact of several key parameters to image retrieval, including image pixel count, fiber core diameter and numerical aperture. We further demonstrated with experiments and numerical simulations that a wide variety of input binary and gray scale images could be faithfully retrieved from the corresponding output speckle patterns. Thus, it promises to be useful for highly miniaturized endoscopy in biomedicine and spatial-mode-division multiplexing in telecommunications.

Video-rate lensless endoscope with self-calibration using wavefront shaping

Lensless fiber endoscopes are of great importance for keyhole imaging. Coherent fiber bundles (CFB) can be used in endoscopes as remote phased arrays to capture images. One challenge is to image at high speed while correcting aberrations induced by the CFB. We propose the combination of digital optical phase conjugation, using a spatial light modulator, with fast scanning, for which a 2D galvo scanner and an adaptive lens are employed. We achieve the transmission of laser and image scanning through the CFB. Video-rate imaging at 20 Hz in 2D with subcellular resolution is demonstrated in 3D with 1 Hz. The sub-millimeter-diameter scanning endoscope has a great potential in biomedicine, for manipulation, e.g., in optogenetics, as well as in imaging.