High-speed photoacoustic-guided wavefront shaping for focusing light in scattering media

Seeing through multimode fibers using real-valued intensity transmission matrix with deep learning

Multimode fibers (MMFs) are emerging as a highly attractive technology for applications in biomedical endoscopy and telecommunications, thanks to their ability to transmit images and data through a large number of transverse optical modes. However, light transmission through MMFs suffers from distortions caused by mode dispersion and coupling. While recent deep learning advancements have shown potential for improving image transmission through MMFs, these methods typically require an extensive training dataset and often exhibit limited generalization capability. In this work, we propose a hybrid approach that combines a real-valued intensity transmission matrix (RVITM) with deep learning for enhanced image retrieval through MMFs. Our approach first characterizes the MMF and retrieves images using a RVITM algorithm, followed by refinement with a hierarchical, parallel multi-scale (HPM)-attention U-Net to improve image quality. Experimental results demonstrated that our approach achieved high-quality reconstructions, with structural similarity index (SSIM) and peak signal-to-noise ratio (PSNR) values of up to 0.9524 and 33.244 dB, respectively. This approach also offers strong generalization capabilities, requires fewer training samples and converges more quickly compared to purely deep learning-based methods reported in the literature. These results highlight the potential of our method for ultrathin endoscopy applications and spatial-mode multiplexing in telecommunications using MMFs.

Scattering Correction through Fourier-Domain Intensity Coupling in Two-Photon Microscopy (2P-FOCUS)

Light penetration depth in biological tissue is limited by tissue scattering. Correcting scattering becomes particularly challenging in scenarios with limited photon availability and when access to the transmission side of the scattering tissue is not possible. Here, we introduce a new two-photon microscopy system with Fourier-domain intensity coupling for scattering correction (2P-FOCUS). 2P-FOCUS corrects scattering by intensity modulation in the Fourier domain, leveraging the nonlinearity of multiple-beam interference and two-photon excitation, eliminating the need for a guide star, iterative optimization, or measuring transmission or reflection matrices. 2P-FOCUS uses random patterns to probe scattering properties, combined with a single-shot algorithm to rapidly generate the correction mask. 2P-FOCUS can also correct scattering beyond the limitation of the memory effect by automatically customizing correction masks for each subregion in a large field-of-view. We provide several proof-of-principle demonstrations here, including focusing and imaging through a bone sample, and imaging neurons and cerebral blood vessels in the mouse brain ex vivo. 2P-FOCUS significantly enhances two-photon fluorescence signals by several tens of folds compared to cases without scattering correction at the same excitation power. 2P-FOCUS can also correct tissue scattering over a 230×230×510 μm3 volume, which is beyond the memory effect range. 2P-FOCUS is able to measure, calculate, and correct scattering within a few seconds, effectively delivering more light deep into the scattering tissue. 2P-FOCUS could be broadly adopted for deep tissue imaging owing to its powerful combination of effectiveness, speed, and cost.

Wavefront shaping improves the transparency of the scattering media: a review

Significance

Wavefront shaping (WFS) can compensate for distortions by optimizing the wavefront of the input light or reversing the transmission matrix of the media. It is a promising field of research. A thorough understanding of principles and developments of WFS is important for optical research.

Aim

To provide insight into WFS for researchers who deal with scattering in biomedicine, imaging, and optical communication, our study summarizes the basic principles and methods of WFS and reviews recent progress.

Approach

The basic principles, methods of WFS, and the latest applications of WFS in focusing, imaging, and multimode fiber (MMF) endoscopy are described. The practical challenges and prospects of future development are also discussed.

Results

Data-driven learning-based methods are opening up new possibilities for WFS. High-resolution imaging through MMFs can support small-diameter endoscopy in the future.

Conclusion

The rapid development of WFS over the past decade has shown that the best solution is not to avoid scattering but to find ways to correct it or even use it. WFS with faster speed, more optical modes, and more modulation degrees of freedom will continue to drive exciting developments in various fields.

Light scattering control with the two-step focusing method based on neural networks and multi-pixel coding

Focusing light through scattering media is essential for high-resolution optical imaging and deep penetration. Here, a two-step focusing method based on neural networks (NNs) and multi-pixel coding is proposed to achieve high-quality focusing with theoretical maximum enhancement. In the first step, a single-layer neural network (SLNN) is used to obtain the initial mask, which can be used to focus with a moderate enhancement. In the second step, we use multi-pixel coding to encode the initial mask. The coded masks and their corresponding speckle patterns are used to train another SLNN to get the final mask and achieve high-quality focusing. In this experiment, for a mask of 16 × 16 modulation units, in the case of using 8 pixels in a modulation unit, focus with the enhancement of 40.3 (only 0.44 less than the theoretical value) has been achieved with 3000 pictures (1000 pictures in the first step and 2000 pictures in the second step). Compared with the case of employing only the initial mask and the direct multi-pixel encoded mask, the enhancement is increased by 220% and 24%. The proposed method provides a new idea for improving the focusing effect through the scattering media using NNs.

Ultrathin, high-speed, all-optical photoacoustic endomicroscopy probe for guiding minimally invasive surgery

Photoacoustic (PA) endoscopy has shown significant potential for clinical diagnosis and surgical guidance. Multimode fibres (MMFs) are becoming increasingly attractive for the development of miniature endoscopy probes owing to their ultrathin size, low cost and diffraction-limited spatial resolution enabled by wavefront shaping. However, current MMF-based PA endomicroscopy probes are either limited by a bulky ultrasound detector or a low imaging speed that hindered their usability. In this work, we report the development of a highly miniaturised and high-speed PA endomicroscopy probe that is integrated within the cannula of a 20 gauge medical needle. This probe comprises a MMF for delivering the PA excitation light and a single-mode optical fibre with a plano-concave microresonator for ultrasound detection. Wavefront shaping with a digital micromirror device enabled rapid raster-scanning of a focused light spot at the distal end of the MMF for tissue interrogation. High-resolution PA imaging of mouse red blood cells covering an area 100 µm in diameter was achieved with the needle probe at ∼3 frames per second. Mosaicing imaging was performed after fibre characterisation by translating the needle probe to enlarge the field-of-view in real-time. The developed ultrathin PA endomicroscopy probe is promising for guiding minimally invasive surgery by providing functional, molecular and microstructural information of tissue in real-time.

Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields

Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media. Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution. However, the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications. In addition, the components of an optical system are usually designed and manufactured for a fixed function or performance. Recent advances in wavefront shaping have demonstrated that scattering- or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium. This offers unprecedented opportunities in many applications to achieve controllable optical delivery or detection at depths or dynamically configurable functionalities by using scattering media to substitute conventional optical components. In this article, the recent progress of wavefront shaping in multidisciplinary fields is reviewed, from optical focusing and imaging with scattering media, functionalized devices, modulation of mode coupling, and nonlinearity in multimode fiber to multimode fiber-based applications. Apart from insights into the underlying principles and recent advances in wavefront shaping implementations, practical limitations and roadmap for future development are discussed in depth. Looking back and looking forward, it is believed that wavefront shaping holds a bright future that will open new avenues for noninvasive or minimally invasive optical interactions and arbitrary control inside deep tissues. The high degree of freedom with multiple scattering will also provide unprecedented opportunities to develop novel optical devices based on a single scattering medium (generic or customized) that can outperform traditional optical components.

Light-field focusing and modulation through scattering media based on dual-polarization-encoded digital optical phase conjugation

Digital optical phase conjugation (DOPC) can be applied for light-field focusing and imaging through or within scattering media. Traditional DOPC only recovers the phase but loses the polarization information of the original incident beam. In this Letter, we propose a dual-polarization-encoded DOPC to recover the full information (both phase and polarization) of the incident beam. The phase distributions of two orthogonal polarization components of the speckle field coming from a multimode fiber are first measured by using digital holography. Then, the phase distributions are separately modulated on two beams and their conjugations are superposed to recover the incident beam through the fiber. By changing the phase difference or amplitude ratio between the two conjugate beams, light fields with complex polarization distribution can also be generated. This method will broaden the application scope of DOPC in imaging through scattering media.

High-resolution photoacoustic microscopy with deep penetration through learning

Optical-resolution photoacoustic microscopy (OR-PAM) enjoys superior spatial resolution and has received intense attention in recent years. The application, however, has been limited to shallow depths because of strong scattering of light in biological tissues. In this work, we propose to achieve deep-penetrating OR-PAM performance by using deep learning enabled image transformation on blurry living mouse vascular images that were acquired with an acoustic-resolution photoacoustic microscopy (AR-PAM) setup. A generative adversarial network (GAN) was trained in this study and improved the imaging lateral resolution of AR-PAM from 54.0 µm to 5.1 µm, comparable to that of a typical OR-PAM (4.7 µm). The feasibility of the network was evaluated with living mouse ear data, producing superior microvasculature images that outperforms blind deconvolution. The generalization of the network was validated with in vivo mouse brain data. Moreover, it was shown experimentally that the deep-learning method can retain high resolution at tissue depths beyond one optical transport mean free path. Whilst it can be further improved, the proposed method provides new horizons to expand the scope of OR-PAM towards deep-tissue imaging and wide applications in biomedicine.

Video-rate dual-modal photoacoustic and fluorescence imaging through a multimode fibre towards forward-viewing endomicroscopy

Multimode fibres (MMFs) are becoming increasingly attractive in optical endoscopy as they promise to enable unparallelled miniaturisation, spatial resolution and cost. However, high-speed imaging with wavefront shaping has been challenging. Here, we report the development of a video-rate dual-modal photoacoustic (PA) and fluorescence microscopy probe with a high-speed digital micromirror device (DMD) towards forward-viewing endomicroscopy. Optimal DMD patterns were obtained using a real-valued intensity transmission matrix algorithm to raster-scan a 1.5  μ m-diameter focused beam at the distal fibre tip for imaging. The PA imaging speed and spatial resolution were varied from  ∼ 2 to 57 frames per second and from 1.7 to 3  μ m, respectively. Further, high-fidelity PA images of carbon fibres and mouse red blood cells were acquired at unprecedented speed. The capability of dual-modal imaging was demonstrated with phantoms. We anticipate that with further miniaturisation of the ultrasound detector, this probe could be integrated into medical needles to guide minimally invasive procedures.

Hybrid optimization algorithm based on neural networks and its application in wavefront shaping

The scattering effect of turbid media can lead to optical wavefront distortion. Focusing light through turbid media can be achieved using wavefront shaping techniques. Intelligent optimization algorithms and neural network algorithms are two powerful types of algorithms in the field of wavefront shaping but have their advantages and disadvantages. In this paper, we propose a new hybrid algorithm that combines the particle swarm optimization algorithm (PSO) and single-layer neural network (SLNN) to achieve the complementary advantages of both. A small number of training sets are used to train the SLNN to obtain preliminary focusing results, after which the PSO continues to optimize to the global optimum. The hybrid algorithm achieves faster convergence and higher enhancement than the PSO, while reducing the size of training samples required for SLNN training. SLNN trained with 1700 training sets can speed up the convergence of the PSO by about 50% and boost the final enhancement by about 24%. This hybrid algorithm will be of great significance in fields such as biomedicine and particle manipulation.

Focusing light through multimode fibres using a digital micromirror device: a comparison study of non-holographic approaches

Focusing light through a multimode fibre (MMF) has attracted significant research interest, mainly driven by the need for miniature endoscopes in biomedicine. In recent years, digital micromirror devices (DMD) have become increasingly popular as a high-speed alternative to liquid-crystal spatial light modulators for light focusing via wavefront shaping based on binary amplitude modulations. To exploit the potentials and limitations of the state-of-the-art DMD-based wavefront shaping methods, in this study, for the first time, we compared four representative, non-holographic and DMD-based methods that are reported so far in literature with the same experimental and simulation conditions, including a real-valued intensity transmission matrix (RVITM)-based algorithm, a complex-valued transmission matrix (TM)-based algorithm, a conditional probability algorithm and a genetic algorithm. We investigated the maximum achievable peak-to-background ratio (PBR) in comparison to theoretical expectations, and further improved the performance of the RVITM-based method. With both numerical simulations and experiments, we found that the genetic algorithm offered the highest PBR but suffered from the lowest focusing speed, while the RVITM-based algorithm provided a comparable PBR to that of the genetic algorithm, and the highest focusing speed.