Reference-less measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques

A novel transcranial MR Guided focused ultrasound method based on the ultrashort echo time skull acoustic model and phase retrieval techniques

Transcranial ultrasound stimulation (TUS) has been clinically applied as a neuromodulation tool. Particularly, the phase array ultrasound can be applied in TUS to non-invasively focus on the cortex or deep brain. However, the vital phase distortion of the ultrasound induced by the skull limits its clinical application. In the current study, we aimed to develop a hybrid method that combines the ultrashort echo time (UTE) magnetic resonance imaging (MRI) sequences with the prDeep technique to achieve focusing ventral intermediate thalamic nucleus (VIM). The time-reversal (TR) approach of the UTE numerical acoustic model of the skull combined with the prDeep algorithm was used to reduce the number of iterations. The skull acoustic model simulation therapy process was establish to valid this method's prediction and focus performance, and the classical TR method were considered as the gold standard (GS). Our approach could restore 75% of the GS intensity in 25 iteration steps, with a superior the noise immunity. Our findings demonstrate that the phase aberration caused by the skull can be estimated using phase retrieval techniques to achieve a fast and accurate transcranial focus. The method has excellent adaptability and anti-noise capacity for satisfying complex and changeable scenarios.

Terahertz Spatiotemporal Wave Synthesis in Random Systems

Complex media have emerged as a powerful and robust framework to control light-matter interactions designed for task-specific optical functionalities. Studies on wavefront shaping through disordered systems have demonstrated optical wave manipulation capabilities beyond conventional optics, including aberration-free and subwavelength focusing. However, achieving arbitrary and simultaneous control over the spatial and temporal features of light remains challenging. In particular, no practical solution exists for field-level arbitrary spatiotemporal control of wave packets. A new paradigm shift has emerged in the terahertz frequency domain, offering methods for absolute time-domain measurements of the scattered electric field, enabling direct field-based wave synthesis. In this work, we report the experimental demonstration of field-level control of single-cycle terahertz pulses on arbitrary spatial points through complex disordered media.

Tight focusing through scattering media via a Bessel-basis transmission matrix

The transmission matrix (TM) is a powerful tool for focusing light through scattering media. Here, we demonstrate a Bessel-basis TM that enables tight focusing through the scattering media and reduces the full width at half maximum of the focus by 23% on average, as compared to the normally used Hadamard-basis TM. To measure the Bessel-basis TM, we establish a common-path inter-mode interferometer (IMI), which can fully utilize the pixels of the spatial light modulator, leading to an enhancement in the peak-to-background intensity ratio (PBR) of the focus. Experimental results suggest that the Bessel-basis TM can achieve a tighter focus behind the scattering media, and the PBR of the focus obtained by the IMI is around 14.3% higher than that achieved using the normal peripheral reference interferometry.

High-fidelity multi-channel optical information transmission through scattering media

High-fidelity optical information transmission through strongly scattering media is challenging, but is crucial for the applications such as the free-space optical communication in a haze or fog. Binarizing optical information can somehow suppress the disruptions caused by light scattering. However, this method gives a compromised communication throughput. Here, we propose high-fidelity multiplexing anti-scattering transmission (MAST). MAST encodes multiple bits into a complex-valued pattern, loads the complex-valued pattern to an optical field through modulation, and finally employs a scattering matrix-assisted retrieval technique to reconstruct the original information from the speckle patterns. In our demonstration, we multiplexed three channels and MAST achieved a high-fidelity transmission of 3072 (= 1024× 3) bits data per transmission and average transmission error as small as 0.06%.

Wavefront shaping improves the transparency of the scattering media: a review

Significance

Wavefront shaping (WFS) can compensate for distortions by optimizing the wavefront of the input light or reversing the transmission matrix of the media. It is a promising field of research. A thorough understanding of principles and developments of WFS is important for optical research.

Aim

To provide insight into WFS for researchers who deal with scattering in biomedicine, imaging, and optical communication, our study summarizes the basic principles and methods of WFS and reviews recent progress.

Approach

The basic principles, methods of WFS, and the latest applications of WFS in focusing, imaging, and multimode fiber (MMF) endoscopy are described. The practical challenges and prospects of future development are also discussed.

Results

Data-driven learning-based methods are opening up new possibilities for WFS. High-resolution imaging through MMFs can support small-diameter endoscopy in the future.

Conclusion

The rapid development of WFS over the past decade has shown that the best solution is not to avoid scattering but to find ways to correct it or even use it. WFS with faster speed, more optical modes, and more modulation degrees of freedom will continue to drive exciting developments in various fields.

High-throughput imaging through dynamic scattering media based on speckle de-blurring

Effectively imaging through dynamic scattering media is of great importance and challenge. Some imaging methods based on physical or learning models have been designed for object reconstruction. However, with an increase in exposure time or more drastic changes in the scattering medium, the speckle pattern superimposed during camera integration time undergoes more significant changes, resulting in a modification of the collected speckle structure and increased blurring, which brings significant challenges to the reconstruction. Here, the clearer structural information of blurred speckles is unearthed with a presented speckle de-blurring algorithm, and a high-throughput imaging method through rapidly changing scattering media is proposed for reconstruction under long exposure. For the problem of varying blur degrees in different regions of the speckle, a block-based method is proposed to divide the speckle into distinct sub-speckles, which can realize the reconstruction of hidden objects. The imaging of hidden objects with different complexity through dynamic scattering media is demonstrated, and the reconstruction results are improved significantly for speckles with different blur degrees, which verifies the effectiveness of the method. This method is a high-throughput approach that enables non-invasive imaging solely through the collection of a single speckle. It directly operates on blurred speckles, making it suitable for traditional speckle-correlation methods and deep learning (DL) methods. This provides a new way of thinking about solving practical scattering imaging challenges.

Imaging through a scattering medium via model-driven deep learning

Imaging through a scattering medium is of great significance in many areas. Especially, speckle correlation imaging has been valued for its noninvasiveness. In this work, we report a deep learning solution that incorporates the physical model and an additional regularization for high-fidelity speckle correlation imaging. Without large-scale data to train, the physical model and regularization prior provide a correct direction for neural network to precisely reconstruct hidden objects from speckle under different scattering scenarios and noise levels. Experimental results demonstrate that the proposed method presents a significant advance in improving generalization and combating the invasion of noise.

Referenceless characterization of complex media using physics-informed neural networks

In this work, we present a method to characterize the transmission matrices of complex scattering media using a physics-informed, multi-plane neural network (MPNN) without the requirement of a known optical reference field. We use this method to accurately measure the transmission matrix of a commercial multi-mode fiber without the problems of output-phase ambiguity and dark spots, leading to up to 58% improvement in focusing efficiency compared with phase-stepping holography. We demonstrate how our method is significantly more noise-robust than phase-stepping holography and show how it can be generalized to characterize a cascade of transmission matrices, allowing one to control the propagation of light between independent scattering media. This work presents an essential tool for accurate light control through complex media, with applications ranging from classical optical networks, biomedical imaging, to quantum information processing.

Reference-less wavefront shaping in a Hopfield-like rough intensity landscape

This study introduces a new digital-micromirror based binary-phase wavefront shaping technique, which allows the measurement of the full coupling matrix of a disordered medium without a reference and enables to focusing transmitted light. The coupling matrix takes on a bi-dyadic structure, similar to a Hopfield memory matrix containing two memory patterns. Sequential wavefront optimization in this configuration often stalls due to a rough intensity landscape, resulting in a non-optimal state. To overcome this issue, we propose the Complete Couplings Mapping method, which consistently reaches the theoretically expected maximum intensity.

Focusing through scattering media by a single polarization transmission matrix with binary polarization modulation

Wavefront distortion induced by scattering media seriously affects optical focusing. Wavefront shaping based on a transmission matrix (TM) is useful in controlling light propagation in highly scattering media. Traditional TM generally studies amplitude and phase, but the stochastic nature of the light propagation in the scattering medium also affects its polarization. Based on the binary polarization modulation, we propose a single polarization transmission matrix (SPTM) and achieve single-spot focusing through scattering media. We anticipate that the SPTM will be widely used in wavefront shaping.

Deep learning for efficiently imaging through the localized speckle field of a multimode fiber

Due to the occurrence of redundant speckle, multimode fiber (MMF) imaging is extremely challenging. Our work studies the relationship between the effective feature distribution of the speckle field and the local spatial position and area, and proves that the information distribution of the speckle is highly redundant. The effective feature refers to the phase and amplitude information of the optical field carrying the image point information and the co-exciting very redundant information due to mode dispersion, interference, coupling, and entrained noise through transmission. The neural network Swin-Unet can well learn the association information between global and local features, greatly simplifies the fitting of the MMF end-to-end global mapping relationship, and achieves high-fidelity reconstruction from the local speckle field to the global image. This work will contribute to the realization of MMF real-time large-field endoscopic imaging.

Light scattering control with the two-step focusing method based on neural networks and multi-pixel coding

Focusing light through scattering media is essential for high-resolution optical imaging and deep penetration. Here, a two-step focusing method based on neural networks (NNs) and multi-pixel coding is proposed to achieve high-quality focusing with theoretical maximum enhancement. In the first step, a single-layer neural network (SLNN) is used to obtain the initial mask, which can be used to focus with a moderate enhancement. In the second step, we use multi-pixel coding to encode the initial mask. The coded masks and their corresponding speckle patterns are used to train another SLNN to get the final mask and achieve high-quality focusing. In this experiment, for a mask of 16 × 16 modulation units, in the case of using 8 pixels in a modulation unit, focus with the enhancement of 40.3 (only 0.44 less than the theoretical value) has been achieved with 3000 pictures (1000 pictures in the first step and 2000 pictures in the second step). Compared with the case of employing only the initial mask and the direct multi-pixel encoded mask, the enhancement is increased by 220% and 24%. The proposed method provides a new idea for improving the focusing effect through the scattering media using NNs.

Locating through dynamic scattering media based on speckle correlations

In complex imaging settings, optical scattering often prohibits the formation of a clear target image, and instead, only a speckle without the original spatial structure information is obtained. Scattering seriously interferes with the locating of targets; especially, when the scattering medium is dynamic, the dynamic nature leads to rapid decorrelation of optical information in time, and the challenge increases. Here, a locating method is proposed to detect the target hidden behind a dynamic scattering medium, which uses the a priori information of a known reference object in the neighborhood of the target. The research further designs an automatic calibration method to simplify the locating process, and analyzes the factors affecting positioning accuracy. The proposed method enables us to predict the position of a target from the autocorrelation of the captured speckle pattern; the angle and distance deviations of the target are all within 2.5%. This approach can locate a target using only a single-shot speckle pattern, and it is beneficial for target localization in dynamic scattering conditions.

Imaging through scattering media using differential intensity transmission matrices with different Hadamard orderings

A transmission matrix (TM) is a powerful tool for light focusing and imaging through scattering media. For measuring it, the normal way requires establishing a multiple-step phase-shifting interferometer, which makes the TM measurement not only complex and sensitive but also time-consuming. Imaging through scattering media using an intensity TM method can make the setup for TM measurement without the phase-shifting interferometer, thus it is much simple, more stable, and several times faster. Here, based upon a differential intensity TM method, we demonstrated it to do imaging through scattering media using different Hadamard orderings. To accelerate the TM measuring speed while degrading as little as possible of the imaging quality, a relatively reasonable strategy to plan Hadamard orderings for the TM measurement is designed since it can suggest us to preferentially measure the components in TM that are more important to the imaging quality. Thanks to the different Hadamard orderings, their influences on the imaging quality at different measuring ratios are investigated, thus an optimal measuring ordering for accelerating the TM measurement can be obtained, while only sacrificing as little as possible of the image fidelity. Simulations and experiments verify the effectiveness of the proposed method.

One-to-all lightweight Fourier channel attention convolutional neural network for speckle reconstructions

Speckle reconstruction is a classical inverse problem in computational imaging. Inspired by the memory effect of the scattering medium, deep learning methods reveal excellent performance in extracting the correlation of speckle patterns. Nowadays, advanced models generally include more than 10M parameters and mostly pay more attention to the spatial feature information. However, the frequency domain of images also contains precise hierarchical representations. Here we propose a one-to-all lightweight Fourier channel attention convolutional neural network (FCACNN) with Fourier channel attention and the res-connected bottleneck structure. Compared with the state-of-the-art model, i.e., self-attention armed convolutional neural network (SACNN), our architecture has better feature extraction and reconstruction ability. The Pearson correlation coefficient and Jaccard index scores of FCACNN increased by at least 5.2% and 13.6% compared with task-related models. And the parameter number of the lightweight FCACNN is only 1.15M. Furthermore, the validation results show that the one-to-all model, FCACNN, has excellent generalization capability on unseen speckle patterns such as handwritten letters and Quickdraws.

Non-invasive imaging through scattering medium and around corners beyond 3D memory effect

The three-dimensional (3D) memory effect (ME) has been shown to exist in a variety of scattering scenes. Limited by the scope of ME, speckle correlation technology only can be applied in a small imaging field of view (FOV) with a small depth of field (DOF). In this Letter, an untrained neural network is constructed and used as an optimization tool to restore the targets beyond the 3D ME range. The autocorrelation consistency relationship and the generative adversarial strategy are combined. Only single frame speckle and unaligned real targets are needed for online optimization; therefore, the neural network does not need to train in advance. Furthermore, the proposed method does not need to conduct additional modulation for the system. This method can reconstruct not only hidden targets behind the scattering medium, but also targets around corners. The combination strategy of the generative adversarial framework with physical priors used to decouple the aliasing information and reconstruct the target will provide inspiration for the field of computational imaging.

Anti-scattering light focusing with full-polarization digital optical phase conjugation based on digital micromirror devices

The high resolution of optical imaging and optogenetic stimulation in the deep tissue requires focusing light against strong scattering with high contrast. Digital optical phase conjugation (DOPC) has emerged recently as a promising solution for this requirement, because of its short latency. A digital micromirror device (DMD) in the implementation of DOPC enables a large number of modulation modes and a high speed of modulation both of which are important when dealing with a highly dynamic scattering medium. Here, we propose full-polarization DOPC (fpDOPC) in which two DMDs simultaneously modulate the two orthogonally polarized components of the optical field, respectively, to mitigate the effect of depolarization caused by strong scattering. We designed a simple system to overcome the difficulty of alignment encountered when modulating two polarized components independently. Our simulation and experiment showed that fpDOPC could generate a high-contrast focal spot, even though the polarization of light had been highly randomized by scattering. In comparison with the conventional method of modulating the polarization along a particular direction, fpDOPC can improve the peak to background ratio of the focal spot by a factor of two. This new technique has good potential in applications such as high-contrast light focusing in vivo.

Wavefront shaping: A versatile tool to conquer multiple scattering in multidisciplinary fields

Optical techniques offer a wide variety of applications as light-matter interactions provide extremely sensitive mechanisms to probe or treat target media. Most of these implementations rely on the usage of ballistic or quasi-ballistic photons to achieve high spatial resolution. However, the inherent scattering nature of light in biological tissues or tissue-like scattering media constitutes a critical obstacle that has restricted the penetration depth of non-scattered photons and hence limited the implementation of most optical techniques for wider applications. In addition, the components of an optical system are usually designed and manufactured for a fixed function or performance. Recent advances in wavefront shaping have demonstrated that scattering- or component-induced phase distortions can be compensated by optimizing the wavefront of the input light pattern through iteration or by conjugating the transmission matrix of the scattering medium. This offers unprecedented opportunities in many applications to achieve controllable optical delivery or detection at depths or dynamically configurable functionalities by using scattering media to substitute conventional optical components. In this article, the recent progress of wavefront shaping in multidisciplinary fields is reviewed, from optical focusing and imaging with scattering media, functionalized devices, modulation of mode coupling, and nonlinearity in multimode fiber to multimode fiber-based applications. Apart from insights into the underlying principles and recent advances in wavefront shaping implementations, practical limitations and roadmap for future development are discussed in depth. Looking back and looking forward, it is believed that wavefront shaping holds a bright future that will open new avenues for noninvasive or minimally invasive optical interactions and arbitrary control inside deep tissues. The high degree of freedom with multiple scattering will also provide unprecedented opportunities to develop novel optical devices based on a single scattering medium (generic or customized) that can outperform traditional optical components.

Imaging Complex Targets through a Scattering Medium Based on Adaptive Encoding

The scattering of light after passing through a complex medium poses challenges in many fields. Any point in the collected speckle will contain information from the entire target plane because of the randomness of scattering. The detailed information of complex targets is submerged in the aliased signal caused by random scattering, and the aliased signal causes the quality of the recovered target to be degraded. In this paper, a new neural network named Adaptive Encoding Scattering Imaging ConvNet (AESINet) is constructed by analyzing the physical prior of speckle image redundancy to recover complex targets hidden behind the opaque medium. AESINet reduces the redundancy of speckle through adaptive encoding which effectively improves the separability of data; the encoded speckle makes it easier for the network to extract features, and helps restore the detailed information of the target. The necessity for adaptive encoding is analyzed, and the ability of this method to reconstruct complex targets is tested. The peak signal-to-noise ratio (PSNR) of the reconstructed target after adaptive encoding can be improved by 1.8 dB. This paper provides an effective reference for neural networks combined with other physical priors in scattering processes.

Prior-free imaging unknown target through unknown scattering medium

Imaging through scattering medium based on deep learning has been extensively studied. However, existing methods mainly utilize paired data-prior and lack physical-process fusion, and it is difficult to reconstruct hidden targets without the trained networks. This paper proposes an unsupervised neural network that integrates the universal physical process. The reconstruction process of the network is irrelevant to the system and only requires one frame speckle pattern and unpaired targets. The proposed network enables online optimization by using physical process instead of fitting data. Thus, large-scale paired data no longer need to be obtained to train the network in advance, and the proposed method does not need prior information. The optimization of the network is a physical-based process rather than a data mapping process, and the proposed method also increases the insufficient generalization ability of the learning-based method in scattering medium and targets. The universal applicability of the proposed method to different optical systems increases the likelihood that the method will be used in practice.

On-Demand Phase Control of a 7-Fiber Amplifiers Array with Neural Network and Quasi-Reinforcement Learning

We report a coherent beam combining technique using a specific quasi-reinforcement learning scheme. A neural network learned by this method enables the tailoring and locking of a tiled beam array on any phase map. We present the experimental implementation of on-demand phase control by a neural network in a seven-fiber laser array. This servo loop needs only six phase corrections to converge to the desired phase set at any profile, with a bandwidth higher than 1 kHz. Moreover, we demonstrate the dynamical feature of adaptive phase control, performing sequences of controlled phase sets. It is the first time, to the best of our knowledge, that an actual array of seven-fiber amplifiers has been successfully phase-locked and controlled by machine learning.

Efficient sorting for an orbital angular momentum multiplexing communication link based on a digital micromirror device and a diffuser

Efficient sorting multiple orbital angular momentum (OAM) spatial modes is a significant step in OAM multiplexing communications. Recently, wavefront shaping (WS) techniques have been implemented to manipulate light scattering through a diffuser. We reported a novel scheme for sorting multiplexed OAM modes faster and more accurately, using the complex amplitude WS based on a digital micromirror device (DMD) through a diffuser to shape the full field (phase and amplitude) of the OAM modes. In this study, we simulate this complex sorter for demultiplexing multiple modes and make a performance comparison with the previous sorter using the phase-only WS. Our results showed that for arbitrary two multiplexed modes, the sorter could achieve a high detection probability of more than 0.99. As the number of the multiplexed modes increases, the detection probability decreases to ∼0.82 when sorting seven modes, which contrasts the ∼0.71 of the phase-only sorters. We also experimentally verified the feasibility, that for arbitrary two modes, the sorter could reach a high detection probability of more than 0.99, and the complex sorter is capable of higher detection probability than the phase-only sorter under the same conditions. Hence, we anticipate that this sorter may potentially be demultiplexing multiple OAM spatial modes efficiently and quickly.

Efficient color imaging through unknown opaque scattering layers via physics-aware learning

Color imaging with scattered light is crucial to many practical applications and becomes one of the focuses in optical imaging fields. More physics theories have been introduced in the deep learning (DL) approach for the optical tasks and improve the imaging capability a lot. Here, an efficient color imaging method is proposed in reconstructing complex objects hidden behind unknown opaque scattering layers, which can obtain high reconstruction fidelity in spatial structure and accurate restoration in color information by training with only one diffuser. More information is excavated by utilizing the scattering redundancy and promotes the physics-aware DL approach to reconstruct the color objects hidden behind unknown opaque scattering layers with robust generalization capability by an efficient means. This approach gives impetus to color imaging through dynamic scattering media and provides an enlightening reference for solving complex inverse problems based on physics-aware DL methods.

Feedback-assisted transmission matrix measurement of a multimode fiber in a referenceless system

Recent development in wavefront shaping shows the promise to employ multimode fibers (MMFs) to deliver images in endoscopy. In these applications, retrieving the transmission matrix (TM) of the MMF is especially important. Among existing non-holographic approaches, feedback-based wavefront shaping requires a large number of measurements, while directly measuring the TM can be easily trapped into local optimums if the constraints are insufficient. To reduce the required number of measurements, we combine the concepts of these two approaches and develop a scheme termed feedback-assisted TM measurements. We show that under such a hybrid scheme, less than 3N intensity measurements are sufficient to accurately retrieve one row of the TM that contains N unknown complex elements. As a proof of concept, we experimentally demonstrated retrieving multiple rows of the TM of an MMF using the proposed scheme with high fidelity. In particular, a single focus and dual foci through the MMF with enhancements larger than 75% of the theoretical values were reported.

High speed, complex wavefront shaping using the digital micro-mirror device

Digital micro-mirror devices (DMDs) have been deployed in many optical applications. As compared to spatial light modulators (SLMs), they are characterized by their much faster refresh rates (full-frame refresh rates up to 32 kHz for binary patterns) compared to 120 Hz for most liquid crystal SLMs. DMDs however can only display binary, unipolar patterns and utilize temporal modulation to represent with excellent accuracy multiple gray-levels in display applications. We used the built-in time domain dynamic range representation of the DMD to project 8-bit complex-fields. With this method, we demonstrated 8-bit complex field modulation with a frame time of 38.4 ms (around 0.15 s for the entire complex-field). We performed phase conjugation by compensating the distortions incurred due to propagation through free-space and a scattering medium. For faster modulation speed, an electro-optic modulator was used in synchronization with the DMD in an amplitude modulation mode to create grayscale patterns with frame rate ~ 833 Hz with display time of only 1.2 ms instead of 38.4 ms for time multiplexing gaining a speed up by a factor of 32.

De-noising imaging through diffusers with autocorrelation

Recovering targets through diffusers is an important topic as well as a general problem in optical imaging. The difficulty of recovering is increased due to the noise interference caused by an imperfect imaging environment. Existing approaches generally require a high-signal-to-noise-ratio (SNR) speckle pattern to recover the target, but still have limitations in de-noising or generalizability. Here, featuring information of high-SNR autocorrelation as a physical constraint, we propose a two-stage (de-noising and reconstructing) method to improve robustness based on data driving. Specifically, a two-stage convolutional neural network (CNN) called autocorrelation reconstruction (ACR) CNN is designed to de-noise and reconstruct targets from low-SNR speckle patterns. We experimentally demonstrate the robustness through various diffusers with different levels of noise, from simulative Gaussian noise to the detector and photon noise captured by the actual optical system. The de-noising stage improves the peak SNR from 20 to 38 dB in the system data, and the reconstructing stage, compared with the unconstrained method, successfully recovers targets hidden in unknown diffusers with the detector and photon noise. With the help of the physical constraint to optimize the learning process, our two-stage method is realized to improve generalizability and has potential in various fields such as imaging in low illumination.

Experimental generation of perfect optical vortices through strongly scattering media

Perfect optical vortices enable the unprecedented optical multiplexing utilizing orbital angular momentum of light, which, however, suffer from distortion when they propagate in inhomogeneous media. Herein, we report on the experimental demonstration of perfect optical vortice generation through strongly scattering media. The transmission-matrix-based point-spread-function engineering is applied to encode the targeted mask in the Fourier domain before focusing. We experimentally demonstrate the perfect optical vortice generation either through a multimode fiber or a ground glass, where the numerical results agree well with the measured one. Our results might facilitate the manipulation of orbital angular momentum of light through disordered scattering media and shed new light on the optical multiplexing utilizing perfect optical vortices.

Time-averaged image projection through a multimode fiber

Many disciplines, ranging from lithography to opto-genetics, require high-fidelity image projection. However, not all optical systems can display all types of images with equal ease. Therefore, the image projection quality is dependent on the type of image. In some circumstances, this can lead to a catastrophic loss of intensity or image quality. For complex optical systems, it may not be known in advance which types of images pose a problem. Here we show a new method called Time-Averaged image Projection (TAP), allowing us to mitigate these limitations by taking the entire image projection system into account despite its complexity and building the desired intensity distribution up from multiple illumination patterns. Using a complex optical setup, consisting of a wavefront shaper and a multimode optical fiber illuminated by coherent light, we succeeded to suppress any speckle-related background. Further, we can display independent images at multiple distances simultaneously, and alter the effective sharpness depth through the algorithm. Our results demonstrate that TAP can significantly enhance the image projection quality in multiple ways. We anticipate that our results will greatly complement any application in which the response to light irradiation is relatively slow (one microsecond with current technology) and where high-fidelity spatial distribution of optical power is required.

Focusing light through multimode fibres using a digital micromirror device: a comparison study of non-holographic approaches

Focusing light through a multimode fibre (MMF) has attracted significant research interest, mainly driven by the need for miniature endoscopes in biomedicine. In recent years, digital micromirror devices (DMD) have become increasingly popular as a high-speed alternative to liquid-crystal spatial light modulators for light focusing via wavefront shaping based on binary amplitude modulations. To exploit the potentials and limitations of the state-of-the-art DMD-based wavefront shaping methods, in this study, for the first time, we compared four representative, non-holographic and DMD-based methods that are reported so far in literature with the same experimental and simulation conditions, including a real-valued intensity transmission matrix (RVITM)-based algorithm, a complex-valued transmission matrix (TM)-based algorithm, a conditional probability algorithm and a genetic algorithm. We investigated the maximum achievable peak-to-background ratio (PBR) in comparison to theoretical expectations, and further improved the performance of the RVITM-based method. With both numerical simulations and experiments, we found that the genetic algorithm offered the highest PBR but suffered from the lowest focusing speed, while the RVITM-based algorithm provided a comparable PBR to that of the genetic algorithm, and the highest focusing speed.

Compressively sampling the optical transmission matrix of a multimode fibre

The measurement of the optical transmission matrix (TM) of an opaque material is an advanced form of space-variant aberration correction. Beyond imaging, TM-based methods are emerging in a range of fields, including optical communications, micro-manipulation, and computing. In many cases, the TM is very sensitive to perturbations in the configuration of the scattering medium it represents. Therefore, applications often require an up-to-the-minute characterisation of the fragile TM, typically entailing hundreds to thousands of probe measurements. Here, we explore how these measurement requirements can be relaxed using the framework of compressive sensing, in which the incorporation of prior information enables accurate estimation from fewer measurements than the dimensionality of the TM we aim to reconstruct. Examples of such priors include knowledge of a memory effect linking the input and output fields, an approximate model of the optical system, or a recent but degraded TM measurement. We demonstrate this concept by reconstructing the full-size TM of a multimode fibre supporting 754 modes at compression ratios down to ∼5% with good fidelity. We show that in this case, imaging is still possible using TMs reconstructed at compression ratios down to ∼1% (eight probe measurements). This compressive TM sampling strategy is quite general and may be applied to a variety of other scattering samples, including diffusers, thin layers of tissue, fibre optics of any refractive profile, and reflections from opaque walls. These approaches offer a route towards the measurement of high-dimensional TMs either quickly or with access to limited numbers of measurements.

Locating and Imaging through Scattering Medium in a Large Depth

Scattering medium brings great difficulties to locate and reconstruct objects especially when the objects are distributed in different positions. In this paper, a novel physics and learning-heuristic method is presented to locate and image the object through a strong scattering medium. A novel physics-informed framework, named DINet, is constructed to predict the depth and the image of the hidden object from the captured speckle pattern. With the phase-space constraint and the efficient network structure, the proposed method enables to locate the object with a depth mean error less than 0.05 mm, and image the object with an average peak signal-to-noise ratio (PSNR) above 24 dB, ranging from 350 mm to 1150 mm. The constructed DINet firstly solves the problem of quantitative locating and imaging via a single speckle pattern in a large depth. Comparing with the traditional methods, it paves the way to the practical applications requiring multi-physics through scattering media.

Non-invasive focusing and imaging in scattering media with a fluorescence-based transmission matrix

In biological microscopy, light scattering represents the main limitation to image at depth. Recently, a set of wavefront shaping techniques has been developed in order to manipulate coherent light in strongly disordered materials. The Transmission Matrix approach has shown its capability to inverse the effect of scattering and efficiently focus light. In practice, the matrix is usually measured using an invasive detector or low-resolution acoustic guide stars. Here, we introduce a non-invasive and all-optical strategy based on linear fluorescence to reconstruct the transmission matrices, to and from a fluorescent object placed inside a scattering medium. It consists in demixing the incoherent patterns emitted by the object using low-rank factorizations and phase retrieval algorithms. We experimentally demonstrate the efficiency of this method through robust and selective focusing. Additionally, from the same measurements, it is possible to exploit memory effect correlations to image and reconstruct extended objects. This approach opens up a new route towards imaging in scattering media with linear or non-linear contrast mechanisms.

Light scattering represents the main limitation to image at depth in biological microscopy. The authors present a strategy to characterize light propagation in and out of a scattering medium based on linear fluorescence feedback and from the same measurements exploit memory effect correlations to image and reconstruct extended objects.

Self-reference method for measuring the transmission matrices of scattering media

A significant approach for manipulating light propagation through scattering media consists of the measurement of transmission matrices (TMs). Here we propose a TM-measurement method with high stability and universal applicability, which we call the self-reference method. This method uses a new, to the best of our knowledge, way to perform holographic measurement, where the reference light is superimposed directly to the signal light. This method does not pose any restriction on the signal light, so it is applicable to nearly all types of input bases. The effectivity of this method in accurately measuring the TM is verified by experimentally achieving high-quality light focusing through a scattering medium. We believe that the self-reference method provides an ideal way for TM measurement and wavefront shaping, which will be of great significance to imaging and communication technologies in scattering environments.

Lensless light-field imaging through diffuser encoding

Microlens array-based light-field imaging has been one of the most commonly used and effective technologies to record high-dimensional optical signals for developing various potential high-performance applications in many fields. However, the use of a microlens array generally suffers from an intrinsic trade-off between the spatial and angular resolutions. In this paper, we concentrate on exploiting a diffuser to explore a novel modality for light-field imaging. We demonstrate that the diffuser can efficiently angularly couple incident light rays into a detected image without needing any lens. To characterize and analyse this phenomenon, we establish a diffuser-encoding light-field transmission model, in which four-dimensional light fields are mapped into two-dimensional images via a transmission matrix describing the light propagation through the diffuser. Correspondingly, a calibration strategy is designed to flexibly determine the transmission matrix, so that light rays can be computationally decoupled from a detected image with adjustable spatio-angular resolutions, which are unshackled from the resolution limitation of the sensor. The proof-of-concept approach indicates the possibility of using scattering media for lensless four-dimensional light-field recording and processing, not just for two- or three-dimensional imaging.

Seeing through multimode fibers with real-valued intensity transmission matrices

Image transmission through multimode optical fibers has been an area of immense interests driven by the demand for miniature endoscopes in biomedicine and higher speed and capacity in telecommunications. Conventionally, a complex-valued transmission matrix is obtained experimentally to link the input and output light fields of a multimode fiber for image retrieval, which complicates the experimental setup and increases the computational complexity. Here, we report a simple and high-speed method for image retrieval based on our demonstration of a pseudo-linearity between the input and output light intensity distributions of multimode fibers. We studied the impact of several key parameters to image retrieval, including image pixel count, fiber core diameter and numerical aperture. We further demonstrated with experiments and numerical simulations that a wide variety of input binary and gray scale images could be faithfully retrieved from the corresponding output speckle patterns. Thus, it promises to be useful for highly miniaturized endoscopy in biomedicine and spatial-mode-division multiplexing in telecommunications.

Speckle-Correlation Scattering Matrix Approaches for Imaging and Sensing through Turbidity

The development of optical and computational techniques has enabled imaging without the need for traditional optical imaging systems. Modern lensless imaging techniques overcome several restrictions imposed by lenses, while preserving or even surpassing the capability of lens-based imaging. However, existing lensless methods often rely on a priori information about objects or imaging conditions. Thus, they are not ideal for general imaging purposes. The recent development of the speckle-correlation scattering matrix (SSM) techniques facilitates new opportunities for lensless imaging and sensing. In this review, we present the fundamentals of SSM methods and highlight recent implementations for holographic imaging, microscopy, optical mode demultiplexing, and quantification of the degree of the coherence of light. We conclude with a discussion of the potential of SSM and future research directions.

OAM-basis transmission matrix in optics: a novel approach to manipulate light propagation through scattering media

Transmission matrix (TM) is an ideal theoretical model describing light propagation through scattering media. Until now, most of the present TMs utilize the eigenstates of spatial position as input and output bases. Thus, they describe the relationship between the spatial distributions of two light fields. Here, we demonstrate that wider relationships between the light fields could be described by a TM. As a significant example, we propose a generalized TM with the eigenstates of OAM as input bases - OAM-basis TM. With the measured OAM-basis TM, we achieved single-spot and multiple-spot focusing, verifying its availability in light propagation manipulation. The distinct eigenchannels property was also discussed. The OAM-basis TM has broadened the definition of TM. Meanwhile, it will open new perspectives for OAM-based communication, as well as the applications of wavefront shaping technology in biomedical photonics and optical communication.

Retrieving the optical transmission matrix of a multimode fiber using the extended Kalman filter

Characterizing the transmission matrix (TM) of a multimode fiber (MMF) benefits many fiber-based applications and allows in-depth studies on the physical properties. For example, by modulating the incident field, the knowledge of the TM allows one to synthesize any optical field at the distill end of the MMF. However, the extraction of optical fields usually requires holographic measurements with interferometry, which complicates the system design and introduces additional noise. In this work, we developed an efficient method to retrieve the TM of the MMF in a referenceless optical system. With pure intensity measurements, this method uses the extended Kalman filter (EKF) to recursively search for the optimum solution. To facilitate the computational process, a modified speckle-correlation scatter matrix (MSSM) is constructed as a low-fidelity initial estimation. This method, termed EKF-MSSM, only requires 4N intensity measurements to precisely solve for N unknown complex variables in the TM. Experimentally, we successfully retrieved the TM of the MMF with high precision, which allows optical focusing with the enhancement (>70%) close to the theoretical value. We anticipate that this method will serve as a useful tool for studying physical properties of the MMFs and potentially open new possibilities in a variety of applications in fiber optics.

Learning-based method to reconstruct complex targets through scattering medium beyond the memory effect

Strong scattering medium brings great difficulties to image objects. Optical memory effect makes it possible to image through strong random scattering medium in a limited angle field-of-view (FOV). The limitation of FOV results in a limited optical memory effect range, which prevents the optical memory effect to be applied to real imaging applications. In this paper, a kind of practical convolutional neural network called PDSNet (Pragmatic De-scatter ConvNet) is constructed to image objects hidden behind different scattering media. The proposed method can expand at least 40 times of the optical memory effect range with a average PSNR above 24dB, and enable to image complex objects in real time, even for objects with untrained scales. The provided experiments can verify its accurateness and efficiency.

Shaping of amplified beam from a highly multimode Yb-doped fiber using transmission matrix

The transmission matrix of an ytterbium doped multimode fiber with gain was measured. It was shown to vary owing to the pump power level. Amplified beam focusing, beam steering and shaping were demonstrated using the measured matrix for input wavefront shaping, with an efficiency similar to the case of a passive fiber. The impact of weak gain saturation was lastly investigated.

Wavefront-shaping-based pattern regeneration through the scattering medium

Generating a specific pattern through a scattering medium is of utmost interest, but remains mostly restricted to weakly scattering or transparent samples. We present a wavefront-shaping-based approach to achieve a user-specified pattern through a strong scattering medium. We show that the virtual transmission matrices can be computed by convolving the transmission matrices with a virtual input wavefront. Interestingly, rather than a focused point, a pattern, which is the square of the modulus of the virtual input wavefront, can be regenerated after the scattering medium at different locations. As a proof of concept, we set up the experiments using a phase-only spatial light modulator and calibrate the transmission matrices of the scattering medium. We demonstrated that a user-specified pattern can be generated after the scattering medium using the virtual transmission matrices. Our method does not rely on the memory effect of the scattering medium and is effective for a strong scattering medium. Our work is expected to be applied in structured light illumination or coherent control.

Deep learning the high variability and randomness inside multimode fibers

Multimode fibers (MMF) are remarkable high-capacity information channels. However, the MMF transmission is highly sensitive to external perturbations and environmental changes. Here, we show the successful binary image transmission using deep learning through a single MMF subject to dynamic shape variations. As a proof-of-concept experiment, we find that a convolutional neural network has excellent generalization capability with various MMF transmission states to accurately predict unknown information at the other end of the MMF at any of these states. Our results demonstrate that deep learning is a promising solution to address the high variability and randomness challenge of MMF based information channels. This deep-learning approach is the starting point of developing future high-capacity MMF optical systems and devices and is applicable to optical systems concerning other diffusing media.

Manipulating the transmission matrix of scattering media for nonlinear imaging beyond the memory effect

The measurement of the transmission matrix (TM) of a scattering medium is of great interest for imaging. It can be acquired directly by interferometry using an internal reference wavefront. Unfortunately, internal reference fields are scattered by the medium, which results in a speckle that makes the TM measurement heterogeneous across the output field of view. We demonstrate how to correct for this effect using the intrinsic properties of the TM. For thin scattering media, we exploit the memory effect of the medium and the reference speckle to create a corrected TM. For highly scattering media where the memory effect is negligible, we use complementary reference speckles to compose a new TM, not compromised by the speckled reference anymore. Using this correction, we demonstrate large field of view second harmonic generation imaging through thick biological media.

Fast Retrieval Method of Forestry Information Features Based on Symmetry Function in Communication Network

Aiming at the problem of insufficient integration and sharing of forestry information resources under the current communication network and the lack of the concept set of forestry information attributes, which leads to poor information retrieval performance, a fast retrieval method of forestry information features based on symmetry function is studied in depth, and the method is implemented by PDA (Personal Digital Assistant)-BA (Buliding Automation). Using the SED (Stream Editor) forestry information acquisition method under a communication network to collect forestry information, a forestry signal noise cancellation method based on symmetric function method is obtained. In order to improve the accuracy of forestry information acquisition, denoising of the signal in the information was carried out. Constructing forestry information data ontology, integrating forestry resources, establishing a conceptual set of forestry information attributes, distinguishing forestry information attributes, establishing a fast retrieval model of forestry information features based on the synonym library, and completing the fast retrieval of forestry information features. The experimental results show that the recall and precision of this method are 99.25% and 99.24%, respectively, and the retrieval performance is superior, which has a certain application value.

Controlling light in complex media beyond the acoustic diffraction-limit using the acousto-optic transmission matrix

Studying the internal structure of complex samples with light is an important task but a difficult challenge due to light scattering. While the complex optical distortions induced by scattering can be effectively undone if the medium’s scattering-matrix is known, this matrix generally cannot be retrieved without the presence of an invasive detector or guide-star at the target points of interest. To overcome this limitation, the current state-of-the-art approaches utilize focused ultrasound for generating acousto-optic guide-stars, in a variety of different techniques. Here, we introduce the acousto-optic transmission matrix (AOTM), which is an ultrasonically-encoded, spatially-resolved, optical scattering-matrix. The AOTM provides both a generalized framework to describe any acousto-optic based technique, and a tool for light control and focusing beyond the acoustic diffraction-limit inside complex samples. We experimentally demonstrate complex light control using the AOTM singular vectors, and utilize the AOTM framework to analyze the resolution limitation of acousto-optic guided focusing approaches.

Various techniques combine light and ultrasound to study the inside of strongly scattering samples, beyond the reach of purely optical imaging. Here, Katz et al. introduce the acousto-optic transmission matrix framework that allows to control and focus light beyond the acoustic diffraction limit.

Light scattering control in transmission and reflection with neural networks

Scattering often limits the controlled delivery of light in applications such as biomedical imaging, optogenetics, optical trapping, and fiber-optic communication or imaging. Such scattering can be controlled by appropriately shaping the light wavefront entering the material. Here, we develop a machine-learning approach for light control. Using pairs of binary intensity patterns and intensity measurements we train neural networks (NNs) to provide the wavefront corrections necessary to shape the beam after the scatterer. Additionally, we demonstrate that NNs can be used to find a functional relationship between transmitted and reflected speckle patterns. Establishing the validity of this relationship, we focus and scan in transmission through opaque media using reflected light. Our approach shows the versatility of NNs for light shaping, for efficiently and flexibly correcting for scattering, and in particular the feasibility of transmission control based on reflected light.

Imaging through scattering media using semidefinite programming

A clear image of an observed object may deteriorate into unrecognizable speckle when encountering heterogeneous scattering media, thus it is necessary to recover the object image from the speckle. A method combining least square and semidefinite programming is proposed, which can be used for imaging through scattering media. The proposed method consists of two main stages, that is, media scattering characteristics (SCs) estimation and image reconstruction. SCs estimation is accomplished through LS concept after establishing a database of known object-and-speckle pairs. Image reconstruction is realized by solving an SDP problem to obtain the product of the unknown object image and its Hermitian transposition. Finally, the unknown object image can be reconstructed by extracting the largest rank-1 component of the product. Structural similarity (SSIM) index is employed as a performance indicator in speckle prediction and image reconstruction. Numerical simulations and physical experiments are performed to verify the feasibility and practicality of the proposed method. Compared with the existing phase shift interferometry mean square optimization method and the single-shot phase retrieval algorithm, the proposed method is the most precise to obtain the best reconstruction results with highest SSIM index value. The work can be used for exploring the potential applications of scattering media, especially for imaging through turbid media in biomedical, scattering property measurement, and optical image encryption.

Bayes' theorem-based binary algorithm for fast reference-less calibration of a multimode fiber

In this paper, we present a Bayes' theorem-based high-speed algorithm, to measure the binary transmission matrix of a multimode fiber using a digital micromirror device, in a reference-less multimode fiber imaging system. Based on conditional probability, we define a preset threshold to locate those digital-micromirror-device pixels that can be switched 'ON' to form a focused spot at the output. This leads to a binary transmission matrix consisting of '0' and '1' elements. High-enhancement-factor light focusing and raster-scanning at the distal end of the fiber are demonstrated experimentally. The key advantage of our algorithm is its capability for fast calibration of a MMF to form a tightly focused spot. In our experiment, for 5000 input-output pairs, we only need 0.26 s to calibrate one row of the transmission matrix to achieve a focused spot with an enhancement factor of 28. This is more than 10 times faster than the prVBEM algorithm. The proposed Bayes' theorem-based binary algorithm can be applied not only in multimode optical fiber focusing but also to other disordered media. Particularly, it will be valuable in fast multimode fiber calibration for endoscopic imaging.

Characterization of an imaging multimode optical fiber using a digital micro-mirror device based single-beam system

This work demonstrates experimental approaches to characterize a single multimode fiber imaging system without a reference beam. Spatial light modulation is performed with a digital micro-mirror device that enables high-speed binary amplitude modulation. Intensity-only images are recorded by the camera and processed by a Bayesian inference based algorithm to retrieve the phase of the output optical field as well as the transmission matrix of the fiber. The calculated transmission matrix is validated by three standards: prediction accuracy, transmission imaging, and focus generation. Also, it is found that information on mode count and eigenchannels can be extracted from the transmission matrix by singular value decomposition. This paves the way for a more compact and cheaper single multimode fiber imaging system for many demanding imaging tasks.

Doubling the pixel count limitation of single-pixel imaging via sinusoidal amplitude modulation

We demonstrate a single-pixel imaging (SPI) method that can achieve pixel resolution beyond the physical limitation of the spatial light modulator (SLM), by adopting sinusoidal amplitude modulation and frequency filtering. Through light field analysis, we observe that the induced intensity with a squared value of the amplitude contains higher frequency components. By filtering out the zero frequency of the sinusoidal amplitude in the Fourier domain, we can separate out the higher frequency components, which enables SPI with higher resolving ability and thus beyond the limitation of the SLM. Further, to address the speed issue in grayscale spatial light modulation, we propose a fast implementation scheme with tens-of-kilohertz refresh rate. Specifically, we use a digital micromirror device (DMD) working at the full frame rate to conduct binarized sinusoidal patterning in the spatial domain and pinhole filtering eliminating the binarization error in the Fourier domain. For experimental validation, we build a single-pixel microscope to retrieve 1200 × 1200-pixel images via a sub-megapixel DMD, and the setup achieves comparable performance to array sensor microscopy and provides additional sectioning ability.