Image reconstruction through a multimode fiber with a simple neural network architecture

Exploiting holographically encoded variance to transmit labelled images through a multimode optical fiber

Artificial intelligence has emerged as promising tool to decode an image transmitted through a multimode fiber (MMF) by applying deep learning techniques. By transmitting thousands of images through the MMF, deep neural networks (DNNs) are able to decipher the seemingly random output speckle patterns and unveil the intrinsic input-output relationship. High fidelity reconstruction is obtained for datasets with a large degree of homogeneity, which underutilizes the capacity of the combined MMF-DNN system. Here, we show that holographic modulation can encode an additional layer of variance on the output speckle pattern, improving the overall transmissive capabilities of the system. Operatively, we have implemented this by adding a holographic label to the original dataset and injecting the resulting phase image into the fiber facet through a Fourier transform lens. The resulting speckle pattern dataset can be clustered primarily by holographic label, and can be reconstructed without loss of fidelity. As an application, we describe how color images may be segmented into RGB components and each color component may then be labelled by distinct hologram. A ResUNet architecture was then used to decode each class of speckle patterns and reconstruct the color image without the need for temporal synchronization between sender and receiver.

Adaptive noise-resilient deep learning for image reconstruction in multimode fiber scattering

This research offers a comprehensive exploration of three pivotal aspects within the realm of fiber optics and piezoelectric materials. The study delves into the influence of voltage variation on piezoelectric displacement, examines the effects of bending multimode fiber (MMF) on data transmission, and scrutinizes the performance of an autoencoder in MMF image reconstruction with and without additional noise. To assess the impact of voltage variation on piezoelectric displacement, experiments were conducted by applying varying voltages to a piezoelectric material, meticulously measuring its radial displacement. The results revealed a notable increase in displacement with higher voltage, presenting implications for fiber stability and overall performance. Additionally, the investigation into the effects of bending MMF on data transmission highlighted that the bending process causes the fiber to become leaky and radiate power radially, potentially affecting data transmission. This crucial insight emphasizes the necessity for further research to optimize data transmission in practical fiber systems. Furthermore, the performance of an autoencoder model was evaluated using a dataset of MMF images, in diverse scenarios. The autoencoder exhibited impressive accuracy in reconstructing MMF images with high fidelity. The results underscore the significance of ongoing research in these domains, propelling advancements in fiber optic technology.

Spatial pilot-aided fast-adapted framework for stable image transmission over long multi-mode fiber

Multi-mode fiber (MMF) has emerged as a promising platform for spatial information transmission attributed to its high capacity. However, the scattering characteristic and time-varying nature of MMF pose challenges for long-term stable transmission. In this study, we propose a spatial pilot-aided learning framework for MMF image transmission, which effectively addresses these challenges and maintains accurate performance in practical applications. By inserting a few reference image frames into the transmitting image sequence and leveraging a fast-adapt network training scheme, our framework adaptively accommodates to the physical channel variations and enables online model update for continuous transmission. Experimented on 100 m length unstable MMFs, we demonstrate transmission accuracy exceeding 92% over hours, with pilot frame overhead around 2%. Our fast-adapt learning scheme requires training of less than 2% of network parameters and reduces the computation time by 70% compared to conventional tuning approaches. Additionally, we propose two pilot-insertion strategies and elaborately compare their applicability to a wide range of scenarios including continuous transmission, burst transmission and transmission after fiber re-plugging. The proposed spatial pilot-aided fast-adapt framework opens up the possibility for MMF spatial transmission in practical complicated applications.

Projecting colorful images through scattering media via deep learning

The existence of scatterers in the optical path has been the major obstacle that prohibits one from projecting images through solid walls, turbid water, clouds, and fog. Recent developments in wavefront shaping and neural networks demonstrate effective compensation for scattering effects, showing the promise to project clear images against strong scattering. However, previous studies were mainly restricted to projecting greyscale images using monochromatic light, mainly due to the increased complexity of simultaneously controlling multiple wavelengths. In this work, we fill this blank by developing a projector network, which enables the projection of colorful images through scattering media with three primary colors. To validate the performance of the projector network, we experimentally demonstrated projecting colorful images obtained from the MINST dataset through two stacked diffusers. Quantitatively, the averaged intensity Pearson's correlation coefficient for 1,000 test colorful images reaches about 90.6%, indicating the superiority of the developed network. We anticipate that the projector network can be beneficial to a variety of display applications in scattering environments.

High-quality color image restoration from a disturbed graded-index imaging system by deep neural networks

Imaging transmission plays an important role in endoscopic clinical diagnosis involved in modern medical treatment. However, image distortion due to various reasons has been a major obstacle to state-of-art endoscopic development. Here, as a preliminary study we demonstrate ultra-efficient recovery of exemplary 2D color images transmitted by a disturbed graded-index (GRIN) imaging system through the deep neural networks (DNNs). Indeed, the GRIN imaging system can preserve analog images through the GRIN waveguides with high quality, while the DNNs serve as an efficient tool for imaging distortion correction. Combining GRIN imaging systems and DNNs can greatly reduce the training process and achieve ideal imaging transmission. We consider imaging distortion under different realistic conditions and use both pix2pix and U-net type DNNs to restore the images, indicating the suitable network in each condition. This method can automatically cleanse the distorted images with superior robustness and accuracy, which can potentially be used in minimally invasive medical applications.

High-definition image transmission through dynamically perturbed multimode fiber by a self-attention based neural network

We implement faithful multimode fiber (MMF) image transmission by a self-attention-based neural network. Compared with a real-valued artificial neural network (ANN) based on a convolutional neural network (CNN), our method utilizes a self-attention mechanism to achieve a higher image quality. The enhancement measure (EME) and structural similarity (SSIM) of the dataset collected in the experiment improved by 0.79 and 0.04; the total number of parameters can be reduced by up to 25%. To enhance the robustness of the neural network to MMF bending in image transmission, we use a simulation dataset to prove that the hybrid training method is helpful in MMF transmission of a high-definition image. Our findings may pave the way for simpler and more robust single-MMF image transmission schemes with hybrid training; SSIM on datasets under different disturbances improve by 0.18. This system has the potential to be applied to various high-demand image transmission tasks, such as endoscopy.

Multimode fiber-based greyscale image projector enabled by neural networks with high generalization ability

Multimode fibers (MMFs) are emerging as promising transmission media for delivering images. However, strong mode coupling inherent in MMFs induces difficulties in directly projecting two-dimensional images through MMFs. By training two subnetworks named Actor-net and Model-net synergetically, [Nature Machine Intelligence2, 403 (2020)10.1038/s42256-020-0199-9] alleviated this issue and demonstrated projecting images through MMFs with high fidelity. In this work, we make a step further by improving the generalization ability to greyscale images. The modified projector network contains three subnetworks, namely forward-net, backward-net, and holography-net, accounting for forward propagation, backward propagation, and the phase-retrieval process. As a proof of concept, we experimentally trained the projector network using randomly generated phase maps and their corresponding resultant speckle images output from a 1-meter-long MMF. With the network being trained, we successfully demonstrated projecting binary images from MNIST and EMNIST and greyscale images from Fashion-MNIST, exhibiting averaged Pearson's correlation coefficients of 0.91, 0.92, and 0.87, respectively. Since all these projected images have never been seen by the projector network before, a strong generalization ability in projecting greyscale images is confirmed.

Unsupervised Data-Driven Classification of Topological Gapped Systems with Symmetries

A remarkable breakthrough in topological phase classification is the establishment of the topological periodic table, which is mainly based on the classifying space analysis or K theory, but not based on concrete Hamiltonians that possess finite bands or arise in a lattice. As a result, it is still difficult to identify the topological phase of an arbitrary Hamiltonian; the common practice is, instead, to check the incomplete and still growing list of topological invariants one by one, very often by trial and error. Here, we develop unsupervised classifications of topological gapped systems with symmetries, and demonstrate the data-driven construction of the topological periodic table without a priori knowledge of topological invariants. This unsupervised data-driven strategy can take into account spatial symmetries, and further classify phases that were previously classified as trivial in the past. Our Letter introduces machine learning into topological phase classification and paves the way for intelligent explorations of new phases of topological matter.

Dark soliton detection using persistent homology

Classifying images often requires manual identification of qualitative features. Machine learning approaches including convolutional neural networks can achieve accuracy comparable to human classifiers but require extensive data and computational resources to train. We show how a topological data analysis technique, persistent homology, can be used to rapidly and reliably identify qualitative features in experimental image data. The identified features can be used as inputs to simple supervised machine learning models, such as logistic regression models, which are easier to train. As an example, we consider the identification of dark solitons using a dataset of 6257 labeled atomic Bose-Einstein condensate density images.

All-fiber high-speed image detection enabled by deep learning

Ultra-high-speed imaging serves as a foundation for modern science. While in biomedicine, optical-fiber-based endoscopy is often required for in vivo applications, the combination of high speed with the fiber endoscopy, which is vital for exploring transient biomedical phenomena, still confronts some challenges. We propose all-fiber imaging at high speeds, which is achieved based on the transformation of two-dimensional spatial information into one-dimensional temporal pulsed streams by leveraging high intermodal dispersion in a multimode fiber. Neural networks are trained to reconstruct images from the temporal waveforms. It can not only detect content-aware images with high quality, but also detect images of different kinds from the training images with slightly reduced quality. The fiber probe can detect micron-scale objects with a high frame rate (15.4 Mfps) and large frame depth (10,000). This scheme combines high speeds with high mechanical flexibility and integration and may stimulate future research exploring various phenomena in vivo.

Surveillance of few-mode fiber-communication channels with a single hidden layer neural network

Multi- and few-mode fibers (FMFs) promise to enhance the capacity of optical communication networks by orders of magnitude. The key for this evolution was the strong advancement of computational approaches that allowed inherent complex light transmission to be surpassed, learned, or controlled, reined in by modal crosstalk and mode-dependent losses. However, complex light transmission through FMFs can be learned by a single hidden layer neural network (NN). The emerging developments in NNs additionally allow the implementation of novel concepts for security enhancements in optical communication. Once the transmission characteristics of FMFs are learned, it is possible to survey the incoming and outgoing light fields via monitoring channels during data transmission. If an eavesdropper tries to gain unauthorized access to the FMF, its transmission properties are impaired through sensitive modal crosstalk. This process is registered by the NN and thus the eavesdropper is revealed. With our solution, the security of optical communication can be improved.

Deeply Subwavelength Localization with Reverberation-Coded Aperture

Accessing subwavelength information about a scene from the far-field without invasive near-field manipulations is a fundamental challenge in wave engineering. Yet it is well understood that the dwell time of waves in complex media sets the scale for the waves' sensitivity to perturbations. Modern coded-aperture imagers leverage the degrees of freedom (d.o.f.) offered by complex media as natural multiplexor but do not recognize and reap the fundamental difference between placing the object of interest outside or within the complex medium. Here, we show that the precision of localizing a subwavelength object can be improved by several orders of magnitude simply by enclosing it in its far field with a reverberant passive chaotic cavity. We identify deep learning as a suitable noise-robust tool to extract subwavelength localization information encoded in multiplexed measurements, achieving resolutions well beyond those available in the training data. We demonstrate our finding in the microwave domain: harnessing the configurational d.o.f. of a simple programmable metasurface, we localize a subwavelength object along a curved trajectory inside a chaotic cavity with a resolution of λ/76 using intensity-only single-frequency single-pixel measurements. Our results may have important applications in photoacoustic imaging as well as human-machine interaction based on reverberating elastic waves, sound, or microwaves.