Optical analog computing of spatial differentiation and edge detection with dielectric metasurfaces

Tunable optical spatial differential operation via photonic spin Hall effect in a Weyl semimetal

Optical differential operation is the basic principle of optical image edge detection, which has the advantages of high efficiency, simple structure and markerless compared with the traditional digital image processing methods. In this paper, we propose an optical differential operation with high contrast based on the photonic spin Hall effect in a Weyl semimetal, which enables to switch between one- and two-dimensional edge detection. Due to the unique optical and electrical properties of the Weyl semimetal, a transport model for the differential operation is established, which is closely related to the beam shifts. By tuning the incidence conditions, we effectively manipulate the in-plane and transverse shifts to switch differential operations between one and two dimensions. The contrast of the differential operation is further regulated by changing the physical parameters of the Weyl semimetal, and can be improved by two orders of magnitude compared to the conventional differentiator. This study provides new possibilities in edge detection and image processing owing to the advantages of switchable dimension and high contrast.

Metalens for Accelerated Optoelectronic Edge Detection under Ambient Illumination

Analog systems may allow image processing, such as edge detection, with low computational power. However, most demonstrated analog systems, based on either conventional 4-f imaging systems or nanophotonic structures, rely on coherent laser sources for illumination, which significantly restricts their use in routine imaging tasks with ambient, incoherent illumination. Here, we demonstrated a metalens-assisted imaging system that can allow optoelectronic edge detection under ambient illumination conditions. The metalens was designed to generate polarization-dependent optical transfer functions (OTFs), resulting in a synthetic OTF with an isotropic high-pass frequency response after digital subtraction. We integrated the polarization-multiplexed metalens with a polarization camera and experimentally demonstrated single-shot edge detection of indoor and outdoor scenes, including a flying airplane, under ambient sunlight illumination. The proposed system showcased the potential of using polarization multiplexing for the construction of complex optical convolution kernels toward accelerated machine vision tasks such as object detection and classification under ambient illumination.

Analog signal metasurface processor supporting mathematical operator reconfiguration

Electromagnetic wave analog computing is an effective method to overcome the bottleneck of electronic computing, which has attracted the attention of scientists. However, many spatial analog signal processing systems based on electromagnetic waves can only execute one unique mathematical operator and cannot provide multiple operators for users to choose arbitrarily. In order to enhance the function of the current spatial analog computing system, we design a coding structure with amplitude-phase decoupling modulation to realize the analog signal processor that supports the switching of mathematical operators and demonstrate the precise switching from the first-order spatial differential operator to the first-order spatial integral operator. Our design idea can be used as a paradigm for designing small reconfigurable analog computing systems, paving the way for the construction of high-speed, multifunctional, and universal signal processing systems. This idea can be extended to any other range of waves.

Metasurface enabled broadband all optical edge detection in visible frequencies

Image processing is of fundamental importance for numerous modern technologies. In recent years, due to increasing demand for real-time and continuous data processing, metamaterial and metasurface based all-optical computation techniques emerged as a promising alternative to digital computation. Most of the pioneer research focused on all-optical edge detection as a fundamental step of image processing. Metasurfaces have been shown to enable real time edge detection with low to no power consumption. However, the previous demonstrations were subjected to the several limitations such as need for oblique-incidence, polarization dependence, need for additional polarizers, narrow operation bandwidth, being limited with processing in 1D, operation with coherent light only, and requiring digital post-processing. Here, we propose and experimentally demonstrate 2D isotropic, polarization-independent, broadband edge detection with high transmission efficiency under both coherent and incoherent illumination along the visible frequency range using a metasurface based on Fourier optics principles.

Surface topography detection based on an optical differential metasurface

Surface topography detection can extract critical characteristics from objects, playing an important role in target identification and precision measurement. Here, an optical method with the advantages of low power consumption, high speed, and simple devices is proposed to realize the surface topography detection of low-contrast phase objects. By constructing reflected light paths, a metasurface can perform spatial differential operation via receiving the light directly reflected from a target. Therefore, our scheme is experimentally demonstrated as having remarkable universality, which can be used not only for opaque objects, but also for transparent pure phase objects. It provides a new, to the best of our knowledge, application for optical differential metasurfaces in precise detection of microscale surface topography.

Electrically tunable optical spatial differentiation with graphene

In recent years, optical analog computing has experienced rapid development, among which optical differential operation has attracted great attention. Here, based on the unique optical properties of graphene, we propose an electrically tunable optical spatial differentiation by introducing a graphene layer at a quartz substrate. It is found that the output light field is sensitive to the graphene layer near the Brewster angle for small polarization output at the graphene-quartz substrate interface and can be modulated by changing the Fermi energy of graphene. In this case, the result of the optical differential operation can be dynamically regulated. Almost strict one-dimensional differential operations in different directions and almost perfect two-dimensional differential operations can be achieved. In addition, two-dimensional edge detection with different degrees of distortion in different directions can also be realized when applied to image processing. This new modulation method may provide more possibilities for tunable image edge detection and provide a potential way for developing more versatile optical simulators in the future.

Two-dimensional optical differentiator for broadband edge detection based on dielectric metasurface

Image edge processing has widespread adoption in a variety of scientific and industrial scenarios. To date, implementations of image edge processing have mostly been done electronically, but there are still difficulties to achieve real-time, high-throughput, and low power consumption image edge processing. The advantages of optical analog computing include low power consumption, fast transmission speed, and high parallel processing capability, and optical analog differentiators make this process possible. However, the proposed analog differentiators can hardly meet the requirements of broadband, polarization insensitive, high contrast, and high efficiency at the same time. Moreover, they are limited to one-dimensional differentiation or work in reflection mode. To be better compatible with two-dimensional image processing or image recognition systems, two-dimensional optical differentiators that integrate the above advantages are urgently needed. In this Letter, a two-dimensional analog optical differentiator with edge detection operating in transmission mode is proposed. It can cover the visible band, is polarization uncorrelated, and has a resolution that reaches 1.7 μm. The efficiency of the metasurface is higher than 88%.

Thin film notch filters as platforms for biological image processing

Many image processing operations involve the modification of the spatial frequency content of images. Here we demonstrate object-plane spatial frequency filtering utilizing the angular sensitivity of a commercial spectral bandstop filter. This approach to all-optical image processing is shown to generate real-time pseudo-3D images of transparent biological and other samples, such as human cervical cancer cells. This work demonstrates the potential of non-local, non-interferometric approaches to image processing for uses in label-free biological cell imaging and dynamical monitoring.

Visualization of transparent particles based on optical spatial differentiation

Optical analog computing operates on the amplitude, phase, polarization, and frequency distributions of the electromagnetic field through the interaction of light and matter. The differentiation operation is widely used in all-optical image processing technology, such as edge detection. Here, we propose a concise way to observe transparent particles, incorporating the optical differential operation that occurs on a single particle. The particle's scattering and cross-polarization components combine into our differentiator. We achieve high-contrast optical images of transparent liquid crystal molecules. The visualization of aleurone grains (the structures that store protein particles in plant cells) in maize seed was experimentally demonstrated with a broadband incoherent light source. Avoiding the interference of stains, our designed method provides the possibility to observe protein particles directly in complex biological tissues.

Realization of all-optical higher-order spatial differentiators based on cascaded operations

Cascaded operations play an important role in traditional electronic computing systems for the realization of advanced strategies. Here, we introduce the idea of cascaded operations into all-optical spatial analog computing. The single function of the first-order operation has difficulty meeting the requirements of practical applications in image recognition. The all-optical second-order spatial differentiators are implemented by cascading two first-order differential operation units, and the image edge detection of amplitude and phase objects are demonstrated. Our scheme provides a possible pathway toward the development of compact multifunctional differentiators and advanced optical analog computing networks.

Designable optical differential operation based on surface plasmon resonance

Various optical differential computing devices have been designed, which have advantages of high speed and low power consumption compared with traditional digital computing. In this paper, considering the reflection of a light beam through a three-layer structure composed of glass, metal and air, we propose a designable optical differential operation based on surface plasmon resonance (SPR). When the SPR is excited under certain conditions, the spin-dependent splitting in the photonic spin Hall effect (SHE) changes dramatically. We first prove theoretically that this three-layer structure can realize one-dimensional optical differential operation. By discussing the transverse beam displacement under different conditions, it is found that the designable differential operation with high sensitivity can be realized by slightly adjusting the incident angle and the thickness of metal film. We design the differentiator which can obtain the image of measured target edge in real time and get different edge effects at different times. This will provide more possible applications for autonomous driving and target recognition.

Optical differentiation based on weak measurements

Optical differentiation shows much potential to be applied in computation due to its strong parallelizability. Currently, each optical differential method can only obtain partial differential information. Here, we propose a general approach to obtain complete differentiation. Compared to previous methods, we can separately obtain the differentiation of amplitude and phase, reserve the negative value of the differentiation, and acquire the differentiation in arbitrary directions of the two-dimensional field. We measure the differentiation of the Gaussian beam to demonstrate this method. A practical experiment of identifying the move direction of the motion-blurred image is also presented to verify the practicability of our method. Our method can further be applied to intelligence algorithms, image identification, and optical analog computing.

Integrated photonic metasystem for image classifications at telecommunication wavelength

Miniaturized image classifiers are potential for revolutionizing their applications in optical communication, autonomous vehicles, and healthcare. With subwavelength structure enabled directional diffraction and dispersion engineering, the light propagation through multi-layer metasurfaces achieves wavelength-selective image recognitions on a silicon photonic platform at telecommunication wavelength. The metasystems implement high-throughput vector-by-matrix multiplications, enabled by near 10³ nanoscale phase shifters as weight elements within 0.135 mm² footprints. The diffraction manifested computing capability incorporates the fabrication and measurement related phase fluctuations, and thus the pre-trained metasystem can handle uncertainties in inputs without post-tuning. Here we demonstrate three functional metasystems: a 15-pixel spatial pattern classifier that reaches near 90% accuracy with femtosecond inputs, a multi-channel wavelength demultiplexer, and a hyperspectral image classifier. The diffractive metasystem provides an alternative machine learning architecture for photonic integrated circuits, with densely integrated phase shifters, spatially multiplexed throughput, and data processing capabilities.

Metasystem architectures are attractive alternatives to waveguide-based integrated photonic processors due to the subwavelength structures. Here, the authors report a 1D passive silicon photonic metasystem with near 90% spatial pattern classification accuracy at telecommunication wavelength.

Realization of tunable edge-enhanced images based on computing metasurfaces

Bright-field imaging and edge imaging can extract different characteristics from objects, and therefore play important roles in image processing and pattern recognition. Here, we propose a fast, convenient, and electrically driven adjustable scheme to achieve tunable edge-enhanced images based on computing metasurfaces. The computing metasurface can perform spatial differential operation as optical waves propagate through it. This optical differential operation is polarization-dependent, thus any desirable contrast can be realized by the interplay between two orthogonal polarization components. By regulating the external voltages applied on the liquid-crystal phase plate, different phase retardances between two orthogonal polarization components are introduced; this allows us to quickly switch between the bright-field image and the edge image.

Analog optical deconvolution computing for wavefront coding based on nanoantennas metasurfaces

Analog optical computing based on metasurfaces has attracted much attention for achieving high-speed calculating without the electronic processing unit. Wavefront coding imaging systems involve the joint design of an encoded image-capturing module and decoding postprocessing algorithms to obtain a required final image. The decoding postprocessing algorithms, as a typical deconvolution computation, require an additional electronic processing unit to yield a high-quality decoded image. We demonstrate an analog optical deconvolution computing kernel based on nanoantennas metasurfaces for the postprocessing calculation of wavefront coding systems. Numerical simulations are presented to prove that the encoded point spread function can be refocused through a designed optical computing metasurface. The proposed approach opens an opportunity for real-time recovering images in wavefront coding optical systems.

Switchable imaging between edge-enhanced and bright-field based on a phase-change metasurface

Edge-enhanced imaging and bright-field imaging extract different morphological information from an object, and hence a system capable of switching dynamically between them is of vital importance for various applications. By incorporating an elaborately designed meta-device with a 4f imaging system, we demonstrate dynamic switching between 2D edge-enhanced imaging and bright-field imaging. The dynamically switchable characteristic results from the composed phase-change material meta-atoms, which are optimized to provide two independent phase profiles in amorphous and crystalline states. For dynamically switchable imaging, the meta-device functions as either a high-pass or a low-pass filter in the Fourier frequency spectrum, relying on its phase state. In addition, the dynamically switchable imaging is polarization independent. The proposed meta-device owns ultra-thin architecture and polarization-insensitive dynamically switchable functionality, holding potential applications in integrated biomedical imaging and defect detection.

Isotropic topological second-order spatial differentiator operating in transmission mode

Differentiation has widespread applications, particularly in image processing for edge detection. Significant advances have been made in using nanophotonic structures and metamaterials to perform such operations. In particular, a recent work demonstrated a topological differentiator in which the transfer function exhibited a topological charge, making the differentiation operation robust to variations in operating conditions. The demonstrated topological differentiator, however, operates in reflection mode at off-normal incidence and is difficult to integrate into compact imaging systems. In this work, we design a topological differentiator that operates isotropically in transmission mode at normal incidence. The device exhibits an optical transfer function with a symmetry-protected topological charge of ±2 and performs second-order differentiation. Our work points to the potential of harnessing topological concepts for optical computing applications.

A review of dielectric optical metasurfaces for spatial differentiation and edge detection

Dielectric metasurfaces-based planar optical spatial differentiator and edge detection have recently been proposed to play an important role in the parallel and fast image processing technology. With the development of dielectric metasurfaces of different geometries and resonance mechanisms, diverse on-chip spatial differentiators have been proposed by tailoring the dispersion characteristics of subwavelength structures. This review focuses on the basic principles and characteristic parameters of dielectric metasurfaces as first- and second-order spatial differentiators realized via the Green's function approach. The spatial bandwidth and polarization dependence are emphasized as key properties by comparing the optical transfer functions of metasurfaces for different incident wavevectors and polarizations. To present the operational capabilities of a two-dimensional spatial differentiator in image information acquisition, edge detection is described to illustrate the practicability of the device. As an application example, experimental demonstrations of edge detection for different biological cells and a flower mold are discussed, in which a spatial differentiator and objective lens or camera are integrated in three optical pathway configurations. The realization of spatial differentiators and edge detection with dielectric metasurfaces provides new opportunities for ultrafast information identification in biological imaging and machine vision.

Optical Realization of Wave-Based Analog Computing with Metamaterials

Recently, the study of analog optical computing raised renewed interest due to its natural advantages of parallel, high speed and low energy consumption over conventional digital counterpart, particularly in applications of big data and high-throughput image processing. The emergence of metamaterials or metasurfaces in the last decades offered unprecedented opportunities to arbitrarily manipulate the light waves within subwavelength scale. Metamaterials and metasurfaces with freely controlled optical properties have accelerated the progress of wave-based analog computing and are emerging as a practical, easy-integration platform for optical analog computing. In this review, the recent progress of metamaterial-based spatial analog optical computing is briefly reviewed. We first survey the implementation of classical mathematical operations followed by two fundamental approaches (metasurface approach and Green’s function approach). Then, we discuss recent developments based on different physical mechanisms and the classical optical simulating of quantum algorithms are investigated, which may lead to a new way for high-efficiency signal processing by exploiting quantum behaviors. The challenges and future opportunities in the booming research field are discussed.

Optical analog computing of two-dimensional spatial differentiation based on the Brewster effect

Optical analog computing has attracted widespread attention in recent decades due to its advantages of lower consumption, higher efficiency, and real-time imaging in image processing. Here, we propose a two-dimensional optical analog computing scheme based on the Brewster effect. We experimentally demonstrate two-dimensional edge detection with high efficiency. By combining microscopy, our approach may develop some significant applications in cellular and molecular imaging.