Topological optical differentiator

Nonlocal flat optics for size-selective image processing and denoising

All-optical image processing based on metasurfaces is a swiftly advancing field of technology, due to its high speed, large integrability and inherently low energy requirements. So far, the proposed devices have been focusing on canonical operations, such as differentiations to perform edge detection across all objects in a complex scene. Yet, undesired background noise and clutter can hinder such operations, requiring target selection with digital post-processing which inherently limits the overall accuracy, efficiency and speed. Here, we introduce an optical solution for real-time size-selective image processing and experimentally demonstrate the concept with a metal-dielectric-metal film performing a spatial band-pass filter in momentum space. We show high-resolution (~0.9 μm) edge detection and real-time dynamic denoising, ideally suited for bio-imaging applications and target recognitions. Our demonstrated k-space filtering metasurface expands the scope of nonlocal flat optics for analog image processing, ushering in opportunities for ultra-compact, cost-effective, and multifunctional image processors.

Electrically Tunable Momentum Space Polarization Singularities in Liquid Crystal Microcavities

Momentum space polarization singularities of light appear as vectorial twists in the scattered and radiated far field patterns of exotic photonic structures. They relate to important concepts such as bound states in the continuum, spatiotemporal light steering, polarization Möbius strips, Berry curvature, and associated topological photonic phenomena. Polarization singularities, such as completely circularly polarized C-points, are readily designed in real space through interference of differently polarized beams. In momentum space, they require instead sophisticated patterning of photonic crystal slabs of reduced symmetries in order to appear in the corresponding band structure with scarce in situ tunability. Here, it is shown that momentum space singularities can be generated and, importantly, electrically tuned in the band structure of a highly birefringent planar liquid crystal microcavity that retains many symmetries. The results agree with theoretical predictions and offer exciting possibilities for integration of momentum space polarization singularities in spinoptronic technologies.

A Simple Optical Convolution Strategy Based on Versatile Adjustable Optical Convolution Kernel for All-Optical Convolution Computing

Convolutional neural network (CNN) is currently one of the most important artificial neural networks. However, traditional CNN hardware architectures suffer from significant increases in energy consumption and processing time as the demand for artificial intelligence tasks grows. Here, a novel optical convolution computing strategy is proposed that leverages a continuously adjustable photoluminescent device (CA-PLD) as the optical convolution kernel, enabling parallel, all-optical convolution computing and greatly simplifying the traditional convolution process. Under ultraviolet illumination, the CA-PLD exhibits visible long-afterglow emission characteristics due to the charge trapping and retention effects. This allows for continuously adjustable light weights, facilitating arbitrary convolution operations. Building on this, parallel and efficient multiply-accumulate operations have been successfully demonstrated using CA-PLD arrays with different weight combinations. Moreover, space-transformable CA-PLD units enable applications in dilated convolution. In a semantic segmentation task with 20 categories, the CA-PLD units achieve higher Intersection over Union (IoU) values and accuracy. Therefore, the weight-adjustable and spatial transformable CA-PLD proposed in this work holds promise for future applications in intelligent optical computing systems and optical implementations of non-von Neumann architectures.

Fundamental interdependence between resolution and spatial bandwidth of analog optical spatial differentiators

Recently, in [Opt. Express28, 898 (2020)10.1364/OE.379492], two relations have been obtained between the gain and resolution of two ideal cases of spatial differentiators. The resolution was computed using the Rayleigh criterion. The first case is an ideal differentiator in that the magnitude of its transfer function is limited to unity, and the second case is the ideal form of a typical differentiator. The relation corresponding to case II has been used as a figure of merit (FOM) for comparison purposes between different differentiator performances. In this paper, we show that the Rayleigh criterion cannot properly compute the resolution of these ideal differentiators, especially for case II. The correct resolution is much smaller than that computed by the Rayleigh criterion in [Opt. Express28, 898 (2020)10.1364/OE.379492]. Hence, the mentioned relations between gain and resolution, and accordingly, the FOM in [Opt. Express28, 898 (2020)10.1364/OE.379492] are not correct. Herein, we propose three conditions (two obligatory and one optional) to determine the resolution of an edge detector. We mathematically prove that two of the three criteria (one optional and the other obligatory) are always met by both the ideal differentiators. We then demonstrate that the correct value of the resolution is approximately independent of the gain in case II but dependent on the spatial bandwidth of the ideal differentiator. We also show that similar resolution results are obtained when using a Gaussian light beam. Hence, we introduce a new FOM, which is a trade-off between the correct resolution and spatial bandwidth of the ideal differentiator in case II. We then use this new FOM to compare the performances of some recently proposed differentiators.

All-optical analog differential operation and information processing empowered by meta-devices

The burgeoning demand for high-performance computing, robust data processing, and rapid growth of big data necessitates the emergence of novel optical devices to efficiently execute demanding computational processes. The field of meta-devices, such as metamaterial or metasurface, has experienced unprecedented growth over the past two decades. By manipulating the amplitude, phase, polarization, and dispersion of light wavefronts in spatial, spectral, and temporal domains, viable solutions for the implementation of all-optical analog computation and information processing have been provided. In this review, we summarize the latest developments and emerging trends of computational meta-devices as innovative platforms for spatial optical analog differentiators and information processing. Based on the general concepts of spatial Fourier transform and Green's function, we analyze the physical mechanisms of meta-devices in the application of amplitude differentiation, phase differentiation, and temporal differentiation and summarize their applications in image edge detection, image edge enhancement, and beam shaping. Finally, we explore the current challenges and potential solutions in optical analog differentiators and provide perspectives on future research directions and possible developments.

Metasurface enabled high-order differentiator

Metasurface-enabled optical analog differentiation has garnered significant attention due to its inherent capacity of parallel operation, compactness, and low power consumption. Most previous works focused on the first- and second-order operations, while several significant works have also achieved higher-order differentiation in both real space and k-space. However, how to construct the desired optical transfer function in a practical system to realize scalable and multi-order-parallel high-order differentiation of images in real space, and particularly how to leverage it to tackle practical problems, have not been fully explored. Here, drawing on the basic mathematical feature of the Fourier transform, we theoretically propose universal phase-gradient functions of the Pancharatnam-Berry-phase-based meta-device for performing arbitrary order differentiation. The fifth-order optical differentiations for both intensity and phase images are realized in the experiment. More importantly, by exploring this elaborately designed spatial differentiator, we construct another scheme for optical super-resolution and achieve the estimation of the distance between two incoherent point sources within 0.015 of the Rayleigh distance, which thereby provides a potential toolkit for optical alignment in high-precision semiconductor nano-fabrication. Our findings hold promise for image processing, microscopy imaging, and optical super-resolution imaging.

Optical sorting: past, present and future

Optical sorting combines optical tweezers with diverse techniques, including optical spectrum, artificial intelligence (AI) and immunoassay, to endow unprecedented capabilities in particle sorting. In comparison to other methods such as microfluidics, acoustics and electrophoresis, optical sorting offers appreciable advantages in nanoscale precision, high resolution, non-invasiveness, and is becoming increasingly indispensable in fields of biophysics, chemistry, and materials science. This review aims to offer a comprehensive overview of the history, development, and perspectives of various optical sorting techniques, categorised as passive and active sorting methods. To begin, we elucidate the fundamental physics and attributes of both conventional and exotic optical forces. We then explore sorting capabilities of active optical sorting, which fuses optical tweezers with a diversity of techniques, including Raman spectroscopy and machine learning. Afterwards, we reveal the essential roles played by deterministic light fields, configured with lens systems or metasurfaces, in the passive sorting of particles based on their varying sizes and shapes, sorting resolutions and speeds. We conclude with our vision of the most promising and futuristic directions, including AI-facilitated ultrafast and bio-morphology-selective sorting. It can be envisioned that optical sorting will inevitably become a revolutionary tool in scientific research and practical biomedical applications.

Nonlinear differential interference contrast imaging

Differential interference contrast (DIC) imaging is essential in both biological research and medical diagnostics. Despite considerable progress in theoretical and experimental frameworks, limited by the inefficient cameras, achieving direct DIC imaging with infrared (IR) illumination remains a formidable challenge. However, infrared DIC imaging is urgent for diverse fields. Here, we creatively leverage the walk-off effect, a limitation in nonlinear optics, to solve this obstacle and present the nonlinear DIC imaging. The critical component of our scheme is a nonlinear beam displacer (NBD) made up of two quadrature-cascaded type I nonlinear crystals. When the infrared beam carrying object information passes through the proposed NBD, it undergoes nonlinear coupling with the pump beam and then generates two orthogonally polarized visible beams with a slight spatial displacement dominated by the walk-off effect. Accordingly, by selecting polarization, the lateral shear interference for realizing DIC imaging can be realized, and thus the phase discontinuities of the object can be visualized with infrared illumination. Our finding brings DIC imaging technology into the realm of upconversion infrared imaging, paving the way for infrared phase microscopy imaging.

Dynamically switchable edge-detection and bright-field imaging based on a phase-change nonlocal metasurface

Optical metasurfaces offer significant advantages in enhancing the speed, efficiency, and miniaturization of imaging systems. However, most existing metasurfaces are limited to static functionalities and lack reconfigurability, which is a key feature for practical applications in dynamic environments. In this work, we demonstrate a reconfigurable optical metasurface capable of switching between two distinct imaging functions (edge detection and bright-field imaging) within the visible spectrum. This reconfigurability is achieved by tuning the phase transition of antimony sulfide (Sb2S3), which controls the angular dependence of the magnetic dipole resonance. The phase transition of Sb2S3 from the amorphous phase to the crystalline phase enables different optical transfer functions, achieving high-performance imaging with a numerical aperture of 0.42, isotropic second-order differentiation, and high-resolution imaging, respectively. This approach allows for functional switching on a single surface, opening up possibilities for applications in medical imaging, optical sensing, and microscopy.

Compressed computational imaging based on optical differentiation

In computational imaging, getting better imaging quality with shorter time usage is always a challenging problem. The powerful compressed sensing functions as a backend algorithm, which leaves room for us to develop a methodology of compression in imaging systems. Optical differentiation was widely utilized in direct imaging to highlight the features of an image. We apply optical differentiation to compress information in the correlated imaging system. The experimental results indicate a significant improvement in the signal-to-noise ratio and imaging speed. In addition, this scheme enables phase imaging from the second-order correlation. Our work can spark potential applications in biological microscopic and scattering media imaging.

Beyond Moiré with Spatial Frequency Mastery via δ-Function Expansion Metasurface

Mastering spatial frequency manipulation within momentum space is pivotal yet challenging, particularly in mitigating moiré patterns that significantly impair image quality across diverse applications. Conventional methods often require trade-offs in spatial resolution or fall short of completely eradicating unwanted frequencies, further burdened by complex post-processing demands. In this work, a novel coherent δ-function expansion technique implemented through an all-silicon metasurface, affording unparalleled synergistic control over arbitrarily selected spatial frequencies via refined k-space amplitude and phase modulations is introduced. This approach transcends traditional global methods by harnessing a sophisticated ensemble of multiple δ-functions, enabling a holistic manipulation of spatial frequencies. The periodicity introduced by this approach also enables the feasibility of infinitely spatial stitching expansion for metasurfaces while maintaining high energy utilization efficiency. The methodology excels in the meticulous removal of local moiré frequencies while concurrently facilitating numerous advanced optical functions, including mixed partial differentiation and noise suppression, all within the optical domain. This work heralds a significant leap forward in optical manipulation, presenting a viable, scalable alternative to complex electronic post-processing. Through this work, not only a longstanding challenge is addressed in optical physics but also open new avenues for research and application in photodetection and optical processing technologies.

Broadband and parallel multiple-order optical spatial differentiation enabled by Bessel vortex modulated metalens

Optical analog image processing technology is expected to provide an effective solution for high-throughput and real-time data processing with low power consumption. In various operations, optical spatial differential operations are essential in edge extraction, data compression, and feature classification. Unfortunately, existing methods can only perform low-order or selectively perform a particular high-order differential operation. Here, we propose and experimentally demonstrate a Bessel vortex modulated metalens composed of a single complex amplitude metasurface, which can perform multiple-order radial differential operations over a wide band by presetting the order of the corresponding Bessel vortex. This architecture further enables angle multiplexing to create multiple information channels that synchronously perform multi-order spatial differential operations, indicating the superiority of the proposed devices in parallel processing. Our approach may find various applications in artificial intelligence, machine vision, autonomous driving, and advanced biomedical imaging.

One-dimensional photonic crystal enhancing spin-to-orbital angular momentum conversion for single-particle tracking

Single-particle tracking (SPT) is an immensely valuable technique for studying a variety of processes in the life sciences and physics. It can help researchers better understand the positions, paths, and interactions of single objects in systems that are highly dynamic or require imaging over an extended time. Here, we propose an all-dielectric one-dimensional photonic crystal (1D PC) that enhances spin-to-orbital angular momentum conversion for three-dimensional (3D) SPTs. This well-designed 1D PC can work as a substrate for optical microscopy. We introduce this effect into the interferometric scattering (iSCAT) technique, resulting in a double-helix point spread function (DH-PSF). DH-PSF provides more uniform Fisher information for 3D position estimation than the PSFs of conventional microscopy, such as encoding the axial position of a single particle in the angular orientation of DH-PSF lobes, thus providing a means for 3D SPT. This approach can address the challenge of iSCAT in 3D SPT because DH-PSF iSCAT will not experience multiple contrast inversions when a single particle travels along the axial direction. DH-PSF iSCAT microscopy was used to record the 3D trajectory of a single microbead attached to the flagellum, facilitating precise analysis of fluctuations in motor dynamics. Its ability to track single nanoparticles, such as 3D diffusion trajectories of 20 nm gold nanoparticles in glycerol solution, was also demonstrated. The DH-PSF iSCAT technique enabled by a 1D PC holds potential promise for future applications in physical, biological, and chemical science.

Tailoring Space-Time Nonlocality for Event-Based Image Processing Metasurfaces

Analog computation with passive optical components can enhance processing speeds and reduce power consumption, recently attracting renewed interest thanks to the opportunities enabled by metasurfaces. Basic image processing tasks, such as spatial differentiation, have been recently demonstrated based on engineered nonlocalities in metasurfaces, but next-generation computational schemes require more advanced capabilities. Here, by simultaneously tailoring the nonlocal electromagnetic response of a metasurface in space and time, we demonstrate a passive ultrathin silicon-based device that performs mixed spatiotemporal differentiation of input images, realizing event-based edge detection. The metasurface performs spatial differentiation only when the input image is evolving in time, resulting in spatiotemporal image processing on subpicosecond timescales. Moreover, the metasurface design can be tailored to selectively enhance objects moving at desired speeds. Our results point towards fully passive processing of spatiotemporal signals, for highly compact neuromorphic cameras.

Quarter-wave Pancharatnam-Berry phase gradient liquid crystal-enabled dual-polarization optical edge detection

Here, we present a novel, to the best of our knowledge, optical edge detection scheme that can be operated in both linear and circular polarization modes, leveraging an optical spatial differentiator constructed by quarter-wave Pancharatnam-Berry (P-B) phase gradient element. After explaining the theoretical mechanism, we utilize a quarter-wave P-B phase liquid crystal polarization grating to validate the dual-polarization optical edge detection capability. We demonstrate that the orientation of linear polarization and the spin of circular polarization dictate the transition between edge and bright-field images. Besides, the linear and circular polarization modes exhibit broadband and monochromatic responsive properties, respectively. This mechanism, dependent on wavelength and polarization, holds promise for applications in color image processing, chiral sensing imaging, and polarization-entangled quantum imaging.

Two-dimensional optical multiple-order differentiations based on spatial spectrum modulation

Optical differential operations have recently attracted considerable attention owing to the capabilities of ultrafast speed and low power consumption. The transfer function, which embodies the frequency-domain characteristics of differential systems, plays an important role in differentiator design. Here, we report a super-Gaussian aperture differential filter, and we reveal unique characteristics of odd- and even-order transfer functions and corresponding differential effects via spatial spectrum modulation. We show that the feature of the transfer function is well maintained, and more precise differentiation can be achieved using the designed filter. Two-dimensional first- to fifth-order full and partial differentiations are implemented both theoretically and experimentally. Our work provides an approach for engineering customized multiple-order differentiators and promotes the advancements of related areas such as optical analog computing and image processing.

Reconfigurable image processing metasurfaces with phase-change materials

Optical metasurfaces have enabled analog computing and image processing within sub-wavelength footprints, and with reduced power consumption and faster speeds. While various image processing metasurfaces have been demonstrated, most of the considered devices are static and lack reconfigurability. Yet, the ability to dynamically reconfigure processing operations is key for metasurfaces to be used within practical computing systems. Here, we demonstrate a passive edge-detection metasurface operating in the near-infrared regime whose response can be drastically modified by temperature variations smaller than 10 °C around a CMOS-compatible temperature of 65 °C. Such reconfigurability is achieved by leveraging the insulator-to-metal phase transition of a thin layer of vanadium dioxide, which strongly alters the metasurface nonlocal response. Importantly, this reconfigurability is accompanied by performance metrics-such as numerical aperture, efficiency, isotropy, and polarization-independence - close to optimal, and it is combined with a simple geometry compatible with large-scale manufacturing. Our work paves the way to a new generation of ultra-compact, tunable and passive devices for all-optical computation, with potential applications in augmented reality, remote sensing and bio-medical imaging.

Polarization-independent edge detection based on the spin-orbit interaction of light

In previous edge detection schemes based on the spin-orbit interaction of light, the direction and intensity of the edge-enhanced images are influenced by the incident polarization state. In this study, we develop an edge detection strategy that is insensitive to changes in both the incident polarization and the incident angle. The output intensity and transfer function remain entirely impervious to changes in incident polarization, being explicitly formulated as functions of the incident angle, specifically in terms of cot 2⁡θ i and cot⁡θ i , respectively. This behavior is attributed to the opposing nature of the polarization components E~r H-H and E~r V-V in the x-direction after undergoing mapping through the Glan polarizer, while the sum of polarization components E~r H-V and E~r V-H in the y-direction can be simplified to terms independent of incident polarization. Furthermore, we propose a metasurface design to achieve the required optical properties in order to realize the derived edge detection scheme.

Compact meta-differentiator for achieving isotropically high-contrast ultrasonic imaging

Ultrasonic imaging is crucial in the fields of biomedical engineering for its deep penetration capabilities and non-ionizing nature. However, traditional techniques heavily rely on impedance differences within objects, resulting in poor contrast when imaging acoustically transparent targets. Here, we propose a compact spatial differentiator for underwater isotropic edge-enhanced imaging, which enhances the imaging contrast without the need for contrast agents or external physical fields. This design incorporates an amplitude meta-grating for linear transmission along the radial direction, combined with a phase meta-grating that utilizes focus and spiral phases with a first-order topological charge. Through theoretical analysis, numerical simulations, and experimental validation, we substantiate the effectiveness of our technique in distinguishing amplitude objects with isotropic edge enhancements. Importantly, this method also enables the accurate detection of both phase objects and artificial biological models. This breakthrough creates new opportunities for applications in medical diagnosis and nondestructive testing.

Poincaré sphere trajectory encoding metasurfaces based on generalized Malus' law

As a fundamental property of light, polarization serves as an excellent information encoding carrier, playing significant roles in many optical applications, including liquid crystal displays, polarization imaging, optical computation and encryption. However, conventional polarization information encoding schemes based on Malus' law usually consider 1D polarization projections on a linear basis, implying that their encoding flexibility is largely limited. Here, we propose a Poincaré sphere (PS) trajectory encoding approach with metasurfaces that leverages a generalized form of Malus' law governing universal 2D projections between arbitrary elliptical polarization pairs spanning the entire PS. Arbitrary polarization encodings are realized by engineering PS trajectories governed by either arbitrary analytic functions or aligned modulation grids of interest, leading to versatile polarization image transformation functionalities, including histogram stretching, thresholding and image encryption within non-orthogonal PS loci. Our work significantly expands the encoding dimensionality of polarization information, unveiling new opportunities for metasurfaces in polarization optics for both quantum and classical regimes.

Broadband and large-aperture metasurface edge encoders for incoherent infrared radiation

The prevalence of computer vision systems necessitates hardware-based approaches to relieve the high computational demand of deep neural networks in resource-limited applications. One solution would be to off-load low-level image feature extraction, such as edge detection, from the digital network to the analog imaging system. To that end, this work demonstrates incoherent, broadband, low-noise optical edge detection of real-world scenes by combining the wavefront shaping of a 24-mm aperture metasurface with a refractive lens. An inverse design approach is used to optimize the metasurface for Laplacian-based edge detection across the 7.5- to 13.5-μm LWIR imaging band, allowing for facile integration with uncooled microbolometer-based LWIR imagers to encode edge information. A polarization multiplexed approach leveraging a birefringent metasurface is also demonstrated as a single-aperture implementation. This work could be applied to improve computer vision capabilities of resource-constrained systems by leveraging optical preprocessing to alleviate the computational requirements for high-accuracy image segmentation and classification.

Generation of Spatiotemporal Vortex Pulses by Resonant Diffractive Grating

Spatiotemporal vortex pulses are wave packets that carry transverse orbital angular momentum, exhibiting exotic structured wave fronts that can twist through space and time. Existing methods to generate these pulses require complex setups like spatial light modulators or computer-optimized structures. Here, we demonstrate a new approach to generate spatiotemporal vortex pulses using just a simple diffractive grating. The key is constructing a phase vortex in frequency-momentum space by leveraging symmetry, resonance, and diffraction. Our approach is applicable to any wave system. We use a liquid surface wave (gravity wave) platform to directly demonstrate and observe the real-time generation and evolution of spatiotemporal vortex pulses. This straightforward technique provides opportunities to explore pulse dynamics and potential applications across different disciplines.

Polychromatic Dual-Mode Imaging with Structured Chiral Photonic Crystals

Optical spatial differentiation is a typical operation of optical analog computing and can single out the edge to accelerate the subsequent image processing, but in some cases, overall information about the object needs to be presented synchronously. Here, we propose a multifunctional optical device based on structured chiral photonic crystals for the simultaneous realization of real-time dual-mode imaging. This optical differentiator is realized by self-organized large-birefringence cholesteric liquid crystals, which are photopatterned to encode with a special integrated geometric phase. Two highly spin-selective modes of second-order spatial differentiation and bright-field imaging are exhibited in the reflected and transmitted directions, respectively. Two-dimensional edges of both amplitude and phase objects have been efficiently enhanced in high contrast and the broadband spectrum. This work extends the ingenious building of hierarchical chiral nanostructures, enriches their applications in the emerging frontiers of optical computing, and boasts considerable potential in machine vision and microscopy.

Perovskite Topological Lasers: A Brand New Combination

Nanolasers are the essential components of modern photonic chips due to their low power consumption, high energy efficiency and fast modulation. As nanotechnology has advanced, researchers have proposed a number of nanolasers operating at both wavelength and sub-wavelength scales for application as light sources in photonic chips. Despite the advances in chip technology, the quality of the optical cavity, the operating threshold and the mode of operation of the light source still limit its advanced development. Ensuring high-performance laser operation has become a challenge as device size has been significantly reduced. A potential solution to this problem is the emergence of a novel optical confinement mechanism using photonic topological insulator lasers. In addition, gain media materials with perovskite-like properties have shown great potential for lasers, a role that many other gain materials cannot fulfil. When combined with topological laser modes, perovskite materials offer new possibilities for the operation and emission mechanism of nanolasers. This study introduces the operating mechanism of topological lasers and the optical properties of perovskite materials. It then outlines the key features of their combination and discusses the principles, structures, applications and prospects of perovskite topological lasers, including the scientific hurdles they face. Finally, the future development of low-dimensional perovskite topological lasers is explored.

Photonic signal processor based on a Kerr microcomb for real-time video image processing

Signal processing has become central to many fields, from coherent optical telecommunications, where it is used to compensate signal impairments, to video image processing. Image processing is particularly important for observational astronomy, medical diagnosis, autonomous driving, big data and artificial intelligence. For these applications, signal processing traditionally has mainly been performed electronically. However these, as well as new applications, particularly those involving real time video image processing, are creating unprecedented demand for ultrahigh performance, including high bandwidth and reduced energy consumption. Here, we demonstrate a photonic signal processor operating at 17 Terabits/s and use it to process video image signals in real-time. The system processes 400,000 video signals concurrently, performing 34 functions simultaneously that are key to object edge detection, edge enhancement and motion blur. As compared with spatial-light devices used for image processing, our system is not only ultra-high speed but highly reconfigurable and programable, able to perform many different functions without any change to the physical hardware. Our approach is based on an integrated Kerr soliton crystal microcomb, and opens up new avenues for ultrafast robotic vision and machine learning.

Signal processing is key to communications and video image processing for astronomy, medical diagnosis, autonomous driving, big data and AI. Menxi Tan and colleagues report a photonic processor operating at 17Tb/s for ultrafast robotic vision and machine learning.

Wavelength-division multiplexing optical Ising simulator enabling fully programmable spin couplings and external magnetic fields

Recently various physical systems have been proposed for modeling Ising spin Hamiltonians appealing to solve combinatorial optimization problems with remarkable performance. However, how to implement arbitrary spin-spin interactions is a critical and challenging problem in unconventional Ising machines. Here, we propose a general gauge transformation scheme to enable arbitrary spin-spin interactions and external magnetic fields as well, by decomposing an Ising Hamiltonian into multiple Mattis-type interactions. With this scheme, a wavelength-division multiplexing spatial photonic Ising machine (SPIM) is developed to show the programmable capability of general spin coupling interactions. We exploit the wavelength-division multiplexing SPIM to simulate three spin systems: ±J models, Sherrington-Kirkpatrick models, and only locally connected J1-J2 models and observe the phase transitions. We also demonstrate the ground-state search for solving Max-Cut problem with the wavelength-division multiplexing SPIM. These results promise the realization of ultrafast-speed and high-power efficiency Boltzmann sampling to a generalized large-scale Ising model.

Dispersion engineered metasurfaces for broadband, high-NA, high-efficiency, dual-polarization analog image processing

Optical metasurfaces performing analog image processing - such as spatial differentiation and edge detection - hold the potential to reduce processing times and power consumption, while avoiding bulky 4 F lens systems. However, current designs have been suffering from trade-offs between spatial resolution, throughput, polarization asymmetry, operational bandwidth, and isotropy. Here, we show that dispersion engineering provides an elegant way to design metasurfaces where all these critical metrics are simultaneously optimized. We experimentally demonstrate silicon metasurfaces performing isotropic and dual-polarization edge detection, with numerical apertures above 0.35 and spectral bandwidths of 35 nm around 1500 nm. Moreover, we introduce quantitative metrics to assess the efficiency of these devices. Thanks to the low loss nature and dual-polarization response, our metasurfaces feature large throughput efficiencies, approaching the theoretical maximum for a given NA. Our results pave the way for low-loss, high-efficiency and broadband optical computing and image processing with free-space metasurfaces.

Metasurface enabled broadband all optical edge detection in visible frequencies

Image processing is of fundamental importance for numerous modern technologies. In recent years, due to increasing demand for real-time and continuous data processing, metamaterial and metasurface based all-optical computation techniques emerged as a promising alternative to digital computation. Most of the pioneer research focused on all-optical edge detection as a fundamental step of image processing. Metasurfaces have been shown to enable real time edge detection with low to no power consumption. However, the previous demonstrations were subjected to the several limitations such as need for oblique-incidence, polarization dependence, need for additional polarizers, narrow operation bandwidth, being limited with processing in 1D, operation with coherent light only, and requiring digital post-processing. Here, we propose and experimentally demonstrate 2D isotropic, polarization-independent, broadband edge detection with high transmission efficiency under both coherent and incoherent illumination along the visible frequency range using a metasurface based on Fourier optics principles.

Topologically crafted spatiotemporal vortices in acoustics

Vortices in fluids and gases have piqued the human interest for centuries. Development of classical-wave physics and quantum mechanics highlighted wave vortices characterized by phase singularities and topological charges. In particular, vortex beams have found numerous applications in modern optics and other areas. Recently, optical spatiotemporal vortex states exhibiting the phase singularity both in space and time have been described. Here, we report the topologically robust generation of acoustic spatiotemporal vortex pulses. We utilize an acoustic meta-grating with broken mirror symmetry which exhibits a topological phase transition with a pair of phase singularities with opposite topological charges emerging in the momentum-frequency domain. We show that these vortices are topologically robust against structural perturbations of the meta-grating and can be employed for the generation of spatiotemporal vortex pulses. Our work paves the way for studies and applications of spatiotemporal structured waves in acoustics and other wave systems.

Revisiting the Design Strategies for Metasurfaces: Fundamental Physics, Optimization, and Beyond

Over the last two decades, the capabilities of metasurfaces in light modulation with subwavelength thickness have been proven, and metasurfaces are expected to miniaturize conventional optical components and add various functionalities. Herein, various metasurface design strategies are reviewed thoroughly. First, the scalar diffraction theory is revisited to provide the basic principle of light propagation. Then, widely used design methods based on the unit-cell approach are discussed. The methods include a set of simplified steps, including the phase-map retrieval and meta-atom unit-cell design. Then, recently emerging metasurfaces that may not be accurately designed using unit-cell approach are introduced. Unconventional metasurfaces are examined where the conventional design methods fail and finally potential design methods for such metasurfaces are discussed.

Feature-enhanced X-ray imaging using fused neural network strategy with designable metasurface

Scintillation-based X-ray imaging can provide convenient visual observation of absorption contrast by standard digital cameras, which is critical in a variety of science and engineering disciplines. More efficient scintillators and electronic postprocessing derived from neural networks are usually used to improve the quality of obtained images from the perspective of optical imaging and machine vision, respectively. Here, we propose to overcome the intrinsic separation of optical transmission process and electronic calculation process, integrating the imaging and postprocessing into one fused optical-electronic convolutional autoencoder network by affixing a designable optical convolutional metasurface to the scintillator. In this way, the convolutional autoencoder was directly connected to down-conversion process, and the optical information loss and training cost can be decreased simultaneously. We demonstrate that feature-specific enhancement of incoherent images is realized, which can apply to multi-class samples without additional data precollection. Hard X-ray experimental validations reveal the enhancement of textural features and regional features achieved by adjusting the optical metasurface, indicating a signal-to-noise ratio improvement of up to 11.2 dB. We anticipate that our framework will advance the fundamental understanding of X-ray imaging and prove to be useful for number recognition and bioimaging applications.

Electrically tunable optical spatial differentiation with graphene

In recent years, optical analog computing has experienced rapid development, among which optical differential operation has attracted great attention. Here, based on the unique optical properties of graphene, we propose an electrically tunable optical spatial differentiation by introducing a graphene layer at a quartz substrate. It is found that the output light field is sensitive to the graphene layer near the Brewster angle for small polarization output at the graphene-quartz substrate interface and can be modulated by changing the Fermi energy of graphene. In this case, the result of the optical differential operation can be dynamically regulated. Almost strict one-dimensional differential operations in different directions and almost perfect two-dimensional differential operations can be achieved. In addition, two-dimensional edge detection with different degrees of distortion in different directions can also be realized when applied to image processing. This new modulation method may provide more possibilities for tunable image edge detection and provide a potential way for developing more versatile optical simulators in the future.

Amp-vortex edge-camera: a lensless multi-modality imaging system with edge enhancement

We demonstrate a lensless imaging system with edge-enhanced imaging constructed with a Fresnel zone aperture (FZA) mask placed 3 mm away from a CMOS sensor. We propose vortex back-propagation (vortex-BP) and amplitude vortex-BP algorithms for the FZA-based lensless imaging system to remove the noise and achieve the fast reconstruction of high contrast edge enhancement. Directionally controlled anisotropic edge enhancement can be achieved with our proposed superimposed vortex-BP algorithm. With different reconstruction algorithms, the proposed amp-vortex edge-camera in this paper can achieve 2D bright filed imaging, isotropic, and directional controllable anisotropic edge-enhanced imaging with incoherent light illumination, by a single-shot captured hologram. The effect of edge detection is the same as optical edge detection, which is the re-distribution of light energy. Noise-free in-focus edge detection can be achieved by using back-propagation, without a de-noise algorithm, which is an advantage over other lensless imaging technologies. This is expected to be widely used in autonomous driving, artificial intelligence recognition in consumer electronics, etc.

Topological spatial differentiation via complex amplitude filtering in Fourier space

Various approaches to implementing optical analog differentiation have been studied extensively and applied in edge-based image processing. Here, we report a topological optical differentiation scheme based on complex amplitude filtering, i.e., amplitude and spiral phase modulation in Fourier space. The isotropic and anisotropic multiple-order differentiation operations are demonstrated both theoretically and experimentally. Meanwhile, we also achieve multiline edge detection corresponding to the differential order for the amplitude and phase objects. This proof-of-principle work could open up new avenues for engineering a nanophotonic differentiator and realizing a more compact image-processing system.

Single-shot isotropic differential interference contrast microscopy

Differential interference contrast (DIC) microscopy allows high-contrast, low-phototoxicity, and label-free imaging of transparent biological objects, and has been applied in the field of cellular morphology, cell segmentation, particle tracking, optical measurement and others. Commercial DIC microscopy based on Nomarski or Wollaston prism resorts to the interference of two polarized waves with a lateral differential offset (shear) and axial phase shift (bias). However, the shear generated by these prisms is limited to the rectilinear direction, unfortunately resulting in anisotropic contrast imaging. Here we propose an ultracompact metasurface-assisted isotropic DIC (i-DIC) microscopy based on a grand original pattern of radial shear interferometry, that converts the rectilinear shear into rotationally symmetric along radial direction, enabling single-shot isotropic imaging capabilities. The i-DIC presents a complementary fusion of typical meta-optics, traditional microscopes and integrated optical system, and showcases the promising and synergetic advancements in edge detection, particle motion tracking, and label-free cellular imaging.

Solving integral equations in free space with inverse-designed ultrathin optical metagratings

As standard microelectronic technology approaches fundamental limitations in speed and power consumption, novel computing strategies are strongly needed. Analogue optical computing enables the processing of large amounts of data at a negligible energy cost and high speeds. Based on these principles, ultrathin optical metasurfaces have been recently explored to process large images in real time, in particular for edge detection. By incorporating feedback, it has also recently been shown that metamaterials can be tailored to solve complex mathematical problems in the analogue domain, although these efforts have so far been limited to guided-wave systems and bulky set-ups. Here, we present an ultrathin Si metasurface-based platform for analogue computing that is able to solve Fredholm integral equations of the second kind using free-space visible radiation. A Si-based metagrating was inverse-designed to implement the scattering matrix synthesizing a prescribed kernel corresponding to the mathematical problem of interest. Next, a semitransparent mirror was incorporated into the sample to provide adequate feedback and thus perform the required Neumann series, solving the corresponding equation in the analogue domain at the speed of light. Visible wavelength operation enables a highly compact, ultrathin device that can be interrogated from free space, implying high processing speeds and the possibility of on-chip integration.

Advances in Meta-Optics and Metasurfaces: Fundamentals and Applications

Meta-optics based on metasurfaces that interact strongly with light has been an active area of research in recent years. The development of meta-optics has always been driven by human's pursuits of the ultimate miniaturization of optical elements, on-demand design and control of light beams, and processing hidden modalities of light. Underpinned by meta-optical physics, meta-optical devices have produced potentially disruptive applications in light manipulation and ultra-light optics. Among them, optical metalens are most fundamental and prominent meta-devices, owing to their powerful abilities in advanced imaging and image processing, and their novel functionalities in light manipulation. This review focuses on recent advances in the fundamentals and applications of the field defined by excavating new optical physics and breaking the limitations of light manipulation. In addition, we have deeply explored the metalenses and metalens-based devices with novel functionalities, and their applications in computational imaging and image processing. We also provide an outlook on this active field in the end.

All-optical image edge detection based on the two-dimensional photonic spin Hall effect in anisotropic metamaterial

Optical image processing based on the photonic spin Hall effect (SHE) has been gaining significant attention as a convenient and an accurate way for image edge detection. However, the recent edge imaging techniques depending on optical differentiation is mainly achieved by modulation of one-dimensional photonic SHE. Here, we theoretically predict the two-dimensional photonic SHE in the anisotropic metamaterial, and find that its longitudinal and transverse displacements exhibit spin-dependent property at filling factors with increasing incidence angle. As the transverse and in-plane displacements induced by the photonic SHE can be controlled by the filling factor of the crystal structure, the optical axis angle, and the incident angle, this intrinsical effect can be used to realize a tunable edge imaging. Interestingly, by changing the optical axis of the anisotropic metamaterial, the in-plane displacements are equal to the transverse displacements for a certain filling factor and the corresponding incident angle. Therefore, we propose a two-dimensional image edge detection method based on the photonic SHE in anisotropic metamaterial. Further numerical results validate the theoretical proposal.

Single planar photonic chip with tailored angular transmission for multiple-order analog spatial differentiator

Analog spatial differentiation is used to realize edge-based enhancement, which plays an important role in data compression, microscopy, and computer vision applications. Here, a planar chip made from dielectric multilayers is proposed to operate as both first- and second-order spatial differentiator without any need to change the structural parameters. Third- and fourth-order differentiations that have never been realized before, are also experimentally demonstrated with this chip. A theoretical analysis is proposed to explain the experimental results, which furtherly reveals that more differentiations can be achieved. Taking advantages of its differentiation capability, when this chip is incorporated into conventional imaging systems as a substrate, it enhances the edges of features in optical amplitude and phase images, thus expanding the functions of standard microscopes. This planar chip offers the advantages of a thin form factor and a multifunctional wave-based analogue computing ability, which will bring opportunities in optical imaging and computing.

Visualization of transparent particles based on optical spatial differentiation

Optical analog computing operates on the amplitude, phase, polarization, and frequency distributions of the electromagnetic field through the interaction of light and matter. The differentiation operation is widely used in all-optical image processing technology, such as edge detection. Here, we propose a concise way to observe transparent particles, incorporating the optical differential operation that occurs on a single particle. The particle's scattering and cross-polarization components combine into our differentiator. We achieve high-contrast optical images of transparent liquid crystal molecules. The visualization of aleurone grains (the structures that store protein particles in plant cells) in maize seed was experimentally demonstrated with a broadband incoherent light source. Avoiding the interference of stains, our designed method provides the possibility to observe protein particles directly in complex biological tissues.

Topological spatial differentiators upon reflection of the normally incident light

We theoretically propose topological spatial differentiators by the normal-incidence reflection of light. Firstly, a three-dimensional propagation model is established for the light normally incident on the interface between two media. It is found that due to the spin-orbit interaction of light, a given circularly polarized light always induces oppositely polarized light carrying a topological charge, so the two intrinsic spin components are separated radially or azimuthally. Moreover, the normally reflected fields are approximately proportional to two kinds of second-order spatial differentiations of the input circularly and linearly polarized fields. Further results applying to the two-dimensional image processing for edge detection validate the two topological spatial differentiators.

Basis function approach for diffractive pattern generation with Dammann vortex metasurfaces

In mathematics, general functions can be decomposed into a linear combination of basis functions. This principle can be used for creating an infinite number of distinct geometric patterns based on a finite number of basis patterns. Here, we propose a Dammann vortex metasurface (DVM) for optically generating an array of diverse, diffraction-multiplexed vortex patterns, based on three custom-defined basis patterns. The proposed DVM, with its capability of quantitatively correlating phase and intensity distribution in different diffraction orders, opens up doors for various applications including orbital angular momentum encryptions and quantum entanglement.

Designable optical differential operation based on surface plasmon resonance

Various optical differential computing devices have been designed, which have advantages of high speed and low power consumption compared with traditional digital computing. In this paper, considering the reflection of a light beam through a three-layer structure composed of glass, metal and air, we propose a designable optical differential operation based on surface plasmon resonance (SPR). When the SPR is excited under certain conditions, the spin-dependent splitting in the photonic spin Hall effect (SHE) changes dramatically. We first prove theoretically that this three-layer structure can realize one-dimensional optical differential operation. By discussing the transverse beam displacement under different conditions, it is found that the designable differential operation with high sensitivity can be realized by slightly adjusting the incident angle and the thickness of metal film. We design the differentiator which can obtain the image of measured target edge in real time and get different edge effects at different times. This will provide more possible applications for autonomous driving and target recognition.

All-dielectric metasurface for polarization-selective full-space complex amplitude modulations

Metasurfaces have exhibited powerful capabilities in the modulation of electromagnetic waves. Here, we demonstrate the polarization-selective full-space complex amplitude modulations of incident electromagnetic waves using all-dielectric metasurfaces. This is done via ingeniously designed subwavelength-scale super-pixels. As a proof of concept, we design two metasurfaces working in transmission and reflection spaces: one generates two independent vortex beams and the other generates two pairs of foci of arbitrary intensity ratios. The proposed full-space complex amplitude modulation provides more choices for the manipulation of electromagnetic waves.

Performing calculus with epsilon-near-zero metamaterials

Calculus is a fundamental subject in mathematics and extensively used in physics and astronomy. Performing calculus operations by analog computing has received much recent research interest because of its high speed and large data throughput; however, current analog calculus frameworks suffer from bulky sizes and relatively low integration densities. In this work, we introduce the concept of an epsilon-near-zero (ENZ) metamaterial processing unit (MPU) that performs differentiation and integration on analog signals to achieve extreme miniaturization at the subwavelength scale by generating desired dispersions of the ENZ metamaterials with photonic doping. To show the feasibility of this proposal, we further build an experimental analog image edge extraction system with a differentiating ENZ-MPU as its compute core. With a computing density theoretically analyzed to be several tera-operations per second and square micrometer, the proposed ENZ-MPU is scalable and configurable for more complex computations, providing an effective solution for analog calculus operators with extreme computing density and data throughput.

Metasurface-Based Imagers Enabled Arbitrary Optical Convolution Processing

Using meta-imagers composed of a meta-lens and a complex-amplitude meta-modulator, all-optical convolutional processing that arbitrarily reshapes the point spread function of an optical system can now be implemented.

Topological phase singularities in atomically thin high-refractive-index materials

Atomically thin transition metal dichalcogenides (TMDCs) present a promising platform for numerous photonic applications due to excitonic spectral features, possibility to tune their constants by external gating, doping, or light, and mechanical stability. Utilization of such materials for sensing or optical modulation purposes would require a clever optical design, as by itself the 2D materials can offer only a small optical phase delay - consequence of the atomic thickness. To address this issue, we combine films of 2D semiconductors which exhibit excitonic lines with the Fabry-Perot resonators of the standard commercial SiO2/Si substrate, in order to realize topological phase singularities in reflection. Around these singularities, reflection spectra demonstrate rapid phase changes while the structure behaves as a perfect absorber. Furthermore, we demonstrate that such topological phase singularities are ubiquitous for the entire class of atomically thin TMDCs and other high-refractive-index materials, making it a powerful tool for phase engineering in flat optics. As a practical demonstration, we employ PdSe2 topological phase singularities for a refractive index sensor and demonstrate its superior phase sensitivity compared to typical surface plasmon resonance sensors.

Meta-programmable analog differentiator

We present wave-based signal differentiation with unprecedented fidelity and flexibility by purposefully perturbing overmoded random scattering systems such that zeros of their scattering matrices lie exactly at the desired locations on the real frequency axis. Our technique overcomes limitations of hitherto existing approaches based on few-mode systems, both regarding their extreme vulnerability to fabrication inaccuracies or environmental perturbations and their inability to maintain high fidelity under in-situ adaptability. We demonstrate our technique experimentally by placing a programmable metasurface with hundreds of degrees of freedom inside a 3D disordered metallic box. Regarding the integrability of wave processors, such repurposing of existing enclosures is an enticing alternative to fabricating miniaturized devices. Our over-the-air differentiator can process in parallel multiple signals on distinct carriers and maintains high fidelity when reprogrammed to different carriers. We also perform programmable higher-order differentiation. Conceivable applications include segmentation or compression of communication or radar signals and machine vision.

Ultracompact meta-imagers for arbitrary all-optical convolution

Electronic digital convolutions could extract key features of objects for data processing and information identification in artificial intelligence, but they are time-cost and energy consumption due to the low response of electrons. Although massless photons enable high-speed and low-loss analog convolutions, two existing all-optical approaches including Fourier filtering and Green's function have either limited functionality or bulky volume, thus restricting their applications in smart systems. Here, we report all-optical convolutional computing with a metasurface-singlet or -doublet imager, considered as the third approach, where its point spread function is modified arbitrarily via a complex-amplitude meta-modulator that enables functionality-unlimited kernels. Beyond one- and two-dimensional spatial differentiation, we demonstrate real-time, parallel, and analog convolutional processing of optical and biological specimens with challenging pepper-salt denoising and edge enhancement, which significantly enrich the toolkit of all-optical computing. Such meta-imager approach bridges multi-functionality and high-integration in all-optical convolutions, meanwhile possessing good architecture compatibility with digital convolutional neural networks.

Computing metasurfaces for all-optical image processing: a brief review

Computing metasurfaces are two-dimensional artificial nanostructures capable of performing mathematical operations on the input electromagnetic field, including its amplitude, phase, polarization, and frequency distributions. Rapid progress in the development of computing metasurfaces provide exceptional abilities for all-optical image processing, including the edge-enhanced imaging, which opens a broad range of novel and superior applications for real-time pattern recognition. In this paper, we review recent progress in the emerging field of computing metasurfaces for all-optical image processing, focusing on innovative and promising applications in optical analog operations, image processing, microscopy imaging, and quantum imaging.