Compact single-shot metalens depth sensors inspired by eyes of jumping spiders

Efficient Gradient-Based Metasurface Optimization toward the Limits of Wavelength-Polarization Multiplexing

Polarization and wavelength multiplexing are the two widely employed techniques to improve capacity in metasurfaces. While previous studies have pushed the channel numbers of each technique to its individual limits, achieving simultaneous limits of both techniques still presents challenges. Furthermore, current multiplexing methods often suffer from computational inefficiencies, hindering their applicability in computationally intensive tasks. In this work, we introduce and experimentally validate a gradient-based optimization algorithm using deep neural network (DNN) to achieve the limits of polarization and wavelength multiplexing with high computational efficiency. By leveraging the computational efficiency of the DNN-based method, we further implement nine multiplexed channels (three wavelengths × three polarizations) for large-scale image recognition tasks with a total of 36 classes in the single-layer metasurface. The classification accuracy reaches 96% in simulations and 91.5% in experiments. Our work sets a new benchmark for high-capacity multiplexing with gradient-based inverse design for advanced optical elements.

Baseline-free structured light 3D imaging using a metasurface double-helix dot projector

Structured light is a widely used 3D imaging method with a drawback that it typically requires a long baseline length between the laser projector and the camera sensor, which hinders its utilization in space-constrained scenarios. On the other hand, the application of passive 3D imaging methods, such as depth from depth-dependent point spread functions (PSFs), is impeded by the challenge in measuring textureless scenes. Here, we combine the advantages of both structured light and depth-dependent PSFs and propose a baseline-free structured light 3D imaging system. A metasurface is designed to project a structured dot array and encode depth information in the double-helix pattern of each dot simultaneously. Combined with a straightforward and fast algorithm, we demonstrate accurate 3D point cloud acquisition for various real-world scenarios including multiple cardboard boxes and a living human face. Such a technique may find application in a broad range of areas including consumer electronics and precision metrology.

Beating spectral bandwidth limits for large aperture broadband nano-optics

Flat optics have been proposed as an attractive approach for the implementation of new imaging and sensing modalities to replace and augment refractive optics. However, chromatic aberrations impose fundamental limitations on diffractive flat optics. As such, true broadband high-quality imaging has thus far been out of reach for fast f-numbers, large aperture, flat optics. In this work, we overcome intrinsic spectral bandwidth limitations, achieving broadband imaging in the visible wavelength range with a flat meta-optic, co-designed with computational reconstruction. We derive the necessary conditions for a broadband, 1 cm aperture, f/2 flat optic, with a diagonal field of view of 30° and average system MTF contrast of 20% or larger for a spatial frequency of 100 lp/mm in the visible band (>30% for <70 lp/mm). Finally, we use a coaxial, dual-aperture system to train the broadband imaging meta-optic with a learned reconstruction method operating on pair-wise captured imaging data. Fundamentally, our work challenges the entrenched belief of the inability of capturing high-quality, full-color images using a single large aperture meta-optic.

Polarization-controlled metasurface for simultaneous holographic display and three-dimensional depth perception

Simultaneous optical display and depth perception are crucial in many intelligent technologies but are usually realized by separate bulky systems unfriendly to integration. Metasurfaces, artificial two-dimensional optical surfaces with strong light-matter interaction capabilities at deep subwavelength scales, offer a promising approach for manufacturing highly integrated optical devices performing various complex functions. In this work, we report a polarization-multiplexed metasurface that can functionally switch between holographic display and Dammann gratings. By tailoring the incidence polarization, the metasurface can display high-quality holographic images in the Fresnel region or project a uniform spot cloud nearly covering the entire 180° × 180° transmissive space. For the latter, a projection and three-dimensional (3D) reconstruction experiment is conducted to elaborate the potential in retrieving 3D complex spatial information. The current results provide a prominent way to manufacture lightweight and highly-integrated comprehensive imaging systems especially vital for cutting-edge intelligent visual technologies.

Four-dimensional imaging based on a binocular chiral metalens

In this Letter, we present a binocular chiral metalens (BCM) device designed for four-dimensional (4D) imaging, which integrates both three-dimensional spatial perception and polarization detection. The BCM consists of two identical monocular metalenses that spatially separate left- and right-handed circularly polarized (LCP and RCP) light. When integrated with a commercial camera, the metalenses enable simultaneous measurement of the depth and polarization information. Numerical simulations and experimental results demonstrate that the BCM can achieve a circular polarization extinction ratio (CPER) of 29.2 dB and an average 3D reconstruction error of 4.09%. The proposed system paves a pathway for multi-dimensional imaging, with significant potential in applications in security, surveillance, and future advancements in more complex imaging tasks across other electromagnetic bands, including terahertz and infrared regimes.

Collision of high-resolution wide FOV metalens cameras and vision tasks

Metalenses, with their compact form factor and unique optical capabilities, hold tremendous potential for advancing computer vision applications. In this work, we propose a high-resolution, large field-of-view (FOV) metalens intelligent recognition system, combining the latest YOLO framework, aimed at supporting a range of vision tasks. Specifically, we demonstrate its effectiveness in scanning, pose recognition, and object classification. The metalens we designed to achieve a 100° FOV while operating near the diffraction limit, as confirmed by experimental results. Moreover, the metalenses weigh only 0.1 g and occupy a compact volume of 0.04 cm3, effectively addressing the bulkiness of conventional lenses and overcoming the limitations of traditional metalens in spatial frequency transmission. This work highlights the transformative potential of metalenses in the field of computer vision, The integration of metalenses with computer vision opens exciting possibilities for next-generation imaging systems, offering both enhanced functionality and unprecedented miniaturization.

The Road to Commercializing Optical Metasurfaces: Current Challenges and Future Directions

Optical metasurfaces, components composed of artificial nanostructures, are recognized for pushing boundaries of wavefront manipulation while maintaining a lightweight, compact design that surpasses conventional optics. Such advantages align with the current trends in optical systems, which demand compact communication devices and immersive holographic projectors, driving significant investment from the industry. Although interest in commercialization of optical metasurfaces has steadily grown since the initial breakthrough with diffraction-limited focusing, their practical applications have remained limited by challenges such as, massive-production yield, absence of standardized evaluation methods, and constrained design methodology. Here, this Perspective addresses the challenges in commercialization of optical metasurfaces, particularly focused on mass production, fabrication tolerance, performance evaluation, and integration into commercial systems. Additionally, we select the fields where metasurfaces may soon play significant roles and provide a perspective on their potentials. By addressing the challenges and exploring the solutions, this Perspective aims to foster discussions that will accelerate the utilization of optical metasurfaces and further build near-future metaphotonics platforms.

Filterless vector light field photodetector based on photonic-electronic co-designed non-Hermitian silicon nanostructures

Recent advances in near-field interference detection, inspired by the non-Hermitian coupling-induced directional sensing of Ormia ochracea, have demonstrated the potential of paired semiconductor nanowires for compact light field detection without optical filters. However, practical implementation faces significant challenges including limited active area, architectural scaling constraints, and incomplete characterization of angular and polarization information. Here, we demonstrate a filterless vector light field photodetector, leveraging the angle- and polarization-sensitive near-field interference of non-Hermitian semiconductor nanostructures. Our design unit comprises four devices, each containing identical silicon nanowires but varying in orientation and electric connection configuration, of which the four-dimensional photoconductive output can be uniquely mapped to key vector light field parameters: intensity, polar angle, azimuth angle, and the linear polarization difference (Stokes parameter, S 1). Optimization of the geometry and doping concentration of these optoelectronic nanostructures yields a theoretical polar angle detectivity of 4 × 10-5 °/Hz0.5. This work establishes a paradigm for multi-output photodetectors with full-rank response matrices for multi-dimensional light field characterization, paving the way for integrated vector light field sensing systems.

Colorimetric Thermography by a Long-Infrared Dual-Band Metalens

Infrared (IR) radiation thermography is extensively utilized in diverse fields due to its non-contact capability. Nevertheless, its effectiveness is often compromised by the significant emissivity variations among different objects, limiting its application to specific setups or focused object types. Colorimetric thermography is introduced as an alternative emissivity-independent method of radiation thermometry. This technique involves measuring radiance across two or more spectral bands and calculating the object's temperature based on the signal ratio, thereby mitigating emissivity effects under certain conditions. However, this method has the trade-off of necessitating bulky optical systems, complex filter imaging configurations, and sensor structures. To meet the requirements of IR thermography for compact structure, lightweight design, and customizability, a dual-band metalens is developed for the IR colorimetric thermography. The central wavelengths targeted are 9.5 and 12.5 µm. The dual-band IR imaging by the fabricated dual-band metalens is demonstrated, and the colorimetric thermography of low-emissivity objects is performed without presetting emissivity values. This approach significantly eliminates measurement errors associated with emissivity by an average of 50.16% across a temperature range of 60-180 °C. This innovation paves the way for dynamic and multi-target thermography using compact IR systems in complex environments.

A Review of Cascaded Metasurfaces for Advanced Integrated Devices

This paper reviews the field of cascaded metasurfaces, which are advanced optical devices formed by stacking or serially arranging multiple metasurface layers. These structures leverage near-field and far-field electromagnetic (EM) coupling mechanisms to enhance functionalities beyond single-layer metasurfaces. This review comprehensively discusses the physical principles, design methodologies, and applications of cascaded metasurfaces, focusing on both static and dynamic configurations. Near-field-coupled structures create new resonant modes through strong EM interactions, allowing for efficient control of light properties like phase, polarization, and wave propagation. Far-field coupling, achieved through greater interlayer spacing, enables traditional optical methods for design, expanding applications to aberration correction, spectrometers, and retroreflectors. Dynamic configurations include tunable devices that adjust their optical characteristics through mechanical motion, making them valuable for applications in beam steering, varifocal lenses, and holography. This paper concludes with insights into the potential of cascaded metasurfaces to create multifunctional, compact optical systems, setting the stage for future innovations in miniaturized and integrated optical devices.

Inspiration from Visual Ecology for Advancing Multifunctional Robotic Vision Systems: Bio-inspired Electronic Eyes and Neuromorphic Image Sensors

In robotics, particularly for autonomous navigation and human-robot collaboration, the significance of unconventional imaging techniques and efficient data processing capabilities is paramount. The unstructured environments encountered by robots, coupled with complex missions assigned to them, present numerous challenges necessitating diverse visual functionalities, and consequently, the development of multifunctional robotic vision systems has become indispensable. Meanwhile, rich diversity inherent in animal vision systems, honed over evolutionary epochs to meet their survival demands across varied habitats, serves as a profound source of inspirations. Here, recent advancements in multifunctional robotic vision systems drawing inspiration from natural ocular structures and their visual perception mechanisms are delineated. First, unique imaging functionalities of natural eyes across terrestrial, aerial, and aquatic habitats and visual signal processing mechanism of humans are explored. Then, designs and functionalities of bio-inspired electronic eyes are explored, engineered to mimic key components and underlying optical principles of natural eyes. Furthermore, neuromorphic image sensors are discussed, emulating functional properties of synapses, neurons, and retinas and thereby enhancing accuracy and efficiency of robotic vision tasks. Next, integration examples of electronic eyes with mobile robotic/biological systems are introduced. Finally, a forward-looking outlook on the development of bio-inspired electronic eyes and neuromorphic image sensors is provided.

Large-scale fabrication of meta-axicon with circular polarization on CMOS platform

Metasurfaces, consisting of arrays of subwavelength structures, are lightweight and compact while being capable of implementing the functions of traditional bulky optical components. Furthermore, they have the potential to significantly improve complex optical systems in terms of space and cost, as they can simultaneously implement multiple functions. The wafer-scale mass production method based on the CMOS (complementary metal oxide semiconductor) process plays a crucial role in the modern semiconductor industry. This approach can also be applied to the production of metasurfaces, thereby accelerating the entry of metasurfaces into industrial applications. In this study, we demonstrated the mass production of large-area meta-axicons with a diameter of 2 mm on an 8-inch wafer using DUV (Deep Ultraviolet) photolithography. The proposed meta-axicon designed here is based on PB (Pancharatnam-Berry) phase and is engineered to simultaneously modulate the phase and polarization of light. In practice, the fabricated meta-axicon generated a circularly polarized Bessel beam with a depth of focus (DoF) of approximately 2.3 mm in the vicinity of 980 nm. We anticipate that the mass production of large-area meta-axicons on this CMOS platform can offer various advantages in optical communication, laser drilling, optical trapping, and tweezing applications.

Stereo imaging inspired by bionic optics

Stereo imaging has been a focal point in fields such as robotics and autonomous driving. This Letter discusses the imaging mechanisms of jumping spiders and human eyes from a biomimetic perspective and proposes a monocular stereo imaging solution with low computational cost and high stability. The stereo imaging mechanism of jumping spiders enables monocular imaging without relying on multiple viewpoints, thus avoiding complex large-scale feature point matching and significantly conserving computational resources. The foveal imaging mechanism of the human eye allows for complex imaging tasks to be completed only on the locally interested regions, resulting in more efficient execution of various visual tasks. By combining these two advantages, we have developed a more computationally efficient monocular stereo imaging method that can achieve stereo imaging on only the locally interested regions without sacrificing the performance of wide field-of-view (FOV) imaging. Finally, through experimental validation, we demonstrate that the method proposed in this Letter exhibits excellent stereo imaging performance.

Highly efficient multifunctional metasurface integrating lens, prism, and wave plate

The miniaturization of optical systems is crucial for various applications, including compact augmented reality/virtual reality devices, microelectromechanical system sensors, ranging technologies, and microfabricated atomic clocks. However, reliance on bulky discrete optical elements has been a significant obstacle to achieving this miniaturization. This work introduces a highly efficient multifunctional metasurface (MFMS) that seamlessly integrates a lens, prism, and quarter-wave plate (QWP). This innovation allows simultaneous collimation, beam deflection, and polarization conversion within a singular thin element. Specifically, for the prism-QWP bifunctional integration, we achieved a high diffraction efficiency of 72.8% and a degree of circular polarization of -0.955 under exposure to linearly polarized light at a wavelength of 795 nm, proving its potential for ultracompact atomic clock applications. Moreover, the lens-prism-QWP trifunctional integration successfully showed diffraction-limited focusing performance with a numerical aperture of 0.4, which was sufficient to collimate a beam with a divergence angle of 20 ∘, corresponding to the light emitted from a standard vertical-cavity surface-emitting laser.

MetaHDR: single shot high-dynamic range imaging and sensing using a multifunctional metasurface

We present MetaHDR, which is a single-shot high-dynamic range (HDR) imaging and sensing system using a multifunctional metasurface. The metasurface is capable of splitting an incident beam into multiple focusing beams with different amounts of power, simultaneously forming multiple low dynamic range (LDR) images with distinct irradiance on a photosensor. Then, the LDR images are jointly processed using a gradient-based HDR fusion algorithm, which is shown to be effective in attenuating the residual light artifacts incurred by the metasurface and the lens flare. MetaHDR achieves single-shot HDR photography and videography that increases the dynamic range by at least 50 dB compared to the original dynamic range of the photosensor. It can also perform single-shot HDR sensing, including reflectance calibration and surface curvature estimation of reflective materials. MetaHDR's demonstrated functionalities could be broadly applied in surveillance and security, microscopic imaging, advanced manufacturing, etc.

Multidimensional vision sensors for information processing

The visual scene in the physical world integrates multidimensional information (spatial, temporal, polarization, spectrum and so on) and typically shows unstructured characteristics. Conventional image sensors cannot process this multidimensional vision data, creating a need for vision sensors that can efficiently extract features from substantial multidimensional vision data. Vision sensors are able to transform the unstructured visual scene into featured information without relying on sophisticated algorithms and complex hardware. The response characteristics of sensors can be abstracted into operators with specific functionalities, allowing for the efficient processing of perceptual information. In this Review, we delve into the hardware implementation of multidimensional vision sensors, exploring their working mechanisms and design principles. We exemplify multidimensional vision sensors built on emerging devices and silicon-based system integration. We further provide benchmarking metrics for multidimensional vision sensors and conclude with the principle of device-system co-design and co-optimization.

A method to efficiently and rapidly approximate the vectorial fields generated by large area metasurfaces

In order to calculate the electromagnetic fields that are produced after light passes through a metasurface, simulation methods such as the Finite-Difference Time-Domain method are often employed. While these provide a good approximation to the fields, the level of detail at which the volume of space that the light is propagating in needs to be modelled and the time for which simulations need to run, mean that as the area of the metasurface is increased these simulations rapidly become unwieldy. In this paper we show how the result of a FDTD simulation of a unit cell can be used to generate a good approximation of the vectorial field that large area metasurfaces will generate, but using a fraction of the computational resources. This approach can provide an intermediate design step, allowing potentially interesting designs to be rapidly identified or discarded.

Full-Color Imaging System Based on the Joint Integration of a Metalens and Neural Network

Lenses have been a cornerstone of optical systems for centuries; however, they are inherently limited by the laws of physics, particularly in terms of size and weight. Because of their characteristic light weight, small size, and subwavelength modulation, metalenses have the potential to miniaturize and integrate imaging systems. However, metalenses still face the problem that chromatic aberration affects the clarity and accuracy of images. A high-quality image system based on the end-to-end joint optimization of a neural network and an achromatic metalens is demonstrated in this paper. In the multi-scale encoder-decoder network, both the phase characteristics of the metalens and the hyperparameters of the neural network are optimized to obtain high-resolution images. The average peak-signal-to-noise ratio (PSNR) and average structure similarity (SSIM) of the recovered images reach 28.53 and 0.83. This method enables full-color and high-performance imaging in the visible band. Our approach holds promise for a wide range of applications, including medical imaging, remote sensing, and consumer electronics.

Advanced visual components inspired by animal eyes

Artificial vision systems pervade our daily lives as a foremost sensing apparatus in various digital technologies, from smartphones to autonomous cars and robotics. The broad range of applications for conventional vision systems requires facile adaptation under extreme and dynamic visual environments. However, these current needs have complicated individual visual components for high-quality image acquisition and processing, which indeed leads to a decline in efficiency in the overall system. Here, we review recent advancements in visual components for high-performance visual processing based on strategies of biological eyes that execute diverse imaging functionalities and sophisticated visual processes with simple and concise ocular structures. This review first covers the structures and functions of biological eyes (i.e., single-lens eyes and compound eyes), which contain micro-optic components and nanophotonic structures. After that, we focus on their inspirations in imaging optics/photonics, light-trapping and filtering components, and retinomorphic devices. We discuss the remaining challenges and notable biological structures waiting to be implemented.

Polarization-separating Alvarez metalens

The rapid advancements in optical communication technologies have highlighted traditional optical components' limitations, particularly in size, adaptability, and integration capabilities, underscoring the need for more compact and versatile solutions. Metalenses offer a promising pathway to address these challenges, with their ability to provide high-functionality, miniaturized optical components. We developed a varifocal metalens with a polarization separation function designed for the wavelength of 1550 nm for potential application for next-generation communication technologies. To integrate the varifocal and polarization separation functions, polarization-dependent phase profiles for an off-axis Alvarez lens were derived and encoded by amorphous silicon pillar meta-atoms with rectangular cross sections to provide independent 0-2π phase delays for both orthogonal linear polarization components. The fabricated metalens achieved a varifocal range of 0.75 mm to 10.65 mm and a polarization extinction ratio of 18.5 dB.

Metasurface- and PCSEL-Based Structured Light for Monocular Depth Perception and Facial Recognition

The novel depth-sensing system presented here revolutionizes structured light (SL) technology by employing metasurfaces and photonic crystal surface-emitting lasers (PCSELs) for efficient facial recognition in monocular depth-sensing. Unlike conventional dot projectors relying on diffractive optical elements (DOEs) and collimators, our system projects approximately 45,700 infrared dots from a compact 297-μm-dimention metasurface, drastically more spots (1.43 times) and smaller (233 times) than the DOE-based dot projector in an iPhone. With a measured field-of-view (FOV) of 158° and a 0.611° dot sampling angle, the system is lens-free and lightweight and boasts lower power consumption than vertical-cavity surface-emitting laser (VCSEL) arrays, resulting in a 5-10 times reduction in power. Utilizing a GaAs-based metasurface and a simplified optical architecture, this innovation not only addresses the drawbacks of traditional SL depth-sensing but also opens avenues for compact integration into wearable devices, offering remarkable advantages in size, power efficiency, and potential for widespread adoption.

Circular-target-style bifocal zoom metalens

Optical zoom plays an important role in realizing high-quality image magnification, especially in photography, telescopes, microscopes, etc. Compared to traditional bulky zoom lenses, the high versatility and flexibility of metalens design provide opportunities for modern electronic and photonic systems with demands for miniature and lightweight optical zoom. Here, we propose an ultra-thin, lightweight and compact bifocal zoom metalens, which consists of a conventional circular sub-aperture and a sparse annular sub-aperture with different focal lengths. The imaging resolutions of such single zoom metalens with 164 lp/mm and 117 lp/mm at magnifications of 1× and 2× have been numerically and experimentally demonstrated, respectively. Furthermore, clear zoom images of a dragonfly wing pattern have been also achieved using this zoom metalens, showing its distinctive aspect in biological imaging. Our results provide an approach for potential applications in integrated optical systems, miniaturized imaging devices, and wearable devices.

Realizing depth measurement and edge detection based on a single metasurface

How to simultaneously obtain the depth, edge, and other light information of the scene to accurately perceive the physical world is an important issue for imaging systems. However, such tasks usually require bulky optical components and active illumination methods. Here, we design and experimentally validate a single geometric metasurface that can achieve depth measurement or edge detection under incoherent or coherent light respectively. Double helix point source function is utilized, and three verification experiments are carried out, including double-helix beam calibration, 2D object and 3D object detection, respectively. Additionally, two-dimensional edge detection can also be achieved. This compact imaging system can enable new applications in various fields, from machine vision to microscopy.

A Meta-Device for Intelligent Depth Perception

The optical illusion affects depth-sensing due to the limited and specific light-field information acquired by single-lens imaging. The incomplete depth information or visual deception would cause cognitive errors. To resolve this problem, an intelligent and compact depth-sensing meta-device that is miniaturized, integrated, and applicable for diverse scenes in all light levels is demonstrated. The compact and multifunction stereo vision system adopts an array with 3600 achromatic meta-lenses and a size of 1.2 × 1.2 mm2 to measure the depth over a 30 cm range with deep-learning support. The meta-lens array can act as multiple imaging lenses to collect light field information. It can also work with a light source as an active optical device to project a structured light. The meta-lens array can serve as the core functional component of a light-field imaging system under bright conditions or a structured-light projection system in the dark. The depth information in both ways can be analyzed and extracted by the convolutional neural network. This work provides a new avenue for the applications such as autonomous driving, machine vision, human-computer interaction, augmented reality, biometric identification, etc.

Monocular depth sensing using metalens

3-D depth sensing is essential for many applications ranging from consumer electronics to robotics. Passive depth sensing techniques based on a double-helix (DH) point-spread-function (PSF) feature high depth estimation precision, minimal power consumption, and reduced system complexity compared to active sensing methods. Here, we propose and experimentally implemented a polarization-multiplexed DH metalens designed using an autonomous direct search algorithm, which utilizes two contra-rotating DH PSFs encoded in orthogonal polarization states to enable monocular depth perception. Using a reconstruction algorithm that we developed, concurrent depth calculation and scene reconstruction with minimum distortion and high resolution in all three dimensions were demonstrated.

Dispersion-engineered metasurfaces reaching broadband 90% relative diffraction efficiency

Dispersion results from the variation of index of refraction as well as electric field confinement in sub-wavelength structures. It usually results in efficiency decrease in metasurface components leading to troublesome scattering into unwanted directions. In this letter, by dispersion engineering, we report a set of eight nanostructures whose dispersion properties are nearly identical to each other while being capable of providing 0 to 2π full-phase coverage. Our nanostructure set enables broadband and polarization-insensitive metasurface components reaching 90% relative diffraction efficiency (normalized to the power of transmitted light) from 450 nm to 700 nm in wavelength. Relative diffraction efficiency is important at a system level - in addition to diffraction efficiency (normalized to the power of incident light) - as it considers only the transmitted optical power that can affect the signal to noise ratio. We first illustrate our design principle by a chromatic dispersion-engineered metasurface grating, then show that other metasurface components such as chromatic metalenses can also be implemented by the same set of nanostructures with significantly improved relative diffraction efficiency.

Active 3D positioning and imaging modulated by single fringe projection with compact metasurface device

Three-dimensional (3D) information is vital for providing detailed features of the physical world, which is used in numerous applications such as industrial inspection, automatic navigation and identity authentication. However, the implementations of 3D imagers always rely on bulky optics. Metasurfaces, as the next-generation optics, shows flexible modulation abilities and excellent performance combined with computer vision algorithm. Here, we demonstrate an active 3D positioning and imaging method with large field of view (FOV) by single fringe projection based on metasurface and solve the accurate and robust calibration problem with the depth uncertainty of 4 μm. With a compact metasurface projector, the demonstrated method can achieve submillimeter positioning accuracy under the FOV of 88°, offering robust and fast 3D reconstruction of the texture-less scene due to the modulation characteristic of the fringe. Such scheme may accelerate prosperous engineering applications with the continued growth of flat-optics manufacturing process by using metadevices.

Monocular metasurface camera for passive single-shot 4D imaging

It is a grand challenge for an imaging system to simultaneously obtain multi-dimensional light field information, such as depth and polarization, of a scene for the accurate perception of the physical world. However, such a task would conventionally require bulky optical components, time-domain multiplexing, and active laser illumination. Here, we experimentally demonstrate a compact monocular camera equipped with a single-layer metalens that can capture a 4D image, including 2D all-in-focus intensity, depth, and polarization of a target scene in a single shot under ambient illumination conditions. The metalens is optimized to have a conjugate pair of polarization-decoupled rotating single-helix point-spread functions that are strongly dependent on the depth of the target object. Combined with a straightforward, physically interpretable image retrieval algorithm, the camera can simultaneously perform high-accuracy depth sensing and high-fidelity polarization imaging over an extended depth of field for both static and dynamic scenes in both indoor and outdoor environments. Such a compact multi-dimensional imaging system could enable new applications in diverse areas ranging from machine vision to microscopy.

New Approaches to 3D Vision

New approaches to 3D vision are enabling new advances in artificial intelligence and autonomous vehicles, a better understanding of how animals navigate the 3D world, and new insights into human perception in virtual and augmented reality. Whilst traditional approaches to 3D vision in computer vision (SLAM: simultaneous localization and mapping), animal navigation (cognitive maps), and human vision (optimal cue integration) start from the assumption that the aim of 3D vision is to provide an accurate 3D model of the world, the new approaches to 3D vision explored in this issue challenge this assumption. Instead, they investigate the possibility that computer vision, animal navigation, and human vision can rely on partial or distorted models or no model at all. This issue also highlights the implications for artificial intelligence, autonomous vehicles, human perception in virtual and augmented reality, and the treatment of visual disorders, all of which are explored by individual articles. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

Single-shot 3D imaging with point cloud projection based on metadevice

Three-dimensional (3D) imaging is a crucial information acquisition technology for light detection, autonomous vehicles, gesture recognition, machine vision, and other applications. Metasurface, as a subwavelength scale two-dimensional array, offers flexible control of optical wavefront owing to abundant design freedom. Metasurfaces are promising for use as optical devices because they have large field of view and powerful functionality. In this study, we propose a flat optical device based on a single-layer metasurface to project a coded point cloud in the Fourier space and explore a sophisticated matching algorithm to achieve 3D reconstruction, offering a complete technical roadmap for single-shot detection. We experimentally demonstrate that the depth accuracy of our system is smaller than 0.24 mm at a measurement distance of 300 mm, indicating the feasibility of the submillimetre measurement platform. Our method can pave the way for practical applications such as surface shape detection, gesture recognition, and personal authentication.

Quantitative phase contrast imaging with a nonlocal angle-selective metasurface

Phase contrast microscopy has played a central role in the development of modern biology, geology, and nanotechnology. It can visualize the structure of translucent objects that remains hidden in regular optical microscopes. The optical layout of a phase contrast microscope is based on a 4 f image processing setup and has essentially remained unchanged since its invention by Zernike in the early 1930s. Here, we propose a conceptually new approach to phase contrast imaging that harnesses the non-local optical response of a guided-mode-resonator metasurface. We highlight its benefits and demonstrate the imaging of various phase objects, including biological cells, polymeric nanostructures, and transparent metasurfaces. Our results showcase that the addition of this non-local metasurface to a conventional microscope enables quantitative phase contrast imaging with a 0.02π phase accuracy. At a high level, this work adds to the growing body of research aimed at the use of metasurfaces for analog optical computing.

Metasurface-driven full-space structured light for three-dimensional imaging

Structured light (SL)-based depth-sensing technology illuminates the objects with an array of dots, and backscattered light is monitored to extract three-dimensional information. Conventionally, diffractive optical elements have been used to form laser dot array, however, the field-of-view (FOV) and diffraction efficiency are limited due to their micron-scale pixel size. Here, we propose a metasurface-enhanced SL-based depth-sensing platform that scatters high-density ~10 K dot array over the 180° FOV by manipulating light at subwavelength-scale. As a proof-of-concept, we place face masks one on the beam axis and the other 50° apart from axis within distance of 1 m and estimate the depth information using a stereo matching algorithm. Furthermore, we demonstrate the replication of the metasurface using the nanoparticle-embedded-resin (nano-PER) imprinting method which enables high-throughput manufacturing of the metasurfaces on any arbitrary substrates. Such a full-space diffractive metasurface may afford ultra-compact depth perception platform for face recognition and automotive robot vision applications.

Arbitrary Jones matrix on-demand design in metasurfaces using multiple meta-atoms

Super cells or multi-layer metasurfaces are used to realize various multi-functional and exotic functional devices. In such methods, the design space expands exponentially as more variable parameters are introduced; however, this will necessitate huge computational effort without special treatment. The function of a metasurface can be described mathematically by using a Jones matrix. When the gap between adjacent atoms is sufficiently large, the overall Jones matrix of a 3D lattice which is composed of multiple meta-atoms can be obtained by adding or multiplying each meta-atom's Jones matrix for a parallel or cascaded arrangement, respectively. Reversely, an arbitrary Jones matrix can be decomposed to achieve a combination of diagonal and rotation matrices. This means that the devices with various functions can be constructed by combining, cascading, and rotating a kind of atom, and thus the computation requirements will be reduced significantly. In this work, the feasibility of this approach is demonstrated with two cases, circular polarization selective transmission and resemble optical activity. Both the simulation and experiment are consistent with the hypothesis. This method can manipulate all degrees of freedom in a Jones matrix and reduce design complexity and may find applications to extend the scope of meta-optics.

Thermally reconfigurable metalens

Reconfigurable metalenses are compact optical components composed by arrays of meta-atoms that offer unique opportunities for advanced optical systems, from microscopy to augmented reality platforms. Although poorly explored in the context of reconfigurable metalenses, thermo-optical effects in resonant silicon nanoresonators have recently emerged as a viable strategy to realize tunable meta-atoms. In this work, we report the proof-of-concept design of an ultrathin (300 nm thick) and thermo-optically reconfigurable silicon metalens operating at a fixed, visible wavelength (632 nm). Importantly, we demonstrate continuous, linear modulation of the focal-length up to 21% (from 165 μm at 20 °C to 135 μm at 260 °C). Operating under right-circularly polarized light, our metalens exhibits an average conversion efficiency of 26%, close to mechanically modulated devices, and has a diffraction-limited performance. Overall, we envision that, combined with machine-learning algorithms for further optimization of the meta-atoms, thermally reconfigurable metalenses with improved performance will be possible. Also, the generality of this approach could offer inspiration for the realization of active metasurfaces with other emerging materials within field of thermo-nanophotonics.

Toward a universal metasurface for optical imaging, communication, and computation

In recent years, active metasurfaces have emerged as a reconfigurable nanophotonic platform for the manipulation of light. Here, application of an external stimulus to resonant subwavelength scatterers enables dynamic control over the wavefront of reflected or transmitted light. In principle, active metasurfaces are capable of controlling key characteristic properties of an electromagnetic wave, such as its amplitude, phase, polarization, spectrum, and momentum. A 'universal' active metasurface should be able to provide independent and continuous control over all characteristic properties of light for deterministic wavefront shaping. In this article, we discuss strategies for the realization of this goal. Specifically, we describe approaches for high performance active metasurfaces, examine pathways for achieving two-dimensional control architectures, and discuss operating configurations for optical imaging, communication, and computation applications based on a universal active metasurface.

Robust Achromatic All-Dielectric Metalens for Infrared Detection in Intelligent Inspection

Metalens has the advantages of high design freedom, light weight and easy integration, thus provides a powerful platform for infrared detection. Here, we numerically demonstrated a broadband achromatic infrared all-dielectric metalens over a continuous 800 nm bandwidth, with strong environmental adaptability in air, water and oil. By building a database with multiple 2π phase coverage and anomalous dispersions, optimizing the corrected required phase profiles and designing the sizes and spatial distributions of silicon nanopillars, we numerically realized the design of broadband achromatic metalens. The simulation results of the designed metalens show nearly constant focal lengths and diffraction-limited focal spots over the continuous range of wavelengths from 4.0 to 4.8 μm, indicating the ability of the designed metalens to detect thermal signals over a temperature range from various fault points. Further simulation results show that the metalens maintains good focusing performance under the environment of water or oil. This work may facilitate the application of metalens in ultra-compact infrared detectors for power grid faults detection.

End-to-end metasurface inverse design for single-shot multi-channel imaging

We introduce end-to-end inverse design for multi-channel imaging, in which a nanophotonic frontend is optimized in conjunction with an image-processing backend to extract depth, spectral and polarization channels from a single monochrome image. Unlike diffractive optics, we show that subwavelength-scale "metasurface" designs can easily distinguish similar wavelength and polarization inputs. The proposed technique integrates a single-layer metasurface frontend with an efficient Tikhonov reconstruction backend, without any additional optics except a grayscale sensor. Our method yields multi-channel imaging by spontaneous demultiplexing: the metaoptics front-end separates different channels into distinct spatial domains whose locations on the sensor are optimally discovered by the inverse-design algorithm. We present large-area metasurface designs, compatible with standard lithography, for multi-spectral imaging, depth-spectral imaging, and "all-in-one" spectro-polarimetric-depth imaging with robust reconstruction performance (≲ 10% error with 1% detector noise). In contrast to neural networks, our framework is physically interpretable and does not require large training sets. It can be used to reconstruct arbitrary three-dimensional scenes with full multi-wavelength spectra and polarization textures.

Meta-optic accelerators for object classifiers

Rapid advances in deep learning have led to paradigm shifts in a number of fields, from medical image analysis to autonomous systems. These advances, however, have resulted in digital neural networks with large computational requirements, resulting in high energy consumption and limitations in real-time decision-making when computation resources are limited. Here, we demonstrate a meta-optic-based neural network accelerator that can off-load computationally expensive convolution operations into high-speed and low-power optics. In this architecture, metasurfaces enable both spatial multiplexing and additional information channels, such as polarization, in object classification. End-to-end design is used to co-optimize the optical and digital systems, resulting in a robust classifier that achieves 93.1% accurate classification of handwriting digits and 93.8% accuracy in classifying both the digit and its polarization state. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine vision and artificial intelligence.

Dielectric metalens for miniaturized imaging systems: progress and challenges

Lightweight, miniaturized optical imaging systems are vastly anticipated in these fields of aerospace exploration, industrial vision, consumer electronics, and medical imaging. However, conventional optical techniques are intricate to downscale as refractive lenses mostly rely on phase accumulation. Metalens, composed of subwavelength nanostructures that locally control light waves, offers a disruptive path for small-scale imaging systems. Recent advances in the design and nanofabrication of dielectric metalenses have led to some high-performance practical optical systems. This review outlines the exciting developments in the aforementioned area whilst highlighting the challenges of using dielectric metalenses to replace conventional optics in miniature optical systems. After a brief introduction to the fundamental physics of dielectric metalenses, the progress and challenges in terms of the typical performances are introduced. The supplementary discussion on the common challenges hindering further development is also presented, including the limitations of the conventional design methods, difficulties in scaling up, and device integration. Furthermore, the potential approaches to address the existing challenges are also deliberated.

Pixel-level Bayer-type colour router based on metasurfaces

The three primary colour model, i.e., red, green, and blue model, based on the colour perception of the human eye, has been widely used in colour imaging. The most common approach for obtaining colour information is to use a Bayer colour filter, which filters colour light with four pixels of an imaging sensor to form an effective colour pixel. However, its energy utilization efficiency and colour collection efficiency are limited to a low level due to the three-channel filtering nature. Here, by employing an inverse-design method, we demonstrate a pixel-level metasurface-based Bayer-type colour router that presents peak colour collection efficiencies of 58%, 59%, and 49% for red, green and blue light, and an average energy utilization efficiency as high as 84% over the visible region (400 nm–700 nm), which is twice as high as that of a commercial Bayer colour filter. Furthermore, by using a 200 µm × 200 µm metasurface-based colour router sample working with a monochromatic imaging sensor, colour imaging is further realized, obtaining an image intensity twice that achieved by a commercial Bayer colour filter. Our work innovates the mechanism of high-efficiency spectrum information acquisition, which is expected to have promising applications in the development of next-generation imaging systems.

Though metasurface-based 3D colour routers are attractive for next-generation colour imaging systems, their complex fabrication and poor performance remain a bottleneck. Here, the authors use the inverse-design method to realize high performance metasurface-based Bayer-type colour routers.

Inverse design enables large-scale high-performance meta-optics reshaping virtual reality

Meta-optics has achieved major breakthroughs in the past decade; however, conventional forward design faces challenges as functionality complexity and device size scale up. Inverse design aims at optimizing meta-optics design but has been currently limited by expensive brute-force numerical solvers to small devices, which are also difficult to realize experimentally. Here, we present a general inverse-design framework for aperiodic large-scale (20k × 20k λ2) complex meta-optics in three dimensions, which alleviates computational cost for both simulation and optimization via a fast approximate solver and an adjoint method, respectively. Our framework naturally accounts for fabrication constraints via a surrogate model. In experiments, we demonstrate aberration-corrected metalenses working in the visible with high numerical aperture, poly-chromatic focusing, and large diameter up to the centimeter scale. Such large-scale meta-optics opens a new paradigm for applications, and we demonstrate its potential for future virtual-reality platforms by using a meta-eyepiece and a laser back-illuminated micro-Liquid Crystal Display.

Trilobite-inspired neural nanophotonic light-field camera with extreme depth-of-field

A unique bifocal compound eye visual system found in the now extinct trilobite, Dalmanitina socialis, may enable them to be sensitive to the light-field information and simultaneously perceive both close and distant objects in the environment. Here, inspired by the optical structure of their eyes, we demonstrate a nanophotonic light-field camera incorporating a spin-multiplexed bifocal metalens array capable of capturing high-resolution light-field images over a record depth-of-field ranging from centimeter to kilometer scale, simultaneously enabling macro and telephoto modes in a snapshot imaging. By leveraging a multi-scale convolutional neural network-based reconstruction algorithm, optical aberrations induced by the metalens are eliminated, thereby significantly relaxing the design and performance limitations on metasurface optics. The elegant integration of nanophotonic technology with computational photography achieved here is expected to aid development of future high-performance imaging systems.

Inspired by the optical structure of bifocal compound eyes, the authors demonstrate a nanophotonic light-field camera with large depth of field. By using a spin-multiplexed bifocal metalens array and neural network-based reconstruction, they capture high-resolution images at centimeter to kilometer scale.

Polarization coincidence images from metasurfaces with HOM-type interference

Metasurfaces provide a promising route for structuring light and generating holograms with designed amplitude, phase, and polarization profiles, leading to a versatile platform for integrating and constructing optical components beyond the conventional ones. At the same time, incorporating coincidence in imaging allows a high signal-to-noise ratio for imaging in very low light levels. As beneficial from the recent development in both metasurfaces and single-photon avalanche diode (SPAD) cameras, we combine the polarization-sensitive capability of metasurfaces with Hong-Ou-Mandel (HOM)-type interference in generating images with tailor-made two-photon interference and polarization coincidence signatures. By using orthogonal linear-polarized photons as incidence, correlated, anticorrelated, and uncorrelated polarization coincidence features can be observed within the same image from the pairwise second-order coherence statistics across different pixels of the image. Our work adds polarization to the demonstrated amplitude and phase sensitivity in the domain of "HOM microscopy" and can be useful for biological and security applications.

Reconfigurable Metalens with Phase-Change Switching between Beam Acceleration and Rotation for 3D Depth Imaging

Depth imaging is very important for many emerging technologies, such as artificial intelligence, driverless vehicles and facial recognition. However, all these applications demand compact and low-power systems that are beyond the capabilities of most state-of-art depth cameras. Recently, metasurface-based depth imaging that exploits point spread function (PSF) engineering has been demonstrated to be miniaturized and single shot without requiring active illumination or multiple viewpoint exposures. A pair of spatially adjacent metalenses with an extended depth-of-field (EDOF) PSF and a depth-sensitive double-helix PSF (DH-PSF) were used, using the former metalens to reconstruct clear images of each depth and the latter to accurately estimate depth. However, due to these two metalenses being non-coaxial, parallax in capturing scenes is inevitable, which would limit the depth precision and field of view. In this work, a bifunctional reconfigurable metalens for 3D depth imaging was proposed by dynamically switching between EDOF-PSF and DH-PSF. Specifically, a polarization-independent metalens working at 1550 nm with a compact 1 mm2 aperture was realized, which can generate a focused accelerating beam and a focused rotating beam at the phase transition of crystalline and amorphous Ge2Sb2Te5 (GST), respectively. Combined with the deconvolution algorithm, we demonstrated the good capabilities of scene reconstruction and depth imaging using a theoretical simulation and achieved a depth measurement error of only 3.42%.

Varifocal diffractive lenses for multi-depth microscope imaging

Flat optical elements enable the realization of ultra-thin devices able to either reproduce or overcome the functionalities of standard bulky components. The fabrication of these elements involves the structuration of material surfaces on the light wavelength scale, whose geometry has to be carefully designed to achieve the desired optical functionality. In addition to the limits imposed by lithographic design-performance compromises, their optical behavior cannot be accurately tuned afterward, making them difficult to integrate in dynamic optical systems. Here we show the realization of fully reconfigurable flat varifocal diffractive lens, which can be in-place realized, erased and reshaped directly on the surface of an azopolymer film by an all-optical holographic process. Integrating the lens in the same optical system used as standard refractive microscope, results in a hybrid microscope capable of multi-depth object imaging. Our approach demonstrates that reshapable flat optics can be a valid choice to integrate, or even substitute, modern optical systems for advanced functionalities.

Ultracompact Nanophotonics: Light Emission and Manipulation with Metasurfaces

Internet of Things (IoT) technology is prosperous for the betterment of human well-being. With the expeditious needs of miniature functional devices and systems for adaptive optics and light manipulation at will, relevant sensing techniques are thus in the urgent stage of development. Extensive developments in ultrathin artificial structures, namely metasurfaces, are paving the way for the next-generation devices. A bunch of tunable and reconfigurable metasurfaces with diversified catalogs of mechanisms have been developed recently, enabling dynamic light modulation on demand. On the other hand, monolithic integration of metasurfaces and light-emitting sources form ultracompact meta-devices as well as exhibiting desired functionalities. Photon-matter interaction provides revolution in more compact meta-devices, manipulating light directly at the source. This study presents an outlook on this merging paradigm for ultracompact nanophotonics with metasurfaces, also known as metaphotonics. Recent advances in the field hold great promise for the novel photonic devices with light emission and manipulation in simplicity.

Rotational varifocal moiré metalens made of single-crystal silicon meta-atoms for visible wavelengths

Metasurface lenses (metalenses) offer an ultrathin and simple optical system with dynamic functions that include focal length tuning. In this study, a rotational varifocal (i.e., moiré) metalens based on octagonal single-crystal silicon pillars was designed and fabricated to realize a high transmittance, whole 2π phase coverage, and polarization insensitivity for visible wavelengths. The moiré metalens consists of a pair of cascaded metasurface-based phase lattices and the focal length can be adjusted from negative to positive by mutual rotation. The fabricated moiré metalens demonstrated a focal length that can be tuned from -36 mm to -2 mm and from 2 to 12 mm by mutual rotation from -90° to 90°, and the experimental measurements agreed well with theoretical values at the design wavelength of 633 nm. Imaging was demonstrated at three distinct wavelengths of 633, 532, and 440 nm.

Recent advances of wide-angle metalenses: principle, design, and applications

Optical imaging systems, like microscopes, cameras, and telescopes, continue to expand the scope of human observation of the world. As one of the key indicators of imaging systems, the field-of-view (FOV) is often limited by coma aberration. Expanding it generally relies on a combination of complex lenses, leading to a bulky and cumbersome system. Recently, the emergency of meta-optics provides an alternative to constructing compact and lightweight large-FOV metalens through elaborated phase modulation within a flat surface, showing great potential in surveillance, unmanned vehicles, onboard planes or satellites, medical science, and other new applications. In this article, we review recent advances of wide-angle metalenses, including operation principles, design strategies, and application demos. Firstly, basic principles of wide-angle imaging using a single metalens are interpreted. Secondly, some advanced methods for designing subwavelength structures with high angle robustness and high efficiency are discussed. Thirdly, some representative functional devices and applications are surveyed. Finally, we conclude with an outlook on future potentials and challenges that need to be overcome.

Spectrally Gated Microscopy (SGM) with Meta Optics for Parallel Three-Dimensional Imaging

Volumetric imaging with high spatiotemporal resolution is of utmost importance for various applications ranging from aerospace and defense to real-time imaging of dynamic biological processes. To facilitate three-dimensional sectioning, current technology relies on mechanisms to reject light from adjacent out-of-focus planes either spatially or by other means. Yet, the combination of rapid acquisition time and high axial resolution is still elusive, motivating a persistent pursuit for emerging imaging approaches. Here we introduce and experimentally demonstrate a concept named spectrally gated microscopy (SGM), which enables a single-shot interrogation over the full axial dimension while maintaining a submicron sectioning resolution. SGM utilizes two important features enabled by flat optics (i.e., metalenses or diffractive lenses), namely, a short focal length and strong chromatic aberrations. Using SGM we demonstrate three-dimensional imaging of millimeter-scale samples while scanning only the lateral dimension, presenting a significant advantage over state-of-the-art technology.