Pushing the limits of optical information storage using deep learning

Emerging trends in SERS-based veterinary drug detection: multifunctional substrates and intelligent data approaches

Veterinary drug residues in poultry and livestock products present persistent challenges to food safety, necessitating precise and efficient detection methods. Surface-enhanced Raman scattering (SERS) has been identified as a powerful tool for veterinary drug residue analysis due to its high sensitivity and specificity. However, the development of reliable SERS substrates and the interpretation of complex spectral data remain significant obstacles. This review summarizes the development process of SERS substrates, categorizing them into metal-based, rigid, and flexible substrates, and highlighting the emerging trend of multifunctional substrates. The diverse application scenarios and detection requirements for these substrates are also discussed, with a focus on their use in veterinary drug detection. Furthermore, the integration of deep learning techniques into SERS-based detection is explored, including substrate structure design optimization, optical property prediction, spectral preprocessing, and both qualitative and quantitative spectral analyses. Finally, key limitations are briefly outlined, such as challenges in selecting reporter molecules, data imbalance, and computational demands. Future trends and directions for improving SERS-based veterinary drug detection are proposed.

Programmable electron-induced color router array

The development of color routers (CRs) realizes the splitting of dichromatic components, contributing to the modulation of photon momentum that acts as the information carrier for optical information technology on the frequency and spatial domains. However, CRs with optical stimulation lack active control of photon momentum at deep subwavelength scale because of the optical diffraction limit. Here, we experimentally demonstrate an active manipulation of dichromatic photon momentum at a deep subwavelength scale via electron-induced CRs, where the CRs radiation patterns are manipulated by steering the electron impact position within 60 nm in a single nanoantenna unit. Moreover, an encrypted display device based on programmable modulation of the CR array is designed and implemented. This approach with enhanced security, large information capacity, and high-level integration at a deep subwavelength scale may find applications in photonic devices and emerging areas in quantum information technologies.

A guidance to intelligent metamaterials and metamaterials intelligence

The bidirectional interactions between metamaterials and artificial intelligence have recently attracted immense interest to motivate scientists to revisit respective communities, giving rise to the proliferation of intelligent metamaterials and metamaterials intelligence. Owning to the strong nonlinear fitting and generalization ability, artificial intelligence is poised to serve as a materials-savvy surrogate electromagnetic simulator and a high-speed computing nucleus that drives numerous self-driving metamaterial applications, such as invisibility cloak, imaging, detection, and wireless communication. In turn, metamaterials create a versatile electromagnetic manipulator for wave-based analogue computing to be complementary with conventional electronic computing. In this Review, we stand from a unified perspective to review the recent advancements in these two nascent fields. For intelligent metamaterials, we discuss how artificial intelligence, exemplified by deep learning, streamline the photonic design, foster independent working manner, and unearth latent physics. For metamaterials intelligence, we particularly unfold three canonical categories, i.e., wave-based neural network, mathematical operation, and logic operation, all of which directly execute computation, detection, and inference task in physical space. Finally, future challenges and perspectives are pinpointed, including data curation, knowledge migration, and imminent practice-oriented issues, with a great vision of ushering in the free management of entire electromagnetic space.

Holographic multiplexing metasurface with twisted diffractive neural network

As the cornerstone of AI generated content, data drives human-machine interaction and is essential for developing sophisticated deep learning agents. Nevertheless, the associated data storage poses a formidable challenge from conventional energy-intensive planar storage, high maintenance cost, and the susceptibility to electromagnetic interference. In this work, we introduce the concept of metasurface disk, meta-disk, to expand the capacity limits of optical holographic storage by leveraging uncorrelated structural twist. We develop a physical twisted neural network to describe the optical behavior of the meta-disk and conduct a comprehensive lateral error analysis, where the meta-disk stores large volumes of information through internal structural multiplexing. Two-layer 640 µm x 640 µm meta-disk is sufficient to store over hundreds of high-fidelity images with SSIM of 0.8. By harnessing advanced three-dimensional (3D) printing technology, optical holographic storage is experimentally demonstrated with Pancharatnam-Berry metasurfaces. Our technology provides essential backing for the next generation of optical storage, display, encryption, and multifunctional optical analog computing.

In Situ Loading of ZnS on the PPF-3 Surface for Enhancing Nonlinear Optical Performance

In recent years, with the rapid development of ultrastrong and ultrafast lasers, it has become essential to develop new materials with excellent nonlinear optical (NLO) properties. Porphyrin-based metal-organic frameworks (MOFs) have great potential for application in the field of NLO due to their large conjugated structure and good stability. As a typical porphyrin-based MOF, porphyrin paddle-wheel framework-3 (PPF-3) has been prepared and applied in the fields of catalysis and sensing, yet the investigation of PPF-3 in NLO remains unexplored. In this study, the ZnS/PPF-3 composite was successfully prepared using a solvent thermal method to in situ load ZnS on the surface of PPF-3. Utilizing the Z-scan technique, the NLO properties of ZnS, PPF-3, and ZnS/PPF-3 composite were investigated under different input energy intensities. ZnS/PPF-3 composite material exhibits significantly enhanced NLO properties, with the third-order nonlinear absorption coefficient (βeff) of up to 7.00 × 10-10 m/W and a limiting threshold as low as 1.52 J/cm2, indicating its promising application potential value in the field of optical limiting. To enhance the practical utility, the ZnS/PPF-3/PVA film was prepared via the drop-casting method, achieving a maximum βeff of 5.00 × 10-8 m/W. The smaller optical bandgap of ZnS/PPF-3 and electron transfer from PPF-3 to ZnS are the key factors that enable the ZnS/PPF-3 composite to a superior NLO performance.

Artificial Intelligence-Enhanced Metasurfaces for Instantaneous Measurements of Dispersive Refractive Index

Measurements of the refractive index of liquids are in high demand in numerous fields such as agriculture, food and beverages, and medicine. However, conventional ellipsometric refractive index measurements are too expensive and labor-intensive for consumer devices, while Abbe refractometry is limited to the measurement at a single wavelength. Here, a new approach is proposed using machine learning to unlock the potential of colorimetric metasurfaces for the real-time measurement of the dispersive refractive index of liquids over the entire visible spectrum. The platform with a proof-of-concept experiment for measuring the concentration of glucose is further demonstrated, which holds a profound impact in non-invasive medical sensing. High-index-dielectric metasurfaces are designed and fabricated, while their experimentally measured reflectance and reflected colors, through microscopy and a standard smartphone, are used to train deep-learning models to provide measurements of the dispersive background refractive index with a resolution of ≈10-4, which is comparable to the known index as measured with ellipsometry. These results show the potential of enabling the unique optical properties of metasurfaces with machine learning to create a platform for the quick, simple, and high-resolution measurement of the dispersive refractive index of liquids, without the need for highly specialized experts and optical procedures.

Snapshot computational spectroscopy enabled by deep learning

Spectroscopy is a technique that analyzes the interaction between matter and light as a function of wavelength. It is the most convenient method for obtaining qualitative and quantitative information about an unknown sample with reasonable accuracy. However, traditional spectroscopy is reliant on bulky and expensive spectrometers, while emerging applications of portable, low-cost and lightweight sensing and imaging necessitate the development of miniaturized spectrometers. In this study, we have developed a computational spectroscopy method that can provide single-shot operation, sub-nanometer spectral resolution, and direct materials characterization. This method is enabled by a metasurface integrated computational spectrometer and deep learning algorithms. The identification of critical parameters of optical cavities and chemical solutions is demonstrated through the application of the method, with an average spectral reconstruction accuracy of 0.4 nm and an actual measurement error of 0.32 nm. The mean square errors for the characterization of cavity length and solution concentration are 0.53 % and 1.21 %, respectively. Consequently, computational spectroscopy can achieve the same level of spectral accuracy as traditional spectroscopy while providing convenient, rapid material characterization in a variety of scenarios.

Decoding of compressive data pages for optical data storage utilizing FFDNet

Coded aperture-based compression has proven to be an effective approach for high-density cold data storage. Nevertheless, its limited decoding speed represents a significant challenge for its broader application. We introduce a novel, to the best of our knowledge, decoding method leveraging the fast and flexible denoising network (FFDNet), capable of decoding a coded aperture-based compressive data page within 30.64 s. The practicality of the method has been confirmed in the decoding of monochromatic photo arrays, full-color photos, and dynamic videos. In experimental trials, the variance between decoded results obtained via the FFDNet-based method and the FFDNet-absent method in terms of average PSNR is less than 1 dB, while realizing a decoding speed enhancement of over 100-fold when employing the FFDNet-based method.

Optical color routing enabled by deep learning

Nano-color routing has emerged as an immensely popular and widely discussed subject in the realms of light field manipulation, image sensing, and the integration of deep learning. The conventional dye filters employed in commercial applications have long been hampered by several limitations, including subpar signal-to-noise ratio, restricted upper bounds on optical efficiency, and challenges associated with miniaturization. Nonetheless, the advent of bandpass-free color routing has opened up unprecedented avenues for achieving remarkable optical spectral efficiency and operation at sub-wavelength scales within the area of image sensing applications. This has brought about a paradigm shift, fundamentally transforming the field by offering a promising solution to surmount the constraints encountered with traditional dye filters. This review presents a comprehensive exploration of representative deep learning-driven nano-color routing structure designs, encompassing forward simulation algorithms, photonic neural networks, and various global and local topology optimization methods. A thorough comparison is drawn between the exceptional light-splitting capabilities exhibited by these methods and those of traditional design approaches. Additionally, the existing research on color routing is summarized, highlighting a promising direction for forthcoming development, delivering valuable insights to advance the field of color routing and serving as a powerful reference for future endeavors.

Artificial Intelligence for Surface-Enhanced Raman Spectroscopy

Surface-enhanced Raman spectroscopy (SERS), well acknowledged as a fingerprinting and sensitive analytical technique, has exerted high applicational value in a broad range of fields including biomedicine, environmental protection, food safety among the others. In the endless pursuit of ever-sensitive, robust, and comprehensive sensing and imaging, advancements keep emerging in the whole pipeline of SERS, from the design of SERS substrates and reporter molecules, synthetic route planning, instrument refinement, to data preprocessing and analysis methods. Artificial intelligence (AI), which is created to imitate and eventually exceed human behaviors, has exhibited its power in learning high-level representations and recognizing complicated patterns with exceptional automaticity. Therefore, facing up with the intertwining influential factors and explosive data size, AI has been increasingly leveraged in all the above-mentioned aspects in SERS, presenting elite efficiency in accelerating systematic optimization and deepening understanding about the fundamental physics and spectral data, which far transcends human labors and conventional computations. In this review, the recent progresses in SERS are summarized through the integration of AI, and new insights of the challenges and perspectives are provided in aim to better gear SERS toward the fast track.

Perovskite single-pixel detector for dual-color metasurface imaging recognition in complex environment

Highly efficient multi-dimensional data storage and extraction are two primary ends for the design and fabrication of emerging optical materials. Although metasurfaces show great potential in information storage due to their modulation for different degrees of freedom of light, a compact and efficient detector for relevant multi-dimensional data retrieval is still a challenge, especially in complex environments. Here, we demonstrate a multi-dimensional image storage and retrieval process by using a dual-color metasurface and a double-layer integrated perovskite single-pixel detector (DIP-SPD). Benefitting from the photoelectric response characteristics of the FAPbBr2.4I0.6 and FAPbI3 films and their stacked structure, our filter-free DIP-SPD can accurately reconstruct different colorful images stored in a metasurface within a single-round measurement, even in complex environments with scattering media or strong background noise. Our work not only provides a compact, filter-free, and noise-robust detector for colorful image extraction in a metasurface, but also paves the way for color imaging application of perovskite-like bandgap tunable materials.

Coded aperture-based compressive data page for optical data storage

In an era of data explosion, optical data storage provides an alternative solution for cold data storage due to its energy-saving and cost-effective features. However, its data density is still insufficient for zettabyte-scale cold data storage. Here, a coded aperture-based compressive data page with a compression ratio of ≤0.125 is proposed. Based on two frameworks-weighted nuclear norm minimization (WNNM) and alternating direction method of multipliers (ADMM)-the decoded quality of the compressive data page is ensured by utilizing sparsity priors. In experiments, compressive data pages of a monochromatic photo-array, full-color photo, and dynamic video are accurately decoded.

Multimodal dynamic and unclonable anti-counterfeiting using robust diamond microparticles on heterogeneous substrate

The growing prevalence of counterfeit products worldwide poses serious threats to economic security and human health. Developing advanced anti-counterfeiting materials with physical unclonable functions offers an attractive defense strategy. Here, we report multimodal, dynamic and unclonable anti-counterfeiting labels based on diamond microparticles containing silicon-vacancy centers. These chaotic microparticles are heterogeneously grown on silicon substrate by chemical vapor deposition, facilitating low-cost scalable fabrication. The intrinsically unclonable functions are introduced by the randomized features of each particle. The highly stable signals of photoluminescence from silicon-vacancy centers and light scattering from diamond microparticles can enable high-capacity optical encoding. Moreover, time-dependent encoding is achieved by modulating photoluminescence signals of silicon-vacancy centers via air oxidation. Exploiting the robustness of diamond, the developed labels exhibit ultrahigh stability in extreme application scenarios, including harsh chemical environments, high temperature, mechanical abrasion, and ultraviolet irradiation. Hence, our proposed system can be practically applied immediately as anti-counterfeiting labels in diverse fields.

A Reflectivity Enhanced 3D Optical Storage Nanostructure Application Based on Direct Laser Writing Lithography

To enable high-density optical storage, better storage media structures, diversified recording methods, and improved accuracy of readout schemes should be considered. In this study, we propose a novel three-dimensional (3D) sloppy nanostructure as the optical storage device, and this nanostructure can be fabricated using the 3D laser direct writing technology. It is a 900 nm high, 1 × 2 µm wide Si slope on a 200 nm SiO2 layer with 200 nm Si₃N₄ deposited on top to enhance reflectivity. In this study, we propose a reflected spectrum-based method as the readout recording strategy to stabilize information readout more stable. The corresponding reflected spectrum varied when the side wall angle of the slope and the azimuth angle of the nanostructure were tuned. In addition, an artificial neural network was applied to readout the stored information from the reflected spectrum. To simulate the realistic fabrication error and measurement error, a 20% noise level was added to the study. Our findings showed that the readout accuracy was 99.86% for all 120 data sequences when the slope and azimuth angle were varied. We investigated the possibility of a higher storage density to fully demonstrate the storage superiority of this designed structure. Our findings also showed that the readout accuracy can reach its highest level at 97.25% when the storage step of the encoded structure becomes 7.5 times smaller. The study provides the possibility to further explore different nanostructures to achieve high-density optical storage.

Learning and Predicting Photonic Responses of Plasmonic Nanoparticle Assemblies via Dual Variational Autoencoders

The application of machine learning is demonstrated for rapid and accurate extraction of plasmonic particles cluster geometries from hyperspectral image data via a dual variational autoencoder (dual-VAE). In this approach, the information is shared between the latent spaces of two VAEs acting on the particle shape data and spectral data, respectively, but enforcing a common encoding on the shape-spectra pairs. It is shown that this approach can establish the relationship between the geometric characteristics of nanoparticles and their far-field photonic responses, demonstrating that hyperspectral darkfield microscopy can be used to accurately predict the geometry (number of particles, arrangement) of a multiparticle assemblies below the diffraction limit in an automated fashion with high fidelity (for monomers (0.96), dimers (0.86), and trimers (0.58). This approach of building structure-property relationships via shared encoding is universal and should have applications to a broader range of materials science and physics problems in imaging of both molecular and nanomaterial systems.

Orientation-Controlled Ultralong Assembly of Janus Particles Induced by Bubble-Driven Instant Quasi-1D Interfaces

Constructing precisely oriented assemblies and exploring their orientation-dependent properties remain a challenge for Janus nanoparticles (JNPs) due to their asymmetric characteristics. Herein, we propose a bubble-driven instant quasi-1D interfacial strategy for the oriented assembly of JNP chains in a highly controllable manner. It is found that the rapid formation of templated bubbles can promote the interfacial orientation of JNPs kinetically, while the confined quasi-1D interface in the curved liquid bridge can constrain the disordered rotation of the particles, yielding well-oriented JNP chains in a long range. During the evaporation process, the interfacial orientation of the JNPs can be transferred to the assembled chains. By regulating the amphiphilicity of the JNPs, both heteraxial and coaxial JNP assemblies are obtained, which show different polarization dependences on light scattering, and the related colorimetric logic behaviors are demonstrated. This work demonstrates the great potential of patterned interfacial assembly with a manageable orientation and shows the broad prospect of asymmetric JNP assembly in constructing novel optoelectronic devices.

A Laplace sensitivity operator enhances the calculation efficiency of OCD metrology

In integrated circuit manufacturing, optical critical dimension measurement is an efficient and non-destructive metrology method. It is also a model-based metrology in which a numerical model of the target device is formed to simulate the optical spectrum. The result is then reconstructed by fitting the simulated spectrum to the experimentally measured optical spectrum. Normally, the measured optical spectrum contains a great deal of data points that consume the storage space, and increase the fitting time. Therefore, it is worth finding an appropriate approach to downsample these data points without losing much accuracy. To quickly and accurately extract critical data with high sensitivity, we propose a Laplace sensitivity operator that is widely used for feature extraction. Compared with traditional sensitivity calculation, the Laplace sensitivity operator focuses more on the correlation and coupling between multiple parameters. Thus, the sensitivity can be properly analyzed from different dimensions. To test the feasibility and correctness of the proposed method, three basic structures were used for single-parameter verification: thin film, one-dimensional grating, and two-dimensional grating, and a vertical gate-all-around device used for multi-parameter analysis. Using the Laplace sensitivity operator, the extracted data showed better results in most cases than those achieved by the traditional sensitivity calculation method. The data volume was compressed by approximately 70%, the result matching loss was not significantly increase in terms of the root mean square error, and the calculation speed was increased by a factor of 2.4. Compared to the traditional sensitivity operator, the Laplace sensitivity operator was able to reduce the RMSE by up to 50%.

Versatile full-colour nanopainting enabled by a pixelated plasmonic metasurface

The growing interest to develop modern digital displays and colour printing has driven the advancement of colouration technologies with remarkable speed. In particular, metasurface-based structural colouration shows a remarkable high colour saturation, wide gamut palette, chiaroscuro presentation and polarization tunability. However, previous approaches cannot simultaneously achieve all these features. Here, we design and experimentally demonstrate a surface-relief plasmonic metasurface consisting of shallow nanoapertures that enable the independent manipulation of colour hue, saturation and brightness by individually varying the geometric dimensions and orientation of the nanoapertures. We fabricate microscale artworks using a reusable template-stripping technique that features photorealistic and stereoscopic impressions. In addition, through the meticulous arrangement of differently oriented nanoapertures, kaleidoscopic information states can be decrypted by particular combinations of incident and reflected polarized light.

Efficient Strategy for Investigating the Third-Order Nonlinear Optical (NLO) Properties of Solid-State Coordination Polymers

The investigation of third-order nonlinear optical (NLO) properties of coordination polymers (CPs) based on solid samples is very difficult but is crucial for practical applications. Herein, we show a method for preparing high optical quality CP films in a polymer matrix to study the third-order NLO performance of solid-state CPs. Two novel azobenzene-based CPs, [CdL(DMAc)(H₂O)]_n (1) and {[CuL(4,4'-azobpy)]·3H₂O}_n (2) (H₂L = 5-((4-(phenyldiazenyl)phenoxy)methyl)isophthalic acid), were selected as study subjects. The corresponding microcrystals with a grain size of around 3 μm were doped into poly(vinyl alcohol) (PVA), forming CP films (1-MC/PVA and 2-MC/PVA). 1-MC/PVA and 2-MC/PVA exhibit NLO absorption switching behavior from saturable absorption (SA) to reverse saturable absorption (RSA) with increasing pulse energy. Moreover, their NLO properties can also be efficiently modulated by photostimulation energy due to the trans → cis isomerization of an azobenzene moiety. The density functional theory (DFT) results show that the narrower the band gap between the conduction band minimum and the valence band maximum, the denser the electron density distribution in the central mental and coordination atoms, which is beneficial to exhibit better third-order NLO performance. This work provides a feasible method for the wider practical application of solid materials with excellent third-order NLO performance.

A Surrogate Model Based Multi-Objective Optimization Method for Optical Imaging System

An optimization model for the optical imaging system was established in this paper. It combined the modern design of experiments (DOE) method known as Latin hypercube sampling (LHS), Kriging surrogate model training, and the multi-objective optimization algorithm NSGA-III into the optimization of a triplet optical system. Compared with the methods that rely mainly on optical system simulation, this surrogate model-based multi-objective optimization method can achieve a high-accuracy result with significantly improved optimization efficiency. Using this model, case studies were carried out for two-objective optimizations of a Cooke triplet optical system. The results showed that the weighted geometric spot diagram and the maximum field curvature were reduced 5.32% and 11.59%, respectively, in the first case. In the second case, where the initial parameters were already optimized by Code-V, this model further reduced the weighted geometric spot diagram and the maximum field curvature by another 3.53% and 4.33%, respectively. The imaging quality in both cases was considerably improved compared with the initial design, indicating that the model is suitable for the optimal design of an optical system.

Deep learning in light-matter interactions

The deep-learning revolution is providing enticing new opportunities to manipulate and harness light at all scales. By building models of light-matter interactions from large experimental or simulated datasets, deep learning has already improved the design of nanophotonic devices and the acquisition and analysis of experimental data, even in situations where the underlying theory is not sufficiently established or too complex to be of practical use. Beyond these early success stories, deep learning also poses several challenges. Most importantly, deep learning works as a black box, making it difficult to understand and interpret its results and reliability, especially when training on incomplete datasets or dealing with data generated by adversarial approaches. Here, after an overview of how deep learning is currently employed in photonics, we discuss the emerging opportunities and challenges, shining light on how deep learning advances photonics.

Engineered extracellular vesicles as intelligent nanosystems for next-generation nanomedicine

Extracellular vesicles (EVs), as natural carriers of bioactive cargo, have a unique micro/nanostructure, bioactive composition, and characteristic morphology, as well as fascinating physical, chemical and biochemical features, which have shown promising application in the treatment of a wide range of diseases. However, native EVs have limitations such as lack of or inefficient cell targeting, on-demand delivery, and therapeutic feedback. Recently, EVs have been engineered to contain an intelligent core, enabling them to (i) actively target sites of disease, (ii) respond to endogenous and/or exogenous signals, and (iii) provide treatment feedback for optimal function in the host. These advances pave the way for next-generation nanomedicine and offer promise for a revolution in drug delivery. Here, we summarise recent research on intelligent EVs and discuss the use of "intelligent core" based EV systems for the treatment of disease. We provide a critique about the construction and properties of intelligent EVs, and challenges in their commercialization. We compare the therapeutic potential of intelligent EVs to traditional nanomedicine and highlight key advantages for their clinical application. Collectively, this review aims to provide a new insight into the design of next-generation EV-based theranostic platforms for disease treatment.

Computational spectrometers enabled by nanophotonics and deep learning

A new type of spectrometer that heavily relies on computational technique to recover spectral information is introduced. They are different from conventional optical spectrometers in many important aspects. Traditional spectrometers offer high spectral resolution and wide spectral range, but they are so bulky and expensive as to be difficult to deploy broadly in the field. Emerging applications in machine sensing and imaging require low-cost miniaturized spectrometers that are specifically designed for certain applications. Computational spectrometers are well suited for these applications. They are generally low in cost and offer single-shot operation, with adequate spectral and spatial resolution. The new type of spectrometer combines recent progress in nanophotonics, advanced signal processing and machine learning. Here we review the recent progress in computational spectrometers, identify key challenges, and note new directions likely to develop in the near future.

Design of highly perceptible dual-resonance all-dielectric metasurface colorimetric sensor via deep neural networks

Colorimetric sensing, which provides effective detection of bio-molecular signals with one's naked eye, is an exceptionally promising sensing technique in that it enables convenient detection and simplification of entire sensing system. Though colorimetric sensors based on all-dielectric nanostructures have potential to exhibit distinct color variations enabling manageable detection due to their trivial intrinsic loss, there is crucial limitation that the sensitivity to environmental changes lags behind their plasmonic counterparts because of relatively small region of near field-analyte interaction of the dielectric Mie-type resonator. To overcome this challenge, we proposed all-dielectric metasurface colorimetric sensor which exhibits dual-resonance in the visible region. Thereafter, we confirmed with simulation that, in the elaborately designed dual-Lorentzian-type spectra, highly perceptible variations of structural color were manifested even in minute change of peripheral refractive index. In addition to verifying physical effectiveness of the superior colorimetric sensing performance appearing in the dual-resonance type sensor, by combining advanced optimization technique utilizing deep neural networks, we attempted to maximize sensing performance while obtaining dramatic improvement of design efficiency. Through well-trained deep neural network that accurately simulates the input target spectrum, we numerically verified that designed colorimetric sensor shows a remarkable sensing resolution distinguishable up to change of refractive index of 0.0086.

Nanophotonic Color Routing

Recent advances in low-dimensional materials and nanofabrication technologies have stimulated many breakthroughs in the field of nanophotonics such as metamaterials and plasmonics that provide efficient ways of light manipulation at a subwavelength scale. The representative structure-induced spectral engineering techniques have demonstrated superior design of freedom compared with natural materials such as pigment/dye. In particular, the emerging spectral routing scheme enables extraordinary light manipulation in both frequency-domain and spatial-domain with high-efficiency utilization of the full spectrum, which is critically important for various applications and may open up entirely new operating paradigms. In this review, a comparative introduction on the operating mechanisms of spectral routing and spectral filtering schemes is given and recent progress on various color nanorouters based on metasurfaces, plasmonics, dielectric antennas is reviewed with a focus on the potential application in high-resolution imaging. With a thorough analysis and discussion on the advanced properties and drawbacks of various techniques, this report is expected to provide an overview and vision for the future development and application of nanophotonic color (spectral) routing techniques.

Metallic Plasmonic Array Structures: Principles, Fabrications, Properties, and Applications

The vast development of nanofabrication has spurred recent progress for the manipulation of light down to a region much smaller than the wavelength. Metallic plasmonic array structures are demonstrated to be the most powerful platform to realize controllable light-matter interactions and have found wide applications due to their rich and tunable optical performance through the morphology and parameter engineering. Here, various light-management mechanisms that may exist on metallic plasmonic array structures are described. Then, the typical techniques for fabrication of metallic plasmonic arrays are summarized. Next, some recent applications of plasmonic arrays are reviewed, including plasmonic sensing, surface-enhanced spectroscopies, plasmonic nanolasing, and perfect light absorption. Lastly, the existing challenges and perspectives for metallic plasmonic arrays are discussed. The aim is to provide guidance for future development of metallic plasmonic array structures.

Optical Fingerprint Classification of Single Upconversion Nanoparticles by Deep Learning

Highly controlled synthesis of upconversion nanoparticles (UCNPs) can be achieved in the heterogeneous design, so that a library of optical properties can be arbitrarily produced by depositing multiple lanthanide ions. Such a control offers the potential in creating nanoscale barcodes carrying high-capacity information. With the increasing creation of optical information, it poses more challenges in decoding them in an accurate, high-throughput, and speedy fashion. Here, we reported that the deep-learning approach can recognize the complexity of the optical fingerprints from different UCNPs. Under a wide-field microscope, the lifetime profiles of hundreds of single nanoparticles can be collected at once, which offers a sufficient amount of data to develop deep-learning algorithms. We demonstrated that high accuracies of over 90% can be achieved in classifying 14 kinds of UCNPs. This work suggests new opportunities in handling the diverse properties of nanoscale optical barcodes toward the establishment of vast luminescent information carriers.

Phase-to-pattern inverse design paradigm for fast realization of functional metasurfaces via transfer learning

Metasurfaces have provided unprecedented freedom for manipulating electromagnetic waves. In metasurface design, massive meta-atoms have to be optimized to produce the desired phase profiles, which is time-consuming and sometimes prohibitive. In this paper, we propose a fast accurate inverse method of designing functional metasurfaces based on transfer learning, which can generate metasurface patterns monolithically from input phase profiles for specific functions. A transfer learning network based on GoogLeNet-Inception-V3 can predict the phases of 2^8×8 meta-atoms with an accuracy of around 90%. This method is validated via functional metasurface design using the trained network. Metasurface patterns are generated monolithically for achieving two typical functionals, 2D focusing and abnormal reflection. Both simulation and experiment verify the high design accuracy. This method provides an inverse design paradigm for fast functional metasurface design, and can be readily used to establish a meta-atom library with full phase span.

Decoding Optical Data with Machine Learning

Optical spectroscopy and imaging techniques play important roles in many fields such as disease diagnosis, biological study, information technology, optical science, and materials science. Over the past decade, machine learning (ML) has proved promising in decoding complex data, enabling rapid and accurate analysis of optical spectra and images. This review aims to shed light on various ML algorithms for optical data analysis with a focus on their applications in a wide range of fields. The goal of this work is to sketch the validity of ML-based optical data decoding. The review concludes with an outlook on unaddressed problems and opportunities in this emerging subject that interfaces optics, data science and ML.

Deep learning-enabled framework for automatic lens design starting point generation

We present a simple, highly modular deep neural network (DNN) framework to address the problem of automatically inferring lens design starting points tailored to the desired specifications. In contrast to previous work, our model can handle various and complex lens structures suitable for real-world problems such as Cooke Triplets or Double Gauss lenses. Our successfully trained dynamic model can infer lens designs with realistic glass materials whose optical performance compares favorably to reference designs from the literature on 80 different lens structures. Using our trained model as a backbone, we make available to the community a web application that outputs a selection of varied, high-quality starting points directly from the desired specifications, which we believe will complement any lens designer's toolbox.

Teaching optics to a machine learning network

In this Letter, we demonstrate how harmonic oscillator equations can be integrated in a neural network to improve the spectral response prediction for an optical system. We use the optical properties of a one-dimensional nanoslit array for a practical implementation of the study. This method allows to build more generalizable relations between the input parameters of the array and its optical properties, showing a 20-fold improvement for parameters outside the range used for the training. We also show how this model generates the output spectrum from phenomenological relationships between the input parameters and the output spectrum, indicating how it grasps the physical mechanisms of the optical response of the structure.

Machine Learning in Nanoscience: Big Data at Small Scales

Recent advances in machine learning (ML) offer new tools to extract new insights from large data sets and to acquire small data sets more effectively. Researchers in nanoscience are experimenting with these tools to tackle challenges in many fields. In addition to ML's advancement of nanoscience, nanoscience provides the foundation for neuromorphic computing hardware to expand the implementation of ML algorithms. In this Mini Review, we highlight some recent efforts to connect the ML and nanoscience communities by focusing on three types of interaction: (1) using ML to analyze and extract new insights from large nanoscience data sets, (2) applying ML to accelerate material discovery, including the use of active learning to guide experimental design, and (3) the nanoscience of memristive devices to realize hardware tailored for ML. We conclude with a discussion of challenges and opportunities for future interactions between nanoscience and ML researchers.

Deep Learning Meets Nanophotonics: A Generalized Accurate Predictor for Near Fields and Far Fields of Arbitrary 3D Nanostructures

Deep artificial neural networks are powerful tools with many possible applications in nanophotonics. Here, we demonstrate how a deep neural network can be used as a fast, general purpose predictor of the full near-field and far-field response of plasmonic and dielectric nanostructures. A trained neural network is shown to infer the internal fields of arbitrary three-dimensional nanostructures many orders of magnitude faster compared to conventional numerical simulations. Secondary physical quantities are derived from the deep learning predictions and faithfully reproduce a wide variety of physical effects without requiring specific training. We discuss the strengths and limitations of the neural network approach using a number of model studies of single particles and their near-field interactions. Our approach paves the way for fast, yet universal, methods for design and analysis of nanophotonic systems.

A Bidirectional Deep Neural Network for Accurate Silicon Color Design

Silicon nanostructure color has achieved unprecedented high printing resolution and larger color gamut than sRGB. The exact color is determined by localized magnetic and electric dipole resonance of nanostructures, which are sensitive to their geometric changes. Usually, the design of specific colors and iterative optimization of geometric parameters are computationally costly, and obtaining millions of different structural colors is challenging. Here, a deep neural network is trained, which can accurately predict the color generated by random silicon nanostructures in the forward modeling process and solve the nonuniqueness problem in the inverse design process that can accurately output the device geometries for at least one million different colors. The key results suggest deep learning is a powerful tool to minimize the computation cost and maximize the design efficiency for nanophotonics, which can guide silicon color manufacturing with high accuracy.

Emerging role of machine learning in light-matter interaction

Machine learning has provided a huge wave of innovation in multiple fields, including computer vision, medical diagnosis, life sciences, molecular design, and instrumental development. This perspective focuses on the implementation of machine learning in dealing with light-matter interaction, which governs those fields involving materials discovery, optical characterizations, and photonics technologies. We highlight the role of machine learning in accelerating technology development and boosting scientific innovation in the aforementioned aspects. We provide future directions for advanced computing techniques via multidisciplinary efforts that can help to transform optical materials into imaging probes, information carriers and photonics devices.

Deep learning enabled real time speckle recognition and hyperspectral imaging using a multimode fiber array

We demonstrate the use of deep learning for fast spectral deconstruction of speckle patterns. The artificial neural network can be effectively trained using numerically constructed multispectral datasets taken from a measured spectral transmission matrix. Optimized neural networks trained on these datasets achieve reliable reconstruction of both discrete and continuous spectra from a monochromatic camera image. Deep learning is compared to analytical inversion methods as well as to a compressive sensing algorithm and shows favourable characteristics both in the oversampling and in the sparse undersampling (compressive) regimes. The deep learning approach offers significant advantages in robustness to drift or noise and in reconstruction speed. In a proof-of-principle demonstrator we achieve real time recovery of hyperspectral information using a multi-core, multi-mode fiber array as a random scattering medium.

Deep-learning-powered photonic analog-to-digital conversion

Analog-to-digital converters (ADCs) must be high speed, broadband, and accurate for the development of modern information systems, such as radar, imaging, and communications systems; photonic technologies are regarded as promising technologies for realizing these advanced requirements. Here, we present a deep-learning-powered photonic ADC architecture that simultaneously exploits the advantages of electronics and photonics and overcomes the bottlenecks of the two technologies, thereby overcoming the ADC tradeoff among speed, bandwidth, and accuracy. Via supervised training, the adopted deep neural networks learn the patterns of photonic system defects and recover the distorted data, thereby maintaining the high quality of the electronic quantized data succinctly and adaptively. The numerical and experimental results demonstrate that the proposed architecture outperforms state-of-the-art ADCs with developable high throughput; hence, deep learning performs well in photonic ADC systems. We anticipate that the proposed architecture will inspire future high-performance photonic ADC design and provide opportunities for substantial performance enhancement for the next-generation information systems.