Wave physics as an analog recurrent neural network

Optical neural networks: progress and challenges

Artificial intelligence has prevailed in all trades and professions due to the assistance of big data resources, advanced algorithms, and high-performance electronic hardware. However, conventional computing hardware is inefficient at implementing complex tasks, in large part because the memory and processor in its computing architecture are separated, performing insufficiently in computing speed and energy consumption. In recent years, optical neural networks (ONNs) have made a range of research progress in optical computing due to advantages such as sub-nanosecond latency, low heat dissipation, and high parallelism. ONNs are in prospect to provide support regarding computing speed and energy consumption for the further development of artificial intelligence with a novel computing paradigm. Herein, we first introduce the design method and principle of ONNs based on various optical elements. Then, we successively review the non-integrated ONNs consisting of volume optical components and the integrated ONNs composed of on-chip components. Finally, we summarize and discuss the computational density, nonlinearity, scalability, and practical applications of ONNs, and comment on the challenges and perspectives of the ONNs in the future development trends.

Emergent coherent modes in nonlinear magnonic waveguides detected at ultrahigh frequency resolution

Nonlinearity of dynamic systems plays a key role in neuromorphic computing, which is expected to reduce the ever-increasing power consumption of machine learning and artificial intelligence applications. For spin waves (magnons), nonlinearity combined with phase coherence is the basis of phenomena like Bose-Einstein condensation, frequency combs, and pattern recognition in neuromorphic computing. Yet, the broadband electrical detection of these phenomena with high-frequency resolution remains a challenge. Here, we demonstrate the generation and detection of phase-coherent nonlinear magnons in an all-electrical GHz probe station based on coplanar waveguides connected to a vector network analyzer which we operate in a frequency-offset mode. Making use of an unprecedented frequency resolution, we resolve the nonlocal emergence of a fine structure of propagating nonlinear magnons, which sensitively depends on both power and a magnetic field. These magnons are shown to maintain coherency with the microwave source while propagating over macroscopic distances. We propose a multi-band four-magnon scattering scheme that is in agreement with the field-dependent characteristics of coherent nonlocal signals in the nonlinear excitation regime. Our findings are key to enable the seamless integration of nonlinear magnon processes into high-speed microwave electronics and to advance phase-encoded information processing in magnonic neuronal networks.

Enhanced DBR mirror design via D3QN: A reinforcement learning approach

Modern optical systems are important components of contemporary electronics and communication technologies, and the design of new systems has led to many innovative breakthroughs. This paper introduces a novel application based on deep reinforcement learning, D3QN, which is a combination of the Dueling Architecture and Double Q-Network methods, to design distributed Bragg reflectors (DBRs). Traditional design methods are based on time-consuming iterative simulations, whereas D3QN is designed to optimize the multilayer structure of DBRs. This approach enabled the reflectance performance and compactness of the DBRs to be improved. The reflectance of the DBRs designed using D3QN is 20.5% higher compared to designs derived from the transfer matrix method (TMM), and these DBRs are 61.2% smaller in terms of their size. These advancements suggest that deep reinforcement learning, specifically the D3QN methodology, is a promising new method for optical design and is more efficient than traditional techniques. Future research possibilities include expansion to 2D and 3D design structures, where increased design complexities could likely be addressed using D3QN or similar innovative solutions.

Reconfigurable metamaterial processing units that solve arbitrary linear calculus equations

Calculus equations serve as fundamental frameworks in mathematics, enabling describing an extensive range of natural phenomena and scientific principles, such as thermodynamics and electromagnetics. Analog computing with electromagnetic waves presents an intriguing opportunity to solve calculus equations with unparalleled speed, while facing an inevitable tradeoff in computing density and equation reconfigurability. Here, we propose a reconfigurable metamaterial processing unit (MPU) that solves arbitrary linear calculus equations at a very fast speed. Subwavelength kernels based on inverse-designed pixel metamaterials are used to perform calculus operations on time-domain signals. In addition, feedback mechanisms and reconfigurable components are used to formulate and solve calculus equations with different orders and coefficients. A prototype of this MPU with a compact planar size of 0.93λ0×0.93λ0 (λ0 is the free-space wavelength) is constructed and evaluated in microwave frequencies. Experimental results demonstrate the MPU's ability to successfully solve arbitrary linear calculus equations. With the merits of compactness, easy integration, reconfigurability, and reusability, the proposed MPU provides a potential route for integrated analog computing with high speed of signal processing.

Mechanical Neural Networks with Explicit and Robust Neurons

Mechanical computing provides an information processing method to realize sensing-analyzing-actuation integrated mechanical intelligence and, when combined with neural networks, can be more efficient for data-rich cognitive tasks. The requirement of solving implicit and usually nonlinear equilibrium equations of motion in training mechanical neural networks makes computation challenging and costly. Here, an explicit mechanical neuron is developed of which the response can be directly determined without the need of solving equilibrium equations. A training method is proposed to ensure the robustness of the neuron, i.e., insensitivity to defects and perturbations. The explicitness and robustness of the neurons facilitate the assembly of various network structures. Two exemplified networks, a robust mechanical convolutional neural network and a mechanical recurrent neural network with long short-term memory capabilities for associative learning, are experimentally demonstrated. The introduction of the explicit and robust mechanical neuron streamlines the design of mechanical neural networks fulfilling robotic matter with a level of intelligence.

An optoacoustic field-programmable perceptron for recurrent neural networks

Recurrent neural networks (RNNs) can process contextual information such as time series signals and language. But their tracking of internal states is a limiting factor, motivating research on analog implementations in photonics. While photonic unidirectional feedforward neural networks (NNs) have demonstrated big leaps, bi-directional optical RNNs present a challenge: the need for a short-term memory that (i) programmable and coherently computes optical inputs, (ii) minimizes added noise, and (iii) allows scalability. Here, we experimentally demonstrate an optoacoustic recurrent operator (OREO) which meets (i, ii, iii). OREO contextualizes the information of an optical pulse sequence via acoustic waves. The acoustic waves link different optical pulses, capturing their information and using it to manipulate subsequent operations. OREO's all-optical control on a pulse-by-pulse basis offers simple reconfigurability and is used to implement a recurrent drop-out and pattern recognition of 27 optical pulse patterns. Finally, we introduce OREO as bi-directional perceptron for new classes of optical NNs.

A physics based machine learning model to characterize room temperature semiconductor detectors in 3D

Room temperature semiconductor radiation detectors (RTSD) for X-ray and γ -ray detection are vital tools for medical imaging, astrophysics and other applications. CdZnTe (CZT) has been the main RTSD for more than three decades with desired detection properties. In a typical pixelated configuration, CZT have electrodes on opposite ends. For advanced event reconstruction algorithms at sub-pixel level, detailed characterization of the RTSD is required in three dimensional (3D) space. However, 3D characterization of the material defects and charge transport properties in the sub-pixel regime is a labor intensive process with skilled manpower and novel experimental setups. Presently, state-of-art characterization is done over the bulk of the RTSD considering homogenous properties. In this paper, we propose a novel physics based machine learning (PBML) model to characterize the RTSD over a discretized sub-pixelated 3D volume which is assumed. Our novel approach is the first to characterize a full 3D charge transport model of the RTSD. In this work, we first discretize the RTSD between a pixelated electrodes spatially into 3 dimensions-x, y, and z. The resulting discretizations are termed as voxels in 3D space. In each voxel, the different physics based charge transport properties such as drift, trapping, detrapping and recombination of charges are modeled as trainable model weights. The drift of the charges considers second order non-linear motion which is observed in practice with the RTSDs. Based on the electron-hole pair injections as input to the PBML model, and signals at the electrodes, free and trapped charges (electrons and holes) as outputs of the model, the PBML model determines the trainable weights by backpropagating the loss function. The trained weights of the model represents one-to-one relation to that of the actual physical charge transport properties in a voxelized detector.

Identifying Defects without a priori Knowledge in a Room-Temperature Semiconductor Detector Using Physics Inspired Machine Learning Model

Room-temperature semiconductor radiation detectors (RTSD) such as CdZnTe are popular in Computed Tomography (CT) imaging and other applications. Transport properties and material defects with respect to electron and hole transport often need to be characterized, which is a labor intensive process. However, these defects often vary from one RTSD to another and are not known a priori during characterization of the material. In recent years, physics-inspired machine learning (PI-ML) models have been developed for the RTSDs which have the ability to characterize the defects in a RTSD by discretizing it volumetrically. These learning models capture the heterogeneity of the defects in the RTSD-which arises due to the fabrication process and the energy bands of elements in the RTSD. In those models, the different defects of RTSD-trapping, detrapping and recombination for electrons and holes-are present. However, these defects are often unknown. In this work, we show the capabilities of a PI-ML model which has been developed considering all the material defects to identify certain defects which are present (or absent). Additionally, these models can identify the defects over the volume of the RTSD in a discretized manner.

On-chip phonon-magnon reservoir for neuromorphic computing

Reservoir computing is a concept involving mapping signals onto a high-dimensional phase space of a dynamical system called "reservoir" for subsequent recognition by an artificial neural network. We implement this concept in a nanodevice consisting of a sandwich of a semiconductor phonon waveguide and a patterned ferromagnetic layer. A pulsed write-laser encodes input signals into propagating phonon wavepackets, interacting with ferromagnetic magnons. The second laser reads the output signal reflecting a phase-sensitive mix of phonon and magnon modes, whose content is highly sensitive to the write- and read-laser positions. The reservoir efficiently separates the visual shapes drawn by the write-laser beam on the nanodevice surface in an area with a size comparable to a single pixel of a modern digital camera. Our finding suggests the phonon-magnon interaction as a promising hardware basis for realizing on-chip reservoir computing in future neuromorphic architectures.

Neuromorphic Computing via Fission-based Broadband Frequency Generation

The performance limitations of traditional computer architectures have led to the rise of brain-inspired hardware, with optical solutions gaining popularity due to the energy efficiency, high speed, and scalability of linear operations. However, the use of optics to emulate the synaptic activity of neurons has remained a challenge since the integration of nonlinear nodes is power-hungry and, thus, hard to scale. Neuromorphic wave computing offers a new paradigm for energy-efficient information processing, building upon transient and passively nonlinear interactions between optical modes in a waveguide. Here, an implementation of this concept is presented using broadband frequency conversion by coherent higher-order soliton fission in a single-mode fiber. It is shown that phase encoding on femtosecond pulses at the input, alongside frequency selection and weighting at the system output, makes transient spectro-temporal system states interpretable and allows for the energy-efficient emulation of various digital neural networks. The experiments in a compact, fully fiber-integrated setup substantiate an anticipated enhancement in computational performance with increasing system nonlinearity. The findings suggest that broadband frequency generation, accessible on-chip and in-fiber with off-the-shelf components, may challenge the traditional approach to node-based brain-inspired hardware design, ultimately leading to energy-efficient, scalable, and dependable computing with minimal optical hardware requirements.

Backpropagation-free training of deep physical neural networks

Recent successes in deep learning for vision and natural language processing are attributed to larger models but come with energy consumption and scalability issues. Current training of digital deep learning models primarily relies on backpropagation that is unsuitable for physical implementation. Here, we proposed a simple deep neural network architecture augmented by a physical local learning (PhyLL) algorithm, enabling supervised and unsupervised training of deep physical neural networks, without detailed knowledge of the nonlinear physical layer's properties. We trained diverse wave-based physical neural networks in vowel and image classification experiments, showcasing our approach's universality. Our method shows advantages over other hardware-aware training schemes by improving training speed, enhancing robustness, and reducing power consumption by eliminating the need for system modelling and thus decreasing digital computation.

Training large-scale optoelectronic neural networks with dual-neuron optical-artificial learning

Optoelectronic neural networks (ONN) are a promising avenue in AI computing due to their potential for parallelization, power efficiency, and speed. Diffractive neural networks, which process information by propagating encoded light through trained optical elements, have garnered interest. However, training large-scale diffractive networks faces challenges due to the computational and memory costs of optical diffraction modeling. Here, we present DANTE, a dual-neuron optical-artificial learning architecture. Optical neurons model the optical diffraction, while artificial neurons approximate the intensive optical-diffraction computations with lightweight functions. DANTE also improves convergence by employing iterative global artificial-learning steps and local optical-learning steps. In simulation experiments, DANTE successfully trains large-scale ONNs with 150 million neurons on ImageNet, previously unattainable, and accelerates training speeds significantly on the CIFAR-10 benchmark compared to single-neuron learning. In physical experiments, we develop a two-layer ONN system based on DANTE, which can effectively extract features to improve the classification of natural images.

Quantum-noise-limited optical neural networks operating at a few quanta per activation

A practical limit to energy efficiency in computation is ultimately from noise, with quantum noise [1] as the fundamental floor. Analog physical neural networks [2], which hold promise for improved energy efficiency and speed compared to digital electronic neural networks, are nevertheless typically operated in a relatively high-power regime so that the signal-to-noise ratio (SNR) is large (>10). We study optical neural networks [3] operated in the limit where all layers except the last use only a single photon to cause a neuron activation. In this regime, activations are dominated by quantum noise from the fundamentally probabilistic nature of single-photon detection. We show that it is possible to perform accurate machine-learning inference in spite of the extremely high noise (signal-to-noise ratio ~ 1). We experimentally demonstrated MNIST handwritten-digit classification with a test accuracy of 98% using an optical neural network with a hidden layer operating in the single-photon regime; the optical energy used to perform the classification corresponds to 0.008 photons per multiply-accumulate (MAC) operation, which is equivalent to 0.003 attojoules of optical energy per MAC. Our experiment also used >40× fewer photons per inference than previous state-of-the-art low-optical-energy demonstrations [4, 5] to achieve the same accuracy of >90%. Our training approach, which directly models the system's stochastic behavior, might also prove useful with non-optical ultra-low-power hardware.

Stimulated Amplification of Propagating Spin Waves

Spin-wave amplification techniques are key to the realization of magnon-based computing concepts. We introduce a novel mechanism to amplify spin waves in magnonic nanostructures. Using the technique of rapid cooling, we create a nonequilibrium state in excess of high-energy magnons and demonstrate the stimulated amplification of an externally seeded, propagating spin wave. Using an extended kinetic model, we qualitatively show that the amplification is mediated by an effective energy flux of high energy magnons into the low energy propagating mode, driven by a nonequilibrium magnon distribution.

Interpretable neural networks: principles and applications

In recent years, with the rapid development of deep learning technology, great progress has been made in computer vision, image recognition, pattern recognition, and speech signal processing. However, due to the black-box nature of deep neural networks (DNNs), one cannot explain the parameters in the deep network and why it can perfectly perform the assigned tasks. The interpretability of neural networks has now become a research hotspot in the field of deep learning. It covers a wide range of topics in speech and text signal processing, image processing, differential equation solving, and other fields. There are subtle differences in the definition of interpretability in different fields. This paper divides interpretable neural network (INN) methods into the following two directions: model decomposition neural networks, and semantic INNs. The former mainly constructs an INN by converting the analytical model of a conventional method into different layers of neural networks and combining the interpretability of the conventional model-based method with the powerful learning capability of the neural network. This type of INNs is further classified into different subtypes depending on which type of models they are derived from, i.e., mathematical models, physical models, and other models. The second type is the interpretable network with visual semantic information for user understanding. Its basic idea is to use the visualization of the whole or partial network structure to assign semantic information to the network structure, which further includes convolutional layer output visualization, decision tree extraction, semantic graph, etc. This type of method mainly uses human visual logic to explain the structure of a black-box neural network. So it is a post-network-design method that tries to assign interpretability to a black-box network structure afterward, as opposed to the pre-network-design method of model-based INNs, which designs interpretable network structure beforehand. This paper reviews recent progress in these areas as well as various application scenarios of INNs and discusses existing problems and future development directions.

End-to-End Diverse Metasurface Design and Evaluation Using an Invertible Neural Network

Employing deep learning models to design high-performance metasurfaces has garnered significant attention due to its potential benefits in terms of accuracy and efficiency. A deep learning-based metasurface design framework typically comprises a forward prediction path for predicting optical responses and a backward retrieval path for generating geometrical configurations. In the forward design path, a specific geometrical configuration corresponds to a unique optical response. However, in the inverse design path, a single performance metric can correspond to multiple potential designs. This one-to-many mapping poses a significant challenge for deep learning models and can potentially impede their performance. Although representing the inverse path as a probabilistic distribution is a widely adopted method for tackling this problem, accurately capturing the posterior distribution to encompass all potential solutions remains an ongoing challenge. Furthermore, in most pioneering works, the forward and backward paths are captured using separate models. However, the knowledge acquired from the forward path does not contribute to the training of the backward model. This separation of models adds complexity to the system and can hinder the overall efficiency and effectiveness of the design framework. Here, we utilized an invertible neural network (INN) to simultaneously model both the forward and inverse process. Unlike other frameworks, INN focuses on the forward process and implicitly captures a probabilistic model for the inverse process. Given a specific optical response, the INN enables the recovery of the complete posterior over the parameter space. This capability allows for the generation of novel designs that are not present in the training data. Through the integration of the INN with the angular spectrum method, we have developed an efficient and automated end-to-end metasurface design and evaluation framework. This novel approach eliminates the need for human intervention and significantly speeds up the design process. Utilizing this advanced framework, we have effectively designed high-efficiency metalenses and dual-polarization metasurface holograms. This approach extends beyond dielectric metasurface design, serving as a general method for modeling optical inverse design problems in diverse optical fields.

Experimental realization of convolution processing in photonic synthetic frequency dimensions

Convolution is an essential operation in signal and image processing and consumes most of the computing power in convolutional neural networks. Photonic convolution has the promise of addressing computational bottlenecks and outperforming electronic implementations. Performing photonic convolution in the synthetic frequency dimension, which harnesses the dynamics of light in the spectral degrees of freedom for photons, can lead to highly compact devices. Here, we experimentally realize convolution operations in the synthetic frequency dimension. Using a modulated ring resonator, we synthesize arbitrary convolution kernels using a predetermined modulation waveform with high accuracy. We demonstrate the convolution computation between input frequency combs and synthesized kernels. We also introduce the idea of an additive offset to broaden the kinds of kernels that can be implemented experimentally when the modulation strength is limited. Our work demonstrate the use of synthetic frequency dimension to efficiently encode data and implement computation tasks, leading to a compact and scalable photonic computation architecture.

Physics-informed neural networks for transcranial ultrasound wave propagation

Transcranial ultrasound imaging has been playing an increasingly important role in the non-invasive treatment of brain disorders. However, the conventional mesh-based numerical wave solvers, which are an integral part of imaging algorithms, suffer from limitations such as high computational cost and discretization error in predicting the wavefield passing through the skull. In this paper, we explore the use of physics-informed neural networks (PINNs) for predicting the transcranial ultrasound wave propagation. The wave equation, two sets of time snapshots data and a boundary condition (BC) are embedded as physical constraints in the loss function during training. The proposed approach has been validated by solving the two-dimensional (2D) acoustic wave equation under three increasingly complex spatially varying velocity models. Our cases demonstrate that due to the meshless nature of PINNs, they can be flexibly applied to different wave equations and types of BCs. By adding physics constraints to the loss function, PINNs can predict wavefields far outside the training data, providing ideas for improving the generalization capability of existing deep learning methods. The proposed approach offers exciting perspectives because of the powerful framework and simple implementation. We conclude with a summary of the strengths, limitations and further research directions of this work.

Solving integral equations in free space with inverse-designed ultrathin optical metagratings

As standard microelectronic technology approaches fundamental limitations in speed and power consumption, novel computing strategies are strongly needed. Analogue optical computing enables the processing of large amounts of data at a negligible energy cost and high speeds. Based on these principles, ultrathin optical metasurfaces have been recently explored to process large images in real time, in particular for edge detection. By incorporating feedback, it has also recently been shown that metamaterials can be tailored to solve complex mathematical problems in the analogue domain, although these efforts have so far been limited to guided-wave systems and bulky set-ups. Here, we present an ultrathin Si metasurface-based platform for analogue computing that is able to solve Fredholm integral equations of the second kind using free-space visible radiation. A Si-based metagrating was inverse-designed to implement the scattering matrix synthesizing a prescribed kernel corresponding to the mathematical problem of interest. Next, a semitransparent mirror was incorporated into the sample to provide adequate feedback and thus perform the required Neumann series, solving the corresponding equation in the analogue domain at the speed of light. Visible wavelength operation enables a highly compact, ultrathin device that can be interrogated from free space, implying high processing speeds and the possibility of on-chip integration.

MXene-Nanoflakes-Enabled All-Optical Nonlinear Activation Function for On-Chip Photonic Deep Neural Networks

2D metal carbides and nitrides (MXene) are promising material platforms for on-chip neural networks owing to their nonlinear saturable absorption effect. The localized surface plasmon resonances in metallic MXene nanoflakes may play an important role in enhancing the electromagnetic absorption; however, their contribution is not determined due to the lack of a precise understanding of its localized surface plasmon behavior. Here, a saturable absorber made of MXene thin film and a silicon waveguide with MXene flakes overlayer are developed to perform neuromorphic tasks. The proposed configurations are reconfigurable and can therefore be adjusted for various applications without the need to modify the physical structure of the proposed MXene-based activator configurations via tuning the wavelength of operation. The capability and feasibility of the obtained results of machine-learning applications are confirmed via handwritten digit classification task, with near 99% accuracy. These findings can guide the design of advanced ultrathin saturable absorption materials on a chip for a broad range of applications.

Artificial intelligence techniques for retinal prostheses: a comprehensive review and future direction

Objective. Retinal prostheses are promising devices to restore vision for patients with severe age-related macular degeneration or retinitis pigmentosa disease. The visual processing mechanism embodied in retinal prostheses play an important role in the restoration effect. Its performance depends on our understanding of the retina's working mechanism and the evolvement of computer vision models. Recently, remarkable progress has been made in the field of processing algorithm for retinal prostheses where the new discovery of the retina's working principle and state-of-the-arts computer vision models are combined together.Approach. We investigated the related research on artificial intelligence techniques for retinal prostheses. The processing algorithm in these studies could be attributed to three types: computer vision-related methods, biophysical models, and deep learning models.Main results. In this review, we first illustrate the structure and function of the normal and degenerated retina, then demonstrate the vision rehabilitation mechanism of three representative retinal prostheses. It is necessary to summarize the computational frameworks abstracted from the normal retina. In addition, the development and feature of three types of different processing algorithms are summarized. Finally, we analyze the bottleneck in existing algorithms and propose our prospect about the future directions to improve the restoration effect.Significance. This review systematically summarizes existing processing models for predicting the response of the retina to external stimuli. What's more, the suggestions for future direction may inspire researchers in this field to design better algorithms for retinal prostheses.

Optimizing hyperparameters of Data-driven simulation-assisted-Physics learned AI (DPAI) model to reduce compounding error

In this paper, we propose the study of optimizing the hyperparameters of deep learning Data-driven simulation-assisted-Physics learned AI (DPAI) model to simulate the ultrasonic wave propagation for extended depth with a lower error. DPAI model has layers of encoder-decoder structure with modified convolutional long short-term memory (ConvLSTM). DPAI model is trained using the finite element (FE) simulations dataset of distributed single-point to multi-point excitation sources in the 2D domain. The DPAI is the data-driven approach to apprehending the underlying physics of elastodynamic wave propagation. Six different combinations of hyperparameters (hidden dimensions, kernel size, batch size) are used in the DAPI model to study parameter optimization for lowering compounding error. The effectiveness of the trained DPAI models with varying hyperparameters is demonstrated to reduce the compounding error for modeling the deeper simulations of the single-point excitation and multi-point excitation sources. The maximum MAE on amplitude is 5.0×10^-2, and MAPE is 2.64% on time of flight (TOF) between DPAI and FE simulations.

Learning-based physical models of room-temperature semiconductor detectors with reduced data

Room-temperature semiconductor radiation detectors (RTSD) have broad applications in medical imaging, homeland security, astrophysics and others. RTSDs such as CdZnTe, CdTe are often pixelated, and characterization of these detectors at micron level can benefit 3-D event reconstruction at sub-pixel level. Material defects alongwith electron and hole charge transport properties need to be characterized which requires several experimental setups and is labor intensive. The current state-of-art approaches characterize each detector pixel, considering the detector in bulk. In this article, we propose a new microscopic learning-based physical models of RTSD based on limited data compared to what is dictated by the physical equations. Our learning models uses a physical charge transport considering trapping centers. Our models learn these material properties in an indirect manner from the measurable signals at the electrodes and/or free and/or trapped charges distributed in the RTSD for electron-hole charge pair injections in the material. Based on the amount of data used during training our physical model, our algorithm characterizes the detector for charge drifts, trapping, detrapping and recombination coefficients considering multiple trapping centers or as a single equivalent trapping center. The RTSD is segmented into voxels spatially, and in each voxel, the material properties are modeled as learnable parameters. Depending on the amount of data, our models can characterize the RTSD either completely or in an equivalent manner.

Learning the stress-strain fields in digital composites using Fourier neural operator

Increased demands for high-performance materials have led to advanced composite materials with complex hierarchical designs. However, designing a tailored material microstructure with targeted properties and performance is extremely challenging due to the innumerable design combinations and prohibitive computational costs for physics-based solvers. In this study, we employ a neural operator-based framework, namely Fourier neural operator (FNO), to learn the mechanical response of 2D composites. We show that the FNO exhibits high-fidelity predictions of the complete stress and strain tensor fields for geometrically complex composite microstructures with very few training data and purely based on the microstructure. The model also exhibits zero-shot generalization on unseen arbitrary geometries with high accuracy. Furthermore, the model exhibits zero-shot super-resolution capabilities by predicting high-resolution stress and strain fields directly from low-resolution input configurations. Finally, the model also provides high-accuracy predictions of equivalent measures for stress-strain fields, allowing realistic upscaling of the results.

•

Predict stress and strain tensors directly from the microstructure

•

Demonstrate material and pixel-wise super-resolution of the FNO model

•

Zero-shot generalization to unseen arbitrary geometries

•

Measuring equivalent quantities such as von-mises stress and equivalent strains

Mechanical neural networks: Architected materials that learn behaviors

Aside from some living tissues, few materials can autonomously learn to exhibit desired behaviors as a consequence of prolonged exposure to unanticipated ambient loading scenarios. Still fewer materials can continue to exhibit previously learned behaviors in the midst of changing conditions (e.g., rising levels of internal damage, varying fixturing scenarios, and fluctuating external loads) while also acquiring new behaviors best suited for the situation at hand. Here, we describe a class of architected materials, called mechanical neural networks (MNNs), that achieve such learning capabilities by tuning the stiffness of their constituent beams similar to how artificial neural networks (ANNs) tune their weights. An example lattice was fabricated to demonstrate its ability to learn multiple mechanical behaviors simultaneously, and a study was conducted to determine the effect of lattice size, packing configuration, algorithm type, behavior number, and linear-versus-nonlinear stiffness tunability on MNN learning as proposed. Thus, this work lays the foundation for artificial-intelligent (AI) materials that can learn behaviors and properties.

SDHAR-HOME: A Sensor Dataset for Human Activity Recognition at Home

Nowadays, one of the most important objectives in health research is the improvement of the living conditions and well-being of the elderly, especially those who live alone. These people may experience undesired or dangerous situations in their daily life at home due to physical, sensorial or cognitive limitations, such as forgetting their medication or wrong eating habits. This work focuses on the development of a database in a home, through non-intrusive technology, where several users are residing by combining: a set of non-intrusive sensors which captures events that occur in the house, a positioning system through triangulation using beacons and a system for monitoring the user's state through activity wristbands. Two months of uninterrupted measurements were obtained on the daily habits of 2 people who live with a pet and receive sporadic visits, in which 18 different types of activities were labelled. In order to validate the data, a system for the real-time recognition of the activities carried out by these residents was developed using different current Deep Learning (DL) techniques based on neural networks, such as Recurrent Neural Networks (RNN), Long Short-Term Memory networks (LSTM) or Gated Recurrent Unit networks (GRU). A personalised prediction model was developed for each user, resulting in hit rates ranging from 88.29% to 90.91%. Finally, a data sharing algorithm has been developed to improve the generalisability of the model and to avoid overtraining the neural network.

Application and Prospect Analysis of Artificial Intelligence in the Field of Physical Education

The development of AI technology has a significant impact on every sector of business. Artificial intelligence uses this technology to reduce the amount of work required, duplicate work, and increase the accuracy of work by modelling human behaviour and thought. On the basis of a thorough analysis of artificial intelligence technology, the technology is applied to teaching, specialised training, and specialised testing in response to the current issues in the field of physical education and the business that can be improved through auxiliary tools. The paper discusses how artificial intelligence technology can be used in physical education. After that, it examines how physical education is currently taught in classrooms and discusses how artificial intelligence is affecting this field. Following a thorough demonstration of the aforementioned material, the commonly used artificial intelligence technologies are introduced in turn, along with the methodology used in physical education. The use of computer models in physical education is then explained, and future analysis is conducted on the basis of this information.

Single image dehazing algorithm based on optical diffraction deep neural networks

Single image dehazing is a challenging task because of the hue and brightness distortion problems due to the atmospheric scattering. These problems limit the perceptual fidelity, as well as information integrity, of a given image. In this paper, we propose an image dehazing method based on the optical neural networks dehazing by simulating optical diffraction. The algorithm is trained from a large number of hazy images and their corresponding clean images. The experimental results demonstrate that the proposed method has reached an advanced level in both PSNR and SSIM dehazing performance indicators, and the amount of calculation is less than most artificial neural networks.

Implementing Data-Driven Approach for Modelling Ultrasonic Wave Propagation Using Spatio-Temporal Deep Learning (SDL)

In this paper, we proposed a data-driven spatio-temporal deep learning (SDL) model, to simulate forward and reflected ultrasonic wave propagation in the 2D geometrical domain, by implementing the convolutional long short-term memory (ConvLSTM) algorithm. The SDL model learns underlying wave physics from the spatio-temporal datasets. Two different SDL models are trained, with the following time-domain finite element (FE) simulation datasets, by applying: (1) multi-point excitation sources inside the domain and (2) single-point excitation sources on the edge of the different geometrical domains. The proposed SDL models simulate ultrasonic wave dynamics, for the forward ultrasonic wave propagation in the different geometrical domains and reflected wave propagation phenomenon, from the geometrical boundaries such as curved, T-shaped, triangular, and rectangular domains, with varying frequencies and cycles. The SDL is a reliable model, which generates simulations faster than the conventional finite element solvers.

Robustness of subwavelength devices: a case study of cochlea-inspired rainbow sensors

We derive asymptotic formulae describing how the properties of subwavelength devices are changed by the introduction of errors and imperfections. As a demonstrative example, we study a class of cochlea-inspired rainbow sensors. These are graded metamaterials which have been designed to mimic the frequency separation performed by the cochlea. The device considered here has similar dimensions to the cochlea and has a resonant spectrum that falls within the range of audible frequencies. We show that the device’s properties (including its role as a signal filtering device) are stable with respect to small imperfections in the positions and sizes of the resonators. Additionally, under suitable assumptions, if the number of resonators is sufficiently large, then the device’s properties are stable under the removal of a resonator.

Neural Schrödinger Equation: Physical Law as Deep Neural Network

We show a new family of neural networks based on the Schrödinger equation (SE-NET). In this analogy, the trainable weights of the neural networks correspond to the physical quantities of the Schrödinger equation. These physical quantities can be trained using the complex-valued adjoint method. Since the propagation of the SE-NET can be described by the evolution of physical systems, its outputs can be computed by using a physical solver. The trained network is transferable to actual optical systems. As a demonstration, we implemented the SE-NET with the Crank-Nicolson finite difference method on Pytorch. From the results of numerical simulations, we found that the performance of the SE-NET becomes better when the SE-NET becomes wider and deeper. However, the training of the SE-NET was unstable due to gradient explosions when SE-NET becomes deeper. Therefore, we also introduced phase-only training, which only updates the phase of the potential field (refractive index) in the Schrödinger equation. This enables stable training even for the deep SE-NET model because the unitarity of the system is kept under the training. In addition, the SE-NET enables a joint optimization of physical structures and digital neural networks. As a demonstration, we performed a numerical demonstration of end-to-end machine learning (ML) with an optical frontend toward a compact spectrometer. Our results extend the application field of ML to hybrid physical-digital optimizations.

Polarization multiplexed diffractive computing: all-optical implementation of a group of linear transformations through a polarization-encoded diffractive network

Research on optical computing has recently attracted significant attention due to the transformative advances in machine learning. Among different approaches, diffractive optical networks composed of spatially-engineered transmissive surfaces have been demonstrated for all-optical statistical inference and performing arbitrary linear transformations using passive, free-space optical layers. Here, we introduce a polarization-multiplexed diffractive processor to all-optically perform multiple, arbitrarily-selected linear transformations through a single diffractive network trained using deep learning. In this framework, an array of pre-selected linear polarizers is positioned between trainable transmissive diffractive materials that are isotropic, and different target linear transformations (complex-valued) are uniquely assigned to different combinations of input/output polarization states. The transmission layers of this polarization-multiplexed diffractive network are trained and optimized via deep learning and error-backpropagation by using thousands of examples of the input/output fields corresponding to each one of the complex-valued linear transformations assigned to different input/output polarization combinations. Our results and analysis reveal that a single diffractive network can successfully approximate and all-optically implement a group of arbitrarily-selected target transformations with a negligible error when the number of trainable diffractive features/neurons (N) approaches [Formula: see text], where Ni and No represent the number of pixels at the input and output fields-of-view, respectively, and Np refers to the number of unique linear transformations assigned to different input/output polarization combinations. This polarization-multiplexed all-optical diffractive processor can find various applications in optical computing and polarization-based machine vision tasks.

Electromagnetic wave-based extreme deep learning with nonlinear time-Floquet entanglement

Wave-based analog signal processing holds the promise of extremely fast, on-the-fly, power-efficient data processing, occurring as a wave propagates through an artificially engineered medium. Yet, due to the fundamentally weak non-linearities of traditional electromagnetic materials, such analog processors have been so far largely confined to simple linear projections such as image edge detection or matrix multiplications. Complex neuromorphic computing tasks, which inherently require strong non-linearities, have so far remained out-of-reach of wave-based solutions, with a few attempts that implemented non-linearities on the digital front, or used weak and inflexible non-linear sensors, restraining the learning performance. Here, we tackle this issue by demonstrating the relevance of time-Floquet physics to induce a strong non-linear entanglement between signal inputs at different frequencies, enabling a power-efficient and versatile wave platform for analog extreme deep learning involving a single, uniformly modulated dielectric layer and a scattering medium. We prove the efficiency of the method for extreme learning machines and reservoir computing to solve a range of challenging learning tasks, from forecasting chaotic time series to the simultaneous classification of distinct datasets. Our results open the way for optical wave-based machine learning with high energy efficiency, speed and scalability.

Wave-based analog signal processing has been challenging for complex nonlinear operations such as data forecasting or classification. The authors propose here an analog neuromorphic platform for optical wave-based machine learning characterized by energy efficiency, speed and scalability.

Integrated photonic metasystem for image classifications at telecommunication wavelength

Miniaturized image classifiers are potential for revolutionizing their applications in optical communication, autonomous vehicles, and healthcare. With subwavelength structure enabled directional diffraction and dispersion engineering, the light propagation through multi-layer metasurfaces achieves wavelength-selective image recognitions on a silicon photonic platform at telecommunication wavelength. The metasystems implement high-throughput vector-by-matrix multiplications, enabled by near 10³ nanoscale phase shifters as weight elements within 0.135 mm² footprints. The diffraction manifested computing capability incorporates the fabrication and measurement related phase fluctuations, and thus the pre-trained metasystem can handle uncertainties in inputs without post-tuning. Here we demonstrate three functional metasystems: a 15-pixel spatial pattern classifier that reaches near 90% accuracy with femtosecond inputs, a multi-channel wavelength demultiplexer, and a hyperspectral image classifier. The diffractive metasystem provides an alternative machine learning architecture for photonic integrated circuits, with densely integrated phase shifters, spatially multiplexed throughput, and data processing capabilities.

Metasystem architectures are attractive alternatives to waveguide-based integrated photonic processors due to the subwavelength structures. Here, the authors report a 1D passive silicon photonic metasystem with near 90% spatial pattern classification accuracy at telecommunication wavelength.

DPAI: A Data-driven simulation-assisted-Physics learned AI model for transient ultrasonic wave propagation

In this paper, we propose a deep neural network model to simulate the transient ultrasonic wave propagation in the 2D domain by implementing the Data driven-simulation-assisted-Physics learned AI (DPAI) model. The DPAI model consists of modified convolutional long short-term memory (ConvLSTM) with an encoder-decoder structure, which learns the representation of spatio-temporal dependence from input sequence data. The DPAI uses the data-driven approach to understand the underlying physics of elastic wave propagation in a medium. This model is trained with simulation-assisted finite element simulation datasets consisting of distributed single and multi-point excitation sources in the medium. The effectiveness of the proposed approach is demonstrated by modeling a wide range of scenarios in elastodynamic physics, such as multiple point sources, varying excitation parameters, and wave propagation in a large 2D domain. The trained DPAI model is tested and compared against FE modeling with respect to accuracy and computational time.

Parallel wave-based analog computing using metagratings

Wave-based signal processing has witnessed a significant expansion of interest in a variety of science and engineering disciplines, as it provides new opportunities for achieving high-speed and low-power operations. Although flat optics desires integrable components to perform multiple missions, yet, the current wave-based computational metasurfaces can engineer only the spatial content of the input signal where the processed signal obeys the traditional version of Snell's law. In this paper, we propose a multi-functional metagrating to modulate both spatial and angular properties of the input signal whereby both symmetric and asymmetric optical transfer functions are realized using high-order space harmonics. The performance of the designed compound metallic grating is validated through several investigations where closed-form expressions are suggested to extract the phase and amplitude information of the diffractive modes. Several illustrative examples are demonstrated to show that the proposed metagrating allows for simultaneous parallel analog computing tasks such as first- and second-order spatial differentiation through a single multichannel structured surface. It is anticipated that the designed platform brings a new twist to the field of optical signal processing and opens up large perspectives for simple integrated image processing systems.

Nonlocal Scatterer for Compact Wave-Based Analog Computing

Analog computing based on wave interactions with metamaterials has been raising significant interest as a low-energy, ultrafast platform to process large amounts of data. Engineered materials can be tailored to impart mathematical operations of choice on the spatial distribution of the impinging signals, but they also require extended footprints and precise large-area fabrication, which may hinder their practical applicability. Here we show that the nonlocal response of a compact scatterer can be engineered to impart operations of choice on arbitrary impinging waves, and even to solve integro-differential equations, whose solution is observed in the scattered fields. The lack of strongly resonant phenomena makes the response robust, and the compact nature opens to scalability and cascading of these processes, paving the way to efficient, compact analog computers based on engineered microstructures.

Deep physical neural networks trained with backpropagation

Deep-learning models have become pervasive tools in science and engineering. However, their energy requirements now increasingly limit their scalability1. Deep-learning accelerators2-9 aim to perform deep learning energy-efficiently, usually targeting the inference phase and often by exploiting physical substrates beyond conventional electronics. Approaches so far10-22 have been unable to apply the backpropagation algorithm to train unconventional novel hardware in situ. The advantages of backpropagation have made it the de facto training method for large-scale neural networks, so this deficiency constitutes a major impediment. Here we introduce a hybrid in situ-in silico algorithm, called physics-aware training, that applies backpropagation to train controllable physical systems. Just as deep learning realizes computations with deep neural networks made from layers of mathematical functions, our approach allows us to train deep physical neural networks made from layers of controllable physical systems, even when the physical layers lack any mathematical isomorphism to conventional artificial neural network layers. To demonstrate the universality of our approach, we train diverse physical neural networks based on optics, mechanics and electronics to experimentally perform audio and image classification tasks. Physics-aware training combines the scalability of backpropagation with the automatic mitigation of imperfections and noise achievable with in situ algorithms. Physical neural networks have the potential to perform machine learning faster and more energy-efficiently than conventional electronic processors and, more broadly, can endow physical systems with automatically designed physical functionalities, for example, for robotics23-26, materials27-29 and smart sensors30-32.

Responsive materials architected in space and time

Rationally designed architected materials have attained previously untapped territories in materials property space. The properties and behaviours of architected materials need not be stagnant after fabrication; they can be encoded with a temporal degree of freedom such that they evolve over time. In this Review, we describe the variety of materials architected in both space and time, and their responses to various stimuli, including mechanical actuation, changes in temperature and chemical environment, and variations in electromagnetic fields. We highlight the additive manufacturing methods that can precisely prescribe complex geometries and local inhomogeneities to make such responsiveness possible. We discuss the emergent physics phenomena observed in architected materials that are analogous to those in classical materials, such as the formation and behaviour of defects, phase transformations and topologically protected properties. Finally, we offer a perspective on the future of architected materials that have a degree of intelligence through mechanical logic and artificial neural networks.

Optical meta-waveguides for integrated photonics and beyond

The growing maturity of nanofabrication has ushered massive sophisticated optical structures available on a photonic chip. The integration of subwavelength-structured metasurfaces and metamaterials on the canonical building block of optical waveguides is gradually reshaping the landscape of photonic integrated circuits, giving rise to numerous meta-waveguides with unprecedented strength in controlling guided electromagnetic waves. Here, we review recent advances in meta-structured waveguides that synergize various functional subwavelength photonic architectures with diverse waveguide platforms, such as dielectric or plasmonic waveguides and optical fibers. Foundational results and representative applications are comprehensively summarized. Brief physical models with explicit design tutorials, either physical intuition-based design methods or computer algorithms-based inverse designs, are cataloged as well. We highlight how meta-optics can infuse new degrees of freedom to waveguide-based devices and systems, by enhancing light-matter interaction strength to drastically boost device performance, or offering a versatile designer media for manipulating light in nanoscale to enable novel functionalities. We further discuss current challenges and outline emerging opportunities of this vibrant field for various applications in photonic integrated circuits, biomedical sensing, artificial intelligence and beyond.

Nanoscale neural network using non-linear spin-wave interference

We demonstrate the design of a neural network hardware, where all neuromorphic computing functions, including signal routing and nonlinear activation are performed by spin-wave propagation and interference. Weights and interconnections of the network are realized by a magnetic-field pattern that is applied on the spin-wave propagating substrate and scatters the spin waves. The interference of the scattered waves creates a mapping between the wave sources and detectors. Training the neural network is equivalent to finding the field pattern that realizes the desired input-output mapping. A custom-built micromagnetic solver, based on the Pytorch machine learning framework, is used to inverse-design the scatterer. We show that the behavior of spin waves transitions from linear to nonlinear interference at high intensities and that its computational power greatly increases in the nonlinear regime. We envision small-scale, compact and low-power neural networks that perform their entire function in the spin-wave domain.

Wave based computing has sparked much interest for neuromorphic computing due to the inherent interconnectedness of such wave based approaches. Here, Papp, Porod and Csaba show how neural networks can be implemented using spin-waves, taking advantage of spin-waves intrinsic non-linearity.

A Machine Learning Model to Classify Dynamic Processes in Liquid Water*

The dynamics of water molecules plays a vital role in understanding water. We combined computer simulation and deep learning to study the dynamics of H-bonds between water molecules. Based on ab initio molecular dynamics simulations and a newly defined directed Hydrogen (H-) bond population operator, we studied a typical dynamic process in bulk water: interchange, in which the H-bond donor reverses roles with the acceptor. By designing a recurrent neural network-based model, we have successfully classified the interchange and breakage processes in water. We have found that the ratio between them is approximately 1 : 4, and it hardly depends on temperatures from 280 to 360 K. This work implies that deep learning has the great potential to help distinguish complex dynamic processes containing H-bonds in other systems.

Short- and long-term predictions of chaotic flows and extreme events: a physics-constrained reservoir computing approach

We propose a physics-constrained machine learning method—based on reservoir computing—to time-accurately predict extreme events and long-term velocity statistics in a model of chaotic flow. The method leverages the strengths of two different approaches: empirical modelling based on reservoir computing, which learns the chaotic dynamics from data only, and physical modelling based on conservation laws. This enables the reservoir computing framework to output physical predictions when training data are unavailable. We show that the combination of the two approaches is able to accurately reproduce the velocity statistics, and to predict the occurrence and amplitude of extreme events in a model of self-sustaining process in turbulence. In this flow, the extreme events are abrupt transitions from turbulent to quasi-laminar states, which are deterministic phenomena that cannot be traditionally predicted because of chaos. Furthermore, the physics-constrained machine learning method is shown to be robust with respect to noise. This work opens up new possibilities for synergistically enhancing data-driven methods with physical knowledge for the time-accurate prediction of chaotic flows.

Photonic-dispersion neural networks for inverse scattering problems

Inferring the properties of a scattering objective by analyzing the optical far-field responses within the framework of inverse problems is of great practical significance. However, it still faces major challenges when the parameter range is growing and involves inevitable experimental noises. Here, we propose a solving strategy containing robust neural-networks-based algorithms and informative photonic dispersions to overcome such challenges for a sort of inverse scattering problem-reconstructing grating profiles. Using two typical neural networks, forward-mapping type and inverse-mapping type, we reconstruct grating profiles whose geometric features span hundreds of nanometers with nanometric sensitivity and several seconds of time consumption. A forward-mapping neural network with a parameters-to-point architecture especially stands out in generating analytical photonic dispersions accurately, featured by sharp Fano-shaped spectra. Meanwhile, to implement the strategy experimentally, a Fourier-optics-based angle-resolved imaging spectroscopy with an all-fixed light path is developed to measure the dispersions by a single shot, acquiring adequate information. Our forward-mapping algorithm can enable real-time comparisons between robust predictions and experimental data with actual noises, showing an excellent linear correlation (R2 > 0.982) with the measurements of atomic force microscopy. Our work provides a new strategy for reconstructing grating profiles in inverse scattering problems.

Spatial and temporal super-resolution for fluorescence microscopy by a recurrent neural network

A novel spatial and temporal super-resolution (SR) framework based on a recurrent neural network (RNN) is demonstrated. In this work, we learn the complex yet useful features from the temporal data by taking advantage of structural characteristics of RNN and a skip connection. The usage of supervision mechanism is not only making full use of the intermediate output of each recurrent layer to recover the final output, but also alleviating vanishing/exploding gradients during the back-propagation. The proposed scheme achieves excellent reconstruction results, improving both the spatial and temporal resolution of fluorescence images including the simulated and real tubulin datasets. Besides, robustness against various critical metrics, such as the full-width at half-maximum (FWHM) and molecular density, can also be incorporated. In the validation, the performance can be increased by more than 20% for intensity profile, and 8% for FWHM, and the running time can be saved at least 40% compared with the classic Deep-STORM method, a high-performance net which is popularly used for comparison.

Arbitrary linear transformations for photons in the frequency synthetic dimension

Arbitrary linear transformations are of crucial importance in a plethora of photonic applications spanning classical signal processing, communication systems, quantum information processing and machine learning. Here, we present a photonic architecture to achieve arbitrary linear transformations by harnessing the synthetic frequency dimension of photons. Our structure consists of dynamically modulated micro-ring resonators that implement tunable couplings between multiple frequency modes carried by a single waveguide. By inverse design of these short- and long-range couplings using automatic differentiation, we realize arbitrary scattering matrices in synthetic space between the input and output frequency modes with near-unity fidelity and favorable scaling. We show that the same physical structure can be reconfigured to implement a wide variety of manipulations including single-frequency conversion, nonreciprocal frequency translations, and unitary as well as non-unitary transformations. Our approach enables compact, scalable and reconfigurable integrated photonic architectures to achieve arbitrary linear transformations in both the classical and quantum domains using current state-of-the-art technology.

Deep learning model to predict complex stress and strain fields in hierarchical composites

Materials-by-design is a paradigm to develop previously unknown high-performance materials. However, finding materials with superior properties is often computationally or experimentally intractable because of the astronomical number of combinations in design space. Here we report an AI-based approach, implemented in a game theory-based conditional generative adversarial neural network (cGAN), to bridge the gap between a material's microstructure-the design space-and physical performance. Our end-to-end deep learning model predicts physical fields like stress or strain directly from the material microstructure geometry, and reaches an astonishing accuracy not only for predicted field data but also for derivative material property predictions. Furthermore, the proposed approach offers extensibility by predicting complex materials behavior regardless of component shapes, boundary conditions, and geometrical hierarchy, providing perspectives of performing physical modeling and simulations. The method vastly improves the efficiency of evaluating physical properties of hierarchical materials directly from the geometry of its structural makeup.