Photonic matrix multiplication lights up photonic accelerator and beyond

Probabilistic photonic computing for AI

Probabilistic computing excels in approximating combinatorial problems and modeling uncertainty. However, using conventional deterministic hardware for probabilistic models is challenging: (pseudo) random number generation introduces computational overhead and additional data shuffling. Therefore, there is a pressing need for different probabilistic computing architectures that achieve low latencies with reasonable energy consumption. Physical computing offers a promising solution, as these systems do not rely on an abstract deterministic representation of data but directly encode the information in physical quantities, enabling inherent probabilistic architectures utilizing entropy sources. Photonic computing is a prominent variant of physical computing due to the large available bandwidth, several orthogonal degrees of freedom for data encoding and optimal properties for in-memory computing and parallel data transfer. Here, we highlight key developments in physical photonic computing and photonic random number generation. We further provide insights into the realization of probabilistic photonic processors and their impact on artificial intelligence systems and future challenges.

Flex multimode neural network for complete optical computation

Compact and efficient photonic integrated circuits (PICs) are promising route to solving modern computing challenges. Traditional PICs using cascaded Mach-Zehnder Interferometers (MZIs) or micro-ring resonators (MRRs) are limited to rigid linear matrix operations, requiring electronics for data compression, nonlinear activation, and post-processing. The dependence on electronic processing counteracts the advantages brought by photonics. Here we propose a photonic chip that tackles this problem. The idea is to apply two sets of electrodes on a multimode waveguide: one set for data loading and the other for shaping the neural network by manipulating the multimode light interference flexibly. The shaping process, following a genetic algorithm, resorts again to optical computation to bypass the gradient acquisition problem. Once trained, the chip handles computation completely in the optical domain. Experimentally 91% classification accuracy is achieved on the Iris dataset. Our approach may bring PICs closer to practical computation applications without electronics overload.

Photonic neuromorphic accelerator for convolutional neural networks based on an integrated reconfigurable mesh

Photonic accelerators have risen as energy efficient, low latency counterparts to computational hungry digital modules for machine learning applications. On the other hand, upscaling integrated photonic circuits to meet the demands of state-of-the-art machine learning schemes such as convolutional layers, remains challenging. In this work, we experimentally validate a photonic-integrated neuromorphic accelerator that uses a hardware-friendly optical spectrum slicing technique through a reconfigurable silicon photonic mesh. The proposed scheme acts as an analogue convolutional engine, enabling information preprocessing in the optical domain, dimensionality reduction, and extraction of spatio-temporal features. Numerical results demonstrate that with only 7 photonic nodes, critical modules of a digital convolutional neural network can be replaced. As a result, a 98.6% accuracy on the MNIST dataset was numerically achieved, with an estimation of power consumption reduction up to 30% compared to digital convolutional neural networks. Experimental results using a reconfigurable silicon integrated chip confirm these findings, achieving 97.7% accuracy with only three optical nodes.

An ultra-compact integrated phase shifter via electrically tunable meta-waveguides

Integrated photonics has emerged as a pivotal technology in the advancement of next-generation computing and communication devices. Thermal optical phase shifters (OPSs) have been widely used to realize a tunable Mach-Zehnder interferometer (MZI) and a micro-ring resonator (MRR), which are the building bricks for the LSI/VLSI photonic integrated circuits. Due to the thermal crosstalk and the low modulation efficiency, thermal OPSs have large-scale size and high power consumption. In this work, we embed a Mie resonant metasurface into a waveguide and use the liquid crystal to tune the phase of the propagated light, which could realize a novel integrated phase shifter based on LC meta-waveguides. We use nanofabrication to prepare the meta-waveguide integrated MZI and MRR. By applying voltage, the output of the waveguide's intensity can be changed. Compared with thermo OPSs, this new modulator has the advantages of small size (20 μm × 0.35 μm), no thermal crosstalk, low power consumption (<10 nW), and easy large-scale integration. Additionally, we apply it to the convolutional architecture and verify that it has the potential to accelerate neural network computation.

Complex-valued matrix-vector multiplication using a scalable coherent photonic processor

Matrix-vector multiplication is a fundamental operation in modern signal processing and artificial intelligence. Developing a chip-scale photonic matrix-vector multiplication processor (MVMP) offers the potential for notably enhanced computing speed and energy efficiency beyond microelectronics. Here, we propose and demonstrate a 16-channel programmable on-chip coherent photonic processor capable of performing complex-valued matrix-vector multiplication at a computing speed of 1.28 tera-operations per second (TOPS). Low phase error Mach-Zehnder interferometers mesh and ultralow-loss broadened photonic waveguide delay lines are firstly combined for optical computing, enabling the encoding of amplitude and phase information, along with high-speed coherent detection. The proposed MVMP demonstrates high flexibility in implementing various functions, including arbitrary matrix transformation, parallel image processing, and handwritten digital recognition. Our work demonstrates the MVMP's advantages in scalability and function flexibility, enabled by the low-loss and low phase error designs, making a substantial advancement in high-speed and large-scale photonic computing technologies.

Anisotropic metamaterials for scalable photonic integrated circuits: a review on subwavelength gratings for high-density integration

Photonic integrated circuits (PICs) are transforming optical technology by miniaturizing complex photonic elements and systems onto single chips. However, scaling PICs to higher densities is constrained by optical crosstalk and device separation requirements, limiting both performance and size. Recent advancements in anisotropic metamaterials, particularly subwavelength gratings (SWGs), address these challenges by providing unprecedented control over evanescent fields and anisotropic perturbations in PICs. Here we review the role of anisotropic SWG metamaterials in enhancing integration density, detailing two foundational mechanisms - skin depth engineering and anisotropic perturbation - that mitigate crosstalk and enable advanced modal controls. We summarize their applications within four key functions: confinement manipulation, hetero-anisotropy and zero-birefringence, adiabatic mode conversion, and group velocity and dispersion control, showing how each benefits from distinct SWG properties. Finally, we discuss current limitations and future directions to expand the full potentials of anisotropic SWG metamaterials, toward highly dense and scalable PICs.

All-optical analog differential operation and information processing empowered by meta-devices

The burgeoning demand for high-performance computing, robust data processing, and rapid growth of big data necessitates the emergence of novel optical devices to efficiently execute demanding computational processes. The field of meta-devices, such as metamaterial or metasurface, has experienced unprecedented growth over the past two decades. By manipulating the amplitude, phase, polarization, and dispersion of light wavefronts in spatial, spectral, and temporal domains, viable solutions for the implementation of all-optical analog computation and information processing have been provided. In this review, we summarize the latest developments and emerging trends of computational meta-devices as innovative platforms for spatial optical analog differentiators and information processing. Based on the general concepts of spatial Fourier transform and Green's function, we analyze the physical mechanisms of meta-devices in the application of amplitude differentiation, phase differentiation, and temporal differentiation and summarize their applications in image edge detection, image edge enhancement, and beam shaping. Finally, we explore the current challenges and potential solutions in optical analog differentiators and provide perspectives on future research directions and possible developments.

Embedding matrices in programmable photonic networks with flexible depth and width

We show that programmable photonic circuit architectures composed of alternating mixing layers and active layers offer a high degree of flexibility. This alternating configuration enables the systematic tailoring of both the network's depth (number of layers) and width (size of each layer) without compromising computational capabilities. From a mathematical perspective, our approach can be viewed as embedding an arbitrary target matrix into a higher-dimensional matrix, which can then be represented with fewer layers and a larger number of active elements. We derive a general relation for the width and depth of a network that guarantees representing all N × N complex-valued matrix operations. Remarkably, we show that just two such active layers-interleaved with passive mixing layers-are sufficient to universally implement arbitrary matrix transformations. This result promises a more adaptable and scalable route to photonic matrix processors.

High-performance analog signal processing with photonic integrated circuits

Digital processing is our preferred way to manipulate data, as it gives us unparalleled flexibility. However, as the volume of information increases, fully digital electronic solutions are encountering memory, latency, and power challenges. A renewed interest is growing in analog signal processing, and photonics integrated circuits could really be a game-changing technology.

Cascaded-mode interferometers: Spectral shape and linewidth engineering

Interferometers are essential tools for measuring and shaping optical fields, widely used in optical metrology, sensing, laser physics, and quantum mechanics. They superimpose waves with a mutual phase delay, modifying light intensity. A frequency-dependent phase delay enables spectral shaping for filtering, routing, wave shaping, or multiplexing. Conventional Mach-Zehnder interferometers generate sinusoidal output intensities, limiting spectral engineering capabilities. Here, we propose a framework that uses interference of multiple transverse modes within a single multimode waveguide to achieve arbitrary spectral shapes in a compact geometry. Designed corrugated gratings couple these modes, enabling energy exchange akin to a beam splitter for easy multimode handling. We theoretically and experimentally demonstrate spectra with independently tunable linewidth and free spectral range, along with distinct spectral shapes for various transverse modes. Our method applies to orthogonal modes of different orders, polarization, and angular momentum, offering potential for sensing, calibration, metrology, and computing.

Temporal point-by-point arbitrary waveform synthesis beyond tera sample per second

Arbitrary waveform synthesizers are indispensable in modern information technology, yet electronic counterparts are limited by the speed of analog-to-digital converters to hundreds of GSa/s. While photonic-assisted synthesizers offer potential to surpass this ceiling, scalability and reconfigurability remain challenges. Here, we propose a temporal point-by-point arbitrary waveform synthesizer beyond TSa/s, leveraging an optical temporal Vernier caliper in the photonic synthetic dimension. The system, combining a mode-locked laser and a fiber loop, controls the sampling rate of synthesized waveforms by exploiting a slight detuning between the pulse period and the round-trip delay of the fiber loop. The experiment demonstrates generated waveforms with ultra-high, tunable sampling rate up to 1 TSa/s, an order of magnitude higher than state-of-the-art electronic counterparts. Additionally, the system supports up to 10.4 kilo-points in memory depth. As application examples, the generation of communication waveforms for high-speed wireless communications and linearly chirped microwave waveforms for high-resolution multi-target detection is demonstrated.

Inverse Design of Unitary Transmission Matrices in Silicon Photonic Coupled Waveguide Arrays Using a Neural Adjoint Model

The development of low-loss reconfigurable integrated optical devices enables further research into technologies including photonic signal processing, analogue quantum computing, and optical neural networks. Here, we introduce digital patterning of coupled waveguide arrays as a platform capable of implementing unitary matrix operations. Determining the required device geometry for a specific optical output is computationally challenging and requires a robust and versatile inverse design protocol. In this work we present an approach using high speed neural network surrogate-based gradient optimization, capable of predicting patterns of refractive index perturbations based on switching of the ultralow loss chalcogenide phase change material, antimony triselinide (Sb2Se3). Results for a 3 × 3 silicon waveguide array are presented, demonstrating control of both amplitude and phase for each transmission matrix element. Network performance is studied using neural network optimization tools such as data set augmentation and supplementation with random noise, resulting in an average fidelity of 0.94 for unitary matrix targets. Our results show that coupled waveguide arrays with perturbation patterns offer new routes for achieving programmable unitary operators, or Hamiltonians for quantum simulators, with a reduced footprint compared to conventional interferometer-mesh technology.

Photonic Chip Based on Ultrafast Laser-Induced Reversible Phase Change for Convolutional Neural Network

Photonic computing has emerged as a promising technology for the ever-increasing computational demands of machine learning and artificial intelligence. Due to the advantages in computing speed, integrated photonic chips have attracted wide research attention on performing convolutional neural network algorithm. Programmable photonic chips are vital for achieving practical applications of photonic computing. Herein, a programmable photonic chip based on ultrafast laser-induced phase change is fabricated for photonic computing. Through designing the ultrafast laser pulses, the Sb film integrated into photonic waveguides can be reversibly switched between crystalline and amorphous phase, resulting in a large contrast in refractive index and extinction coefficient. As a consequence, the light transmission of waveguides can be switched between write and erase states. To determine the phase change time, the transient laser-induced phase change dynamics of Sb film are revealed at atomic scale, and the time-resolved transient reflectivity is measured. Based on the integrated photonic chip, photonic convolutional neural networks are built to implement machine learning algorithm, and images recognition task is achieved. This work paves a route for fabricating programmable photonic chips by designed ultrafast laser, which will facilitate the application of photonic computing in artificial intelligence.

Asymmetrical estimator for training encapsulated deep photonic neural networks

Photonic neural networks (PNNs) are fast in-propagation and high bandwidth paradigms that aim to popularize reproducible NN acceleration with higher efficiency and lower cost. However, the training of PNN is known to be challenging, where the device-to-device and system-to-system variations create imperfect knowledge of the PNN. Despite backpropagation (BP)-based training algorithms being the industry standard for their robustness, generality, and fast gradient convergence for digital training, existing PNN-BP methods rely heavily on accurate intermediate state extraction or extensive computational resources for deep PNNs (DPNNs). The truncated photonic signal propagation and the computation overhead bottleneck DPNN's operation efficiency and increase system construction cost. Here, we introduce the asymmetrical training (AsyT) method, tailored for encapsulated DPNNs, where the signal is preserved in the analogue photonic domain for the entire structure. AsyT offers a lightweight solution for DPNNs with minimum readouts, fast and energy-efficient operation, and minimum system footprint. AsyT's ease of operation, error tolerance, and generality aim to promote PNN acceleration in a widened operational scenario despite the fabrication variations and imperfect controls. We demonstrated AsyT for encapsulated DPNN with integrated photonic chips, repeatably enhancing the performance from in-silico BP for different network structures and datasets.

Progress on intelligent metasurfaces for signal relay, transmitter, and processor

Pursuing higher data rate with limited spectral resources is a longstanding topic that has triggered the fast growth of modern wireless communication techniques. However, the massive deployment of active nodes to compensate for propagation loss necessitates high hardware expenditure, energy consumption, and maintenance cost, as well as complicated network interference issues. Intelligent metasurfaces, composed of a number of subwavelength passive or active meta-atoms, have recently found to be a new paradigm to actively reshape wireless communication environment in a green way, distinct from conventional works that passively adapt to the surrounding. In this review, we offer a unified perspective on how intelligent metasurfaces can facilitate wireless communication in three manners: signal relay, signal transmitter, and signal processor. We start by the basic modeling of wireless channel and the evolution of metasurfaces from passive, active to intelligent metasurfaces. Integrated with various deep learning algorithms, intelligent metasurfaces adapt to cater for the ever-changing environments without human intervention. Then, we overview specific experimental advancements using intelligent metasurfaces. We conclude by identifying key issues in the practical implementations of intelligent metasurfaces, and surveying new directions, such as gain metasurfaces and knowledge migration.

A guidance to intelligent metamaterials and metamaterials intelligence

The bidirectional interactions between metamaterials and artificial intelligence have recently attracted immense interest to motivate scientists to revisit respective communities, giving rise to the proliferation of intelligent metamaterials and metamaterials intelligence. Owning to the strong nonlinear fitting and generalization ability, artificial intelligence is poised to serve as a materials-savvy surrogate electromagnetic simulator and a high-speed computing nucleus that drives numerous self-driving metamaterial applications, such as invisibility cloak, imaging, detection, and wireless communication. In turn, metamaterials create a versatile electromagnetic manipulator for wave-based analogue computing to be complementary with conventional electronic computing. In this Review, we stand from a unified perspective to review the recent advancements in these two nascent fields. For intelligent metamaterials, we discuss how artificial intelligence, exemplified by deep learning, streamline the photonic design, foster independent working manner, and unearth latent physics. For metamaterials intelligence, we particularly unfold three canonical categories, i.e., wave-based neural network, mathematical operation, and logic operation, all of which directly execute computation, detection, and inference task in physical space. Finally, future challenges and perspectives are pinpointed, including data curation, knowledge migration, and imminent practice-oriented issues, with a great vision of ushering in the free management of entire electromagnetic space.

Silicon photonic convolution operator exploiting on-chip nonlinear activation function

Nonlinear activation functions (NAFs) are essential in artificial neural networks, enhancing learning capabilities by capturing complex input-output relationships. However, most NAF implementations rely on additional optoelectronic devices or digital computers, reducing the benefits of optical computing. To address this, we propose what we believe to be the first implementation of a nonlinear modulation process using an electro-optic IQ modulator on a silicon photonic convolution operator chip as a novel NAF. We validated this operator by constructing a convolutional neural network for radio machine learning classification, achieving 92.5% accuracy-an improvement of 27% over the case without a NAF. Compared with optoelectronic systems that rely on separate components, this fully integrated silicon photonic chip allows the NAF to execute nearly synchronously with the convolution operation, significantly lowering latency and reducing the complexity of the peripheral control system. This work paves the way for a large-scale on-chip optical neural network computation.

AlGaN/AlN heterostructures: an emerging platform for integrated photonics

We introduce a novel material for integrated photonics and investigate aluminum gallium nitride (AlGaN) on aluminum nitride (AlN) templates as a platform for developing reconfigurable and on-chip nonlinear optical devices. AlGaN combines compatibility with standard photonic fabrication technologies and high electro-optic modulation capabilities with low loss over a broad spectral range, from UVC to long-wave infrared, making it a viable material for complex photonic applications. In this work, we design and grow AlGaN/AlN heterostructures and integrate several photonic components. In particular, we fabricate edge couplers, low-loss waveguides, directional couplers, and tunable high-quality factor ring resonators. These devices will enable nonlinear light-matter interaction and quantum functionality. The comprehensive platform we present in this work paves the way for photon-pair generation applications, on-chip quantum frequency conversion, and fast electro-optic modulation for switching and routing classical and quantum light fields.

Spatial Optical Simulator for Classical Statistical Models

Optical simulators for the Ising model have demonstrated great promise for solving challenging problems in physics and beyond. Here, we develop a spatial optical simulator for a variety of classical statistical systems, including the clock, XY, Potts, and Heisenberg models, utilizing a digital micromirror device composed of a large number of tiny mirrors. Spins, with desired amplitudes or phases of the statistical models, are precisely encoded by a patch of mirrors with a superpixel approach. Then, by modulating the light field in a sequence of designed patterns, the spin-spin interaction is realized in such a way that the Hamiltonian symmetries are preserved. We successfully simulate statistical systems on a fully connected network, with ferromagnetic or Mattis-type random interactions, and observe the corresponding phase transitions between the paramagnetic and the ferromagnetic or spin-glass phases. Our results largely extend the research scope of spatial optical simulators and their versatile applications.

Multimodal In-Sensor Computing System Using Integrated Silicon Photonic Convolutional Processor

Photonic integrated circuits offer miniaturized solutions for multimodal spectroscopic sensory systems by leveraging the simultaneous interaction of light with temperature, chemicals, and biomolecules, among others. The multimodal spectroscopic sensory data is complex and has huge data volume with high redundancy, thus requiring high communication bandwidth associated with high communication power consumption to transfer the sensory data. To circumvent this high communication cost, the photonic sensor and processor are brought into intimacy and propose a photonic multimodal in-sensor computing system using an integrated silicon photonic convolutional processor. A microring resonator crossbar array is used as the photonic processor to implement convolutional operation with 5-bit accuracy, validated through image edge detection tasks. Further integrating the processor with a photonic spectroscopic sensor, the in situ processing of multimodal spectroscopic sensory data is demonstrated, achieving the classification of protein species of different types and concentrations at various temperatures. A classification accuracy of 97.58% across 45 different classes is achieved. The multimodal in-sensor computing system demonstrates the feasibility of integrating photonic processors and photonic sensors to enhance the data processing capability of photonic devices at the edge.

Miniaturized on-chip optical differentiator based on 2F-structured metasurfaces

Analog optical computing based on Fourier optics has attracted ever-growing attention, offering unprecedented low power consumption and high parallelism computation at the speed of light. Typically, classical optical 4F systems have been widely employed as one of the most common approaches for analog optical computing. However, most existing schemes replicate the original architecture relying on two Fourier transforms and one spatial-frequency filtering, resulting in bulky size and complex structure. Here, we propose a novel, to the best of our knowledge, on-chip 2F structure that achieves ultra-miniaturized optical analog computing. Taking advantage of the exceptional design flexibility of metasurfaces, we reduce the optical path length through a combination of phase compensation and complex amplitude modulation, thereby significantly simplifying the system structure without sacrificing accuracy compared to the traditional 4F system. As a proof-of-concept demonstration, we design and fabricate an on-chip optical differentiator on a silicon-on-insulator platform, achieving 84.01% and 79.81% differentiation accuracy in simulation and experiment, respectively.

Cascadable optical nonlinear activation function based on Ge-Si

To augment the capabilities of optical computing, specialized nonlinear devices as optical activation functions are crucial for enhancing the complexity of optical neural networks. However, existing optical nonlinear activation function devices often encounter challenges in preparation, compatibility, and multi-layer cascading. Here, we propose a cascadable optical nonlinear activation function architecture based on Ge-Si structured devices. Leveraging dual-source modulation, this architecture achieves cascading and wavelength switching by compensating for loss. Experimental comparisons with traditional Ge-Si devices validate the cascading capability of the new architecture. We first verified the versatility of this activation function in a MNIST task, and then in a multi-layer optical dense neural network designed for complex gesture recognition classification, the proposed architecture improves accuracy by an average of 23% compared to a linear network and 15% compared to a network with a traditional activation function architecture. With its advantages of cascadability and high compatibility, this work underscores the potential of all-optical activation functions for large-scale optical neural network scaling and complex task handling.

Optical neural networks: progress and challenges

Artificial intelligence has prevailed in all trades and professions due to the assistance of big data resources, advanced algorithms, and high-performance electronic hardware. However, conventional computing hardware is inefficient at implementing complex tasks, in large part because the memory and processor in its computing architecture are separated, performing insufficiently in computing speed and energy consumption. In recent years, optical neural networks (ONNs) have made a range of research progress in optical computing due to advantages such as sub-nanosecond latency, low heat dissipation, and high parallelism. ONNs are in prospect to provide support regarding computing speed and energy consumption for the further development of artificial intelligence with a novel computing paradigm. Herein, we first introduce the design method and principle of ONNs based on various optical elements. Then, we successively review the non-integrated ONNs consisting of volume optical components and the integrated ONNs composed of on-chip components. Finally, we summarize and discuss the computational density, nonlinearity, scalability, and practical applications of ONNs, and comment on the challenges and perspectives of the ONNs in the future development trends.

Harnessing the capabilities of VCSELs: unlocking the potential for advanced integrated photonic devices and systems

Vertical cavity surface emitting lasers (VCSELs) have emerged as a versatile and promising platform for developing advanced integrated photonic devices and systems due to their low power consumption, high modulation bandwidth, small footprint, excellent scalability, and compatibility with monolithic integration. By combining these unique capabilities of VCSELs with the functionalities offered by micro/nano optical structures (e.g. metasurfaces), it enables various versatile energy-efficient integrated photonic devices and systems with compact size, enhanced performance, and improved reliability and functionality. This review provides a comprehensive overview of the state-of-the-art versatile integrated photonic devices/systems based on VCSELs, including photonic neural networks, vortex beam emitters, holographic devices, beam deflectors, atomic sensors, and biosensors. By leveraging the capabilities of VCSELs, these integrated photonic devices/systems open up new opportunities in various fields, including artificial intelligence, large-capacity optical communication, imaging, biosensing, and so on. Through this comprehensive review, we aim to provide a detailed understanding of the pivotal role played by VCSELs in integrated photonics and highlight their significance in advancing the field towards efficient, compact, and versatile photonic solutions.

Revolutionizing breast cancer Ki-67 diagnosis: ultrasound radiomics and fully connected neural networks (FCNN) combination method

PURPOSE

This study aims to assess the diagnostic value of ultrasound habitat sub-region radiomics feature parameters using a fully connected neural networks (FCNN) combination method L2,1-norm in relation to breast cancer Ki-67 status.

METHODS

Ultrasound images from 528 cases of female breast cancer at the Affiliated Hospital of Xiangnan University and 232 cases of female breast cancer at the Affiliated Rehabilitation Hospital of Xiangnan University were selected for this study. We utilized deep learning methods to automatically outline the gross tumor volume and perform habitat clustering. Subsequently, habitat sub-regions were extracted to identify radiomics features and underwent feature engineering using the L1,2-norm. A prediction model for the Ki-67 status of breast cancer patients was then developed using a FCNN. The model's performance was evaluated using accuracy, area under the curve (AUC), specificity (Spe), positive predictive value (PPV), negative predictive value (NPV), Recall, and F1. In addition, calibration curves and clinical decision curves were plotted for the test set to visually assess the predictive accuracy and clinical benefit of the models.

RESULT

Based on the feature engineering using the L1,2-norm, a total of 9 core features were identified. The predictive model, constructed by the FCNN model based on these 9 features, achieved the following scores: ACC 0.856, AUC 0.915, Spe 0.843, PPV 0.920, NPV 0.747, Recall 0.974, and F1 0.890. Furthermore, calibration curves and clinical decision curves of the validation set demonstrated a high level of confidence in the model's performance and its clinical benefit.

CONCLUSION

Habitat clustering of ultrasound images of breast cancer is effectively supported by the combined implementation of the L1,2-norm and FCNN algorithms, allowing for the accurate classification of the Ki-67 status in breast cancer patients.

Tunable optical matrix convolution of 20-Gbit/s QPSK 2-D data with a kernel using optical wave mixing

Compared to its electronic counterpart, optically performed matrix convolution can accommodate phase-encoded data at high rates while avoiding optical-to-electronic-to-optical (OEO) conversions. We experimentally demonstrate a reconfigurable matrix convolution of quadrature phase-shift keying (QPSK)-encoded input data. The two-dimensional (2-D) input data is serialized, and its time-shifted replicas are generated. This 2-D data is convolved with a 1-D kernel with coefficients, which are applied by adjusting the relative phase and amplitude of the kernel pumps. Time-shifted data replicas (TSDRs) and kernel pumps are coherently mixed using nonlinear wave mixing in a periodically poled lithium niobate (PPLN) waveguide. To show the tunability and reconfigurability of this approach, we vary the kernel coefficients, kernel sizes (e.g., 2 × 1 or 3 × 1), and input data rates (e.g., 6-20 Gbit/s). The convolution results are verified to be error-free under an applied: (a) 2 × 1 kernel, resulting in a 16-quadrature amplitude modulation (QAM) output with an error vector magnitude (EVM) of ∼5.1-8.5%; and (b) 3 × 1 kernel, resulting in a 64-QAM output with an EVM of ∼4.9-5.5%.

Partial coherence enhances parallelized photonic computing

Advancements in optical coherence control1-5 have unlocked many cutting-edge applications, including long-haul communication, light detection and ranging (LiDAR) and optical coherence tomography6-8. Prevailing wisdom suggests that using more coherent light sources leads to enhanced system performance and device functionalities9-11. Our study introduces a photonic convolutional processing system that takes advantage of partially coherent light to boost computing parallelism without substantially sacrificing accuracy, potentially enabling larger-size photonic tensor cores. The reduction of the degree of coherence optimizes bandwidth use in the photonic convolutional processing system. This breakthrough challenges the traditional belief that coherence is essential or even advantageous in integrated photonic accelerators, thereby enabling the use of light sources with less rigorous feedback control and thermal-management requirements for high-throughput photonic computing. Here we demonstrate such a system in two photonic platforms for computing applications: a photonic tensor core using phase-change-material photonic memories that delivers parallel convolution operations to classify the gaits of ten patients with Parkinson's disease with 92.2% accuracy (92.7% theoretically) and a silicon photonic tensor core with embedded electro-absorption modulators (EAMs) to facilitate 0.108 tera operations per second (TOPS) convolutional processing for classifying the Modified National Institute of Standards and Technology (MNIST) handwritten digits dataset with 92.4% accuracy (95.0% theoretically).

Multimodal deep learning using on-chip diffractive optics with in situ training capability

Multimodal deep learning plays a pivotal role in supporting the processing and learning of diverse data types within the realm of artificial intelligence generated content (AIGC). However, most photonic neuromorphic processors for deep learning can only handle a single data modality (either vision or audio) due to the lack of abundant parameter training in optical domain. Here, we propose and demonstrate a trainable diffractive optical neural network (TDONN) chip based on on-chip diffractive optics with massive tunable elements to address these constraints. The TDONN chip includes one input layer, five hidden layers, and one output layer, and only one forward propagation is required to obtain the inference results without frequent optical-electrical conversion. The customized stochastic gradient descent algorithm and the drop-out mechanism are developed for photonic neurons to realize in situ training and fast convergence in the optical domain. The TDONN chip achieves a potential throughput of 217.6 tera-operations per second (TOPS) with high computing density (447.7 TOPS/mm2), high system-level energy efficiency (7.28 TOPS/W), and low optical latency (30.2 ps). The TDONN chip has successfully implemented four-class classification in different modalities (vision, audio, and touch) and achieve 85.7% accuracy on multimodal test sets. Our work opens up a new avenue for multimodal deep learning with integrated photonic processors, providing a potential solution for low-power AI large models using photonic technology.

A perspective on the artificial intelligence's transformative role in advancing diffractive optics

Artificial intelligence (AI) is transforming diffractive optics development through its advanced capabilities in design optimization, pattern generation, fabrication enhancement, performance forecasting, and customization. Utilizing AI algorithms like machine learning, generative models, and transformers, researchers can analyze extensive datasets to refine the design of diffractive optical elements (DOEs) tailored to specific applications and performance requirements. AI-driven pattern generation methods enable the creation of intricate and efficient optical structures that manipulate light with exceptional precision. Furthermore, AI optimizes manufacturing processes by fine-tuning fabrication parameters, resulting in higher quality and productivity. AI models also simulate diffractive optics behavior, accelerating design iterations and facilitating rapid prototyping. This integration of AI into diffractive optics holds tremendous potential to revolutionize optical technology applications across diverse sectors, spanning from imaging and sensing to telecommunications and beyond.

I/O-efficient iterative matrix inversion with photonic integrated circuits

Photonic integrated circuits have been extensively explored for optical processing with the aim of breaking the speed and energy efficiency bottlenecks of digital electronics. However, the input/output (IO) bottleneck remains one of the key barriers. Here we report a photonic iterative processor (PIP) for matrix-inversion-intensive applications. The direct reuse of inputted data in the optical domain unlocks the potential to break the IO bottleneck. We demonstrate notable IO advantages with a lossless PIP for real-valued matrix inversion and integral-differential equation solving, as well as a coherent PIP with optical loops integrated on-chip, enabling complex-valued computation and a net inversion time of 1.2 ns. Furthermore, we estimate at least an order of magnitude enhancement in IO efficiency of a PIP over photonic single-pass processors and the state-of-the-art electronic processors for reservoir training tasks and multiple-input and multiple-output (MIMO) precoding tasks, indicating the huge potential of PIP technology in practical applications.

Optoelectronic nonlinear Softmax operator based on diffractive neural networks

Softmax, a pervasive nonlinear operation, plays a pivotal role in numerous statistics and deep learning (DL) models such as ChatGPT. To compute it is expensive especially for at-scale models. Several software and hardware speed-up strategies are proposed but still suffer from low efficiency, poor scalability. Here we propose a photonic-computing solution including massive programmable neurons that is capable to execute such operation in an accurate, computation-efficient, robust and scalable manner. Experimental results show our diffraction-based computing system exhibits salient generalization ability in diverse artificial and real-world tasks (mean square error <10-5). We further analyze its performances against several realistic restricted factors. Such flexible system not only contributes to optimizing Softmax operation mechanism but may provide an inspiration of manufacturing a plug-and-play module for general optoelectronic accelerators.

On-chip silicon photonic micro-ring processor lights up optical image encryption

Optical image encryption has long been an important concept in the fields of photonic network processing and communication. Here, we propose a convolution-like operation-based optical image encryption algorithm exploiting a silicon photonic multiplexing architecture to achieve content security. Particularly, the encryption process is completed in a 3 × 3 cross-shaped photonic micro-ring resonator (MRR) array on chip. For the first time, to the best of our knowledge, this algorithm encodes information in an integrated intensity modulation, effectively reducing the encoding difficulty. Moreover, the high reliability and scalability of optical encryption are ensured using both linear and nonlinear operations on photonic chips according to characteristics of MRRs. As the encryption and decryption experiments show, the image restoration accuracy of our optical encryption algorithm exceeds 99% under real system noise at the pixel level, indicating its noise-robust property. Meanwhile, the peak signal-to-noise ratios of the restored and encrypted images are >60 and <15 dB, respectively, revealing both the high accuracy of the restored image and the small correlation between the encrypted and original images. This work adds to the rapidly expanding field of optical image encryption on photonic chips.

The Goldilocks principle of learning unitaries by interlacing fixed operators with programmable phase shifters on a photonic chip

Programmable photonic integrated circuits represent an emerging technology that amalgamates photonics and electronics, paving the way for light-based information processing at high speeds and low power consumption. Programmable photonics provides a flexible platform that can be reconfigured to perform multiple tasks, thereby holding great promise for revolutionizing future optical networks and quantum computing systems. Over the past decade, there has been constant progress in developing several different architectures for realizing programmable photonic circuits that allow for realizing arbitrary discrete unitary operations with light. Here, we systematically investigate a general family of photonic circuits for realizing arbitrary unitaries based on a simple architecture that interlaces a fixed intervening layer with programmable phase shifter layers. We introduce a criterion for the intervening operator that guarantees the universality of this architecture for representing arbitrary N × N unitary operators with N + 1 phase layers. We explore this criterion for different photonic components, including photonic waveguide lattices and meshes of directional couplers, which allows the identification of several families of photonic components that can serve as the intervening layers in the interlacing architecture. Our findings pave the way for efficiently designing and realizing novel families of programmable photonic integrated circuits for multipurpose analog information processing.

Multiplexable all-optical nonlinear activator for optical computing

As an alternative solution to surpass electronic neural networks, optical neural networks (ONNs) offer significant advantages in terms of energy consumption and computing speed. Despite the optical hardware platform could provide an efficient approach to realizing neural network algorithms than traditional hardware, the lack of optical nonlinearity limits the development of ONNs. Here, we proposed and experimentally demonstrated an all-optical nonlinear activator based on the stimulated Brillouin scattering (SBS). Utilizing the exceptional carrier dynamics of SBS, our activator supports two types of nonlinear functions, saturable absorption and rectified linear unit (Relu) models. Moreover, the proposed activator exhibits large dynamic response bandwidth (∼11.24 GHz), low nonlinear threshold (∼2.29 mW), high stability, and wavelength division multiplexing identities. These features have potential advantages for the physical realization of optical nonlinearities. As a proof of concept, we verify the performance of the proposed activator as an ONN nonlinear mapping unit via numerical simulations. Simulation shows that our approach achieves comparable performance to the activation functions commonly used in computers. The proposed approach provides support for the realization of all-optical neural networks.

Integrated multi-operand optical neurons for scalable and hardware-efficient deep learning

Optical neural networks (ONNs) are promising hardware platforms for next-generation neuromorphic computing due to their high parallelism, low latency, and low energy consumption. However, previous integrated photonic tensor cores (PTCs) consume numerous single-operand optical modulators for signal and weight encoding, leading to large area costs and high propagation loss to implement large tensor operations. This work proposes a scalable and efficient optical dot-product engine based on customized multi-operand photonic devices, namely multi-operand optical neuron (MOON). We experimentally demonstrate the utility of a MOON using a multi-operand-Mach-Zehnder-interferometer (MOMZI) in image recognition tasks. Specifically, our MOMZI-based ONN achieves a measured accuracy of 85.89 % in the street view house number (SVHN) recognition dataset with 4-bit voltage control precision. Furthermore, our performance analysis reveals that a 128 × 128 MOMZI-based PTCs outperform their counterparts based on single-operand MZIs by one to two order-of-magnitudes in propagation loss, optical delay, and total device footprint, with comparable matrix expressivity.

Spatio-spectral control of coherent nanophotonics

Fast modulation of optical signals that carry multidimensional information in the form of wavelength, phase or polarization has fueled an explosion of interest in integrated photonics. This interest however masks a significant challenge which is that independent modulation of multi-wavelength carrier signals in a single waveguide is not trivial. Such challenge is attributed to the longitudinal direction of guided-mode propagation, limiting the spatial separation and modulation of electric-field. Here, we overcome this using a single photonic element that utilizes active coherent (near) perfect absorption. We make use of standing wave patterns to exploit the spatial-degrees-of-freedom of in-plane modes and individually address elements according to their mode number. By combining the concept of coherent absorption in spatio-spectral domain with active phase-change nanoantennas, we engineer and test an integrated, reconfigurable and multi-spectral modulator operating within a single element. Our approach demonstrates for the first time, a non-volatile, wavelength-addressable element, providing a pathway for exploring the tunable capabilities in both spatial and spectral domains of coherent nanophotonics.

On-chip photonic convolution by phase-change in-memory computing cells with quasi-continuous tuning

Matrix multiplication acceleration by on-chip photonic integrated circuits (PICs) is emerging as one of the attractive and promising solutions, offering outstanding benefits in speed and bandwidth as compared to non-photonic approaches. Incorporating nonvolatile phase-change materials into PICs or devices enables optical storage and computing, surpassing their electrical counterparts. In this paper, we propose a design of on-chip photonic convolution for optical in-memory computing by integrating the phase change chalcogenide of Ge2Sb2Se4Te1 (GSST) into an asymmetric directional coupler for constructions of an in-memory computing cell, marrying the advantages of both the large bandwidth of Mach-Zehnder interferometers (MZIs) and the small size of micro-ring resonators (MRRs). Through quasi-continuous electro-thermal tuning of the GSST-integrated in-memory computing cells, numerical calculations about the optical and electro-thermal behaviors during GSST phase transition confirm the tunability of the programmable elements stored in the in-memory computing cells within [-1, 1]. For proof-of-concept verification, we apply the proposed optical convolutional kernel to a typical image edge detection application. As evidenced by the evaluation results, the prototype achieves the same accuracy as the convolution kernel implemented on a common digital computer, demonstrating the feasibility of the proposed scheme for on-chip photonic convolution and optical in-memory computing.

Programmable Photonic Time Circuits for Highly Scalable Universal Unitaries

Programmable photonic circuits (PPCs) have garnered substantial interest for their potential in facilitating deep learning accelerations and universal quantum computations. Although photonic computation using PPCs offers ultrafast operation, energy-efficient matrix calculations, and room-temperature quantum states, its poor scalability hinders integration. This challenge arises from the temporally one-shot operation of propagating light in conventional PPCs, resulting in a light-speed increase in device footprints. Here we propose the concept of programmable photonic time circuits, utilizing time-cycle-based computations analogous to gate cycling in the von Neumann architecture and quantum computation. Our building block is a reconfigurable SU(2) time gate, consisting of two resonators with tunable resonances, and coupled via time-coded dual-channel gauge fields. We demonstrate universal U(N) operations with high fidelity using an assembly of the SU(2) time gates, substantially improving scalability from O(N^{2}) to O(N) in terms of both the footprint and the number of gates. This result paves the way for PPC implementation in very large-scale integration.

Application of a reconfigurable all-optical activation unit based on optical injection into a bistable Fabry-Perot laser in multilayer perceptron neural networks

In this Letter, we theoretically investigate the application of a bistable Fabry-Perot semiconductor laser under optical injection as an all-optical activation unit for multilayer perceptron optical neural networks. The proposed device is programmed to provide reconfigurable sigmoid-like activation functions with adjustable thresholds and saturation points and benchmarked on machine learning image recognition problems. Due to the reconfigurability of the activation unit, the accuracy can be increased by up to 2% simply by adjusting the control parameter of the activation unit to suit the specific problem. For a simple two-layer perceptron neural network, we achieve inference accuracies of up to 95% and 85%, for the MNIST and Fashion-MNIST datasets, respectively.

Conflict-free joint decision by lag and zero-lag synchronization in laser network

With the end of Moore's Law and the increasing demand for computing, photonic accelerators are garnering considerable attention. This is due to the physical characteristics of light, such as high bandwidth and multiplicity, and the various synchronization phenomena that emerge in the realm of laser physics. These factors come into play as computer performance approaches its limits. In this study, we explore the application of a laser network, acting as a photonic accelerator, to the competitive multi-armed bandit problem. In this context, conflict avoidance is key to maximizing environmental rewards. We experimentally demonstrate cooperative decision-making using zero-lag and lag synchronization within a network of four semiconductor lasers. Lag synchronization of chaos realizes effective decision-making and zero-lag synchronization is responsible for the realization of the collision avoidance function. We experimentally verified a low collision rate and high reward in a fundamental 2-player, 2-slot scenario, and showed the scalability of this system. This system architecture opens up new possibilities for intelligent functionalities in laser dynamics.

Parallel edge extraction operators on chip speed up photonic convolutional neural networks

We experimentally establish a 3 × 3 cross-shaped micro-ring resonator (MRR) array-based photonic multiplexing architecture relying on silicon photonics to achieve parallel edge extraction operations in images for photonic convolution neural networks. The main mathematical operations involved are convolution. Precisely, a faster convolutional calculation speed of up to four times is achieved by extracting four feature maps simultaneously with the same photonic hardware's structure and power consumption, where a maximum computility of 0.742 TOPS at an energy cost of 48.6 mW and a convolution accuracy of 95.1% is achieved in an MRR array chip. In particular, our experimental results reveal that this system using parallel edge extraction operators instead of universal operators can improve the imaging recognition accuracy for CIFAR-10 dataset by 6.2% within the same computing time, reaching a maximum of 78.7%. This work presents high scalability and efficiency of parallel edge extraction chips, furnishing a novel, to the best of our knowledge, approach to boost photonic computing speed.

Optical ReLU-like activation function based on a semiconductor laser with optical injection

Artificial neural networks usually consist of successive linear multiply-accumulate operations and nonlinear activation functions. However, most optical neural networks only achieve the linear operation in the optical domain, while the optical implementation of activation function remains challenging. Here we present an optical ReLU-like activation function (with 180° rotation) based on a semiconductor laser subject to the optical injection in an experiment. The ReLU-like function is achieved in a broad regime above the Hopf bifurcation of the injection-locking diagram and is operated in the continuous-wave mode. In particular, the slope of the activation function is reconfigurable by tuning the frequency difference between the master laser and the slave laser.

Orbital angular momentum-mediated machine learning for high-accuracy mode-feature encoding

Machine learning with optical neural networks has featured unique advantages of the information processing including high speed, ultrawide bandwidths and low energy consumption because the optical dimensions (time, space, wavelength, and polarization) could be utilized to increase the degree of freedom. However, due to the lack of the capability to extract the information features in the orbital angular momentum (OAM) domain, the theoretically unlimited OAM states have never been exploited to represent the signal of the input/output nodes in the neural network model. Here, we demonstrate OAM-mediated machine learning with an all-optical convolutional neural network (CNN) based on Laguerre-Gaussian (LG) beam modes with diverse diffraction losses. The proposed CNN architecture is composed of a trainable OAM mode-dispersion impulse as a convolutional kernel for feature extraction, and deep-learning diffractive layers as a classifier. The resultant OAM mode-dispersion selectivity can be applied in information mode-feature encoding, leading to an accuracy as high as 97.2% for MNIST database through detecting the energy weighting coefficients of the encoded OAM modes, as well as a resistance to eavesdropping in point-to-point free-space transmission. Moreover, through extending the target encoded modes into multiplexed OAM states, we realize all-optical dimension reduction for anomaly detection with an accuracy of 85%. Our work provides a deep insight to the mechanism of machine learning with spatial modes basis, which can be further utilized to improve the performances of various machine-vision tasks by constructing the unsupervised learning-based auto-encoder.

Analog spatiotemporal feature extraction for cognitive radio-frequency sensing with integrated photonics

Analog feature extraction (AFE) is an appealing strategy for low-latency and efficient cognitive sensing systems since key features are much sparser than the Nyquist-sampled data. However, applying AFE to broadband radio-frequency (RF) scenarios is challenging due to the bandwidth and programmability bottlenecks of analog electronic circuitry. Here, we introduce a photonics-based scheme that extracts spatiotemporal features from broadband RF signals in the analog domain. The feature extractor structure inspired by convolutional neural networks is implemented on integrated photonic circuits to process RF signals from multiple antennas, extracting valid features from both temporal and spatial dimensions. Because of the tunability of the photonic devices, the photonic spatiotemporal feature extractor is trainable, which enhances the validity of the extracted features. Moreover, a digital-analog-hybrid transfer learning method is proposed for the effective and low-cost training of the photonic feature extractor. To validate our scheme, we demonstrate a radar target recognition task with a 4-GHz instantaneous bandwidth. Experimental results indicate that the photonic analog feature extractor tackles broadband RF signals and reduces the sampling rate of analog-to-digital converters to 1/4 of the Nyquist sampling while maintaining a high target recognition accuracy of 97.5%. Our scheme offers a promising path for exploiting the AFE strategy in the realm of cognitive RF sensing, with the potential to contribute to the efficient signal processing involved in applications such as autonomous driving, robotics, and smart factories.

Correlated optical convolutional neural network with "quantum speedup"

Compared with electrical neural networks, optical neural networks (ONNs) have the potentials to break the limit of the bandwidth and reduce the consumption of energy, and therefore draw much attention in recent years. By far, several types of ONNs have been implemented. However, the current ONNs cannot realize the acceleration as powerful as that indicated by the models like quantum neural networks. How to construct and realize an ONN with the quantum speedup is a huge challenge. Here, we propose theoretically and demonstrate experimentally a new type of optical convolutional neural network by introducing the optical correlation. It is called the correlated optical convolutional neural network (COCNN). We show that the COCNN can exhibit "quantum speedup" in the training process. The character is verified from the two aspects. One is the direct illustration of the faster convergence by comparing the loss function curves of the COCNN with that of the traditional convolutional neural network (CNN). Such a result is compatible with the training performance of the recently proposed quantum convolutional neural network (QCNN). The other is the demonstration of the COCNN's capability to perform the QCNN phase recognition circuit, validating the connection between the COCNN and the QCNN. Furthermore, we take the COCNN analog to the 3-qubit QCNN phase recognition circuit as an example and perform an experiment to show the soundness and the feasibility of it. The results perfectly match the theoretical calculations. Our proposal opens up a new avenue for realizing the ONNs with the quantum speedup, which will benefit the information processing in the era of big data.

Effects of cavity nonlinearities and linear losses on silicon microring-based reservoir computing

Microring resonators (MRRs) are promising devices for time-delay photonic reservoir computing, but the impact of the different physical effects taking place in the MRRs on the reservoir computing performance is yet to be fully understood. We numerically analyze the impact of linear losses as well as thermo-optic and free-carrier effects relaxation times on the prediction error of the time-series task NARMA-10. We demonstrate the existence of three regions, defined by the input power and the frequency detuning between the optical source and the microring resonance, that reveal the cavity transition from linear to nonlinear regimes. One of these regions offers very low error in time-series prediction under relatively low input power and number of nodes while the other regions either lack nonlinearity or become unstable. This study provides insight into the design of the MRR and the optimization of its physical properties for improving the prediction performance of time-delay reservoir computing.

Iterative eigensolver using fixed-point photonic primitive

Photonic computing has potential advantages in speed and energy consumption yet is subject to inaccuracy due to the limited equivalent bitwidth of the analog signal. In this Letter, we demonstrate a configurable, fixed-point coherent photonic iterative solver for numerical eigenvalue problems using shifted inverse iteration. The photonic primitive can accommodate arbitrarily sized sparse matrix-vector multiplication and is deployed to solve eigenmodes in a photonic waveguide structure. The photonic iterative eigensolver does not accumulate errors from each iteration, providing a path toward implementing scientific computing applications on photonic primitives.

Redundancy-free integrated optical convolver for optical neural networks based on arrayed waveguide grating

Optical neural networks (ONNs) have gained significant attention due to their potential for high-speed and energy-efficient computation in artificial intelligence. The implementation of optical convolutions plays a vital role in ONNs, as they are fundamental operations within neural network architectures. However, state-of-the-art convolution architectures often suffer from redundant inputs, leading to substantial resource waste. Here, we demonstrate an integrated optical convolution architecture that leverages the inherent routing principles of arrayed waveguide grating (AWG) to execute the sliding of convolution kernel and summation of results. M × N multiply-accumulate (MAC) operations are facilitated by M + N units within a single clock cycle, thus eliminating the redundancy. In the experiment, we achieved 5 bit precision and 91.9 % accuracy in the handwritten digit recognition task confirming the reliability of our approach. Its redundancy-free architecture, low power consumption, high compute density (8.53 teraOP mm-1 s-1) and scalability make it a valuable contribution to the field of optical neural networks, thereby paving the way for future advancements in high-performance computing and artificial intelligence applications.

A Design of High-Efficiency: Vertical Accumulation Modulators Based on Silicon Photonics

On-chip optical modulators, which are capable of converting electrical signals into optical signals, constitute the foundational components of photonic devices. Photonics modulators exhibiting high modulation efficiency and low insertion loss are highly sought after in numerous critical applications, such as optical phase steering, optical coherent imaging, and optical computing. This paper introduces a novel accumulation-type vertical modulator structure based on a silicon photonics platform. By incorporating a high-K dielectric layer of ZrO2, we have observed an increase in modulation efficiency while maintaining relatively low levels of modulation loss. Through meticulous study and optimization, the simulation results of the final device structure demonstrate a modulation efficiency of 0.16 V·cm, with a mere efficiency-loss product of 8.24 dB·V.