Feedback control for microring weight banks

Multi-wavelength optical information processing with deep reinforcement learning

Multi-wavelength optical information processing systems are commonly utilized in optical neural networks and broadband signal processing. However, their effectiveness is often compromised by frequency-selective responses caused by fabrication, transmission, and environmental factors. To mitigate these issues, this study introduces a deep reinforcement learning calibration (DRC) method inspired by the deep deterministic policy gradient training strategy. This method continuously and autonomously learns from the system, effectively accumulating experiential knowledge for calibration strategies and demonstrating superior adaptability compared to traditional methods. In systems based on dispersion compensating fiber, micro-ring resonator array, and Mach-Zehnder interferometer array that use multi-wavelength optical carriers as the light source, the DRC method enables the completion of the corresponding signal processing functions within 21 iterations. This method provides efficient and accurate control, making it suitable for applications such as optical convolution computation acceleration, microwave photonic signal processing, and optical network routing.

On-chip real-time detection of optical frequency variations with ultrahigh resolution using the sine-cosine encoder approach

Real-time measurement of optical frequency variations (OFVs) is crucial for various applications including laser frequency control, optical computing, and optical sensing. Traditional devices, though accurate, are often too large, slow, and costly. Here we present a photonic integrated circuit (PIC) chip, utilizing the sine-cosine encoder principle, for high-speed and high-resolution real-time OFV measurement. Fabricated on a thin film lithium niobate (TFLN) platform, this chip-sized optical frequency detector (OFD) (5.5 mm × 2.7 mm) achieves a speed of up to 2500 THz/s and a resolution as fine as 2 MHz over a range exceeding 160 nm. Our robust algorithm overcomes the device imperfections and ensures precise quantification of OFV parameters. As a practical demonstration, the PIC OFD surpasses existing fiber Bragg grating (FBG) interrogators in sensitivity and speed for strain and vibration measurements. This work opens new avenues for on-chip OFV detection and offers significant potential for diverse applications involving OFV measurement.

Addressing optical modulator non-linearities for photonic neural networks

Within the context of neuromorphic computing, analog photonics, especially after the advent of photonic integrated technologies, offers unparalleled computing speeds per core, and the reduction of size and power consumption compared to digital electronics. However, the functionality of analog systems is limited by noise and non-linear distortions, which degrade signal resolution. Here, a method is presented for analyzing and minimizing the effect of non-linearities associated with the optical power transfer function of a generic modulator, to inform choices of design and operation conditions. The Mach-Zehnder interferometer, micro-ring modulator, and ring-assisted Mach-Zehnder interferometer are compared using this method. The analysis is applied to compare three analog photonic processor architectures for machine learning applications, based on wavelength, space, and time division multiplexing. Our results indicate that despite the lower maximum resolution exhibited by Mach-Zehnder interferometers, they are the most balanced choice for space and time division multiplexing architectures due to stability and power consumption.

Multimodal In-Sensor Computing System Using Integrated Silicon Photonic Convolutional Processor

Photonic integrated circuits offer miniaturized solutions for multimodal spectroscopic sensory systems by leveraging the simultaneous interaction of light with temperature, chemicals, and biomolecules, among others. The multimodal spectroscopic sensory data is complex and has huge data volume with high redundancy, thus requiring high communication bandwidth associated with high communication power consumption to transfer the sensory data. To circumvent this high communication cost, the photonic sensor and processor are brought into intimacy and propose a photonic multimodal in-sensor computing system using an integrated silicon photonic convolutional processor. A microring resonator crossbar array is used as the photonic processor to implement convolutional operation with 5-bit accuracy, validated through image edge detection tasks. Further integrating the processor with a photonic spectroscopic sensor, the in situ processing of multimodal spectroscopic sensory data is demonstrated, achieving the classification of protein species of different types and concentrations at various temperatures. A classification accuracy of 97.58% across 45 different classes is achieved. The multimodal in-sensor computing system demonstrates the feasibility of integrating photonic processors and photonic sensors to enhance the data processing capability of photonic devices at the edge.

Optical neural networks: progress and challenges

Artificial intelligence has prevailed in all trades and professions due to the assistance of big data resources, advanced algorithms, and high-performance electronic hardware. However, conventional computing hardware is inefficient at implementing complex tasks, in large part because the memory and processor in its computing architecture are separated, performing insufficiently in computing speed and energy consumption. In recent years, optical neural networks (ONNs) have made a range of research progress in optical computing due to advantages such as sub-nanosecond latency, low heat dissipation, and high parallelism. ONNs are in prospect to provide support regarding computing speed and energy consumption for the further development of artificial intelligence with a novel computing paradigm. Herein, we first introduce the design method and principle of ONNs based on various optical elements. Then, we successively review the non-integrated ONNs consisting of volume optical components and the integrated ONNs composed of on-chip components. Finally, we summarize and discuss the computational density, nonlinearity, scalability, and practical applications of ONNs, and comment on the challenges and perspectives of the ONNs in the future development trends.

Multimodal deep learning using on-chip diffractive optics with in situ training capability

Multimodal deep learning plays a pivotal role in supporting the processing and learning of diverse data types within the realm of artificial intelligence generated content (AIGC). However, most photonic neuromorphic processors for deep learning can only handle a single data modality (either vision or audio) due to the lack of abundant parameter training in optical domain. Here, we propose and demonstrate a trainable diffractive optical neural network (TDONN) chip based on on-chip diffractive optics with massive tunable elements to address these constraints. The TDONN chip includes one input layer, five hidden layers, and one output layer, and only one forward propagation is required to obtain the inference results without frequent optical-electrical conversion. The customized stochastic gradient descent algorithm and the drop-out mechanism are developed for photonic neurons to realize in situ training and fast convergence in the optical domain. The TDONN chip achieves a potential throughput of 217.6 tera-operations per second (TOPS) with high computing density (447.7 TOPS/mm2), high system-level energy efficiency (7.28 TOPS/W), and low optical latency (30.2 ps). The TDONN chip has successfully implemented four-class classification in different modalities (vision, audio, and touch) and achieve 85.7% accuracy on multimodal test sets. Our work opens up a new avenue for multimodal deep learning with integrated photonic processors, providing a potential solution for low-power AI large models using photonic technology.

Roadmapping the next generation of silicon photonics

Silicon photonics has developed into a mainstream technology driven by advances in optical communications. The current generation has led to a proliferation of integrated photonic devices from thousands to millions-mainly in the form of communication transceivers for data centers. Products in many exciting applications, such as sensing and computing, are around the corner. What will it take to increase the proliferation of silicon photonics from millions to billions of units shipped? What will the next generation of silicon photonics look like? What are the common threads in the integration and fabrication bottlenecks that silicon photonic applications face, and which emerging technologies can solve them? This perspective article is an attempt to answer such questions. We chart the generational trends in silicon photonics technology, drawing parallels from the generational definitions of CMOS technology. We identify the crucial challenges that must be solved to make giant strides in CMOS-foundry-compatible devices, circuits, integration, and packaging. We identify challenges critical to the next generation of systems and applications-in communication, signal processing, and sensing. By identifying and summarizing such challenges and opportunities, we aim to stimulate further research on devices, circuits, and systems for the silicon photonics ecosystem.

Redundancy-free integrated optical convolver for optical neural networks based on arrayed waveguide grating

Optical neural networks (ONNs) have gained significant attention due to their potential for high-speed and energy-efficient computation in artificial intelligence. The implementation of optical convolutions plays a vital role in ONNs, as they are fundamental operations within neural network architectures. However, state-of-the-art convolution architectures often suffer from redundant inputs, leading to substantial resource waste. Here, we demonstrate an integrated optical convolution architecture that leverages the inherent routing principles of arrayed waveguide grating (AWG) to execute the sliding of convolution kernel and summation of results. M × N multiply-accumulate (MAC) operations are facilitated by M + N units within a single clock cycle, thus eliminating the redundancy. In the experiment, we achieved 5 bit precision and 91.9 % accuracy in the handwritten digit recognition task confirming the reliability of our approach. Its redundancy-free architecture, low power consumption, high compute density (8.53 teraOP mm-1 s-1) and scalability make it a valuable contribution to the field of optical neural networks, thereby paving the way for future advancements in high-performance computing and artificial intelligence applications.

Human emotion recognition with a microcomb-enabled integrated optical neural network

State-of-the-art deep learning models can converse and interact with humans by understanding their emotions, but the exponential increase in model parameters has triggered an unprecedented demand for fast and low-power computing. Here, we propose a microcomb-enabled integrated optical neural network (MIONN) to perform the intelligent task of human emotion recognition at the speed of light and with low power consumption. Large-scale tensor data can be independently encoded in dozens of frequency channels generated by the on-chip microcomb and computed in parallel when flowing through the microring weight bank. To validate the proposed MIONN, we fabricated proof-of-concept chips and a prototype photonic-electronic artificial intelligence (AI) computing engine with a potential throughput up to 51.2 TOPS (tera-operations per second). We developed automatic feedback control procedures to ensure the stability and 8 bits weighting precision of the MIONN. The MIONN has successfully recognized six basic human emotions, and achieved 78.5 % accuracy on the blind test set. The proposed MIONN provides a high-speed and energy-efficient neuromorphic computing hardware for deep learning models with emotional interaction capabilities.

Comparison of Microcomb-Based Radio-Frequency Photonic Transversal Signal Processors Implemented with Discrete Components Versus Integrated Chips

RF photonic transversal signal processors, which combine reconfigurable electrical digital signal processing and high-bandwidth photonic processing, provide a powerful solution for achieving adaptive high-speed information processing. Recent progress in optical microcomb technology provides compelling multi-wavelength sources with a compact footprint, yielding a variety of microcomb-based RF photonic transversal signal processors with either discrete or integrated components. Although they operate based on the same principle, the processors in these two forms exhibit distinct performances. This paper presents a comparative investigation of their performances. First, we compare the performances of state-of-the-art processors, focusing on the processing accuracy. Next, we analyze various factors that contribute to the performance differences, including the tap number and imperfect response of experimental components. Finally, we discuss the potential for future improvement. These results provide a comprehensive comparison of microcomb-based RF photonic transversal signal processors implemented using discrete and integrated components and provide insights for their future development.

Broadband physical layer cognitive radio with an integrated photonic processor for blind source separation

The expansion of telecommunications incurs increasingly severe crosstalk and interference, and a physical layer cognitive method, called blind source separation (BSS), can effectively address these issues. BSS requires minimal prior knowledge to recover signals from their mixtures, agnostic to the carrier frequency, signal format, and channel conditions. However, previous electronic implementations did not fulfil this versatility due to the inherently narrow bandwidth of radio-frequency (RF) components, the high energy consumption of digital signal processors (DSP), and their shared weaknesses of low scalability. Here, we report a photonic BSS approach that inherits the advantages of optical devices and fully fulfils its "blindness" aspect. Using a microring weight bank integrated on a photonic chip, we demonstrate energy-efficient, wavelength-division multiplexing (WDM) scalable BSS across 19.2 GHz processing bandwidth. Our system also has a high (9-bit) resolution for signal demixing thanks to a recently developed dithering control method, resulting in higher signal-to-interference ratios (SIR) even for ill-conditioned mixtures.

Microcomb-based integrated photonic processing unit

The emergence of parallel convolution-operation technology has substantially powered the complexity and functionality of optical neural networks (ONN) by harnessing the dimension of optical wavelength. However, this advanced architecture faces remarkable challenges in high-level integration and on-chip operation. In this work, convolution based on time-wavelength plane stretching approach is implemented on a microcomb-driven chip-based photonic processing unit (PPU). To support the operation of this processing unit, we develop a dedicated control and operation protocol, leading to a record high weight precision of 9 bits. Moreover, the compact architecture and high data loading speed enable a preeminent photonic-core compute density of over 1 trillion of operations per second per square millimeter (TOPS mm^-2). Two proof-of-concept experiments are demonstrated, including image edge detection and handwritten digit recognition, showing comparable processing capability compared to that of a digital computer. Due to the advanced performance and the great scalability, this parallel photonic processing unit can potentially revolutionize sophisticated artificial intelligence tasks including autonomous driving, video action recognition and image reconstruction.

Quantization-aware training for low precision photonic neural networks

Recent advances in Deep Learning (DL) fueled the interest in developing neuromorphic hardware accelerators that can improve the computational speed and energy efficiency of existing accelerators. Among the most promising research directions towards this is photonic neuromorphic architectures, which can achieve femtojoule per MAC efficiencies. Despite the benefits that arise from the use of neuromorphic architectures, a significant bottleneck is the use of expensive high-speed and precision analog-to-digital (ADCs) and digital-to-analog conversion modules (DACs) required to transfer the electrical signals, originating from the various Artificial Neural Networks (ANNs) operations (inputs, weights, etc.) in the photonic optical engines. The main contribution of this paper is to study quantization phenomena in photonic models, induced by DACs/ADCs, as an additional noise/uncertainty source and to provide a photonics-compliant framework for training photonic DL models with limited precision, allowing for reducing the need for expensive high precision DACs/ADCs. The effectiveness of the proposed method is demonstrated using different architectures, ranging from fully connected and convolutional networks to recurrent architectures, following recent advances in photonic DL.

Two-layer integrated photonic architectures with multiport photodetectors for high-fidelity and energy-efficient matrix multiplications

Photonic integrated circuits (PICs) are emerging as a promising tool for accelerating matrix multiplications in deep learning. Previous PIC architectures, primarily focusing on the matrix-vector multiplication (MVM), have large hardware errors that increase with the device scale. In this work, we propose a novel PIC architecture for MVM, which features an intrinsically small hardware error that does not increase with the device scale. Moreover, we further develop this concept and propose a PIC architecture for the general matrix-matrix multiplication (GEMM), which allows the GEMM to be directly performed on a photonic chip with a high energy efficiency unattainable by parallel or sequential MVMs. This work provides a promising approach to realize a high fidelity and high energy efficiency optical computing platform.

Design automation of photonic resonator weights

Neuromorphic photonic processors based on resonator weight banks are an emerging candidate technology for enabling modern artificial intelligence (AI) in high speed analog systems. These purpose-built analog devices implement vector multiplications with the physics of resonator devices, offering efficiency, latency, and throughput advantages over equivalent electronic circuits. Along with these advantages, however, often come the difficult challenges of compensation for fabrication variations and environmental disturbances. In this paper, we review sources of variation and disturbances from our experiments, as well as mathematically define quantities that model them. Then, we introduce how the physics of resonators can be exploited to weight and sum multiwavelength signals. Finally, we outline automated design and control methodologies necessary to create practical, manufacturable, and high accuracy/precision resonator weight banks that can withstand operating conditions in the field. This represents a road map for unlocking the potential of resonator weight banks in practical deployment scenarios.

Photonic matrix multiplication lights up photonic accelerator and beyond

Matrix computation, as a fundamental building block of information processing in science and technology, contributes most of the computational overheads in modern signal processing and artificial intelligence algorithms. Photonic accelerators are designed to accelerate specific categories of computing in the optical domain, especially matrix multiplication, to address the growing demand for computing resources and capacity. Photonic matrix multiplication has much potential to expand the domain of telecommunication, and artificial intelligence benefiting from its superior performance. Recent research in photonic matrix multiplication has flourished and may provide opportunities to develop applications that are unachievable at present by conventional electronic processors. In this review, we first introduce the methods of photonic matrix multiplication, mainly including the plane light conversion method, Mach-Zehnder interferometer method and wavelength division multiplexing method. We also summarize the developmental milestones of photonic matrix multiplication and the related applications. Then, we review their detailed advances in applications to optical signal processing and artificial neural networks in recent years. Finally, we comment on the challenges and perspectives of photonic matrix multiplication and photonic acceleration.

Wideband Anti-Jamming Based on Free Space Optical Communication and Photonic Signal Processing

We propose and demonstrate an anti-jamming system to defend against wideband jamming attack. Free space optical communication is deployed to provide a reference for jamming cancellation. The mixed signal is processed and separated with photonic signal processing method to achieve large bandwidth. As an analog signal processing method, the cancellation system introduces zero latency. The radio frequency signals are modulated on optical carriers to achieve wideband and unanimous frequency response. With wideband and zero latency, the system meets the key requirements of high speed and real-time communications in transportation systems.

Blind source separation with integrated photonics and reduced dimensional statistics

Microwave communications have witnessed an incipient proliferation of multi-antenna and opportunistic technologies in the wake of an ever-growing demand for spectrum resources, while facing increasingly difficult network management over widespread channel interference and heterogeneous wireless broadcasting. Radio frequency (RF) blind source separation (BSS) is a powerful technique for demixing mixtures of unknown signals with minimal assumptions, but relies on frequency dependent RF electronics and prior knowledge of the target frequency band. We propose photonic BSS with unparalleled frequency agility supported by the tremendous bandwidths of photonic channels and devices. Specifically, our approach adopts an RF photonic front-end to process RF signals at various frequency bands within the same array of integrated microring resonators, and implements a novel two-step photonic BSS pipeline to reconstruct source identities from the reduced dimensional statistics of front-end output. We verify the feasibility and robustness of our approach by performing the first proof-of-concept photonic BSS experiments on mixed-over-the-air RF signals across multiple frequency bands. The proposed technique lays the groundwork for further research in interference cancellation, radio communications, and photonic information processing.

Measuring on-chip waveguide losses using a single, two-point coupled microring resonator

We demonstrate a method for measuring on-chip waveguide losses using a single microring resonator with a tunable coupler. By tuning the power coupling to the microring and measuring the microring's through-port transmission at each power coupling, one can separate the waveguide propagation loss and the effects of the coupling to the microring. This method is tolerant of fiber-chip coupling/alignment errors and does not require the use of expensive instruments for phase response measurements. In addition, this method offers a compact solution for measuring waveguide propagation losses, only using a single microring (230 µm×190 µm, including the metal pads). We demonstrate this method by measuring the propagation losses of silicon-on-insulator rib waveguides, yielding propagation losses of 3.1-1.3 dB/cm for core widths varying from 400-600 nm.

Photonic independent component analysis using an on-chip microring weight bank

Independent component analysis (ICA) is a general-purpose technique for analyzing multi-dimensional data to reveal the underlying hidden factors that are maximally independent from each other. We report the first photonic ICA on mixtures of unknown signals by employing an on-chip microring (MRR) weight bank. The MRR weight bank performs so-called weighted addition (i.e., multiply-accumulate) operations on the received mixtures, and outputs a single reduced-dimensional representation of the signal of interest. We propose a novel ICA algorithm to recover independent components solely based on the statistical information of the weighted addition output, while remaining blind to not only the original sources but also the waveform information of the mixtures. We investigate both channel separability and near-far problems, and our two-channel photonic ICA experiment demonstrates our scheme holds comparable performance with the conventional software-based ICA method. Our numerical simulation validates the fidelity of the proposed approach, and studies noise effects to identify the operating regime of our method. The proposed technique could open new domains for future research in blind source separation, microwave photonics, and on-chip information processing.

Photonic principal component analysis using an on-chip microring weight bank

Photonic principal component analysis (PCA) enables high-performance dimensionality reduction in wideband analog systems. In this paper, we report a photonic PCA approach using an on-chip microring (MRR) weight bank to perform weighted addition operations on correlated wavelength-division multiplexed (WDM) inputs. We are able to configure the MRR weight bank with record-high accuracy and precision, and generate multi-channel correlated input signals in a controllable manner. We also consider the realistic scenario in which the PCA procedure remains blind to the waveforms of both the input signals and weighted addition output, and propose a novel PCA algorithm that is able to extract principal components (PCs) solely based on the statistical information of the weighted addition output. Our experimental demonstration of two-channel photonic PCA produces PCs holding consistently high correspondence to those computed by a conventional software-based PCA method. Our numerical simulation further validates that our scheme can be generalized to high-dimensional (up to but not limited to eight-channel) PCA with good convergence. The proposed technique could bring new solutions to problems in microwave communications, ultrafast control, and on-chip information processing.