A high performance photonic pulse processing device

Emerging photoelectric devices for neuromorphic vision applications: principles, developments, and outlooks

The traditional von Neumann architecture is gradually failing to meet the urgent need for highly parallel computing, high-efficiency, and ultra-low power consumption for the current explosion of data. Brain-inspired neuromorphic computing can break the inherent limitations of traditional computers. Neuromorphic devices are the key hardware units of neuromorphic chips to implement the intelligent computing. In recent years, the development of optogenetics and photosensitive materials has provided new avenues for the research of neuromorphic devices. The emerging optoelectronic neuromorphic devices have received a lot of attentions because they have shown great potential in the field of visual bionics. In this paper, we summarize the latest visual bionic applications of optoelectronic synaptic memristors and transistors based on different photosensitive materials. The basic principle of bio-vision formation is first introduced. Then the device structures and operating mechanisms of optoelectronic memristors and transistors are discussed. Most importantly, the recent progresses of optoelectronic synaptic devices based on various photosensitive materials in the fields of visual perception are described. Finally, the problems and challenges of optoelectronic neuromorphic devices are summarized, and the future development of visual bionics is also proposed.

A photonic artificial synapse with a reversible multifaceted photochromic compound

Modern computational technology based on the von Neumann architecture physically partitions memory and the central processing unit, resulting in fundamental speed limitations and high energy consumption. On the other hand, the human brain is an extraordinary multifunctional organ composed of more than a billion neurons capable of simultaneously thinking, processing, and storing information. Neurons are interconnected with synapses that control information flow from pre-synaptic-to-post-synaptic neurons. Therefore, emulating synaptic functionalities and developing neuromorphic computational architecture has recently attracted much interest. Due to their high-speed, large bandwidth, and no interconnect-related power loss, photonic (all-optical) synapses can overcome the existing hurdles with electronic synapses. Here, we show an artificial photonic synapse by utilizing the well-established reversible, high-contrast photochromic organic compound, spiropyran, stimulated by optical pulses. Optical transmission of spiropyran significantly changes during spiropyran-merocyanine isomerization driven by UV-visible optical pulses. Such changes are equivalent to the biological synapses' inhibitory and excitatory synaptic actions. The slow relaxation to the initial state is considered as synaptic plasticity responsible for learning and memory formation. Short-term memory (STM), long-term memory (LTM), and transition from the STM to the LTM are demonstrated in all-optical synapses by modulating the stimuli's strength. The solvatochromic properties of spiropyran are further utilized to augment memory in synapses. Our work shows that photochromic organic compounds are excellent hosts for artificial photonic synapses and can be implemented in neuromorphic applications.

Advances in Emerging Photonic Memristive and Memristive-Like Devices

Possessing the merits of high efficiency, low consumption, and versatility, emerging photonic memristive and memristive-like devices exhibit an attractive future in constructing novel neuromorphic computing and miniaturized bionic electronic system. Recently, the potential of various emerging materials and structures for photonic memristive and memristive-like devices has attracted tremendous research efforts, generating various novel theories, mechanisms, and applications. Limited by the ambiguity of the mechanism and the reliability of the material, the development and commercialization of such devices are still rare and in their infancy. Therefore, a detailed and systematic review of photonic memristive and memristive-like devices is needed to further promote its development. In this review, the resistive switching mechanisms of photonic memristive and memristive-like devices are first elaborated. Then, a systematic investigation of the active materials, which induce a pivotal influence in the overall performance of photonic memristive and memristive-like devices, is highlighted and evaluated in various indicators. Finally, the recent advanced applications are summarized and discussed. In a word, it is believed that this review provides an extensive impact on many fields of photonic memristive and memristive-like devices, and lay a foundation for academic research and commercial applications.

Quantization-aware training for low precision photonic neural networks

Recent advances in Deep Learning (DL) fueled the interest in developing neuromorphic hardware accelerators that can improve the computational speed and energy efficiency of existing accelerators. Among the most promising research directions towards this is photonic neuromorphic architectures, which can achieve femtojoule per MAC efficiencies. Despite the benefits that arise from the use of neuromorphic architectures, a significant bottleneck is the use of expensive high-speed and precision analog-to-digital (ADCs) and digital-to-analog conversion modules (DACs) required to transfer the electrical signals, originating from the various Artificial Neural Networks (ANNs) operations (inputs, weights, etc.) in the photonic optical engines. The main contribution of this paper is to study quantization phenomena in photonic models, induced by DACs/ADCs, as an additional noise/uncertainty source and to provide a photonics-compliant framework for training photonic DL models with limited precision, allowing for reducing the need for expensive high precision DACs/ADCs. The effectiveness of the proposed method is demonstrated using different architectures, ranging from fully connected and convolutional networks to recurrent architectures, following recent advances in photonic DL.

Photonic spiking neural networks with event-driven femtojoule optoelectronic neurons based on Izhikevich-inspired model

Photonic spiking neural networks (PSNNs) potentially offer exceptionally high throughput and energy efficiency compared to their electronic neuromorphic counterparts while maintaining their benefits in terms of event-driven computing capability. While state-of-the-art PSNN designs require a continuous laser pump, this paper presents a monolithic optoelectronic PSNN hardware design consisting of an MZI mesh incoherent network and event-driven laser spiking neurons. We designed, prototyped, and experimentally demonstrated this event-driven neuron inspired by the Izhikevich model incorporating both excitatory and inhibitory optical spiking inputs and producing optical spiking outputs accordingly. The optoelectronic neurons consist of two photodetectors for excitatory and inhibitory optical spiking inputs, electrical transistors' circuits providing spiking nonlinearity, and a laser for optical spiking outputs. Additional inclusion of capacitors and resistors complete the Izhikevich-inspired optoelectronic neurons, which receive excitatory and inhibitory optical spikes as inputs from other optoelectronic neurons. We developed a detailed optoelectronic neuron model in Verilog-A and simulated the circuit-level operation of various cases with excitatory input and inhibitory input signals. The experimental results closely resemble the simulated results and demonstrate how the excitatory inputs trigger the optical spiking outputs while the inhibitory inputs suppress the outputs. The nanoscale neuron designed in our monolithic PSNN utilizes quantum impedance conversion. It shows that estimated 21.09 fJ/spike input can trigger the output from on-chip nanolasers running at a maximum of 10 Gspike/second in the neural network. Utilizing the simulated neuron model, we conducted simulations on MNIST handwritten digits recognition using fully connected (FC) and convolutional neural networks (CNN). The simulation results show 90% accuracy on unsupervised learning and 97% accuracy on a supervised modified FC neural network. The benchmark shows our PSNN can achieve 50 TOP/J energy efficiency, which corresponds to 100 × throughputs and 1000 × energy-efficiency improvements compared to state-of-art electrical neuromorphic hardware such as Loihi and NeuroGrid.

Photonics enabled intelligence system to identify SARS-CoV 2 mutations

Abstract

The COVID-19, MERS-CoV, and SARS-CoV are hazardous epidemics that have resulted in many deaths which caused a worldwide debate. Despite control efforts, SARS-CoV-2 continues to spread, and the fast spread of this highly infectious illness has posed a grave threat to global health. The effect of the SARS-CoV-2 mutation, on the other hand, has been characterized by worrying variations that modify viral characteristics in response to the changing resistance profile of the human population. The repeated transmission of virus mutation indicates that epidemics are likely to occur. Therefore, an early identification system of ongoing mutations of SARS-CoV-2 will provide essential insights for planning and avoiding future outbreaks. This article discussed the following highlights: First, comparing the omicron mutation with other variants; second, analysis and evaluation of the spread rate of the SARS-CoV 2 variations in the countries; third, identification of mutation areas in spike protein; and fourth, it discussed the photonics approaches enabled with artificial intelligence. Therefore, our goal is to identify the SARS-CoV 2 virus directly without the need for sample preparation or molecular amplification procedures. Furthermore, by connecting through the optical network, the COVID-19 test becomes a component of the Internet of healthcare things to improve precision, service efficiency, and flexibility and provide greater availability for the evaluation of the general population.

Key points

• A proposed framework of photonics based on AI for identifying and sorting SARS-CoV 2 mutations.

• Comparative scatter rates Omicron variant and other SARS-CoV 2 variations per country.

• Evaluating mutation areas in spike protein and AI enabled by photonic technologies for SARS-CoV 2 virus detection.

Collective and synchronous dynamics of photonic spiking neurons

Nonlinear dynamics of spiking neural networks have recently attracted much interest as an approach to understand possible information processing in the brain and apply it to artificial intelligence. Since information can be processed by collective spiking dynamics of neurons, the fine control of spiking dynamics is desirable for neuromorphic devices. Here we show that photonic spiking neurons implemented with paired nonlinear optical oscillators can be controlled to generate two modes of bio-realistic spiking dynamics by changing optical-pump amplitude. When the photonic neurons are coupled in a network, the interaction between them induces an effective change in the pump amplitude depending on the order parameter that characterizes synchronization. The experimental results show that the effective change causes spontaneous modification of the spiking modes and firing rates of clustered neurons, and such collective dynamics can be utilized to realize efficient heuristics for solving NP-hard combinatorial optimization problems.

Roadmap on emerging hardware and technology for machine learning

Recent progress in artificial intelligence is largely attributed to the rapid development of machine learning, especially in the algorithm and neural network models. However, it is the performance of the hardware, in particular the energy efficiency of a computing system that sets the fundamental limit of the capability of machine learning. Data-centric computing requires a revolution in hardware systems, since traditional digital computers based on transistors and the von Neumann architecture were not purposely designed for neuromorphic computing. A hardware platform based on emerging devices and new architecture is the hope for future computing with dramatically improved throughput and energy efficiency. Building such a system, nevertheless, faces a number of challenges, ranging from materials selection, device optimization, circuit fabrication and system integration, to name a few. The aim of this Roadmap is to present a snapshot of emerging hardware technologies that are potentially beneficial for machine learning, providing the Nanotechnology readers with a perspective of challenges and opportunities in this burgeoning field.

An all-optical neuron with sigmoid activation function

We present an all-optical neuron that utilizes a logistic sigmoid activation function, using a Wavelength-Division Multiplexing (WDM) input & weighting scheme. The activation function is realized by means of a deeply-saturated differentially-biased Semiconductor Optical Amplifier-Mach-Zehnder Interferometer (SOA-MZI) followed by a SOA-Cross-Gain-Modulation (XGM) gate. Its transfer function is both experimentally and theoretically analyzed, showing excellent agreement between theory and experiment and an almost perfect fitting with a logistic sigmoid function. The optical sigmoid transfer function is then exploited in the experimental demonstration of a photonic neuron, demonstrating successful thresholding over a 100psec-long pulse sequence with 4 different weighted-and-summed power levels.

Artificial neural networks enabled by nanophotonics

The growing demands of brain science and artificial intelligence create an urgent need for the development of artificial neural networks (ANNs) that can mimic the structural, functional and biological features of human neural networks. Nanophotonics, which is the study of the behaviour of light and the light-matter interaction at the nanometre scale, has unveiled new phenomena and led to new applications beyond the diffraction limit of light. These emerging nanophotonic devices have enabled scientists to develop paradigm shifts of research into ANNs. In the present review, we summarise the recent progress in nanophotonics for emulating the structural, functional and biological features of ANNs, directly or indirectly.

Light-Stimulated Synaptic Devices Utilizing Interfacial Effect of Organic Field-Effect Transistors

Synaptic transistors stimulated by light waves or photons may offer advantages to the devices, such as wide bandwidth, ultrafast signal transmission, and robustness. However, previously reported light-stimulated synaptic devices generally require special photoelectric properties from the semiconductors and sophisticated device's architectures. In this work, a simple and effective strategy for fabricating light-stimulated synaptic transistors is provided by utilizing interface charge trapping effect of organic field-effect transistors (OFETs). Significantly, our devices exhibited highly synapselike behaviors, such as excitatory postsynaptic current (EPSC) and pair-pulse facilitation (PPF), and presented memory and learning ability. The EPSC decay, PPF curves, and forgetting behavior can be well expressed by mathematical equations for synaptic devices, indicating that interfacial charge trapping effect of OFETs can be utilized as a reliable strategy to realize organic light-stimulated synapses. Therefore, this work provides a simple and effective strategy for fabricating light-stimulated synaptic transistors with both memory and learning ability, which enlightens a new direction for developing neuromorphic devices.

A lead-free two-dimensional perovskite for a high-performance flexible photoconductor and a light-stimulated synaptic device

Organo-lead halide perovskites have emerged as promising materials for high-performance photodetectors. However, the toxicity of lead cations in these materials limits their further applications. Here, a flexible photoconductor is developed based on lead-free two-dimensional (2D) perovskite (PEA)2SnI4via a one-step solution processing method. The flexible transparent electrodes are patterned from rGO/(PEDOT:PSS) hybrid films. The stability and reproducibility of the devices are significantly improved on adding 30 mol% SnF2 to the perovskite. The flexible photoconductors show a photoresponsivity of 16 A W-1 and a detectivity of 1.92 × 1011 Jones under 470 nm illumination, which are higher than those of most of the similar devices. Besides, the devices possess much better mechanical flexibility and durability than the flexible devices with an Au electrode. Finally, this flexible photoconductor is applied as a light-stimulated synaptic device and can mimic the short-term plasticity of biological synapses. This is the first study to report that lead-free 2D perovskite can be used in flexible photoconductors and synaptic devices.

Photonic implementation of a neuronal algorithm applicable towards angle of arrival detection and localization

A photonic system exemplifying the neurobiological learning algorithm, spike timing dependent plasticity (STDP), is experimentally demonstrated using the cooperative effects of cross gain modulation and nonlinear polarization rotation within an SOA. Furthermore, an STDP-based photonic approach towards the measurement of the angle of arrival (AOA) of a microwave signal is developed, and a three-dimensional AOA localization scheme is explored. Measurement accuracies on the order of tens of centimeters, rivaling that of complex positioning systems that utilize a large distribution of measuring units, are achieved for larger distances and with a simpler setup using just three STDP-based AOA units.

Pulse lead/lag timing detection for adaptive feedback and control based on optical spike-timing-dependent plasticity

Biological neurons perform information processing using a model called pulse processing, which is both computationally efficient and scalable, adopting the best features of both analog and digital computing. Implementing pulse processing with photonics can result in bandwidths that are billions of times faster than biological neurons and substantially faster than electronics. Neurons have the ability to learn and adapt their processing based on experience through a change in the strength of synaptic connections in response to spiking activity. This mechanism is called spike-timing-dependent plasticity (STDP). Functionally, STDP constitutes a mechanism in which strengths of connections between neurons are based on the timing and order between presynaptic spikes and postsynaptic spikes, essentially forming a pulse lead/lag timing detector that is useful in feedback control and adaptation. Here we report for the first time the demonstration of optical STDP that is useful in pulse lead/lag timing detection and apply it to automatic gain control of a photonic pulse processor.

Asynchronous spiking photonic neuron for lightwave neuromorphic signal processing

We developed an asynchronous spiking photonic neuron that forms the basic building block for hybrid analog/digital lightwave neuromorphic processing. Our approach enables completely asynchronous spiking in response to input signals while maximizing the throughput relative to synchronous approaches. Asynchronous operation is achieved by generating the spike source for the photonic neuron through four-wave mixing. This hybrid analog/digital photonic neuron has an electro-absorption modulator as the temporal integration unit for analog processing, while the digital processing portion employs optical thresholding in a highly Ge-doped nonlinear loop mirror.

Ultrafast all-optical implementation of a leaky integrate-and-fire neuron

In this paper, we demonstrate for the first time an ultrafast fully functional photonic spiking neuron. Our experimental setup constitutes a complete all-optical implementation of a leaky integrate-and-fire neuron, a computational primitive that provides a basis for general purpose analog optical computation. Unlike purely analog computational models, spiking operation eliminates noise accumulation and results in robust and efficient processing. Operating at gigahertz speed, which corresponds to at least 108 speed-up compared with biological neurons, the demonstrated neuron provides all functionality required by the spiking neuron model. The two demonstrated prototypes and a demonstrated feedback operation mode prove the feasibility and stability of our approach and show the obtained performance characteristics.

Signal feature recognition based on lightwave neuromorphic signal processing

We developed a hybrid analog/digital lightwave neuromorphic processing device that effectively performs signal feature recognition. The approach, which mimics the neurons in a crayfish responsible for the escape response mechanism, provides a fast and accurate reaction to its inputs. The analog processing portion of the device uses the integration characteristic of an electro-absorption modulator, while the digital processing portion employ optical thresholding in a highly Ge-doped nonlinear loop mirror. The device can be configured to respond to different sets of input patterns by simply varying the weights and delays of the inputs. We experimentally demonstrated the use of the proposed lightwave neuromorphic signal processing device for recognizing specific input patterns.