Spike Encoding with Optic Sensory Neurons Enable a Pulse Coupled Neural Network for Ultraviolet Image Segmentation

Bioinspired Target Detection Pipeline Based on Two-Dimensional Optoelectronic van der Waals Heterostructures

Noncooperative target detection in real-world scenarios relies on large-scale deep neural networks after image capture. However, directly implementing this detection pipeline under conventional optoelectronic sensors and computing units leads to physical bottlenecks in latency and energy consumption. Here, inspired by the biological visual attention mechanism and leveraging fabricated two-dimensional optoelectronic van der Waals heterostructure devices, we present a highly efficient neuromorphic in-sensor target detection pipeline. The inherent physical process of infrared self-driven visible photoresponse in heterostructures is used to simplify the originally complex processing in artificial intelligence (AI) optical detection algorithms. Specifically, the high-cost target localization and image fusion process can be directly implemented in the sensing unit. This manipulation decreases redundant information at the sensor level, reducing the burden on data transmission and backend computation. The results show that our bioinspired pipeline achieves a mean average precision (mAP) of 95.85% in detecting real-world scenes, even in extreme environments. Meanwhile, significant reductions in computing load (31.65%), latency (95.66%), and energy consumption (21.25%) can be attained compared with previous research. Our work provides a scalable material-for-AI solution for real-time, highly efficient target detection applications in real-world settings.

Multifunctional UV photodetect-memristors based on area selective fabricated Ga2S3/graphene/GaN van der Waals heterojunctions

Multifunctional devices based on van der Waals heterojunctions have drawn significant attention owing to their portable size, low power consumption and various application scenarios. However, high fabrication equipment requirements, complex device structures and limited operating conditions hinder their potential value. Herein, multifunctional UV photodetect-memristors based on Ga2S3/graphene/GaN van der Waals heterojunctions via area selective deposition have been proposed for the first time. The Ga2S3/graphene/GaN heterojunctions are firstly grown via area selective deposition (ASD) without a mask plate or lithography process. And the corresponding molecular dynamics (MD) and density functional theory (DFT) simulation further confirmed its feasibility and physical properties. Subsequently, multifunctional devices based on Ga2S3/graphene/GaN heterojunctions are fabricated accordingly, and exhibit ultrafast (<80 μs) response at 0 V and stable, highly sensitive (1150.4 A W-1) memory features at 3 V. Here, the huge hole barriers formed on the two edges of graphene set the foundation of trapping and detecting light-induced carriers. Afterwards, handwriting numeral recognition tasks are carried out based on the performance extracted from the device and a simplified noise filtering and improved recognition accuracy system is proposed, confirming its application potential in the artificial intelligence area. This study proposes a practical way to grow large-size 2D materials selectively, shows the valuable application potential of p-g-n heterojunctions in various application fields, and expands an innovative path of device development in the post-Moorish era.

Light-to-Spike Encoding Using Indium-Gallium-Zinc Oxide Phototransistor for all-Color Image Recognition with Dynamic Range and Precision Tunability

To enhance the efficiency of machine vision system, physical hardware capable of sensing and encoding is essential. However, sensing and encoding color information has been overlooked. Therefore, this work utilizes an indium-gallium-zinc oxide (IGZO) phototransistor to detect varying densities of red, green, and blue (RGB) light, converting them into corresponding drain current (ID) states. By applying stochastic gate voltage (VG) pulses to the IGZO phototransistor, the fluctuations are generated in these ID states. When the ID exceeds the threshold current (ITC), a spike signal is generated. This approach enables the conversion of light densities into spike signals, achieving spike-rate encoding. Moreover, adjusting the standard deviation (σ) of the VG pulses controls the range of light densities converted into spike rates, while altering the mean (μ) of the VG pulses changes the baseline level of spike rates. Remarkably, separate RGB channels offer a tunable encoding process, which can emphasize individual colors and correct color bias. The encoded spike rates are also fed into a spiking neural network (SNN) for CIFAR-10 pattern recognition, achieving an accuracy of 86%. The method allows the operation of SNN and shows the tunability in the process of light-to-spike encoding, opening possibilities for color image processing.

Memristive neuromorphic interfaces: integrating sensory modalities with artificial neural networks

The advent of the Internet of Things (IoT) has led to exponential growth in data generated from sensors, requiring efficient methods to process complex and unstructured external information. Unlike conventional von Neumann sensory systems with separate data collection and processing units, biological sensory systems integrate sensing, memory, and computing to process environmental information in real time with high efficiency. Memristive neuromorphic sensory systems using memristors as their basic components have emerged as promising alternatives to CMOS-based systems. Memristors can closely replicate the key characteristics of biological receptors, neurons, and synapses by integrating the threshold and adaptation properties of receptors, the action potential firing in neurons, and the synaptic plasticity of synapses. Furthermore, through careful engineering of their switching dynamics, the electrical properties of memristors can be tailored to emulate specific functions, while benefiting from high operational speed, low power consumption, and exceptional scalability. Consequently, their integration with high-performance sensors offers a promising pathway toward realizing fully integrated artificial sensory systems that can efficiently process and respond to diverse environmental stimuli in real time. In this review, we first introduce the fundamental principles of memristive neuromorphic technologies for artificial sensory systems, explaining how each component is structured and what functions it performs. We then discuss how these principles can be applied to replicate the four traditional senses, highlighting the underlying mechanisms and recent advances in mimicking biological sensory functions. Finally, we address the remaining challenges and provide prospects for the continued development of memristor-based artificial sensory systems.

Light-induced negative differential resistance and neural oscillations in neuromorphic photonic semiconductor micropillar sensory neurons

Neuromorphic systems, inspired by nature, are sought to efficiently process analogue inputs in real and complex environments. This could lead to ultralow-power in-sensor intelligent edge computers. Here, we present an artificial sensory oscillator neuron consisting of a III-V semiconductor micropillar quantum resonant tunnelling diode (RTD) with GaAs photosensitive absorption layers. The oscillatory optical neuron encodes incoming analogue optical data into spatiotemporal oscillatory signals. We demonstrate that near-infrared light within a certain intensity range activates a region of negative differential resistance, and subsequently, large-amplitude voltage oscillations. As a result, optic analogue information is encoded into electrical oscillations resulting in amplification of sensory light inputs. Under pulse-modulated light, excitation and inhibition of burst firing patterns can be controlled within a single oscillatory neuron, simulating neural activity in networks in the form of breather-type oscillatory phenomena. Such spatiotemporal oscillatory patterns (burst firing) form the basis for the combined sensing, pre-processing, and encoding abilities of the vision-nervous system found in biological organisms. This work paves the way for future artificial visual systems using III-V semiconductor nano-optoelectronic circuits in applications for light-driven neurorobotics, bioinspired optoelectronics, and in-sensor neuromorphic computing systems for real-time processing of sensory data.

Artificial Photothermal Nociceptor Using Mott Oscillators

Bioinspired sensory systems based on spike neural networks have received considerable attention in resolving high energy consumption and limited bandwidth in current sensory systems. To efficiently produce spike signals upon exposure to external stimuli, compact neuron devices are required for signal detection and their encoding into spikes in a single device. Herein, it is demonstrated that Mott oscillative spike neurons can integrate sensing and ceaseless spike generation in a compact form, which emulates the process of evoking photothermal sensing in the features of biological photothermal nociceptors. Interestingly, frequency-tunable and repetitive spikes are generated above the threshold value (Pth = 84 mW cm-2) as a characteristic of "threshold" in leaky-integrate-and-fire (LIF) neurons; the neuron devices successfully mimic a crucial feature of biological thermal nociceptors, including modulation of frequency coding and startup latency depending on the intensity of photothermal stimuli. Furthermore, Mott spike neurons are self-adapted after sensitization upon exposure to high-intensity electromagnetic radiation, which can replicate allodynia and hyperalgesia in a biological sensory system. Thus, this study presents a unique approach to capturing and encoding environmental source data into spikes, enabling efficient sensing of environmental sources for the application of adaptive sensory systems.

A stochastic photo-responsive memristive neuron for an in-sensor visual system based on a restricted Boltzmann machine

In-sensor computing has gained attention as a solution to overcome the von Neumann computing bottlenecks inherent in conventional sensory systems. This attention is due to the ability of sensor elements to directly extract meaningful information from external signals, thereby simplifying complex data. The advantage of in-sensor computing can be maximized with the sampling principle of a restricted Boltzmann machine (RBM) to extract significant features. In this study, a stochastic photo-responsive neuron is developed using a TiN/In-Ga-Zn-O/TiN optoelectronic memristor and an Ag/HfO2/Pt threshold-switching memristor, which can be configured as an input neuron in an in-sensor RBM. It demonstrates a sigmoidal switching probability depending on light intensity. The stochastic properties allow for the simultaneous exploration of various neuron states within the network, making identifying optimal features in complex images easier. Based on semi-empirical simulations, high recognition accuracies of 90.9% and 95.5% are achieved using handwritten digit and face image datasets, respectively. In addition, the in-sensor RBM effectively reconstructs abnormal face images, indicating that integrating in-sensor computing with probabilistic neural networks can lead to reliable and efficient image recognition under unpredictable real-world conditions.

Recent Advances in Artificial Sensory Neurons: Biological Fundamentals, Devices, Applications, and Challenges

Spike-based neural networks, which use spikes or action potentials to represent information, have gained a lot of attention because of their high energy efficiency and low power consumption. To fully leverage its advantages, converting the external analog signals to spikes is an essential prerequisite. Conventional approaches including analog-to-digital converters or ring oscillators, and sensors suffer from high power and area costs. Recent efforts are devoted to constructing artificial sensory neurons based on emerging devices inspired by the biological sensory system. They can simultaneously perform sensing and spike conversion, overcoming the deficiencies of traditional sensory systems. This review summarizes and benchmarks the recent progress of artificial sensory neurons. It starts with the presentation of various mechanisms of biological signal transduction, followed by the systematic introduction of the emerging devices employed for artificial sensory neurons. Furthermore, the implementations with different perceptual capabilities are briefly outlined and the key metrics and potential applications are also provided. Finally, we highlight the challenges and perspectives for the future development of artificial sensory neurons.

Inspiration from Visual Ecology for Advancing Multifunctional Robotic Vision Systems: Bio-inspired Electronic Eyes and Neuromorphic Image Sensors

In robotics, particularly for autonomous navigation and human-robot collaboration, the significance of unconventional imaging techniques and efficient data processing capabilities is paramount. The unstructured environments encountered by robots, coupled with complex missions assigned to them, present numerous challenges necessitating diverse visual functionalities, and consequently, the development of multifunctional robotic vision systems has become indispensable. Meanwhile, rich diversity inherent in animal vision systems, honed over evolutionary epochs to meet their survival demands across varied habitats, serves as a profound source of inspirations. Here, recent advancements in multifunctional robotic vision systems drawing inspiration from natural ocular structures and their visual perception mechanisms are delineated. First, unique imaging functionalities of natural eyes across terrestrial, aerial, and aquatic habitats and visual signal processing mechanism of humans are explored. Then, designs and functionalities of bio-inspired electronic eyes are explored, engineered to mimic key components and underlying optical principles of natural eyes. Furthermore, neuromorphic image sensors are discussed, emulating functional properties of synapses, neurons, and retinas and thereby enhancing accuracy and efficiency of robotic vision tasks. Next, integration examples of electronic eyes with mobile robotic/biological systems are introduced. Finally, a forward-looking outlook on the development of bio-inspired electronic eyes and neuromorphic image sensors is provided.

A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO2 Mott Memristor for the Spike-Based Electronic Retina

The neuromorphic vision system that utilizes spikes as information carriers is crucial for the formation of spiking neural networks. Here, we present a bioinspired flexible artificial spiking photoreceptor (ASP), which is realized by using a single VO2 Mott memristor that can simultaneously sense and encode the stimulus light into spikes. The ASP has high spike-encoded photosensitivity and ultrawide photosensing range (405-808 nm) with good endurance (>7 × 107) and high flexibility (bending radius ∼5 mm). Then, we put forward an all-spike electronic retina architecture that comprises one layer of ASPs and one layer of artificial optical nerves (AONs) to process the spike information. Each AON consists of a single Mott memristor connected in series with a neuro-transistor that is a multiple-input floating-gate MOS transistor. Simulation results demonstrate that the all-spike electronic retina can successfully segment images with high Shannon entropy, thus laying the foundation for the development of a spike-based neuromorphic vision system.

Toward a Brain-Neuromorphics Interface

Brain-computer interfaces (BCIs) that enable human-machine interaction have immense potential in restoring or augmenting human capabilities. Traditional BCIs are realized based on complementary metal-oxide-semiconductor (CMOS) technologies with complex, bulky, and low biocompatible circuits, and suffer with the low energy efficiency of the von Neumann architecture. The brain-neuromorphics interface (BNI) would offer a promising solution to advance the BCI technologies and shape the interactions with machineries. Neuromorphic devices and systems are able to provide substantial computation power with extremely high energy-efficiency by implementing in-materia computing such as in situ vector-matrix multiplication (VMM) and physical reservoir computing. Recent progresses on integrating neuromorphic components with sensing and/or actuating modules, give birth to the neuromorphic afferent nerve, efferent nerve, sensorimotor loop, and so on, which has advanced the technologies for future neurorobotics by achieving sophisticated sensorimotor capabilities as the biological system. With the development on the compact artificial spiking neuron and bioelectronic interfaces, the seamless communication between a BNI and a bioentity is reasonably expectable. In this review, the upcoming BNIs are profiled by introducing the brief history of neuromorphics, reviewing the recent progresses on related areas, and discussing the future advances and challenges that lie ahead.

An Ion-Mediated Spiking Chemical Neuron based on Mott Memristor

Artificial spiking neurons capable of interpreting ionic information into electrical spikes are critical to mimic biological signaling systems. Mott memristors are attractive for constructing artificial spiking neurons due to their simple structure, low energy consumption, and rich neural dynamics. However, challenges remain in achieving ion-mediated spiking and biohybrid-interfacing in Mott neurons. Here, a biomimetic spiking chemical neuron (SCN) utilizing an NbOx Mott memristor and oxide field-effect transistor-type chemical sensor is introduced. The SCN exhibits both excitation and inhibition spiking behaviors toward ionic concentrations akin to biological neural systems. It demonstrates spiking responses across physiological and pathological Na+ concentrations (1-200 × 10-3 m). The Na+-mediated SCN enables both frequency encoding and time-to-first-spike coding schemes, illustrating the rich neural dynamics of Mott neuron. In addition, the SCN interfaced with L929 cells facilitates real-time modulation of ion-mediated spiking under both normal and salty cellular microenvironments.

Augmented reality navigation method based on image segmentation and sensor tracking registration technology

With the rapid development of modern science and technology, navigation technology provides great convenience for people's life, but the problem of inaccurate localization in complex environments has always been a challenge that navigation technology needs to be solved urgently. To address this challenge, this paper proposes an augmented reality navigation method that combines image segmentation and multi-sensor fusion tracking registration. The method optimizes the image processing process through the GA-OTSU-Canny algorithm and combines high-precision multi-sensor information in order to achieve accurate tracking of positioning and guidance in complex environments. Experimental results show that the GA-OTSU-Canny algorithm has a faster image edge segmentation rate, and the fastest start speed is only 1.8 s, and the fastest intersection selection time is 1.2 s. The navigation system combining the image segmentation and sensor tracking and registration techniques has a highly efficient performance in real-world navigation, and its building recognition rates are all above 99%. The augmented reality navigation system not only improves the navigation accuracy in high-rise and urban canyon environments, but also significantly outperforms traditional navigation solutions in terms of navigation startup time and target building recognition accuracy. In summary, this research not only provides a new framework for the theoretical integration of image processing and multi-sensor data, but also brings innovative technical solutions for the development and application of practical navigation systems.

Human somatosensory systems based on sensor-memory-integrated technology

As a representative artificial neural network (ANN) for incorporating sensing functions and memory functions into one system to achieve highly miniaturized and highly integrated devices or systems, artificial sensory systems (ASSs) can have a far-reaching influence on precise instrumentation, sensing, and automation engineering. Artificial sensory systems have enjoyed considerable progress in recent years, from low degree integrations to highly advanced sophisticated integrations, from single-modal perceptions to multimode-fused perceptions. However, there are issues around the large hardware area, power consumption, and communication bandwidth needed during the processes where multimodal sensing signals are converted into a digital mode before they can be processed by a digital processor. Therefore, deepening the research into sensory integration is of great importance. In this review, we briefly introduce fundamental knowledge about the memristor mechanism, describe some representative human somatosensory systems, and elucidate the relationship between the properties of memristor devices and the structure. The electronic character of the sensors, future prospects, and key challenges surrounding sensor-memory integrated technologies are also discussed.

Optically Tunable Electrical Oscillations in Oxide-Based Memristors for Neuromorphic Computing

The application of hardware-based neural networks can be enhanced by integrating sensory neurons and synapses that enable direct input from external stimuli. This work reports direct optical control of an oscillatory neuron based on volatile threshold switching in V3O5. The devices exhibit electroforming-free operation with switching parameters that can be tuned by optical illumination. Using temperature-dependent electrical measurements, conductive atomic force microscopy (C-AFM), in situ thermal imaging, and lumped element modelling, it is shown that the changes in switching parameters, including threshold and hold voltages, arise from overall conductivity increase of the oxide film due to the contribution of both photoconductive and bolometric characteristics of V3O5, which eventually affects the oscillation dynamics. Furthermore, V3O5 is identified as a new bolometric material with a temperature coefficient of resistance (TCR) as high as -4.6% K-1 at 423 K. The utility of these devices is illustrated by demonstrating in-sensor reservoir computing with reduced computational effort and an optical encoding layer for spiking neural network (SNN), respectively, using a simulated array of devices.

Ovonic threshold switching-based artificial afferent neurons for thermal in-sensor computing

Artificial afferent neurons in the sensory nervous system inspired by biology have enormous potential for efficiently perceiving and processing environmental information. However, the previously reported artificial afferent neurons suffer from two prominent challenges: considerable power consumption and limited scalability efficiency. Herein, addressing these challenges, a bioinspired artificial thermal afferent neuron based on a N-doped SiTe ovonic threshold switching (OTS) device is presented for the first time. The engineered OTS device shows remarkable uniformity and robust endurance, ensuring the reliability and efficacy of the artificial afferent neurons. A substantially decreased leakage current of the SiTe OTS device by nitrogen doping results in ultra-low power consumption less than 0.3 nJ per spike for artificial afferent neurons. The inherent temperature response exhibited by N-doped SiTe OTS materials allows us to construct a highly compact artificial thermal afferent neuron over a wide temperature range. An edge detection task is performed to further verify its thermal perceptual computing function. Our work provides an insight into OTS-based artificial afferent neurons for electronic skin and sensory neurorobotics.

An artificial visual neuron with multiplexed rate and time-to-first-spike coding

Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial visual neurons for use in spiking neural network (SNN). However, the lack of multiplexed data coding schemes reduces the ability of artificial visual neurons in SNN to emulate the visual perception ability of biological systems. Here, we present an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The artificial neuron can code visual information at different spiking frequencies (rate coding) and enables precise and energy-efficient time-to-first-spike (TTFS) coding. This multiplexed sensory coding scheme could improve the computing capability and efficacy of artificial visual neurons. A hardware-based SNN with the RTF coding scheme exhibits good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The multiplexed RTF coding scheme demonstrates the feasibility of developing highly efficient spike-based neuromorphic hardware.

High-order sensory processing nanocircuit based on coupled VO2 oscillators

Conventional circuit elements are constrained by limitations in area and power efficiency at processing physical signals. Recently, researchers have delved into high-order dynamics and coupled oscillation dynamics utilizing Mott devices, revealing potent nonlinear computing capabilities. However, the intricate yet manageable population dynamics of multiple artificial sensory neurons with spatiotemporal coupling remain unexplored. Here, we present an experimental hardware demonstration featuring a capacitance-coupled VO2 phase-change oscillatory network. This network serves as a continuous-time dynamic system for sensory pre-processing and encodes information in phase differences. Besides, a decision-making module for special post-processing through software simulation is designed to complete a bio-inspired dynamic sensory system. Our experiments provide compelling evidence that this transistor-free coupling network excels in sensory processing tasks such as touch recognition and gesture recognition, achieving significant advantages of fewer devices and lower energy-delay-product compared to conventional methods. This work paves the way towards an efficient and compact neuromorphic sensory system based on nano-scale nonlinear dynamics.

Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision

The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.

Artificial sensory system based on memristive devices

In the biological nervous system, the integration and cooperation of parallel system of receptors, neurons, and synapses allow efficient detection and processing of intricate and disordered external information. Such systems acquire and process environmental data in real-time, efficiently handling complex tasks with minimal energy consumption. Memristors can mimic typical biological receptors, neurons, and synapses by implementing key features of neuronal signal-processing functions such as selective adaption in receptors, leaky integrate-and-fire in neurons, and synaptic plasticity in synapses. External stimuli are sensitively detected and filtered by "artificial receptors," encoded into spike signals via "artificial neurons," and integrated and stored through "artificial synapses." The high operational speed, low power consumption, and superior scalability of memristive devices make their integration with high-performance sensors a promising approach for creating integrated artificial sensory systems. These integrated systems can extract useful data from a large volume of raw data, facilitating real-time detection and processing of environmental information. This review explores the recent advances in memristor-based artificial sensory systems. The authors begin with the requirements of artificial sensory elements and then present an in-depth review of such elements demonstrated by memristive devices. Finally, the major challenges and opportunities in the development of memristor-based artificial sensory systems are discussed.

Noisy image segmentation based on synchronous dynamics of coupled photonic spiking neurons

The collective dynamics in neural networks is essential for information processing and has attracted much interest on the application in artificial intelligence. Synchronization is one of the most dominant phenomenon in the collective dynamics of neural network. Here, we propose to use the spiking dynamics and collective synchronization of coupled photonic spiking neurons for noisy image segmentation. Based on the synchronization mechanism and synchronization control, the noised pattern segmentation is demonstrated numerically. This work provides insight into the possible application based on the collective dynamics of large-scale photonic networks and opens a way for ultra-high speed image processing.

Memristors based on NdNiO3 nanocrystals film as sensory neurons for neuromorphic computing

By mimicking the behavior of the human brain, artificial neural systems offer the possibility to further improve computing efficiency and solve the von Neumann bottleneck. In particular, neural systems with perceptual capability expand the application field and lay a good foundation for the construction of perceptual storage and computational systems. However, research on neurons with perceptual functions is still relatively scarce, with most works focusing on optoelectronic synapses. The neuron is important for neuromorphic computing systems because neurons output excitatory or inhibitory stimuli to regulate the weight of synapses. Therefore, the construction of sensory neurons is crucial to expand the application range of brain-like neural computing. Here, an artificial sensory neuron is proposed, which is constructed using a photosensitive bipolar threshold switching memristor based on NdNiO3 (NNO) nanocrystals. These metallic phase nanocrystals can not only enhance the local electric field, but also act as a reservoir for defects (VoS) to guide the growth of conductive filaments and stabilize the performance of the device. They present stable bipolar threshold switching behavior with a low 120 nW set power, and the operating voltages decreased in light due to photocarrier action. A leaky integrate firing (LIF) neuron has been realized, which achieved key biological neuron functions, such as all-or-nothing spiking, threshold-driven firing, refractory period, and spiking frequency modulation. The LIF neurons receiving optical inputs have the properties of an artificial sensory neuron. It could regulate the spiking output frequency at different light densities, which could be used for a ship approaching a port. This work provides a promising hardware implementation towards constructing high-performance artificial intelligence to assist ships at night in a sensory system.

One-Phototransistor-One-Memristor Array with High-Linearity Light-Tunable Weight for Optic Neuromorphic Computing

The recent advances in optic neuromorphic devices have led to a subsequent rise in use for construction of energy-efficient artificial-vision systems. The widespread use can be attributed to their ability to capture, store, and process visual information from the environment. The primary limitations of existing optic neuromorphic devices include nonlinear weight updates, cross-talk issues, and silicon process incompatibility. In this study, a highly linear, light-tunable, cross-talk-free, and silicon-compatible one-phototransistor-one-memristor (1PT1R) optic memristor is experimentally demonstrated for the implementation of an optic artificial neural network (OANN). For optic image recognition in the experiment, an OANN is constructed using a 16 × 3 1PT1R memristor array, and it is trained on an online platform. The model yields an accuracy of 99.3% after only ten training epochs. The 1PT1R memristor, which shows good performance, demonstrates its ability as an excellent hardware solution for highly efficient optic neuromorphic and edge computing.

Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications

Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.

An Artificial Olfactory System Based on a Chemi-Memristive Device

Technologies based on the fusion of gas sensors and neuromorphic computing to mimic the olfactory system have immense potential. However, the implementation of neuromorphic olfactory systems remains in a state of infancy because conventional gas sensors lack the necessary functions. Therefore, this study proposes a hysteretic "chemi-memristive gas sensor" based on oxygen vacancy chemi-memristive dynamics that differ from that of conventional gas sensors. After the memristive switching operation, the redox reaction with the external gas molecules is enhanced, resulting in the generation and elimination of oxygen vacancies that induce rapid current changes. In addition, the pre-generated oxygen vacancies enhance the post-sensing properties. Therefore, fast responses, short recovery times, and hysteretic gas response are achieved by the proposed sensor at room temperature. Based on the advantageous functionality of the sensor, device-level olfactory systems that can monitor the history of input gas stimuli are experimentally demonstrated as a potential application. Moreover, analog conductance modulation induced by oxidizing and reducing gases enables the conversion of external gas stimuli into synaptic weights and hence the realization of typical synaptic functionalities without an additional device or circuit. The proposed chemi-memristive device represents an advance in the bioinspired technology adopted in creating artificial intelligence systems.

Simulation analysis of visual perception model based on pulse coupled neural network

Pulse-coupled neural networks perform well in many fields such as information retrieval, depth estimation and object detection. Based on pulse coupled neural network (PCNN) theory, this paper constructs a visual perception model framework and builds a real image reproduction platform. The model firstly analyzes the structure and generalization ability of neural network multi-class classifier, uses the minimax criterion of feature space as the splitting criterion of visual perception decision node, which solves the generalization problem of neural network learning algorithm. In the simulation process, the initial threshold is optimized by the two-dimensional maximum inter-class variance method, and in order to improve the real-time performance of the algorithm, the fast recurrence formula of neural network is derived and given. The PCNN image segmentation method based on genetic algorithm is analyzed. The genetic algorithm improves the loop termination condition and the adaptive setting of model parameters of PCNN image segmentation algorithm, but the PCNN image segmentation algorithm still has the problem of complexity. In order to solve this problem, this paper proposed an IGA-PCNN image segmentation method combining the improved algorithm and PCNN model. Firstly, it used the improved immune genetic algorithm to adaptively obtain the optimal threshold, and then replaced the dynamic threshold in PCNN model with the optimal threshold, and finally used the pulse coupling characteristics of PCNN model to complete the image segmentation. From the coupling characteristics of PCNN, junction close space of image and gray level characteristics, it determined the local gray mean square error of image connection strength coefficient. The feature extraction and object segmentation properties of PCNN come from the spike frequency of neurons, and the number of neurons in PCNN is equal to the number of pixels in the input image. In addition, the spatial and gray value differences of pixels should be considered comprehensively to determine their connection matrix. Digital experiments show that the multi-scale multi-task pulse coupled neural network model can shorten the total training time by 17 h, improve the comprehensive accuracy of the task test data set by 1.04%, and shorten the detection time of each image by 4.8 s compared with the series network model of multiple single tasks. Compared with the traditional PCNN algorithm, it has the advantages of fast visual perception and clear target contour segmentation, and effectively improves the anti-interference performance of the model.

Vertically integrated spiking cone photoreceptor arrays for color perception

The cone photoreceptors in our eyes selectively transduce the natural light into spiking representations, which endows the brain with high energy-efficiency color vision. However, the cone-like device with color-selectivity and spike-encoding capability remains challenging. Here, we propose a metal oxide-based vertically integrated spiking cone photoreceptor array, which can directly transduce persistent lights into spike trains at a certain rate according to the input wavelengths. Such spiking cone photoreceptors have an ultralow power consumption of less than 400 picowatts per spike in visible light, which is very close to biological cones. In this work, lights with three wavelengths were exploited as pseudo-three-primary colors to form 'colorful' images for recognition tasks, and the device with the ability to discriminate mixed colors shows better accuracy. Our results would enable hardware spiking neural networks with biologically plausible visual perception and provide great potential for the development of dynamic vision sensors.

Thermal transport in metal-NbOx-metal cross-point devices and its effect on threshold switching characteristics

Volatile threshold switching and current-controlled negative differential resistance (NDR) in metal-oxide-metal (MOM) devices result from thermally driven conductivity changes induced by local Joule heating and are therefore influenced by the thermal properties of the device-structure. In this study, we investigate the effect of the metal electrodes on the threshold switching response of NbO_x-based cross-point devices. The electroforming and switching characteristics are shown to be strongly influenced by the thickness and thermal conductivity of the top-electrode due to its effect on heat loss from the NbO_x film. Specifically, we demonstrate a 40% reduction in threshold voltage and a 75% reduction in threshold power as the thickness of the top Au electrode is reduced from 125 nm to 25 nm, and a 24% reduction in threshold voltage and 64% reduction in threshold power when the Au electrode is replaced by a Pt electrode of the same thickness of NbO_x film, due to its lower thermal conductivity. Lumped element and finite element modelling of the devices show that these improvements are due to a reduction in heat loss to the electrodes, which is dominated by lateral heat flow within the electrode. These results clearly demonstrate the importance of the electrodes in determining the electroforming and threshold switching characteristics of MOM cross point devices.

A two-dimensional mid-infrared optoelectronic retina enabling simultaneous perception and encoding

Infrared machine vision system for object perception and recognition is becoming increasingly important in the Internet of Things era. However, the current system suffers from bulkiness and inefficiency as compared to the human retina with the intelligent and compact neural architecture. Here, we present a retina-inspired mid-infrared (MIR) optoelectronic device based on a two-dimensional (2D) heterostructure for simultaneous data perception and encoding. A single device can perceive the illumination intensity of a MIR stimulus signal, while encoding the intensity into a spike train based on a rate encoding algorithm for subsequent neuromorphic computing with the assistance of an all-optical excitation mechanism, a stochastic near-infrared (NIR) sampling terminal. The device features wide dynamic working range, high encoding precision, and flexible adaption ability to the MIR intensity. Moreover, an inference accuracy more than 96% to MIR MNIST data set encoded by the device is achieved using a trained spiking neural network (SNN).

Artificial Multisensory Neuron with a Single Transistor for Multimodal Perception through Hybrid Visual and Thermal Sensing

An artificial multisensory device applicable to in-sensor computing is demonstrated with a single-transistor neuron (1T-neuron) for multimodal perception. It simultaneously receives two sensing signals from visual and thermal stimuli. The 1T-neuron transforms these signals into electrical signals in the form of spiking and then fires them for a spiking neural network at the same time. This feature makes it feasible to realize input neurons for multimodal sensing. Visual and thermal sensing is achieved due to the inherent optical and thermal behaviors of the 1T-neuron. To demonstrate a neuromorphic multimodal sensing system with the artificial multisensory 1T-neuron, fingerprint recognition, widely used for biometric security, is implemented. Owing to the simultaneous sensing of heat as well as light, the proposed fingerprint recognition system composed of multisensory 1T-neurons not only identifies a genuine pattern but also judges whether or not it is forged.

Advances in Emerging Photonic Memristive and Memristive-Like Devices

Possessing the merits of high efficiency, low consumption, and versatility, emerging photonic memristive and memristive-like devices exhibit an attractive future in constructing novel neuromorphic computing and miniaturized bionic electronic system. Recently, the potential of various emerging materials and structures for photonic memristive and memristive-like devices has attracted tremendous research efforts, generating various novel theories, mechanisms, and applications. Limited by the ambiguity of the mechanism and the reliability of the material, the development and commercialization of such devices are still rare and in their infancy. Therefore, a detailed and systematic review of photonic memristive and memristive-like devices is needed to further promote its development. In this review, the resistive switching mechanisms of photonic memristive and memristive-like devices are first elaborated. Then, a systematic investigation of the active materials, which induce a pivotal influence in the overall performance of photonic memristive and memristive-like devices, is highlighted and evaluated in various indicators. Finally, the recent advanced applications are summarized and discussed. In a word, it is believed that this review provides an extensive impact on many fields of photonic memristive and memristive-like devices, and lay a foundation for academic research and commercial applications.

A Bioinspired Artificial Gustatory Neuron for a Neuromorphic Based Electronic Tongue

A novel biomimicked neuromorphic sensor for an energy efficient and highly scalable electronic tongue (E-tongue) is demonstrated with a metal-oxide-semiconductor field-effect transistor (MOSFET). By mimicking a biological gustatory neuron, the proposed E-tongue can simultaneously detect ion concentrations of chemicals on an extended gate and encode spike signals on the MOSFET, which acts as an input neuron in a spiking neural network (SNN). Such in-sensor neuromorphic functioning can reduce the energy and area consumption of the conventional E-tongue hardware. pH-sensitive and sodium-sensitive artificial gustatory neurons are implemented by using two different sensing materials: Al₂O₃ for pH sensing and sodium ionophore X for sodium ion sensing. In addition, a sensitivity control function inspired by the biological sensory neuron is demonstrated. After the unit device characterization of the artificial gustatory neuron, a fully hardware-based E-tongue that can classify two distinct liquids is demonstrated to show a practical application of the artificial gustatory neurons.

A calibratable sensory neuron based on epitaxial VO2 for spike-based neuromorphic multisensory system

Neuromorphic perception systems inspired by biology have tremendous potential in efficiently processing multi-sensory signals from the physical world, but a highly efficient hardware element capable of sensing and encoding multiple physical signals is still lacking. Here, we report a spike-based neuromorphic perception system consisting of calibratable artificial sensory neurons based on epitaxial VO₂, where the high crystalline quality of VO₂ leads to significantly improved cycle-to-cycle uniformity. A calibration resistor is introduced to optimize device-to-device consistency, and to adapt the VO₂ neuron to different sensors with varied resistance level, a scaling resistor is further incorporated, demonstrating cross-sensory neuromorphic perception component that can encode illuminance, temperature, pressure and curvature signals into spikes. These components are utilized to monitor the curvatures of fingers, thereby achieving hand gesture classification. This study addresses the fundamental cycle-to-cycle and device-to-device variation issues of sensory neurons, therefore promoting the construction of neuromorphic perception systems for e-skin and neurorobotics.

Emerging Memristive Devices for Brain-Inspired Computing and Artificial Perception

Brain-inspired computing is an emerging field that aims at building a compact and massively parallel architecture, to reduce power consumption in conventional Von Neumann Architecture. Recently, memristive devices have gained great attention due to their immense potential in implementing brain-inspired computing and perception. The conductance of a memristor can be modulated by a voltage pulse, enabling emulations of both essential synaptic and neuronal functions, which are considered as the important building blocks for artificial neural networks. As a result, it is critical to review recent developments of memristive devices in terms of neuromorphic computing and perception applications, waiting for new thoughts and breakthroughs. The device structures, operation mechanisms, and materials are introduced sequentially in this review; additionally, late advances in emergent neuromorphic computing and perception based on memristive devices are summed up. Finally, the challenges that memristive devices toward high-performance brain-inspired computing and perception are also briefly discussed. We believe that the advances and challenges will lead to significant advancements in artificial neural networks and intelligent humanoid robots.

Artificial Olfactory Neuron for an In-Sensor Neuromorphic Nose

A neuromorphic module of an electronic nose (E-nose) is demonstrated by hybridizing a chemoresistive gas sensor made of a semiconductor metal oxide (SMO) and a single transistor neuron (1T-neuron) made of a metal-oxide-semiconductor field-effect transistor (MOSFET). By mimicking a biological olfactory neuron, it simultaneously detects a gas and encoded spike signals for in-sensor neuromorphic functioning. It identifies an odor source by analyzing the complicated mixed signals using a spiking neural network (SNN). The proposed E-nose does not require conversion circuits, which are essential for processing the sensory signals between the sensor array and processors in the conventional bulky E-nose. In addition, they do not have to include a central processing unit (CPU) and memory, which are required for von Neumann computing. The spike transmission of the biological olfactory system, which is known to be the main factor for reducing power consumption, is realized with the SNN for power savings compared to the conventional E-nose with a deep neural network (DNN). Therefore, the proposed neuromorphic E-nose is promising for application to Internet of Things (IoT), which demands a highly scalable and energy-efficient system. As a practical example, it is employed as an electronic sommelier by classifying different types of wines.

A Photoelectric Spiking Neuron for Visual Depth Perception

The biological visual system encodes optical information into spikes and processes them by the neural network, which enables the perception with high throughput of visual processing with ultralow energy budget. This has inspired a wide spectrum of devices to imitate such neural process, while precise mimicking such procedure is still highly required. Here, a highly bio-realistic photoelectric spiking neuron for visual depth perception is presented. The firing spikes generated by the TaO_X memristive spiking encoders have a biologically similar frequency range of 1-200 Hz and sub-micro watts power. Such spiking encoder is integrated with a photodetector and a network of neuromorphic transistors, for information collection and recognition tasks, respectively. The distance-dependent response and eye fatigue of biological visual systems have been mimicked based on such photoelectric spiking neuron. The simulated depth perception shows a recognition improvement by adapting to sights at different distances. The results can advance the technologies in bioinspired or robotic systems that may be endowed with depth perception and power efficiency at the same time.

A Skin-Inspired Artificial Mechanoreceptor for Tactile Enhancement and Integration

Mechanoreceptors endow humans with the sense of touch by translating the external stimuli into coded spikes, inspiring the rise of artificial mechanoreceptor systems. However, to incorporate slow adaptive receptors-like pressure sensors with artificial neurons remains a challenge. Here we demonstrate an artificial mechanoreceptor by rationally integrating a polypyrrole-based resistive pressure sensor with a volatile NbO_x memristor, to mimic the tactile sensation and perception in natural skin, respectively. The artificial mechanoreceptor enables the tactile sensory coding by converting the external mechanical stimuli into strength-modulated electrical spikes. Also, tactile sensation enhancement is achieved by processing the spike frequency characteristics with the pulse coupled neural network. Furthermore, the artificial mechanoreceptor can integrate signals from parallel sensor channels and encode them into unified electrical spikes, resembling the coding of intensity in tactile neural processing. These results provide simple and efficient strategies for constructing future bio-inspired electronic systems.

Composite Memristors by Nanoscale Modification of Hf/Ta Anodic Oxides

Composite memristors based on anodic oxidation of Hf superimposed on Ta thin films are studied. A layered structure is obtained by successive sputtering of Ta and Hf thin films. The deposition geometry ensured components' thickness gradient profiles (wedges) aligned in opposite directions. Anodization in citrate buffer electrolyte leads to a nanoscale columnar structuring of Ta₂O₅ in HfO₂ due to the higher electrical resistance of the latter. Following the less resistive path, the ionic current forces Ta oxide to locally grow toward the electrolyte interface according to the Rayleigh-Taylor principle. The obtained composite oxide memristive properties are studied as a function of the Hf/Ta thickness ratio. One pronounced zone prominent for memristive applications is found for ratios between 4 and 5. Here, unipolar and bipolar memristors are found, with remarkable endurance and retention capabilities. This is discussed in the frame of conductive filament formation preferentially along the interfaces between oxides.

Artificial Intelligence Pulse Coupled Neural Network Algorithm in the Diagnosis and Treatment of Severe Sepsis Complicated with Acute Kidney Injury under Ultrasound Image

The objective of this study was to explore the diagnosis of severe sepsis complicated with acute kidney injury (AKI) by ultrasonic image information based on the artificial intelligence pulse coupled neural network (PCNN) algorithm. In this study, an algorithm of ultrasonic image information enhancement based on the artificial intelligence PCNN was constructed and compared with the histogram equalization algorithm and linear transformation algorithm. After that, it was applied to the ultrasonic image diagnosis of 20 cases of severe sepsis combined with AKI in hospital. The condition of each patient was diagnosed by ultrasound image performance, change of renal resistance index (RRI), ultrasound score, and receiver operator characteristic curve (ROC) analysis. It was found that the histogram distribution of this algorithm was relatively uniform, and the information of each gray level was obviously retained and enhanced, which had the best effect in this algorithm; there was a marked individual difference in the values of RRI. Overall, the values of RRI showed a slight upward trend after admission to the intensive care unit (ICU). The RRI was taken as the dependent variable, time as the fixed-effect model, and patients as the random effect; the parameter value of time was between 0.012 and 0.015, p=0.000 < 0.05. Besides, there was no huge difference in the ultrasonic score among different time measurements (t = 1.348 and p=0.128 > 0.05). The area under the ROC curve of the RRI for the diagnosis of AKI at the 2nd day, 4th day, and 6th day was 0.758, 0.841, and 0.856, respectively, which was all greater than 0.5 (p < 0.05). In conclusion, the proposed algorithm in this study could significantly enhance the amount of information in ultrasound images. In addition, the change of RRI values measured by ultrasound images based on the artificial intelligence PCNN was associated with AKI.