Artificial Multisensory Neuron with a Single Transistor for Multimodal Perception through Hybrid Visual and Thermal Sensing

Recent Advances in Artificial Sensory Neurons: Biological Fundamentals, Devices, Applications, and Challenges

Spike-based neural networks, which use spikes or action potentials to represent information, have gained a lot of attention because of their high energy efficiency and low power consumption. To fully leverage its advantages, converting the external analog signals to spikes is an essential prerequisite. Conventional approaches including analog-to-digital converters or ring oscillators, and sensors suffer from high power and area costs. Recent efforts are devoted to constructing artificial sensory neurons based on emerging devices inspired by the biological sensory system. They can simultaneously perform sensing and spike conversion, overcoming the deficiencies of traditional sensory systems. This review summarizes and benchmarks the recent progress of artificial sensory neurons. It starts with the presentation of various mechanisms of biological signal transduction, followed by the systematic introduction of the emerging devices employed for artificial sensory neurons. Furthermore, the implementations with different perceptual capabilities are briefly outlined and the key metrics and potential applications are also provided. Finally, we highlight the challenges and perspectives for the future development of artificial sensory neurons.

An evaluation of recent advancements in biological sensory organ-inspired neuromorphically tuned biomimetic devices

In the field of neuroscience, significant progress has been made regarding how the brain processes information. Unlike computer processors, the brain comprises neurons and synapses instead of memory blocks and transistors. Despite advancements in artificial neural networks, a complete understanding concerning brain functions remains elusive. For example, to achieve more accurate neuron replication, we must better understand signal transmission during synaptic processes, neural network tunability, and the creation of nanodevices featuring neurons and synapses. This study discusses the latest algorithms utilized in neuromorphic systems, the production of synaptic devices, differences between single and multisensory gadgets, recent advances in multisensory devices, and the promising research opportunities available in this field. We also explored the ability of an artificial synaptic device to mimic biological neural systems across diverse applications. Despite existing challenges, neuroscience-based computing technology holds promise for attracting scientists seeking to enhance solutions and augment the capabilities of neuromorphic devices, thereby fostering future breakthroughs in algorithms and the widespread application of cutting-edge technologies.

A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO2 Mott Memristor for the Spike-Based Electronic Retina

The neuromorphic vision system that utilizes spikes as information carriers is crucial for the formation of spiking neural networks. Here, we present a bioinspired flexible artificial spiking photoreceptor (ASP), which is realized by using a single VO2 Mott memristor that can simultaneously sense and encode the stimulus light into spikes. The ASP has high spike-encoded photosensitivity and ultrawide photosensing range (405-808 nm) with good endurance (>7 × 107) and high flexibility (bending radius ∼5 mm). Then, we put forward an all-spike electronic retina architecture that comprises one layer of ASPs and one layer of artificial optical nerves (AONs) to process the spike information. Each AON consists of a single Mott memristor connected in series with a neuro-transistor that is a multiple-input floating-gate MOS transistor. Simulation results demonstrate that the all-spike electronic retina can successfully segment images with high Shannon entropy, thus laying the foundation for the development of a spike-based neuromorphic vision system.

Multimodal Artificial Synapses for Neuromorphic Application

The rapid development of neuromorphic computing has led to widespread investigation of artificial synapses. These synapses can perform parallel in-memory computing functions while transmitting signals, enabling low-energy and fast artificial intelligence. Robots are the most ideal endpoint for the application of artificial intelligence. In the human nervous system, there are different types of synapses for sensory input, allowing for signal preprocessing at the receiving end. Therefore, the development of anthropomorphic intelligent robots requires not only an artificial intelligence system as the brain but also the combination of multimodal artificial synapses for multisensory sensing, including visual, tactile, olfactory, auditory, and taste. This article reviews the working mechanisms of artificial synapses with different stimulation and response modalities, and presents their use in various neuromorphic tasks. We aim to provide researchers in this frontier field with a comprehensive understanding of multimodal artificial synapses.

Mimicking the retinal neuron functions by a photoresponsive single transistor with a double gate

To realize a low-cost neuromorphic visual system, employing an artificial neuron capable of mimicking the retinal neuron functions is essential. A photoresponsive single transistor neuron composed of a vertical silicon nanowire is proposed. Similar to retinal neurons, various photoresponsive characteristics of the single transistor neuron can be modulated by light intensity as well as wavelength and have a high responsivity to green light like the human eye. The device is designed with a cylindrical surrounding double-gate structure, enclosed by an independently controlled outer gate and inner gate. The outer gate has the function of selectively inhibiting neuron activity, which can mimic lateral inhibition of amacrine cells to ganglion cells, and the inner gate can be utilized for the adjustment of the firing threshold voltage, which can be used to mimic the regulation of photoresponsivity by horizontal cells for adaptive visual perception. Furthermore, a myelination function that controls the speed of information transmission is obtained according to the inherent asymmetric source/drain structure of a vertical silicon nanowire. This work can enable photoresponsive neuronal function using only a single transistor, providing a promising hardware implementation for building miniaturized neuromorphic vision systems at low cost.

PBDB-T/Pentacene-Based Organic Optoelectronic Synaptic Transistor with Adjustable Critical Flicker Fusion Frequency for Dynamic Vision

In the era of the Internet of Things and the rapid progress of artificial intelligence, there is a growing demand for advanced dynamic vision systems. Vision systems are no longer confined to static object detection and recognition, as the detection and recognition of moving objects are becoming increasingly important. To meet the requirements for more precise and efficient dynamic vision, the development of adaptive multimodal motion detection devices becomes imperative. Inspired by the varied response rates in biological vision, we introduce the concept of critical flicker fusion frequency (cFFF) and develop an organic optoelectronic synaptic transistor with adjustable cFFF. In situ Kelvin probe force microscopy analysis reveals that light signal recognition in this device originates from charge transfer in the poly[(2,6-(4,8-bis(5-(2-ethylhexyl)thiophen-2-yl)benzo[1,2-b:4,5-b']dithiophene)-co-(1,3-di(5-thiophene-2-yl)-5,7-bis(2-ethylhexyl)-benzo[1,2-c:4,5-c']dithiophene-4,8-dione)] (PBDB-T)/pentacene heterojunction, which can be effectively modulated by gate voltage. Building upon this, we implement different cFFF within a single device to facilitate the detection and recognition of objects moving at different speeds. This approach allows for resource allocation during dynamic detection, resulting in a reduction in power consumption. Our research holds great potential for enhancing the capabilities of dynamic visual systems.

Recent Progress in Wearable Near-Sensor and In-Sensor Intelligent Perception Systems

As the Internet of Things (IoT) becomes more widespread, wearable smart systems will begin to be used in a variety of applications in people's daily lives, not only requiring the devices to have excellent flexibility and biocompatibility, but also taking into account redundant data and communication delays due to the use of a large number of sensors. Fortunately, the emerging paradigms of near-sensor and in-sensor computing, together with the proposal of flexible neuromorphic devices, provides a viable solution for the application of intelligent low-power wearable devices. Therefore, wearable smart systems based on new computing paradigms are of great research value. This review discusses the research status of a flexible five-sense sensing system based on near-sensor and in-sensor architectures, considering material design, structural design and circuit design. Furthermore, we summarize challenging problems that need to be solved and provide an outlook on the potential applications of intelligent wearable devices.

A Butterfly-Inspired Multisensory Neuromorphic Platform for Integration of Visual and Chemical Cues

Unisensory cues are often insufficient for animals to effectively engage in foraging, mating, and predatory activities. In contrast, integration of cues collected from multiple sensory organs enhances the overall perceptual experience and thereby facilitates better decision-making. Despite the importance of multisensory integration in animals, the field of artificial intelligence (AI) and neuromorphic computing has primarily focused on processing unisensory information. This lack of emphasis on multisensory integration can be attributed to the absence of a miniaturized hardware platform capable of co-locating multiple sensing modalities and enabling in-sensor and near-sensor processing. In this study, this limitation is addressed by utilizing the chemo-sensing properties of graphene and the photo-sensing capability of monolayer molybdenum disulfide (MoS2 ) to create a multisensory platform for visuochemical integration. Additionally, the in-memory-compute capability of MoS2 memtransistors is leveraged to develop neural circuits that facilitate multisensory decision-making. The visuochemical integration platform is inspired by intricate courtship of Heliconius butterflies, where female species rely on the integration of visual cues (such as wing color) and chemical cues (such as pheromones) generated by the male butterflies for mate selection. The butterfly-inspired visuochemical integration platform has significant implications in both robotics and the advancement of neuromorphic computing, going beyond unisensory intelligence and information processing.

A bio-inspired visuotactile neuron for multisensory integration

Multisensory integration is a salient feature of the brain which enables better and faster responses in comparison to unisensory integration, especially when the unisensory cues are weak. Specialized neurons that receive convergent input from two or more sensory modalities are responsible for such multisensory integration. Solid-state devices that can emulate the response of these multisensory neurons can advance neuromorphic computing and bridge the gap between artificial and natural intelligence. Here, we introduce an artificial visuotactile neuron based on the integration of a photosensitive monolayer MoS2 memtransistor and a triboelectric tactile sensor which minutely captures the three essential features of multisensory integration, namely, super-additive response, inverse effectiveness effect, and temporal congruency. We have also realized a circuit which can encode visuotactile information into digital spiking events, with probability of spiking determined by the strength of the visual and tactile cues. We believe that our comprehensive demonstration of bio-inspired and multisensory visuotactile neuron and spike encoding circuitry will advance the field of neuromorphic computing, which has thus far primarily focused on unisensory intelligence and information processing.