Optoelectronic graded neurons for bioinspired in-sensor motion perception

Avian eye-inspired perovskite artificial vision system for foveated and multispectral imaging

Avian eyes have deep central foveae as a result of extensive evolution. Deep foveae efficiently refract incident light, creating a magnified image of the target object and making it easier to track object motion. These features are essential for detecting and tracking remote objects in dynamic environments. Furthermore, avian eyes respond to a wide spectrum of light, including visible and ultraviolet light, allowing them to efficiently distinguish the target object from complex backgrounds. Despite notable advances in artificial vision systems that mimic animal vision, the exceptional object detection and targeting capabilities of avian eyes via foveated and multispectral imaging remain underexplored. Here, we present an artificial vision system that capitalizes on these aspects of avian vision. We introduce an artificial fovea and vertically stacked perovskite photodetector arrays whose designs were optimized by theoretical simulations for the demonstration of foveated and multispectral imaging. The artificial vision system successfully identifies colored and mixed-color objects and detects remote objects through foveated imaging. The potential for use in uncrewed aerial vehicles that need to detect, track, and recognize distant targets in dynamic environments is also discussed. Our avian eye-inspired perovskite artificial vision system marks a notable advance in bioinspired artificial visions.

Bionic Scotopic Adaptation Transistors for Nighttime Low Illumination Imaging

Human vision excels in perceiving nighttime low illumination due to biological feedforward adaptation. Replicating this ability in biomimetic vision using solid-state devices has been highly sought after. However, emulating scotopic adaptation, entailing a confluence of efficient photoexcitation and dynamic carrier modulation, presents formidable challenges. Here, we demonstrate a low-power and bionic scotopic adaptation transistor by coupling a light-absorption layer and an electron-trapping layer at the bottom of the semiconducting channel, enabling simultaneous achievement of efficient generation of free photocarriers and adaptive carrier accumulation within a single device. This innovation empowers our transistor to exhibit sensitivity-potentiated characteristics after adaptation, detecting scotopic-level illumination (0.001 lx) with exceptional photosensitivity up to 103 at low voltages below 2 V. Moreover, we have successfully replicated diverse scotopic vision functions, encompassing time-dependent visual threshold enhancement, light intensity-dependent adaptation index, imaging contrast enhancement for nighttime low illumination imaging, opening an opportunity for artificial night vision.

Bioinspired In-Sensor Multimodal Fusion for Enhanced Spatial and Spatiotemporal Association

Multimodal perception can capture more precise and comprehensive information compared with unimodal approaches. However, current sensory systems typically merge multimodal signals at computing terminals following parallel processing and transmission, which results in the potential loss of spatial association information and requires time stamps to maintain temporal coherence for time-series data. Here we demonstrate bioinspired in-sensor multimodal fusion, which effectively enhances comprehensive perception and reduces the level of data transfer between sensory terminal and computation units. By adopting floating gate phototransistors with reconfigurable photoresponse plasticity, we realize the agile spatial and spatiotemporal fusion under nonvolatile and volatile photoresponse modes. To realize an optimal spatial estimation, we integrate spatial information from visual-tactile signals. For dynamic events, we capture and fuse in real time spatiotemporal information from visual-audio signals, realizing a dance-music synchronization recognition task without a time-stamping process. This in-sensor multimodal fusion approach provides the potential to simplify the multimodal integration system, extending the in-sensor computing paradigm.

A Reconfigurable Bipolar Image Sensor for High-Efficiency Dynamic Vision Recognition

Dynamic vision perception and processing (DVPP) is in high demand by booming edge artificial intelligence. However, existing imaging systems suffer from low efficiency or low compatibility with advanced machine vision techniques. Here, we propose a reconfigurable bipolar image sensor (RBIS) for in-sensor DVPP based on a two-dimensional WSe2/GeSe heterostructure device. Owing to the gate-tunable and reversible built-in electric field, its photoresponse shows bipolarity as being positive or negative. High-efficiency DVPP incorporating front-end RBIS and back-end CNN is then demonstrated. It shows a high recognition accuracy of over 94.9% on the derived DVS128 data set and requires much fewer neural network parameters than that without RBIS. Moreover, we demonstrate an optimized device with a vertically stacked structure and a stable nonvolatile bipolarity, which enables more efficient DVPP hardware. Our work demonstrates the potential of fabricating DVPP devices with a simple structure, high efficiency, and outputs compatible with advanced algorithms.

Gap State-Modulated Van Der Waals Short-Term Memory with Broad Band Negative Photoconductance

Floating gate memory (FGM), composed of van der Waals (vdW) junctions with an atomically thin floating layer for charge storage, is widely employed to develop logic-in memories and in-sensor computing devices. Most research efforts of FGM are spent on achieving long-term charge storage and fast reading/writing for flash and random-access memory. However, dynamic modulation of memory time via a tunneling barrier and vdW interfaces, which is critical for synaptic computing and machine vision, is still lacking. Here, a van der Waals short-term memory with tunable memory windows and retention times from milliseconds to thousands of seconds is reported, which is approximately exponentially proportional to the thickness h-BN (hexagonal boron nitride) barrier. The specific h-BN barrier with fruitful gap states provides charge release channels for trapped charges, making the vdW device switchable between positive photoconductance and negative photoconductance with a broadband light from IR to UV range. The dynamic short-term memory with tunable photo response highlights the design strategy of novel vdW memory vis interface engineering for further intelligent information storage and optoelectronic detection.

A Spiking Artificial Vision Architecture Based on Fully Emulating the Human Vision

Intelligent vision necessitates the deployment of detectors that are always-on and low-power, mirroring the continuous and uninterrupted responsiveness characteristic of human vision. Nonetheless, contemporary artificial vision systems attain this goal by the continuous processing of massive image frames and executing intricate algorithms, thereby expending substantial computational power and energy. In contrast, biological data processing, based on event-triggered spiking, has higher efficiency and lower energy consumption. Here, this work proposes an artificial vision architecture consisting of spiking photodetectors and artificial synapses, closely mirroring the intricacies of the human visual system. Distinct from previously reported techniques, the photodetector is self-powered and event-triggered, outputting light-modulated spiking signals directly, thereby fulfilling the imperative for always-on with low-power consumption. With the spiking signals processing through the integrated synapse units, recognition of graphics, gestures, and human action has been implemented, illustrating the potent image processing capabilities inherent within this architecture. The results prove the 90% accuracy rate in human action recognition within a mere five epochs utilizing a rudimentary artificial neural network. This novel architecture, grounded in spiking photodetectors, offers a viable alternative to the extant models of always-on low-power artificial vision system.

Fully Memristive Elementary Motion Detectors for a Maneuver Prediction

Insects can efficiently perform object motion detection via a specialized neural circuit, called an elementary motion detector (EMD). In contrast, conventional machine vision systems require significant computational resources for dynamic motion processing. Here, a fully memristive EMD (M-EMD) is presented that implements the Hassenstein-Reichardt (HR) correlator, a biological model of the EMD. The M-EMD consists of a simple Wye (Y) configuration, including a static resistor, a dynamic memristor, and a Mott memristor. The resistor and dynamic memristor introduce different signal delays, enabling spatio-temporal signal integration in the subsequent Mott memristor, resulting in a direction-selective response. In addition, a neuromorphic system is developed employing the M-EMDs to predict a lane-changing maneuver by vehicles on the road. The system achieved a high accuracy (> 87%) in predicting future lane-changing maneuvers on the Next Generation Simulation (NGSIM) dataset while reducing the computational cost by 92.9% compared to the conventional neuromorphic system without the M-EMD, suggesting its strong potential for edge-level computing.

Schottky barrier reduction on optoelectronic responses in heavy ion irradiated WSe2 memtransistors

Two-dimensional transition metal dichalcogenide-based memtransistors provide simulation, sensing, and storage capabilities for applications in a remotely operated aerospace environment. Swift heavy ion (SHI) irradiation technology is a common method to simulate the influences of radiation ions on electronic devices in space environments. Here, SHI irradiation technology under different conditions was utilized to produce complex defects in WSe2-based memtransistors. Low-resistance state to low-resistance state (LRS-LRS) switching behaviors under light illumination were achieved and photocurrent responses with different spike trains were observed in SHI-irradiated memtransistors, which facilitated the design of devices with enriched analog functions. Reduction of the Schottky barrier height due to the introduced defects at the metal/WSe2 interface was confirmed to be the major factor responsible for the observed behaviors. 1T phase and concentric circle-type vacancies were also created in the SHI-irradiated 2H-WSe2 channel besides the amorphous structure; these complex defects could seriously affect the transport properties of the devices. We believe that this work serves as a foundation for aerospace radiation applications of all-in-one devices. It also opens a new application field of heavy ion irradiation technology for the development of multiterminal memtransistor-based optoelectronic artificial synapses for neuromorphic computing.

Ultralow-power optoelectronic synaptic transistors based on polyzwitterion dielectrics for in-sensor reservoir computing

Bio-inspired transistor synapses use solid electrolytes to achieve low-power operation and rich synaptic behaviors via ion diffusion and trapping. While these neuromorphic devices hold great promise, they still suffer from challenges such as high leakage currents and power consumption, electrolysis risk, and irreversible conductance changes due to long-range ion migrations and permanent ion trapping. In addition, their response to light is generally limited because of "exciton-polaron quenching", which restricts their potential in in-sensor neuromorphic visions. To address these issues, we propose replacing solid electrolytes with polyzwitterions, where the cation and anion are covalently concatenated via a flexible alkyl chain, thus preventing long-range ion migrations while inducing good photoresponses to the transistors via interfacial charge trapping. Our detailed studies reveal that polyzwitterion-based transistors exhibit optoelectronic synaptic behavior with ultralow-power consumption (~250 aJ per spike) and enable high-performance in-sensor reservoir computing, achieving 95.56% accuracy in perceiving the trajectory of moving basketballs.

A bionic self-driven retinomorphic eye with ionogel photosynaptic retina

Bioinspired bionic eyes should be self-driving, repairable and conformal to arbitrary geometries. Such eye would enable wide-field detection and efficient visual signal processing without requiring external energy, along with retinal transplantation by replacing dysfunctional photoreceptors with healthy ones for vision restoration. A variety of artificial eyes have been constructed with hemispherical silicon, perovskite and heterostructure photoreceptors, but creating zero-powered retinomorphic system with transplantable conformal features remains elusive. By combining neuromorphic principle with retinal and ionoelastomer engineering, we demonstrate a self-driven hemispherical retinomorphic eye with elastomeric retina made of ionogel heterojunction as photoreceptors. The receptor driven by photothermoelectric effect shows photoperception with broadband light detection (365 to 970 nm), wide field-of-view (180°) and photosynaptic (paired-pulse facilitation index, 153%) behaviors for biosimilar visual learning. The retinal photoreceptors are transplantable and conformal to any complex surface, enabling visual restoration for dynamic optical imaging and motion tracking.

Inorganic Halide Perovskite Nanowires/Conjugated Polymer Heterojunction-Based Optoelectronic Synaptic Transistors for Dynamic Machine Vision

Inspired by the retina, artificial optoelectronic synapses have groundbreaking potential for machine vision. The field-effect transistor is a crucial platform for optoelectronic synapses that is highly sensitive to external stimuli and can modulate conductivity. On the basis of the decent optical absorption, perovskite materials have been widely employed for constructing optoelectronic synaptic transistors. However, the reported optoelectronic synaptic transistors focus on the static processing of independent stimuli at different moments, while the natural visual information consists of temporal signals. Here, we report CsPbBrI2 nanowire-based optoelectronic synaptic transistors to study the dynamic responses of artificial synaptic transistors to time-varying visual information for the first time. Moreover, on the basis of the dynamic synaptic behavior, a hardware system with an accuracy of 85% is built to the trajectory of moving objects. This work offers a new way to develop artificial optoelectronic synapses for the construction of dynamic machine vision systems.

PBDB-T/Pentacene-Based Organic Optoelectronic Synaptic Transistor with Adjustable Critical Flicker Fusion Frequency for Dynamic Vision

In the era of the Internet of Things and the rapid progress of artificial intelligence, there is a growing demand for advanced dynamic vision systems. Vision systems are no longer confined to static object detection and recognition, as the detection and recognition of moving objects are becoming increasingly important. To meet the requirements for more precise and efficient dynamic vision, the development of adaptive multimodal motion detection devices becomes imperative. Inspired by the varied response rates in biological vision, we introduce the concept of critical flicker fusion frequency (cFFF) and develop an organic optoelectronic synaptic transistor with adjustable cFFF. In situ Kelvin probe force microscopy analysis reveals that light signal recognition in this device originates from charge transfer in the poly[(2,6-(4,8-bis(5-(2-ethylhexyl)thiophen-2-yl)benzo[1,2-b:4,5-b']dithiophene)-co-(1,3-di(5-thiophene-2-yl)-5,7-bis(2-ethylhexyl)-benzo[1,2-c:4,5-c']dithiophene-4,8-dione)] (PBDB-T)/pentacene heterojunction, which can be effectively modulated by gate voltage. Building upon this, we implement different cFFF within a single device to facilitate the detection and recognition of objects moving at different speeds. This approach allows for resource allocation during dynamic detection, resulting in a reduction in power consumption. Our research holds great potential for enhancing the capabilities of dynamic visual systems.

Toward Intelligent Display with Neuromorphic Technology

In the era of the Internet and the Internet of Things, display technology has evolved significantly toward full-scene display and realistic display. Incorporating "intelligence" into displays is a crucial technical approach to meet the demands of this development. Traditional display technology relies on distributed hardware systems to achieve intelligent displays but encounters challenges stemming from the physical separation of sensing, processing, and light-emitting modules. The high energy consumption and data transformation delays limited the development of intelligence display, breaking the physical separation is crucial to overcoming the bottlenecks of intelligence display technology. Inspired by the biological neural system, neuromorphic technology with all-in-one features is widely employed across various fields. It proves effective in reducing system power consumption, facilitating frequent data transformation, and enabling cross-scene integration. Neuromorphic technology shows great potential to overcome display technology bottlenecks, realizing the full-scene display and realistic display with high efficiency and low power consumption. This review offers a comprehensive summary of recent advancements in the application of neuromorphic technology in displays, with a focus on interoperability. This work delves into its state-of-the-art designs and potential future developments aimed at revolutionizing display technology.

Green Alga-Inspired Underwater Vision Based on Light-Driven Active Ion Transport across Janus Dual-Field Heterostructures

Natural organisms have evolved various biological ion channels to make timely responses toward different physical and/or chemical stimuli, giving guidance to construct artificial counterparts and expand the corresponding applications. They have also shown promising potential to overcome disadvantages of traditional electronic devices (e.g., energy-consuming operation and adverse humidity interference). Herein, we constructed a green alga-inspired nanofluidic system based on a Janus dual-field heterogeneous membrane (i.e., J-HM), which can function underwater as an artificial visual platform for light perception through enhanced active ion transport. The J-HM was obtained through sequentially assembled MXene and Cu-HHTP (i.e., a metal-organic framework based on the reaction between 2,3,6,7,10,11-hexahydroxytriphenylene hydrate (HHTP) and Cu2+) building units. Due to the formed temperature gradient and intramembrane electric field caused by the localized thermal excitation and efficient charge separation of J-HM under illumination, thermo-osmotic and photo-driven forces are generated for preferential cation transport from Cu-HHTP to MXene. Furthermore, unidirectional active transport can be enhanced by self-diffusion under a concentration gradient. Then, the corresponding underwater light perceptions at various light illumination conditions are explored, showing nearly a linear correlation with the light intensity. Finally, it is demonstrated that the visual platform can achieve object shape, definition, and distance recognition using a defined pixelated matrix, giving impetus to develop ionic signal transmission based sensing systems.

High performance artificial visual perception and recognition with a plasmon-enhanced 2D material neural network

The development of neuromorphic visual systems has recently gained momentum due to their potential in areas such as autonomous vehicles and robotics. However, current machine visual systems based on silicon technology usually contain photosensor arrays, format conversion, memory and processing modules. As a result, the redundant data shuttling between each unit, resulting in large latency and high-power consumption, seriously limits the performance of neuromorphic vision chips. Here, we demonstrate an artificial neural network (ANN) architecture based on an integrated 2D MoS2/Ag nanograting phototransistor array, which can simultaneously sense, pre-process and recognize optical images without latency. The pre-processing function of the device under photoelectric synergy ensures considerable improvement of efficiency and accuracy of subsequent image recognition. The comprehensive performance of the proof-of-concept device demonstrates great potential for machine vision applications in terms of large dynamic range (180 dB), high speed (500 ns) and low energy consumption per spike (2.4 × 10-17 J).

Neuromorphic antennal sensory system

Insect antennae facilitate the nuanced detection of vibrations and deflections, and the non-contact perception of magnetic or chemical stimuli, capabilities not found in mammalian skin. Here, we report a neuromorphic antennal sensory system that emulates the structural, functional, and neuronal characteristics of ant antennae. Our system comprises electronic antennae sensor with three-dimensional flexible structures that detects tactile and magnetic stimuli. The integration of artificial synaptic devices adsorbed with solution-processable MoS2 nanoflakes enables synaptic processing of sensory information. By emulating the architecture of receptor-neuron pathway, our system realizes hardware-level, spatiotemporal perception of tactile contact, surface pattern, and magnetic field (detection limits: 1.3 mN, 50 μm, 9.4 mT). Vibrotactile-perception tasks involving profile and texture classifications were accomplished with high accuracy (> 90%), surpassing human performance in "blind" tactile explorations. Magneto-perception tasks including magnetic navigation and touchless interaction were successfully completed. Our work represents a milestone for neuromorphic sensory systems and biomimetic perceptual intelligence.

Non-volatile rippled-assisted optoelectronic array for all-day motion detection and recognition

In-sensor processing has the potential to reduce the energy consumption and hardware complexity of motion detection and recognition. However, the state-of-the-art all-in-one array integration technologies with simultaneous broadband spectrum image capture (sensory), image memory (storage) and image processing (computation) functions are still insufficient. Here, macroscale (2 × 2 mm2) integration of a rippled-assisted optoelectronic array (18 × 18 pixels) for all-day motion detection and recognition. The rippled-assisted optoelectronic array exhibits remarkable uniformity in the memory window, optically stimulated non-volatile positive and negative photoconductance. Importantly, the array achieves an extensive optical storage dynamic range exceeding 106, and exceptionally high room-temperature mobility up to 406.7 cm2 V-1 s-1, four times higher than the International Roadmap for Device and Systems 2028 target. Additionally, the spectral range of each rippled-assisted optoelectronic processor covers visible to near-infrared (405 nm-940 nm), achieving function of motion detection and recognition.

Thin-film transistor for temporal self-adaptive reservoir computing with closed-loop architecture

Reservoir computing is a powerful neural network-based computing paradigm for spatiotemporal signal processing. Recently, physical reservoirs have been explored based on various electronic devices with outstanding efficiency. However, the inflexible temporal dynamics of these reservoirs have posed fundamental restrictions in processing spatiotemporal signals with various timescales. Here, we fabricated thin-film transistors with controllable temporal dynamics, which can be easily tuned with electrical operation signals and showed excellent cycle-to-cycle uniformity. Based on this, we constructed a temporal adaptive reservoir capable of extracting temporal information of multiple timescales, thereby achieving improved accuracy in the human-activity-recognition task. Moreover, by leveraging the former computing output to modify the hyperparameters, we constructed a closed-loop architecture that equips the reservoir computing system with temporal self-adaptability according to the current input. The adaptability is demonstrated by accurate real-time recognition of objects moving at diverse speed levels. This work provides an approach for reservoir computing systems to achieve real-time processing of spatiotemporal signals with compound temporal characteristics.

Bionic visual-audio photodetectors with in-sensor perception and preprocessing

Serving as the "eyes" and "ears" of the Internet of Things, optical and acoustic sensors are the fundamental components in hardware systems. Nowadays, mainstream hardware systems, often comprising numerous discrete sensors, conversion modules, and processing units, tend to result in complex architectures that are less efficient compared to human sensory pathways. Here, a visual-audio photodetector inspired by the human perception system is proposed to enable all-in-one visual and acoustic signal detection with computing capability. This device not only captures light but also optically records sound waves, thus achieving "watching" and "listening" within a single unit. The gate-tunable positive, negative, and zero photoresponses lead to highly programmable responsivities. This programmability enables the execution of diverse functions, including visual feature extraction, object classification, and sound wave manipulation. These results showcase the potential of expanding perception approaches in neuromorphic devices, opening up new possibilities to craft intelligent and compact hardware systems.

2D Materials in Flexible Electronics: Recent Advances and Future Prospectives

Flexible electronics have recently gained considerable attention due to their potential to provide new and innovative solutions to a wide range of challenges in various electronic fields. These electronics require specific material properties and performance because they need to be integrated into a variety of surfaces or folded and rolled for newly formatted electronics. Two-dimensional (2D) materials have emerged as promising candidates for flexible electronics due to their unique mechanical, electrical, and optical properties, as well as their compatibility with other materials, enabling the creation of various flexible electronic devices. This article provides a comprehensive review of the progress made in developing flexible electronic devices using 2D materials. In addition, it highlights the key aspects of materials, scalable material production, and device fabrication processes for flexible applications, along with important examples of demonstrations that achieved breakthroughs in various flexible and wearable electronic applications. Finally, we discuss the opportunities, current challenges, potential solutions, and future investigative directions about this field.

A high-dimensional in-sensor reservoir computing system with optoelectronic memristors for high-performance neuromorphic machine vision

In-sensor reservoir computing (RC) is a promising technology to reduce power consumption and training costs of machine vision systems by processing optical signals temporally. This study demonstrates a high-dimensional in-sensor RC system with optoelectronic memristors to enhance the performance of the in-sensor RC system. Because optoelectronic memristors can respond to both optical and electrical stimuli, optical and electrical masks are proposed to improve the dimensionality and performance of the in-sensor RC system. An optical mask is employed to regulate the wavelength of light, while an electrical mask is used to control the initial conductance of zinc oxide optoelectronic memristors. The distinct characteristics of these two masks contribute to the representation of various distinguishable reservoir states, making it possible to implement diverse reservoir configurations with minimal correlation and to increase the dimensionality of the in-sensor RC system. Using the high-dimensional in-sensor RC system, handwritten digits are successfully classified with an accuracy of 94.1%. Furthermore, human action pattern recognition is achieved with a high accuracy of 99.4%. These high accuracies are achieved with the use of a single-layer readout network, which can significantly reduce the network size and training costs.

Nanomaterial-Based Artificial Vision Systems: From Bioinspired Electronic Eyes to In-Sensor Processing Devices

High-performance robotic vision empowers mobile and humanoid robots to detect and identify their surrounding objects efficiently, which enables them to cooperate with humans and assist human activities. For error-free execution of these robots' tasks, efficient imaging and data processing capabilities are essential, even under diverse and complex environments. However, conventional technologies fall short of meeting the high-standard requirements of robotic vision under such circumstances. Here, we discuss recent progress in artificial vision systems with high-performance imaging and data processing capabilities enabled by distinctive electrical, optical, and mechanical characteristics of nanomaterials surpassing the limitations of traditional silicon technologies. In particular, we focus on nanomaterial-based electronic eyes and in-sensor processing devices inspired by biological eyes and animal visual recognition systems, respectively. We provide perspectives on key nanomaterials, device components, and their functionalities, as well as explain the remaining challenges and future prospects of the artificial vision systems.

Polarized Tunneling Transistor for Ultralow-Energy-Consumption Artificial Synapse toward Neuromorphic Computing

Neural networks based on low-power artificial synapses can significantly reduce energy consumption, which is of great importance in today's era of artificial intelligence. Two-dimensional (2D) material-based floating-gate transistors (FGTs) have emerged as compelling candidates for simulating artificial synapses owing to their multilevel and nonvolatile data storage capabilities. However, the low erasing/programming speed of FGTs renders them unsuitable for low-energy-consumption artificial synapses, thereby limiting their potential in high-energy-efficient neuromorphic computing. Here, we introduce a FGT-inspired MoS2/Trap/PZT heterostructure-based polarized tunneling transistor (PTT) with a simple fabrication process and significantly enhanced erasing/programming speed. Distinct from the FGT, the PTT lacks a tunnel layer, leading to a marked improvement in its erasing/programming speed. The PTT's highest erasing/programming (operation) speed can reach ∼20 ns, which outperforms the performance of most FGTs based on 2D heterostructures. Furthermore, the PTT has been utilized as an artificial synapse, and its weight-update energy consumption can be as low as 0.0002 femtojoule (fJ), which benefits from the PTT's ultrahigh operation speed. Additionally, PTT-based artificial synapses have been employed in constructing artificial neural network simulations, achieving facial-recognition accuracy (95%). This groundbreaking work makes it possible for fabricating future high-energy-efficient neuromorphic transistors utilizing 2D materials.

Achieving Reliable and Ultrafast Memristors via Artificial Filaments in Silk Fibroin

The practical implementation of memristors in neuromorphic computing and biomimetic sensing suffers from unexpected temporal and spatial variations due to the stochastic formation and rupture of conductive filaments (CFs). Here, the biocompatible silk fibroin (SF) is patterned with an on-demand nanocone array by using thermal scanning probe lithography (t-SPL) to guide and confine the growth of CFs in the silver/SF/gold (Ag/SF/Au) memristor. Benefiting from the high fabrication controllability, cycle-to-cycle (temporal) standard deviation of the set voltage for the structured memristor is significantly reduced by ≈95.5% (from 1.535 to 0.0686 V) and the device-to-device (spatial) standard deviation is also reduced to 0.0648 V. Besides, the statistical relationship between the structural nanocone design and the resultant performance is confirmed, optimizing at the small operation voltage (≈0.5 V) and current (100 nA), ultrafast switching speed (sub-100 ns), large on/off ratio (104 ), and the smallest switching slope (SS < 0.01 mV dec-1 ). Finally, the short-term plasticity and leaky integrated-and-fire behavior are emulated, and a reliable thermal nociceptor system is demonstrated for practical neuromorphic applications.

Energy-Efficient ReS2-Based Optoelectronic Synapse for 3D Object Reconstruction and Recognition

The neuromorphic vision system (NVS) equipped with optoelectronic synapses integrates perception, storage, and processing and is expected to address the issues of traditional machine vision. However, owing to their lack of stereo vision, existing NVSs focus on 2D image processing, which makes it difficult to solve problems such as spatial cognition errors and low-precision interpretation. Consequently, inspired by the human visual system, an NVS with stereo vision is developed to achieve 3D object recognition, depending on the prepared ReS2 optoelectronic synapse with 12.12 fJ ultralow power consumption. This device exhibits excellent optical synaptic plasticity derived from the persistent photoconductivity effect. As the cornerstone for 3D vision, color planar information is successfully discriminated and stored in situ at the sensor end, benefiting from its wavelength-dependent plasticity in the visible region. Importantly, the dependence of the channel conductance on the target distance is experimentally revealed, implying that the structure information on the object can be directly captured and stored by the synapse. The 3D image of the object is successfully reconstructed via fusion of its planar and depth images. Therefore, the proposed 3D-NVS based on ReS2 synapses for 3D objects achieves a recognition accuracy of 97.0%, which is much higher than that for 2D objects (32.6%), demonstrating its strong ability to prevent 2D-photo spoofing in applications such as face payment, entrance guard systems, and others.

Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing

In-sensor and near-sensor computing are becoming the next-generation computing paradigm for high-density and low-power sensory processing. To fulfil a high-density and efficient neuromorphic visual system with fully hierarchical emulation of the retina and visual cortex, emerging multimodal neuromorphic devices for multi-stage processing and a fully hardware-implemented system with versatile image processing functions are still lacking and highly desirable. Here we demonstrate an emerging multimodal-multifunctional resistive random-access memory (RRAM) device array based on modified silk fibroin protein (MSFP), exhibiting both optoelectronic RRAM (ORRAM) mode featured by unique negative and positive photoconductance memory and electrical RRAM (ERRAM) mode featured by analogue resistive switching. A full hardware implementation of the artificial visual system with versatile image processing functions is realised for the first time, including ORRAM mode array for the in-sensor image pre-processing (contrast enhancement, background denoising, feature extraction) and ERRAM mode array for near-sensor high-level image recognition, which hugely improves the integration density, and simply the circuit design and the fabrication and integration complexity.

Ferroelectric-defined reconfigurable homojunctions for in-memory sensing and computing

Recently, the increasing demand for data-centric applications is driving the elimination of image sensing, memory and computing unit interface, thus promising for latency- and energy-strict applications. Although dedicated electronic hardware has inspired the development of in-memory computing and in-sensor computing, folding the entire signal chain into one device remains challenging. Here an in-memory sensing and computing architecture is demonstrated using ferroelectric-defined reconfigurable two-dimensional photodiode arrays. High-level cognitive computing is realized based on the multiplications of light power and photoresponsivity through the photocurrent generation process and Kirchhoff's law. The weight is stored and programmed locally by the ferroelectric domains, enabling 51 (>5 bit) distinguishable weight states with linear, symmetric and reversible manipulation characteristics. Image recognition can be performed without any external memory and computing units. The three-in-one paradigm, integrating high-level computing, weight memorization and high-performance sensing, paves the way for a computing architecture with low energy consumption, low latency and reduced hardware overhead.

PZT-Enabled MoS2 Floating Gate Transistors: Overcoming Boltzmann Tyranny and Achieving Ultralow Energy Consumption for High-Accuracy Neuromorphic Computing

Low-power electronic devices play a pivotal role in the burgeoning artificial intelligence era. The study of such devices encompasses low-subthreshold swing (SS) transistors and neuromorphic devices. However, conventional field-effect transistors (FETs) face the inherent limitation of the "Boltzmann tyranny", which restricts SS to 60 mV decade-1 at room temperature. Additionally, FET-based neuromorphic devices lack sufficient conductance states for highly accurate neuromorphic computing due to a narrow memory window. In this study, we propose a pioneering PZT-enabled MoS2 floating gate transistor (PFGT) configuration, demonstrating a low SS of 46 mV decade-1 and a wide memory window of 7.2 V in the dual-sweeping gate voltage range from -7 to 7 V. The wide memory window provides 112 distinct conductance states for PFGT. Moreover, the PFGT-based artificial neural network achieves an outstanding facial-recognition accuracy of 97.3%. This study lays the groundwork for the development of low-SS transistors and highly energy efficient artificial synapses utilizing two-dimensional materials.

Ion trap and release dynamics enables nonintrusive tactile augmentation in monolithic sensory neuron

An iontronic-based artificial tactile nerve is a promising technology for emulating the tactile recognition and learning of human skin with low power consumption. However, its weak tactile memory and complex integration structure remain challenging. We present an ion trap and release dynamics (iTRD)-driven, neuro-inspired monolithic artificial tactile neuron (NeuroMAT) that can achieve tactile perception and memory consolidation in a single device. Through the tactile-driven release of ions initially trapped within iTRD-iongel, NeuroMAT only generates nonintrusive synaptic memory signals when mechanical stress is applied under voltage stimulation. The induced tactile memory is augmented by auxiliary voltage pulses independent of tactile sensing signals. We integrate NeuroMAT with an anthropomorphic robotic hand system to imitate memory-based human motion; the robust tactile memory of NeuroMAT enables the hand to consistently perform reliable gripping motion.

Parallel perception of visual motion using light-tunable memory matrix

Parallel perception of visual motion is of crucial significance to the development of an intelligent machine vision system. However, implementing in-sensor parallel visual motion perception using conventional complementary metal-oxide semiconductor technology is challenging, because the temporal and spatial information embedded in motion cannot be simultaneously encoded and perceived at the sensory level. Here, we demonstrate the parallel perception of diverse motion modes at the sensor level by exploiting light-tunable memory matrix in a van der Waals (vdW) heterostructure array. The optoelectronic characteristics of gate-tunable photoconductivity and light-tunable memory matrix enable devices in the array to realize simultaneous encoding and processing of incoming spatiotemporal light pattern. Furthermore, we implement a visual motion perceptron with the array capable of deciphering multiple motion parameters in parallel, including direction, velocity, acceleration, and angular velocity. Our work opens up a promising venue for the realization of an intelligent machine vision system based on in-sensor motion perception.