A Three-Dimensional Neuromorphic Photosensor Array for Nonvolatile In-Sensor Computing

Gate-Tunable Positive and Negative Photoconductance in Near-Infrared Organic Heterostructures for In-Sensor Computing

The rapid growth of sensor data in the artificial intelligence often causes significant reductions in processing speed and power efficiency. Addressing this challenge, in-sensor computing is introduced as an advanced sensor architecture that simultaneously senses, memorizes, and processes images at the sensor level. However, this is rarely reported for organic semiconductors that possess inherent flexibility and tunable bandgap. Herein, an organic heterostructure that exhibits a robust photoresponse to near-infrared (NIR) light is introduced, making it ideal for in-sensor computing applications. This heterostructure, consisting of partially overlapping p-type and n-type organic thin films, is compatible with conventional photolithography techniques, allowing for high integration density of up to 520 devices cm-2 with a 5 µm channel length. Importantly, by modulating gate voltage, both positive and negative photoresponses to NIR light (1050 nm) are attained, which establishes a linear correlation between responsivity and gate voltage and consequently enables real-time matrix multiplication within the sensor. As a result, this organic heterostructure facilitates efficient and precise NIR in-sensor computing, including image processing and nondestructive reading and classification, achieving a recognition accuracy of 97.06%. This work serves as a foundation for the development of reconfigurable and multifunctional NIR neuromorphic vision systems.

Recent Progress in Wearable Near-Sensor and In-Sensor Intelligent Perception Systems

As the Internet of Things (IoT) becomes more widespread, wearable smart systems will begin to be used in a variety of applications in people's daily lives, not only requiring the devices to have excellent flexibility and biocompatibility, but also taking into account redundant data and communication delays due to the use of a large number of sensors. Fortunately, the emerging paradigms of near-sensor and in-sensor computing, together with the proposal of flexible neuromorphic devices, provides a viable solution for the application of intelligent low-power wearable devices. Therefore, wearable smart systems based on new computing paradigms are of great research value. This review discusses the research status of a flexible five-sense sensing system based on near-sensor and in-sensor architectures, considering material design, structural design and circuit design. Furthermore, we summarize challenging problems that need to be solved and provide an outlook on the potential applications of intelligent wearable devices.

Bionic visual-audio photodetectors with in-sensor perception and preprocessing

Serving as the "eyes" and "ears" of the Internet of Things, optical and acoustic sensors are the fundamental components in hardware systems. Nowadays, mainstream hardware systems, often comprising numerous discrete sensors, conversion modules, and processing units, tend to result in complex architectures that are less efficient compared to human sensory pathways. Here, a visual-audio photodetector inspired by the human perception system is proposed to enable all-in-one visual and acoustic signal detection with computing capability. This device not only captures light but also optically records sound waves, thus achieving "watching" and "listening" within a single unit. The gate-tunable positive, negative, and zero photoresponses lead to highly programmable responsivities. This programmability enables the execution of diverse functions, including visual feature extraction, object classification, and sound wave manipulation. These results showcase the potential of expanding perception approaches in neuromorphic devices, opening up new possibilities to craft intelligent and compact hardware systems.

Nanomaterial-Based Artificial Vision Systems: From Bioinspired Electronic Eyes to In-Sensor Processing Devices

High-performance robotic vision empowers mobile and humanoid robots to detect and identify their surrounding objects efficiently, which enables them to cooperate with humans and assist human activities. For error-free execution of these robots' tasks, efficient imaging and data processing capabilities are essential, even under diverse and complex environments. However, conventional technologies fall short of meeting the high-standard requirements of robotic vision under such circumstances. Here, we discuss recent progress in artificial vision systems with high-performance imaging and data processing capabilities enabled by distinctive electrical, optical, and mechanical characteristics of nanomaterials surpassing the limitations of traditional silicon technologies. In particular, we focus on nanomaterial-based electronic eyes and in-sensor processing devices inspired by biological eyes and animal visual recognition systems, respectively. We provide perspectives on key nanomaterials, device components, and their functionalities, as well as explain the remaining challenges and future prospects of the artificial vision systems.

Broadband sensory networks with locally stored responsivities for neuromorphic machine vision

As the most promising candidates for the implementation of in-sensor computing, retinomorphic vision sensors can constitute built-in neural networks and directly implement multiply-and-accumulation operations using responsivities as the weights. However, existing retinomorphic vision sensors mainly use a sustained gate bias to maintain the responsivity due to its volatile nature. Here, we propose an ion-induced localized-field strategy to develop retinomorphic vision sensors with nonvolatile tunable responsivity in both positive and negative regimes and construct a broadband and reconfigurable sensory network with locally stored weights to implement in-sensor convolutional processing in spectral range of 400 to 1800 nanometers. In addition to in-sensor computing, this retinomorphic device can implement in-memory computing benefiting from the nonvolatile tunable conductance, and a complete neuromorphic visual system involving front-end in-sensor computing and back-end in-memory computing architectures has been constructed, executing supervised and unsupervised learning tasks as demonstrations. This work paves the way for the development of high-speed and low-power neuromorphic machine vision for time-critical and data-intensive applications.