Bioinspired and Low-Power 2D Machine Vision with Adaptive Machine Learning and Forgetting

Strain-insensitive viscoelastic perovskite film for intrinsically stretchable neuromorphic vision-adaptive transistors

Stretchable neuromorphic optoelectronics present tantalizing opportunities for intelligent vision applications that necessitate high spatial resolution and multimodal interaction. Existing neuromorphic devices are either stretchable but not reconcilable with multifunctionality, or discrete but with low-end neurological function and limited flexibility. Herein, we propose a defect-tunable viscoelastic perovskite film that is assembled into strain-insensitive quasi-continuous microsphere morphologies for intrinsically stretchable neuromorphic vision-adaptive transistors. The resulting device achieves trichromatic photoadaptation and a rapid adaptive speed (<150 s) beyond human eyes (3 ~ 30 min) even under 100% mechanical strain. When acted as an artificial synapse, the device can operate at an ultra-low energy consumption (15 aJ) (far below the human brain of 1 ~ 10 fJ) with a high paired-pulse facilitation index of 270% (one of the best figures of merit in stretchable synaptic phototransistors). Furthermore, adaptive optical imaging is achieved by the strain-insensitive perovskite films, accelerating the implementation of next-generation neuromorphic vision systems.

Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision

The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.

Nanomaterial-Based Artificial Vision Systems: From Bioinspired Electronic Eyes to In-Sensor Processing Devices

High-performance robotic vision empowers mobile and humanoid robots to detect and identify their surrounding objects efficiently, which enables them to cooperate with humans and assist human activities. For error-free execution of these robots' tasks, efficient imaging and data processing capabilities are essential, even under diverse and complex environments. However, conventional technologies fall short of meeting the high-standard requirements of robotic vision under such circumstances. Here, we discuss recent progress in artificial vision systems with high-performance imaging and data processing capabilities enabled by distinctive electrical, optical, and mechanical characteristics of nanomaterials surpassing the limitations of traditional silicon technologies. In particular, we focus on nanomaterial-based electronic eyes and in-sensor processing devices inspired by biological eyes and animal visual recognition systems, respectively. We provide perspectives on key nanomaterials, device components, and their functionalities, as well as explain the remaining challenges and future prospects of the artificial vision systems.

A Butterfly-Inspired Multisensory Neuromorphic Platform for Integration of Visual and Chemical Cues

Unisensory cues are often insufficient for animals to effectively engage in foraging, mating, and predatory activities. In contrast, integration of cues collected from multiple sensory organs enhances the overall perceptual experience and thereby facilitates better decision-making. Despite the importance of multisensory integration in animals, the field of artificial intelligence (AI) and neuromorphic computing has primarily focused on processing unisensory information. This lack of emphasis on multisensory integration can be attributed to the absence of a miniaturized hardware platform capable of co-locating multiple sensing modalities and enabling in-sensor and near-sensor processing. In this study, this limitation is addressed by utilizing the chemo-sensing properties of graphene and the photo-sensing capability of monolayer molybdenum disulfide (MoS2 ) to create a multisensory platform for visuochemical integration. Additionally, the in-memory-compute capability of MoS2 memtransistors is leveraged to develop neural circuits that facilitate multisensory decision-making. The visuochemical integration platform is inspired by intricate courtship of Heliconius butterflies, where female species rely on the integration of visual cues (such as wing color) and chemical cues (such as pheromones) generated by the male butterflies for mate selection. The butterfly-inspired visuochemical integration platform has significant implications in both robotics and the advancement of neuromorphic computing, going beyond unisensory intelligence and information processing.

Tailoring Wavelength-Adaptive Visual Neuroplasticity Transitions of Synaptic Transistors Comprising Rod-Coil Block Copolymers for Dual-Mode Photoswitchable Learning/Forgetting Neural Functions

The vision-inspired artificial neural network based on optical synapses has drawn a tremendous amount of attention for emulating biological senses. Although photoexcitation-induced synaptic functionalities have been widely studied, optical habituation via the photoinhibitory pathway is yet to be demonstrated for sophisticated biomimetic visual adaptive systems. Here, the first optical neuromorphic block copolymer (BCP) phototransistor is demonstrated as an all-optical operation responding to various wavelengths, fulfilling photoassisted dynamic learning/forgetting cycles via optical potentiation without gate bias. The polyfluorene BCPs were precisely designed to enable wavelength-adaptive responses, benefiting from interfacial semiconductor/electret morphology and the crystallinity/electron affinity of the BCPs. Notably, this is the first work to simultaneously exhibit fully light-controlled short- and long-term memory based on organic material systems. The device presents a high current contrast above 100-fold and long-term retention over 104 s. As a proof-of-concept for neural networks, a 6 × 6 array of photosynapses performed outstanding visual pattern learning/forgetting with high accuracy. This study exploits the design strategy of a conjugated BCP electret to unleash the full potential of wavelength-adaptive visual neuroplasticity transitions. It provides an effective architecture for designing high-performance and high-storage capacity required applications in next-generation neuromorphic systems.

A bio-inspired visuotactile neuron for multisensory integration

Multisensory integration is a salient feature of the brain which enables better and faster responses in comparison to unisensory integration, especially when the unisensory cues are weak. Specialized neurons that receive convergent input from two or more sensory modalities are responsible for such multisensory integration. Solid-state devices that can emulate the response of these multisensory neurons can advance neuromorphic computing and bridge the gap between artificial and natural intelligence. Here, we introduce an artificial visuotactile neuron based on the integration of a photosensitive monolayer MoS2 memtransistor and a triboelectric tactile sensor which minutely captures the three essential features of multisensory integration, namely, super-additive response, inverse effectiveness effect, and temporal congruency. We have also realized a circuit which can encode visuotactile information into digital spiking events, with probability of spiking determined by the strength of the visual and tactile cues. We believe that our comprehensive demonstration of bio-inspired and multisensory visuotactile neuron and spike encoding circuitry will advance the field of neuromorphic computing, which has thus far primarily focused on unisensory intelligence and information processing.

Dielectrics for Two-Dimensional Transition-Metal Dichalcogenide Applications

Despite over a decade of intense research efforts, the full potential of two-dimensional transition-metal dichalcogenides continues to be limited by major challenges. The lack of compatible and scalable dielectric materials and integration techniques restrict device performances and their commercial applications. Conventional dielectric integration techniques for bulk semiconductors are difficult to adapt for atomically thin two-dimensional materials. This review provides a brief introduction into various common and emerging dielectric synthesis and integration techniques and discusses their applicability for 2D transition metal dichalcogenides. Dielectric integration for various applications is reviewed in subsequent sections including nanoelectronics, optoelectronics, flexible electronics, valleytronics, biosensing, quantum information processing, and quantum sensing. For each application, we introduce basic device working principles, discuss the specific dielectric requirements, review current progress, present key challenges, and offer insights into future prospects and opportunities.