1
|
Lin Z, Chen J, Zheng Z, Lai Q, Liu Z, Liu L, Xiao J, Wang W. Multifunctional UV photodetect-memristors based on area selective fabricated Ga 2S 3/graphene/GaN van der Waals heterojunctions. MATERIALS HORIZONS 2025; 12:3091-3104. [PMID: 39878536 DOI: 10.1039/d4mh01711k] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/31/2025]
Abstract
Multifunctional devices based on van der Waals heterojunctions have drawn significant attention owing to their portable size, low power consumption and various application scenarios. However, high fabrication equipment requirements, complex device structures and limited operating conditions hinder their potential value. Herein, multifunctional UV photodetect-memristors based on Ga2S3/graphene/GaN van der Waals heterojunctions via area selective deposition have been proposed for the first time. The Ga2S3/graphene/GaN heterojunctions are firstly grown via area selective deposition (ASD) without a mask plate or lithography process. And the corresponding molecular dynamics (MD) and density functional theory (DFT) simulation further confirmed its feasibility and physical properties. Subsequently, multifunctional devices based on Ga2S3/graphene/GaN heterojunctions are fabricated accordingly, and exhibit ultrafast (<80 μs) response at 0 V and stable, highly sensitive (1150.4 A W-1) memory features at 3 V. Here, the huge hole barriers formed on the two edges of graphene set the foundation of trapping and detecting light-induced carriers. Afterwards, handwriting numeral recognition tasks are carried out based on the performance extracted from the device and a simplified noise filtering and improved recognition accuracy system is proposed, confirming its application potential in the artificial intelligence area. This study proposes a practical way to grow large-size 2D materials selectively, shows the valuable application potential of p-g-n heterojunctions in various application fields, and expands an innovative path of device development in the post-Moorish era.
Collapse
Affiliation(s)
- Zhengliang Lin
- School of Materials Scicence and Engineering, South China University of Technology, Guangzhou, 510640, China.
| | - Junrui Chen
- School of Materials Scicence and Engineering, South China University of Technology, Guangzhou, 510640, China.
| | - Zhuohang Zheng
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510640, China
| | - Quanguang Lai
- School of Materials Scicence and Engineering, South China University of Technology, Guangzhou, 510640, China.
| | - Zhiqi Liu
- School of Materials Scicence and Engineering, South China University of Technology, Guangzhou, 510640, China.
| | - Liwei Liu
- School of Materials Scicence and Engineering, South China University of Technology, Guangzhou, 510640, China.
| | - Jiaying Xiao
- School of Materials Scicence and Engineering, South China University of Technology, Guangzhou, 510640, China.
| | - Wenliang Wang
- School of Materials Scicence and Engineering, South China University of Technology, Guangzhou, 510640, China.
| |
Collapse
|
2
|
Huang YC, Chen YC, Chen KT, Chen CT, Shih LC, Chen JS. Light-to-Spike Encoding Using Indium-Gallium-Zinc Oxide Phototransistor for all-Color Image Recognition with Dynamic Range and Precision Tunability. SMALL METHODS 2025; 9:e2401502. [PMID: 39648497 DOI: 10.1002/smtd.202401502] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/13/2024] [Revised: 11/18/2024] [Indexed: 12/10/2024]
Abstract
To enhance the efficiency of machine vision system, physical hardware capable of sensing and encoding is essential. However, sensing and encoding color information has been overlooked. Therefore, this work utilizes an indium-gallium-zinc oxide (IGZO) phototransistor to detect varying densities of red, green, and blue (RGB) light, converting them into corresponding drain current (ID) states. By applying stochastic gate voltage (VG) pulses to the IGZO phototransistor, the fluctuations are generated in these ID states. When the ID exceeds the threshold current (ITC), a spike signal is generated. This approach enables the conversion of light densities into spike signals, achieving spike-rate encoding. Moreover, adjusting the standard deviation (σ) of the VG pulses controls the range of light densities converted into spike rates, while altering the mean (μ) of the VG pulses changes the baseline level of spike rates. Remarkably, separate RGB channels offer a tunable encoding process, which can emphasize individual colors and correct color bias. The encoded spike rates are also fed into a spiking neural network (SNN) for CIFAR-10 pattern recognition, achieving an accuracy of 86%. The method allows the operation of SNN and shows the tunability in the process of light-to-spike encoding, opening possibilities for color image processing.
Collapse
Affiliation(s)
- Ya-Chi Huang
- Department of Materials Science and Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
| | - Yu-Chieh Chen
- Department of Materials Science and Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
| | - Kuan-Ting Chen
- Department of Materials Science and Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
| | - Chun-Tao Chen
- Department of Materials Science and Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
| | - Li-Chung Shih
- Department of Materials Science and Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
| | - Jen-Sue Chen
- Department of Materials Science and Engineering, National Cheng Kung University, Tainan, 70101, Taiwan
- Academy of Innovative Semiconductor and Sustainable Manufacturing, National Cheng Kung University, Tainan, 70101, Taiwan
| |
Collapse
|
3
|
Kim JE, Soh K, Hwang SI, Yang DY, Yoon JH. Memristive neuromorphic interfaces: integrating sensory modalities with artificial neural networks. MATERIALS HORIZONS 2025. [PMID: 40104909 DOI: 10.1039/d5mh00038f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/20/2025]
Abstract
The advent of the Internet of Things (IoT) has led to exponential growth in data generated from sensors, requiring efficient methods to process complex and unstructured external information. Unlike conventional von Neumann sensory systems with separate data collection and processing units, biological sensory systems integrate sensing, memory, and computing to process environmental information in real time with high efficiency. Memristive neuromorphic sensory systems using memristors as their basic components have emerged as promising alternatives to CMOS-based systems. Memristors can closely replicate the key characteristics of biological receptors, neurons, and synapses by integrating the threshold and adaptation properties of receptors, the action potential firing in neurons, and the synaptic plasticity of synapses. Furthermore, through careful engineering of their switching dynamics, the electrical properties of memristors can be tailored to emulate specific functions, while benefiting from high operational speed, low power consumption, and exceptional scalability. Consequently, their integration with high-performance sensors offers a promising pathway toward realizing fully integrated artificial sensory systems that can efficiently process and respond to diverse environmental stimuli in real time. In this review, we first introduce the fundamental principles of memristive neuromorphic technologies for artificial sensory systems, explaining how each component is structured and what functions it performs. We then discuss how these principles can be applied to replicate the four traditional senses, highlighting the underlying mechanisms and recent advances in mimicking biological sensory functions. Finally, we address the remaining challenges and provide prospects for the continued development of memristor-based artificial sensory systems.
Collapse
Affiliation(s)
- Ji Eun Kim
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul 02791, Republic of Korea
- Department of Materials Science and Engineering, Korea University, Seoul 02841, Republic of Korea
| | - Keunho Soh
- School of Advanced Materials and Engineering, Sungkyunkwan University (SKKU), Suwon 16419, Republic of Korea.
| | - Su In Hwang
- School of Advanced Materials and Engineering, Sungkyunkwan University (SKKU), Suwon 16419, Republic of Korea.
| | - Do Young Yang
- School of Advanced Materials and Engineering, Sungkyunkwan University (SKKU), Suwon 16419, Republic of Korea.
| | - Jung Ho Yoon
- School of Advanced Materials and Engineering, Sungkyunkwan University (SKKU), Suwon 16419, Republic of Korea.
| |
Collapse
|
4
|
Jacob B, Silva J, Figueiredo JML, Nieder JB, Romeira B. Light-induced negative differential resistance and neural oscillations in neuromorphic photonic semiconductor micropillar sensory neurons. Sci Rep 2025; 15:6805. [PMID: 40000706 PMCID: PMC11862094 DOI: 10.1038/s41598-025-90265-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/23/2024] [Accepted: 02/11/2025] [Indexed: 02/27/2025] Open
Abstract
Neuromorphic systems, inspired by nature, are sought to efficiently process analogue inputs in real and complex environments. This could lead to ultralow-power in-sensor intelligent edge computers. Here, we present an artificial sensory oscillator neuron consisting of a III-V semiconductor micropillar quantum resonant tunnelling diode (RTD) with GaAs photosensitive absorption layers. The oscillatory optical neuron encodes incoming analogue optical data into spatiotemporal oscillatory signals. We demonstrate that near-infrared light within a certain intensity range activates a region of negative differential resistance, and subsequently, large-amplitude voltage oscillations. As a result, optic analogue information is encoded into electrical oscillations resulting in amplification of sensory light inputs. Under pulse-modulated light, excitation and inhibition of burst firing patterns can be controlled within a single oscillatory neuron, simulating neural activity in networks in the form of breather-type oscillatory phenomena. Such spatiotemporal oscillatory patterns (burst firing) form the basis for the combined sensing, pre-processing, and encoding abilities of the vision-nervous system found in biological organisms. This work paves the way for future artificial visual systems using III-V semiconductor nano-optoelectronic circuits in applications for light-driven neurorobotics, bioinspired optoelectronics, and in-sensor neuromorphic computing systems for real-time processing of sensory data.
Collapse
Affiliation(s)
- Bejoys Jacob
- INL-International Iberian Nanotechnology Laboratory, Av. Mestre José Veiga S/N, 4715-330, Braga, Portugal.
| | - Juan Silva
- INL-International Iberian Nanotechnology Laboratory, Av. Mestre José Veiga S/N, 4715-330, Braga, Portugal
| | - José M L Figueiredo
- LIP - Laboratório de Instrumentação e Física Experimental de Partículas, Departamento de Física, Faculdade de Ciências, Universidade de Lisboa, 1749-016, Lisboa, Portugal
| | - Jana B Nieder
- INL-International Iberian Nanotechnology Laboratory, Av. Mestre José Veiga S/N, 4715-330, Braga, Portugal
| | - Bruno Romeira
- INL-International Iberian Nanotechnology Laboratory, Av. Mestre José Veiga S/N, 4715-330, Braga, Portugal.
| |
Collapse
|
5
|
Hur P, Yoon D, Yoon M, Park Y, Son J. Artificial Photothermal Nociceptor Using Mott Oscillators. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2025; 12:e2409353. [PMID: 39692203 PMCID: PMC11809409 DOI: 10.1002/advs.202409353] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/08/2024] [Revised: 12/04/2024] [Indexed: 12/19/2024]
Abstract
Bioinspired sensory systems based on spike neural networks have received considerable attention in resolving high energy consumption and limited bandwidth in current sensory systems. To efficiently produce spike signals upon exposure to external stimuli, compact neuron devices are required for signal detection and their encoding into spikes in a single device. Herein, it is demonstrated that Mott oscillative spike neurons can integrate sensing and ceaseless spike generation in a compact form, which emulates the process of evoking photothermal sensing in the features of biological photothermal nociceptors. Interestingly, frequency-tunable and repetitive spikes are generated above the threshold value (Pth = 84 mW cm-2) as a characteristic of "threshold" in leaky-integrate-and-fire (LIF) neurons; the neuron devices successfully mimic a crucial feature of biological thermal nociceptors, including modulation of frequency coding and startup latency depending on the intensity of photothermal stimuli. Furthermore, Mott spike neurons are self-adapted after sensitization upon exposure to high-intensity electromagnetic radiation, which can replicate allodynia and hyperalgesia in a biological sensory system. Thus, this study presents a unique approach to capturing and encoding environmental source data into spikes, enabling efficient sensing of environmental sources for the application of adaptive sensory systems.
Collapse
Affiliation(s)
- Pyeongkang Hur
- Department of Materials Science and EngineeringPohang University of Science and Technology (POSTECH)Pohang37683Republic of Korea
| | - Daseob Yoon
- Department of Electrical EngineeringPukyong National UniversityBusan48513Republic of Korea
| | - Minwook Yoon
- Department of Materials Science and EngineeringSeoul National UniversitySeoul08826Republic of Korea
- Research Institute of Advanced MaterialsSeoul National UniversitySeoul08826Republic of Korea
- Institute of Applied PhysicsSeoul National UniversitySeoul08826Republic of Korea
| | - Yunkyu Park
- Materials Science and Technology DivisionOak Ridge National LaboratoryOak RidgeTN37830USA
| | - Junwoo Son
- Department of Materials Science and EngineeringSeoul National UniversitySeoul08826Republic of Korea
- Research Institute of Advanced MaterialsSeoul National UniversitySeoul08826Republic of Korea
- Institute of Applied PhysicsSeoul National UniversitySeoul08826Republic of Korea
| |
Collapse
|
6
|
Kim JH, Kim HW, Chung MJ, Shin DH, Kim YR, Kim J, Jang YH, Cheong SW, Lee SH, Han J, Park HJ, Han JK, Hwang CS. A stochastic photo-responsive memristive neuron for an in-sensor visual system based on a restricted Boltzmann machine. NANOSCALE HORIZONS 2024; 9:2248-2258. [PMID: 39376201 DOI: 10.1039/d4nh00421c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/09/2024]
Abstract
In-sensor computing has gained attention as a solution to overcome the von Neumann computing bottlenecks inherent in conventional sensory systems. This attention is due to the ability of sensor elements to directly extract meaningful information from external signals, thereby simplifying complex data. The advantage of in-sensor computing can be maximized with the sampling principle of a restricted Boltzmann machine (RBM) to extract significant features. In this study, a stochastic photo-responsive neuron is developed using a TiN/In-Ga-Zn-O/TiN optoelectronic memristor and an Ag/HfO2/Pt threshold-switching memristor, which can be configured as an input neuron in an in-sensor RBM. It demonstrates a sigmoidal switching probability depending on light intensity. The stochastic properties allow for the simultaneous exploration of various neuron states within the network, making identifying optimal features in complex images easier. Based on semi-empirical simulations, high recognition accuracies of 90.9% and 95.5% are achieved using handwritten digit and face image datasets, respectively. In addition, the in-sensor RBM effectively reconstructs abnormal face images, indicating that integrating in-sensor computing with probabilistic neural networks can lead to reliable and efficient image recognition under unpredictable real-world conditions.
Collapse
Affiliation(s)
- Jin Hong Kim
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Hyun Wook Kim
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Min Jung Chung
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Dong Hoon Shin
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Yeong Rok Kim
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Jaehyun Kim
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Yoon Ho Jang
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Sun Woo Cheong
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Soo Hyung Lee
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Janguk Han
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Hyung Jun Park
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| | - Joon-Kyu Han
- System Semiconductor Engineering and Department of Electronic Engineering, Sogang University, 35 Baekbeom-ro, Mapo-gu, Seoul 04107, Republic of Korea.
| | - Cheol Seong Hwang
- Department of Materials Science and Engineering and Inter-University Semiconductor Research Center, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Republic of Korea.
| |
Collapse
|
7
|
Zhong S, Su L, Xu M, Loke D, Yu B, Zhang Y, Zhao R. Recent Advances in Artificial Sensory Neurons: Biological Fundamentals, Devices, Applications, and Challenges. NANO-MICRO LETTERS 2024; 17:61. [PMID: 39537845 PMCID: PMC11561216 DOI: 10.1007/s40820-024-01550-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/31/2024] [Accepted: 09/28/2024] [Indexed: 11/16/2024]
Abstract
Spike-based neural networks, which use spikes or action potentials to represent information, have gained a lot of attention because of their high energy efficiency and low power consumption. To fully leverage its advantages, converting the external analog signals to spikes is an essential prerequisite. Conventional approaches including analog-to-digital converters or ring oscillators, and sensors suffer from high power and area costs. Recent efforts are devoted to constructing artificial sensory neurons based on emerging devices inspired by the biological sensory system. They can simultaneously perform sensing and spike conversion, overcoming the deficiencies of traditional sensory systems. This review summarizes and benchmarks the recent progress of artificial sensory neurons. It starts with the presentation of various mechanisms of biological signal transduction, followed by the systematic introduction of the emerging devices employed for artificial sensory neurons. Furthermore, the implementations with different perceptual capabilities are briefly outlined and the key metrics and potential applications are also provided. Finally, we highlight the challenges and perspectives for the future development of artificial sensory neurons.
Collapse
Affiliation(s)
- Shuai Zhong
- Guangdong Institute of Intelligence Science and Technology, Hengqin, Zhuhai, 519031, People's Republic of China.
| | - Lirou Su
- Guangdong Institute of Intelligence Science and Technology, Hengqin, Zhuhai, 519031, People's Republic of China
| | - Mingkun Xu
- Guangdong Institute of Intelligence Science and Technology, Hengqin, Zhuhai, 519031, People's Republic of China
| | - Desmond Loke
- Department of Science, Mathematics and Technology, Singapore University of Technology and Design, Singapore, 487372, Singapore
| | - Bin Yu
- College of Integrated Circuits, Zhejiang University, Hangzhou, 3112000, People's Republic of China
- ZJU-Hangzhou Global Scientific and Technological Innovation Center, Hangzhou, 310027, People's Republic of China
| | - Yishu Zhang
- College of Integrated Circuits, Zhejiang University, Hangzhou, 3112000, People's Republic of China.
- ZJU-Hangzhou Global Scientific and Technological Innovation Center, Hangzhou, 310027, People's Republic of China.
| | - Rong Zhao
- Department of Precision Instruments, Tsinghua University, Beijing, 100084, People's Republic of China
- Center for Brain-Inspired Computing Research, Tsinghua University, Beijing, 100084, People's Republic of China
- IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, 100084, People's Republic of China
| |
Collapse
|
8
|
Choi C, Lee GJ, Chang S, Song YM, Kim DH. Inspiration from Visual Ecology for Advancing Multifunctional Robotic Vision Systems: Bio-inspired Electronic Eyes and Neuromorphic Image Sensors. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2412252. [PMID: 39402806 DOI: 10.1002/adma.202412252] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/19/2024] [Revised: 09/18/2024] [Indexed: 11/29/2024]
Abstract
In robotics, particularly for autonomous navigation and human-robot collaboration, the significance of unconventional imaging techniques and efficient data processing capabilities is paramount. The unstructured environments encountered by robots, coupled with complex missions assigned to them, present numerous challenges necessitating diverse visual functionalities, and consequently, the development of multifunctional robotic vision systems has become indispensable. Meanwhile, rich diversity inherent in animal vision systems, honed over evolutionary epochs to meet their survival demands across varied habitats, serves as a profound source of inspirations. Here, recent advancements in multifunctional robotic vision systems drawing inspiration from natural ocular structures and their visual perception mechanisms are delineated. First, unique imaging functionalities of natural eyes across terrestrial, aerial, and aquatic habitats and visual signal processing mechanism of humans are explored. Then, designs and functionalities of bio-inspired electronic eyes are explored, engineered to mimic key components and underlying optical principles of natural eyes. Furthermore, neuromorphic image sensors are discussed, emulating functional properties of synapses, neurons, and retinas and thereby enhancing accuracy and efficiency of robotic vision tasks. Next, integration examples of electronic eyes with mobile robotic/biological systems are introduced. Finally, a forward-looking outlook on the development of bio-inspired electronic eyes and neuromorphic image sensors is provided.
Collapse
Affiliation(s)
- Changsoon Choi
- Center for Quantum Technology, Korea Institute of Science and Technology, Seoul, 02792, Republic of Korea
| | - Gil Ju Lee
- School of Electrical and Electronics Engineering, Pusan National University, Busan, 46241, Republic of Korea
| | - Sehui Chang
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju, 61005, Republic of Korea
| | - Young Min Song
- School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology, Gwangju, 61005, Republic of Korea
- AI Graduate School, Gwangju Institute of Science and Technology, Gwangju, 61005, Republic of Korea
- Department of Semiconductor Engineering, Gwangju Institute of Science and Technology, Gwangju, 61005, Republic of Korea
| | - Dae-Hyeong Kim
- Center for Nanoparticle Research, Institute for Basic Science (IBS), Seoul, 08826, Republic of Korea
- School of Chemical and Biological Engineering, Institute of Chemical Processes, Seoul National University, Seoul, 08826, Republic of Korea
| |
Collapse
|
9
|
Han CY, Zhao S, Fang SL, Liu W, Tang WM, Lai PT, Li C, Ma YX, Song JQ, Li X, Wang XL, Ren WJ, Wang RL, Huang XD, Zhang GH, Geng L. A Flexible Artificial Spiking Photoreceptor Enabled by a Single VO 2 Mott Memristor for the Spike-Based Electronic Retina. ACS APPLIED MATERIALS & INTERFACES 2024; 16:57404-57411. [PMID: 39380467 DOI: 10.1021/acsami.4c12874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/10/2024]
Abstract
The neuromorphic vision system that utilizes spikes as information carriers is crucial for the formation of spiking neural networks. Here, we present a bioinspired flexible artificial spiking photoreceptor (ASP), which is realized by using a single VO2 Mott memristor that can simultaneously sense and encode the stimulus light into spikes. The ASP has high spike-encoded photosensitivity and ultrawide photosensing range (405-808 nm) with good endurance (>7 × 107) and high flexibility (bending radius ∼5 mm). Then, we put forward an all-spike electronic retina architecture that comprises one layer of ASPs and one layer of artificial optical nerves (AONs) to process the spike information. Each AON consists of a single Mott memristor connected in series with a neuro-transistor that is a multiple-input floating-gate MOS transistor. Simulation results demonstrate that the all-spike electronic retina can successfully segment images with high Shannon entropy, thus laying the foundation for the development of a spike-based neuromorphic vision system.
Collapse
Affiliation(s)
- Chuan Yu Han
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| | - Shujing Zhao
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| | - Sheng Li Fang
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| | - Weihua Liu
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| | - Wing Man Tang
- The Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong 999077, P.R. China
| | - Peter To Lai
- The Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong 999077, P.R. China
| | - Can Li
- The Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong 999077, P.R. China
| | - Yuan Xiao Ma
- School of Integrated Circuits and Electronics, Beijing Institute of Technology, Beijing 100081, P.R. China
| | - Jia Qi Song
- Physics Teaching and Experiment Center, Shenzhen Technology University, Shenzhen 518118, P.R. China
| | - Xin Li
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| | - Xiao Li Wang
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| | - Wen Jun Ren
- PE Department, Xi' an Jiaotong University, Xi'an 710049, China
| | - Rui Lin Wang
- PE Department, Xi' an Jiaotong University, Xi'an 710049, China
| | - Xiao Dong Huang
- Key Laboratory of MEMS of the Ministry of Education, School of Electronic Science and Engineering, Southeast University, Nanjing 211189, China
| | - Guo He Zhang
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| | - Li Geng
- School of Microelectronics, Xi'an Jiaotong University, Xi'an 710049, P.R. China
| |
Collapse
|
10
|
Wan C, Pei M, Shi K, Cui H, Long H, Qiao L, Xing Q, Wan Q. Toward a Brain-Neuromorphics Interface. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2311288. [PMID: 38339866 DOI: 10.1002/adma.202311288] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Revised: 01/17/2024] [Indexed: 02/12/2024]
Abstract
Brain-computer interfaces (BCIs) that enable human-machine interaction have immense potential in restoring or augmenting human capabilities. Traditional BCIs are realized based on complementary metal-oxide-semiconductor (CMOS) technologies with complex, bulky, and low biocompatible circuits, and suffer with the low energy efficiency of the von Neumann architecture. The brain-neuromorphics interface (BNI) would offer a promising solution to advance the BCI technologies and shape the interactions with machineries. Neuromorphic devices and systems are able to provide substantial computation power with extremely high energy-efficiency by implementing in-materia computing such as in situ vector-matrix multiplication (VMM) and physical reservoir computing. Recent progresses on integrating neuromorphic components with sensing and/or actuating modules, give birth to the neuromorphic afferent nerve, efferent nerve, sensorimotor loop, and so on, which has advanced the technologies for future neurorobotics by achieving sophisticated sensorimotor capabilities as the biological system. With the development on the compact artificial spiking neuron and bioelectronic interfaces, the seamless communication between a BNI and a bioentity is reasonably expectable. In this review, the upcoming BNIs are profiled by introducing the brief history of neuromorphics, reviewing the recent progresses on related areas, and discussing the future advances and challenges that lie ahead.
Collapse
Affiliation(s)
- Changjin Wan
- Yongjiang Laboratory (Y-LAB), Ningbo, Zhejiang, 315202, China
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
- Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, 315201, China
| | - Mengjiao Pei
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Kailu Shi
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Hangyuan Cui
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Haotian Long
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Lesheng Qiao
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Qianye Xing
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Qing Wan
- Yongjiang Laboratory (Y-LAB), Ningbo, Zhejiang, 315202, China
- School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
- Ningbo Institute of Materials Technology and Engineering, Chinese Academy of Sciences, Ningbo, 315201, China
| |
Collapse
|
11
|
Ren H, Li F, Wang M, Liu G, Li D, Wang R, Chen Y, Tang Y, Wang Y, Jin R, Huang Q, Xing L, Chen X, Wang J, Guo C, Zhu B. An Ion-Mediated Spiking Chemical Neuron based on Mott Memristor. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2403678. [PMID: 38887824 DOI: 10.1002/adma.202403678] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/12/2024] [Revised: 05/31/2024] [Indexed: 06/20/2024]
Abstract
Artificial spiking neurons capable of interpreting ionic information into electrical spikes are critical to mimic biological signaling systems. Mott memristors are attractive for constructing artificial spiking neurons due to their simple structure, low energy consumption, and rich neural dynamics. However, challenges remain in achieving ion-mediated spiking and biohybrid-interfacing in Mott neurons. Here, a biomimetic spiking chemical neuron (SCN) utilizing an NbOx Mott memristor and oxide field-effect transistor-type chemical sensor is introduced. The SCN exhibits both excitation and inhibition spiking behaviors toward ionic concentrations akin to biological neural systems. It demonstrates spiking responses across physiological and pathological Na+ concentrations (1-200 × 10-3 m). The Na+-mediated SCN enables both frequency encoding and time-to-first-spike coding schemes, illustrating the rich neural dynamics of Mott neuron. In addition, the SCN interfaced with L929 cells facilitates real-time modulation of ion-mediated spiking under both normal and salty cellular microenvironments.
Collapse
Affiliation(s)
- Huihui Ren
- School of Materials and Engineering, Zhejiang University, Hangzhou, 310027, China
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Fanfan Li
- School of Materials and Engineering, Zhejiang University, Hangzhou, 310027, China
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Min Wang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Guolei Liu
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Dingwei Li
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Rui Wang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Yitong Chen
- School of Materials and Engineering, Zhejiang University, Hangzhou, 310027, China
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Yingjie Tang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Yan Wang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Ran Jin
- School of Materials and Engineering, Zhejiang University, Hangzhou, 310027, China
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Qi Huang
- Westlake Institute for Optoelectronics, Hangzhou, 311421, China
| | - Lixiang Xing
- Westlake Institute for Optoelectronics, Hangzhou, 311421, China
| | - Xiaopeng Chen
- Enovated3D (Hangzhou) Technology Development Co. Ltd., Hangzhou, 310051, China
| | - Juan Wang
- College of Environmental & Resource Sciences, Zhejiang University, Hangzhou, 310058, China
| | - Chengchen Guo
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
| | - Bowen Zhu
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, 310024, China
- Westlake Institute for Optoelectronics, Hangzhou, 311421, China
| |
Collapse
|
12
|
Zhang X, Zhu Y, Chen L, Duan P, Zhou M. Augmented reality navigation method based on image segmentation and sensor tracking registration technology. Sci Rep 2024; 14:15281. [PMID: 38961095 PMCID: PMC11222374 DOI: 10.1038/s41598-024-65204-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2023] [Accepted: 06/18/2024] [Indexed: 07/05/2024] Open
Abstract
With the rapid development of modern science and technology, navigation technology provides great convenience for people's life, but the problem of inaccurate localization in complex environments has always been a challenge that navigation technology needs to be solved urgently. To address this challenge, this paper proposes an augmented reality navigation method that combines image segmentation and multi-sensor fusion tracking registration. The method optimizes the image processing process through the GA-OTSU-Canny algorithm and combines high-precision multi-sensor information in order to achieve accurate tracking of positioning and guidance in complex environments. Experimental results show that the GA-OTSU-Canny algorithm has a faster image edge segmentation rate, and the fastest start speed is only 1.8 s, and the fastest intersection selection time is 1.2 s. The navigation system combining the image segmentation and sensor tracking and registration techniques has a highly efficient performance in real-world navigation, and its building recognition rates are all above 99%. The augmented reality navigation system not only improves the navigation accuracy in high-rise and urban canyon environments, but also significantly outperforms traditional navigation solutions in terms of navigation startup time and target building recognition accuracy. In summary, this research not only provides a new framework for the theoretical integration of image processing and multi-sensor data, but also brings innovative technical solutions for the development and application of practical navigation systems.
Collapse
Affiliation(s)
- Xiaoying Zhang
- College of Mechanical Engineering, Zhengzhou University of Science and Technology, Zhengzhou, 450064, China
| | - Yonggang Zhu
- College of Mechanical Engineering, Zhengzhou University of Science and Technology, Zhengzhou, 450064, China.
| | - Lumin Chen
- College of Mechanical Engineering, Zhengzhou University of Science and Technology, Zhengzhou, 450064, China
| | - Peng Duan
- College of Mechanical Engineering, Zhengzhou University of Science and Technology, Zhengzhou, 450064, China
| | - Meijuan Zhou
- College of Mechanical Engineering, Zhengzhou University of Science and Technology, Zhengzhou, 450064, China
| |
Collapse
|
13
|
Meng Y, Cheng G. Human somatosensory systems based on sensor-memory-integrated technology. NANOSCALE 2024; 16:11928-11958. [PMID: 38847091 DOI: 10.1039/d3nr06521a] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/28/2024]
Abstract
As a representative artificial neural network (ANN) for incorporating sensing functions and memory functions into one system to achieve highly miniaturized and highly integrated devices or systems, artificial sensory systems (ASSs) can have a far-reaching influence on precise instrumentation, sensing, and automation engineering. Artificial sensory systems have enjoyed considerable progress in recent years, from low degree integrations to highly advanced sophisticated integrations, from single-modal perceptions to multimode-fused perceptions. However, there are issues around the large hardware area, power consumption, and communication bandwidth needed during the processes where multimodal sensing signals are converted into a digital mode before they can be processed by a digital processor. Therefore, deepening the research into sensory integration is of great importance. In this review, we briefly introduce fundamental knowledge about the memristor mechanism, describe some representative human somatosensory systems, and elucidate the relationship between the properties of memristor devices and the structure. The electronic character of the sensors, future prospects, and key challenges surrounding sensor-memory integrated technologies are also discussed.
Collapse
Affiliation(s)
- Yanfang Meng
- Institute of Intelligent Flexible Mechatronics, School of Mechanical Engineering, Jiangsu University, Zhenjiang, No. 301 Xuefu Road, Zhenjiang, Jiangsu Province, 212013, China.
| | - Guanggui Cheng
- Institute of Intelligent Flexible Mechatronics, School of Mechanical Engineering, Jiangsu University, Zhenjiang, No. 301 Xuefu Road, Zhenjiang, Jiangsu Province, 212013, China.
| |
Collapse
|
14
|
Nath SK, Das SK, Nandi SK, Xi C, Marquez CV, Rúa A, Uenuma M, Wang Z, Zhang S, Zhu RJ, Eshraghian J, Sun X, Lu T, Bian Y, Syed N, Pan W, Wang H, Lei W, Fu L, Faraone L, Liu Y, Elliman RG. Optically Tunable Electrical Oscillations in Oxide-Based Memristors for Neuromorphic Computing. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2400904. [PMID: 38516720 DOI: 10.1002/adma.202400904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 03/18/2024] [Indexed: 03/23/2024]
Abstract
The application of hardware-based neural networks can be enhanced by integrating sensory neurons and synapses that enable direct input from external stimuli. This work reports direct optical control of an oscillatory neuron based on volatile threshold switching in V3O5. The devices exhibit electroforming-free operation with switching parameters that can be tuned by optical illumination. Using temperature-dependent electrical measurements, conductive atomic force microscopy (C-AFM), in situ thermal imaging, and lumped element modelling, it is shown that the changes in switching parameters, including threshold and hold voltages, arise from overall conductivity increase of the oxide film due to the contribution of both photoconductive and bolometric characteristics of V3O5, which eventually affects the oscillation dynamics. Furthermore, V3O5 is identified as a new bolometric material with a temperature coefficient of resistance (TCR) as high as -4.6% K-1 at 423 K. The utility of these devices is illustrated by demonstrating in-sensor reservoir computing with reduced computational effort and an optical encoding layer for spiking neural network (SNN), respectively, using a simulated array of devices.
Collapse
Affiliation(s)
- Shimul Kanti Nath
- Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA, 6009, Australia
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National University, Canberra, ACT, 2601, Australia
- School of Photovoltaic and Renewable Energy Engineering, University of New South Wales (UNSW Sydney), Kensington, NSW, 2052, Australia
| | - Sujan Kumar Das
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National University, Canberra, ACT, 2601, Australia
- Department of Physics, Jahangirnagar Univeristy, Savar, Dhaka, 1342, Bangladesh
| | - Sanjoy Kumar Nandi
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National University, Canberra, ACT, 2601, Australia
| | - Chen Xi
- Department of Electrical and Electronic Engineering, the University of Hong Kong, Pok Fu Lam Rd, Hong Kong Island, Hong Kong
| | | | - Armando Rúa
- Department of Physics, University of Puerto Rico, Mayaguez, PR, 00681, USA
| | - Mutsunori Uenuma
- Information Device Science Laboratory, Nara Institute of Science and Technology (NAIST), Nara, 630-0192, Japan
| | - Zhongrui Wang
- Department of Electrical and Electronic Engineering, the University of Hong Kong, Pok Fu Lam Rd, Hong Kong Island, Hong Kong
| | - Songqing Zhang
- Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA, 6009, Australia
| | - Rui-Jie Zhu
- Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA, 95064, USA
| | - Jason Eshraghian
- Department of Electrical and Computer Engineering, University of California, Santa Cruz, CA, 95064, USA
| | - Xiao Sun
- John de Laeter Centre, Curtin University, Perth, WA, 6102, Australia
| | - Teng Lu
- Research School of Chemistry, The Australian National University, Canberra, ACT, 2601, Australia
| | - Yue Bian
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National University, Canberra, ACT, 2601, Australia
- Australian Research Council Centre of Excellence for Transformative Meta-Optical Systems, Canberra, ACT, 2601, Australia
| | - Nitu Syed
- Australian Research Council Centre of Excellence for Transformative Meta-Optical Systems, School of Physics, University of Melbourne, Melbourne, Victoria, 3010, Australia
| | - Wenwu Pan
- Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA, 6009, Australia
- Australian Research Council Centre of Excellence for Transformative Meta-Optical Systems, Perth, WA, 6009, Australia
| | - Han Wang
- Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA, 6009, Australia
| | - Wen Lei
- Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA, 6009, Australia
| | - Lan Fu
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National University, Canberra, ACT, 2601, Australia
- Australian Research Council Centre of Excellence for Transformative Meta-Optical Systems, Canberra, ACT, 2601, Australia
| | - Lorenzo Faraone
- Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, 35 Stirling Highway, Perth, WA, 6009, Australia
- Australian Research Council Centre of Excellence for Transformative Meta-Optical Systems, Perth, WA, 6009, Australia
| | - Yun Liu
- Research School of Chemistry, The Australian National University, Canberra, ACT, 2601, Australia
| | - Robert G Elliman
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National University, Canberra, ACT, 2601, Australia
| |
Collapse
|
15
|
Li K, Yao J, Zhao P, Luo Y, Ge X, Yang R, Cheng X, Miao X. Ovonic threshold switching-based artificial afferent neurons for thermal in-sensor computing. MATERIALS HORIZONS 2024; 11:2106-2114. [PMID: 38545857 DOI: 10.1039/d4mh00053f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/08/2024]
Abstract
Artificial afferent neurons in the sensory nervous system inspired by biology have enormous potential for efficiently perceiving and processing environmental information. However, the previously reported artificial afferent neurons suffer from two prominent challenges: considerable power consumption and limited scalability efficiency. Herein, addressing these challenges, a bioinspired artificial thermal afferent neuron based on a N-doped SiTe ovonic threshold switching (OTS) device is presented for the first time. The engineered OTS device shows remarkable uniformity and robust endurance, ensuring the reliability and efficacy of the artificial afferent neurons. A substantially decreased leakage current of the SiTe OTS device by nitrogen doping results in ultra-low power consumption less than 0.3 nJ per spike for artificial afferent neurons. The inherent temperature response exhibited by N-doped SiTe OTS materials allows us to construct a highly compact artificial thermal afferent neuron over a wide temperature range. An edge detection task is performed to further verify its thermal perceptual computing function. Our work provides an insight into OTS-based artificial afferent neurons for electronic skin and sensory neurorobotics.
Collapse
Affiliation(s)
- Kai Li
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
| | - Jiaping Yao
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
| | - Peng Zhao
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
| | - Yunhao Luo
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
| | - Xiang Ge
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
| | - Rui Yang
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
- Hubei Yangtze Memory Laboratories, Wuhan 430205, China
| | - Xiaomin Cheng
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
- Hubei Yangtze Memory Laboratories, Wuhan 430205, China
| | - Xiangshui Miao
- School of Integrated Circuits, Hubei Key Laboratory for Advanced Memories, Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan 430074, China.
- Hubei Yangtze Memory Laboratories, Wuhan 430205, China
| |
Collapse
|
16
|
Li F, Li D, Wang C, Liu G, Wang R, Ren H, Tang Y, Wang Y, Chen Y, Liang K, Huang Q, Sawan M, Qiu M, Wang H, Zhu B. An artificial visual neuron with multiplexed rate and time-to-first-spike coding. Nat Commun 2024; 15:3689. [PMID: 38693165 PMCID: PMC11063071 DOI: 10.1038/s41467-024-48103-9] [Citation(s) in RCA: 12] [Impact Index Per Article: 12.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/02/2023] [Accepted: 04/17/2024] [Indexed: 05/03/2024] Open
Abstract
Human visual neurons rely on event-driven, energy-efficient spikes for communication, while silicon image sensors do not. The energy-budget mismatch between biological systems and machine vision technology has inspired the development of artificial visual neurons for use in spiking neural network (SNN). However, the lack of multiplexed data coding schemes reduces the ability of artificial visual neurons in SNN to emulate the visual perception ability of biological systems. Here, we present an artificial visual spiking neuron that enables rate and temporal fusion (RTF) coding of external visual information. The artificial neuron can code visual information at different spiking frequencies (rate coding) and enables precise and energy-efficient time-to-first-spike (TTFS) coding. This multiplexed sensory coding scheme could improve the computing capability and efficacy of artificial visual neurons. A hardware-based SNN with the RTF coding scheme exhibits good consistency with real-world ground truth data and achieves highly accurate steering and speed predictions for self-driving vehicles in complex conditions. The multiplexed RTF coding scheme demonstrates the feasibility of developing highly efficient spike-based neuromorphic hardware.
Collapse
Affiliation(s)
- Fanfan Li
- School of Materials Science and Engineering, Zhejiang University, Hangzhou, China
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Dingwei Li
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Chuanqing Wang
- CenBRAIN Neurotech, School of Engineering, Westlake University, Hangzhou, China
| | - Guolei Liu
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Rui Wang
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Microelectronics, Xidian University, Xi'an, China
| | - Huihui Ren
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Yingjie Tang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Yan Wang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Yitong Chen
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Kun Liang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
| | - Qi Huang
- Westlake Institute for Optoelectronics, Westlake University, Hangzhou, China
| | - Mohamad Sawan
- CenBRAIN Neurotech, School of Engineering, Westlake University, Hangzhou, China
- Westlake Institute for Optoelectronics, Westlake University, Hangzhou, China
- Institute of Advanced Technology, Westlake Institute for Advanced Study, Hangzhou, China
| | - Min Qiu
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China
- Westlake Institute for Optoelectronics, Westlake University, Hangzhou, China
- Institute of Advanced Technology, Westlake Institute for Advanced Study, Hangzhou, China
| | - Hong Wang
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Microelectronics, Xidian University, Xi'an, China.
| | - Bowen Zhu
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou, China.
- Westlake Institute for Optoelectronics, Westlake University, Hangzhou, China.
- Institute of Advanced Technology, Westlake Institute for Advanced Study, Hangzhou, China.
| |
Collapse
|
17
|
Yang K, Wang Y, Tiw PJ, Wang C, Zou X, Yuan R, Liu C, Li G, Ge C, Wu S, Zhang T, Huang R, Yang Y. High-order sensory processing nanocircuit based on coupled VO 2 oscillators. Nat Commun 2024; 15:1693. [PMID: 38402226 PMCID: PMC10894221 DOI: 10.1038/s41467-024-45992-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Accepted: 02/08/2024] [Indexed: 02/26/2024] Open
Abstract
Conventional circuit elements are constrained by limitations in area and power efficiency at processing physical signals. Recently, researchers have delved into high-order dynamics and coupled oscillation dynamics utilizing Mott devices, revealing potent nonlinear computing capabilities. However, the intricate yet manageable population dynamics of multiple artificial sensory neurons with spatiotemporal coupling remain unexplored. Here, we present an experimental hardware demonstration featuring a capacitance-coupled VO2 phase-change oscillatory network. This network serves as a continuous-time dynamic system for sensory pre-processing and encodes information in phase differences. Besides, a decision-making module for special post-processing through software simulation is designed to complete a bio-inspired dynamic sensory system. Our experiments provide compelling evidence that this transistor-free coupling network excels in sensory processing tasks such as touch recognition and gesture recognition, achieving significant advantages of fewer devices and lower energy-delay-product compared to conventional methods. This work paves the way towards an efficient and compact neuromorphic sensory system based on nano-scale nonlinear dynamics.
Collapse
Affiliation(s)
- Ke Yang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Yanghao Wang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Pek Jun Tiw
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Chaoming Wang
- School of Psychological and Cognitive Sciences, IDG/McGovern Institute for Brain Research, PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, 100871, China
| | - Xiaolong Zou
- School of Psychological and Cognitive Sciences, IDG/McGovern Institute for Brain Research, PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, 100871, China
| | - Rui Yuan
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Chang Liu
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Ge Li
- Beijing National Laboratory for Condensed Matter Physics, Institute of Physics, Chinese Academy of Sciences, Beijing, 100190, China
| | - Chen Ge
- Beijing National Laboratory for Condensed Matter Physics, Institute of Physics, Chinese Academy of Sciences, Beijing, 100190, China
| | - Si Wu
- School of Psychological and Cognitive Sciences, IDG/McGovern Institute for Brain Research, PKU-Tsinghua Center for Life Sciences, Peking University, Beijing, 100871, China
| | - Teng Zhang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China.
| | - Ru Huang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Yuchao Yang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China.
- Center for Brain Inspired Chips, Institute for Artificial Intelligence, Frontiers Science Center for Nano-optoelectronics, Peking University, Beijing, 100871, China.
- School of Electronic and Computer Engineering, Peking University, Shenzhen, 518055, China.
- Center for Brain Inspired Intelligence, Chinese Institute for Brain Research (CIBR), Beijing, 102206, China.
| |
Collapse
|
18
|
Zhu S, Xie T, Lv Z, Leng YB, Zhang YQ, Xu R, Qin J, Zhou Y, Roy VAL, Han ST. Hierarchies in Visual Pathway: Functions and Inspired Artificial Vision. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2024; 36:e2301986. [PMID: 37435995 DOI: 10.1002/adma.202301986] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/02/2023] [Revised: 06/28/2023] [Accepted: 07/10/2023] [Indexed: 07/13/2023]
Abstract
The development of artificial intelligence has posed a challenge to machine vision based on conventional complementary metal-oxide semiconductor (CMOS) circuits owing to its high latency and inefficient power consumption originating from the data shuffling between memory and computation units. Gaining more insights into the function of every part of the visual pathway for visual perception can bring the capabilities of machine vision in terms of robustness and generality. Hardware acceleration of more energy-efficient and biorealistic artificial vision highly necessitates neuromorphic devices and circuits that are able to mimic the function of each part of the visual pathway. In this paper, we review the structure and function of the entire class of visual neurons from the retina to the primate visual cortex within reach (Chapter 2) are reviewed. Based on the extraction of biological principles, the recent hardware-implemented visual neurons located in different parts of the visual pathway are discussed in detail in Chapters 3 and 4. Furthermore, valuable applications of inspired artificial vision in different scenarios (Chapter 5) are provided. The functional description of the visual pathway and its inspired neuromorphic devices/circuits are expected to provide valuable insights for the design of next-generation artificial visual perception systems.
Collapse
Affiliation(s)
- Shirui Zhu
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Tao Xie
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Ziyu Lv
- College of Electronics and Information Engineering, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Yan-Bing Leng
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Yu-Qi Zhang
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Runze Xu
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Jingrun Qin
- Institute of Microscale Optoelectronics, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Ye Zhou
- Institute for Advanced Study, Shenzhen University, Shenzhen, 518060, P. R. China
| | - Vellaisamy A L Roy
- School of Science and Technology, Hong Kong Metropolitan University, Hong Kong, 999077, P. R. China
| | - Su-Ting Han
- College of Electronics and Information Engineering, Shenzhen University, Shenzhen, 518060, P. R. China
| |
Collapse
|
19
|
Kwon JY, Kim JE, Kim JS, Chun SY, Soh K, Yoon JH. Artificial sensory system based on memristive devices. EXPLORATION (BEIJING, CHINA) 2024; 4:20220162. [PMID: 38854486 PMCID: PMC10867403 DOI: 10.1002/exp.20220162] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/20/2023] [Accepted: 10/16/2023] [Indexed: 06/11/2024]
Abstract
In the biological nervous system, the integration and cooperation of parallel system of receptors, neurons, and synapses allow efficient detection and processing of intricate and disordered external information. Such systems acquire and process environmental data in real-time, efficiently handling complex tasks with minimal energy consumption. Memristors can mimic typical biological receptors, neurons, and synapses by implementing key features of neuronal signal-processing functions such as selective adaption in receptors, leaky integrate-and-fire in neurons, and synaptic plasticity in synapses. External stimuli are sensitively detected and filtered by "artificial receptors," encoded into spike signals via "artificial neurons," and integrated and stored through "artificial synapses." The high operational speed, low power consumption, and superior scalability of memristive devices make their integration with high-performance sensors a promising approach for creating integrated artificial sensory systems. These integrated systems can extract useful data from a large volume of raw data, facilitating real-time detection and processing of environmental information. This review explores the recent advances in memristor-based artificial sensory systems. The authors begin with the requirements of artificial sensory elements and then present an in-depth review of such elements demonstrated by memristive devices. Finally, the major challenges and opportunities in the development of memristor-based artificial sensory systems are discussed.
Collapse
Affiliation(s)
- Ju Young Kwon
- Electronic Materials Research CenterKorea Institute of Science and Technology (KIST)SeoulRepublic of Korea
| | - Ji Eun Kim
- Electronic Materials Research CenterKorea Institute of Science and Technology (KIST)SeoulRepublic of Korea
- Department of Materials Science and EngineeringKorea UniversitySeoulRepublic of Korea
| | - Jong Sung Kim
- Electronic Materials Research CenterKorea Institute of Science and Technology (KIST)SeoulRepublic of Korea
- Department of Materials Science and EngineeringKorea UniversitySeoulRepublic of Korea
| | - Suk Yeop Chun
- Electronic Materials Research CenterKorea Institute of Science and Technology (KIST)SeoulRepublic of Korea
- KU‐KIST Graduate School of Converging Science and TechnologyKorea UniversitySeoulRepublic of Korea
| | - Keunho Soh
- Electronic Materials Research CenterKorea Institute of Science and Technology (KIST)SeoulRepublic of Korea
- Department of Materials Science and EngineeringKorea UniversitySeoulRepublic of Korea
| | - Jung Ho Yoon
- Electronic Materials Research CenterKorea Institute of Science and Technology (KIST)SeoulRepublic of Korea
| |
Collapse
|
20
|
Han Y, Xiang S, Song Z, Gao S, Zhang Y, Guo X, Hao Y. Noisy image segmentation based on synchronous dynamics of coupled photonic spiking neurons. OPTICS EXPRESS 2023; 31:35484-35492. [PMID: 38017717 DOI: 10.1364/oe.498191] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/20/2023] [Accepted: 09/29/2023] [Indexed: 11/30/2023]
Abstract
The collective dynamics in neural networks is essential for information processing and has attracted much interest on the application in artificial intelligence. Synchronization is one of the most dominant phenomenon in the collective dynamics of neural network. Here, we propose to use the spiking dynamics and collective synchronization of coupled photonic spiking neurons for noisy image segmentation. Based on the synchronization mechanism and synchronization control, the noised pattern segmentation is demonstrated numerically. This work provides insight into the possible application based on the collective dynamics of large-scale photonic networks and opens a way for ultra-high speed image processing.
Collapse
|
21
|
Zhao J, Ran Y, Pei Y, Wei Y, Sun J, Zhang Z, Wang J, Zhou Z, Wang Z, Sun Y, Yan X. Memristors based on NdNiO 3 nanocrystals film as sensory neurons for neuromorphic computing. MATERIALS HORIZONS 2023; 10:4521-4531. [PMID: 37555245 DOI: 10.1039/d3mh00835e] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/10/2023]
Abstract
By mimicking the behavior of the human brain, artificial neural systems offer the possibility to further improve computing efficiency and solve the von Neumann bottleneck. In particular, neural systems with perceptual capability expand the application field and lay a good foundation for the construction of perceptual storage and computational systems. However, research on neurons with perceptual functions is still relatively scarce, with most works focusing on optoelectronic synapses. The neuron is important for neuromorphic computing systems because neurons output excitatory or inhibitory stimuli to regulate the weight of synapses. Therefore, the construction of sensory neurons is crucial to expand the application range of brain-like neural computing. Here, an artificial sensory neuron is proposed, which is constructed using a photosensitive bipolar threshold switching memristor based on NdNiO3 (NNO) nanocrystals. These metallic phase nanocrystals can not only enhance the local electric field, but also act as a reservoir for defects (VoS) to guide the growth of conductive filaments and stabilize the performance of the device. They present stable bipolar threshold switching behavior with a low 120 nW set power, and the operating voltages decreased in light due to photocarrier action. A leaky integrate firing (LIF) neuron has been realized, which achieved key biological neuron functions, such as all-or-nothing spiking, threshold-driven firing, refractory period, and spiking frequency modulation. The LIF neurons receiving optical inputs have the properties of an artificial sensory neuron. It could regulate the spiking output frequency at different light densities, which could be used for a ship approaching a port. This work provides a promising hardware implementation towards constructing high-performance artificial intelligence to assist ships at night in a sensory system.
Collapse
Affiliation(s)
- Jianhui Zhao
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Yunfeng Ran
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Yifei Pei
- Hebei Key Laboratory of Optic-Electronic Information Materials, College of Physics Science and Technology, Hebei University, Baoding 071002, People's Republic of China
| | - Yiheng Wei
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Jiameng Sun
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Zixuan Zhang
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Jiacheng Wang
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Zhenyu Zhou
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Zhongrong Wang
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Yong Sun
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| | - Xiaobing Yan
- Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, College of Electronic and Information Engineering, Hebei University, Baoding 071002, People's Republic of China.
| |
Collapse
|
22
|
Dang B, Liu K, Wu X, Yang Z, Xu L, Yang Y, Huang R. One-Phototransistor-One-Memristor Array with High-Linearity Light-Tunable Weight for Optic Neuromorphic Computing. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2023; 35:e2204844. [PMID: 35917248 DOI: 10.1002/adma.202204844] [Citation(s) in RCA: 21] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/29/2022] [Revised: 07/21/2022] [Indexed: 06/15/2023]
Abstract
The recent advances in optic neuromorphic devices have led to a subsequent rise in use for construction of energy-efficient artificial-vision systems. The widespread use can be attributed to their ability to capture, store, and process visual information from the environment. The primary limitations of existing optic neuromorphic devices include nonlinear weight updates, cross-talk issues, and silicon process incompatibility. In this study, a highly linear, light-tunable, cross-talk-free, and silicon-compatible one-phototransistor-one-memristor (1PT1R) optic memristor is experimentally demonstrated for the implementation of an optic artificial neural network (OANN). For optic image recognition in the experiment, an OANN is constructed using a 16 × 3 1PT1R memristor array, and it is trained on an online platform. The model yields an accuracy of 99.3% after only ten training epochs. The 1PT1R memristor, which shows good performance, demonstrates its ability as an excellent hardware solution for highly efficient optic neuromorphic and edge computing.
Collapse
Affiliation(s)
- Bingjie Dang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Keqin Liu
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Xulei Wu
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Zhen Yang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Liying Xu
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
| | - Yuchao Yang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
- Center for Brain Inspired Chips, Institute for Artificial Intelligence, Peking University, Beijing, 100871, China
- Center for Brain Inspired Intelligence, Chinese Institute for Brain Research (CIBR), Beijing, Beijing, 102206, China
- Beijing Academy of Artificial Intelligence, Beijing, 100084, China
| | - Ru Huang
- Beijing Advanced Innovation Center for Integrated Circuits, School of Integrated Circuits, Peking University, Beijing, 100871, China
- Center for Brain Inspired Chips, Institute for Artificial Intelligence, Peking University, Beijing, 100871, China
| |
Collapse
|
23
|
Liu H, Qin Y, Chen HY, Wu J, Ma J, Du Z, Wang N, Zou J, Lin S, Zhang X, Zhang Y, Wang H. Artificial Neuronal Devices Based on Emerging Materials: Neuronal Dynamics and Applications. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2023; 35:e2205047. [PMID: 36609920 DOI: 10.1002/adma.202205047] [Citation(s) in RCA: 23] [Impact Index Per Article: 11.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2022] [Revised: 12/02/2022] [Indexed: 06/17/2023]
Abstract
Artificial neuronal devices are critical building blocks of neuromorphic computing systems and currently the subject of intense research motivated by application needs from new computing technology and more realistic brain emulation. Researchers have proposed a range of device concepts that can mimic neuronal dynamics and functions. Although the switching physics and device structures of these artificial neurons are largely different, their behaviors can be described by several neuron models in a more unified manner. In this paper, the reports of artificial neuronal devices based on emerging volatile switching materials are reviewed from the perspective of the demonstrated neuron models, with a focus on the neuronal functions implemented in these devices and the exploitation of these functions for computational and sensing applications. Furthermore, the neuroscience inspirations and engineering methods to enrich the neuronal dynamics that remain to be implemented in artificial neuronal devices and networks toward realizing the full functionalities of biological neurons are discussed.
Collapse
Affiliation(s)
- Hefei Liu
- Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA, 90089, USA
| | - Yuan Qin
- Center for Power Electronics Systems, Bradley Department of Electrical and Computer Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA, 24060, USA
| | - Hung-Yu Chen
- Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA, 90089, USA
| | - Jiangbin Wu
- Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA, 90089, USA
| | - Jiahui Ma
- Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA, 90089, USA
| | - Zhonghao Du
- Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA, 90089, USA
| | - Nan Wang
- Mork Family Department of Chemical Engineering and Materials Science, University of Southern California, Los Angeles, CA, 90089, USA
| | - Jingyi Zou
- Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - Sen Lin
- Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - Xu Zhang
- Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, PA, 15213, USA
| | - Yuhao Zhang
- Center for Power Electronics Systems, Bradley Department of Electrical and Computer Engineering, Virginia Polytechnic Institute and State University, Blacksburg, VA, 24060, USA
| | - Han Wang
- Ming Hsieh Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, CA, 90089, USA
- Mork Family Department of Chemical Engineering and Materials Science, University of Southern California, Los Angeles, CA, 90089, USA
| |
Collapse
|
24
|
Chun SY, Song YG, Kim JE, Kwon JU, Soh K, Kwon JY, Kang CY, Yoon JH. An Artificial Olfactory System Based on a Chemi-Memristive Device. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2023; 35:e2302219. [PMID: 37116944 DOI: 10.1002/adma.202302219] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Revised: 04/18/2023] [Indexed: 06/19/2023]
Abstract
Technologies based on the fusion of gas sensors and neuromorphic computing to mimic the olfactory system have immense potential. However, the implementation of neuromorphic olfactory systems remains in a state of infancy because conventional gas sensors lack the necessary functions. Therefore, this study proposes a hysteretic "chemi-memristive gas sensor" based on oxygen vacancy chemi-memristive dynamics that differ from that of conventional gas sensors. After the memristive switching operation, the redox reaction with the external gas molecules is enhanced, resulting in the generation and elimination of oxygen vacancies that induce rapid current changes. In addition, the pre-generated oxygen vacancies enhance the post-sensing properties. Therefore, fast responses, short recovery times, and hysteretic gas response are achieved by the proposed sensor at room temperature. Based on the advantageous functionality of the sensor, device-level olfactory systems that can monitor the history of input gas stimuli are experimentally demonstrated as a potential application. Moreover, analog conductance modulation induced by oxidizing and reducing gases enables the conversion of external gas stimuli into synaptic weights and hence the realization of typical synaptic functionalities without an additional device or circuit. The proposed chemi-memristive device represents an advance in the bioinspired technology adopted in creating artificial intelligence systems.
Collapse
Affiliation(s)
- Suk Yeop Chun
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
- KU-KIST Graduate School of Converging Science and Technology, Korea University, Seoul, 02841, Republic of Korea
| | - Young Geun Song
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
| | - Ji Eun Kim
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
- Department of Materials Science and Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Jae Uk Kwon
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
- Department of Materials Science and Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Keunho Soh
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
- Department of Materials Science and Engineering, Korea University, Seoul, 02841, Republic of Korea
| | - Ju Young Kwon
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
| | - Chong-Yun Kang
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
- KU-KIST Graduate School of Converging Science and Technology, Korea University, Seoul, 02841, Republic of Korea
| | - Jung Ho Yoon
- Electronic Materials Research Center, Korea Institute of Science and Technology (KIST), Seoul, 02791, Republic of Korea
| |
Collapse
|
25
|
Li M. Simulation analysis of visual perception model based on pulse coupled neural network. Sci Rep 2023; 13:12281. [PMID: 37507535 PMCID: PMC10382568 DOI: 10.1038/s41598-023-39376-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Accepted: 07/25/2023] [Indexed: 07/30/2023] Open
Abstract
Pulse-coupled neural networks perform well in many fields such as information retrieval, depth estimation and object detection. Based on pulse coupled neural network (PCNN) theory, this paper constructs a visual perception model framework and builds a real image reproduction platform. The model firstly analyzes the structure and generalization ability of neural network multi-class classifier, uses the minimax criterion of feature space as the splitting criterion of visual perception decision node, which solves the generalization problem of neural network learning algorithm. In the simulation process, the initial threshold is optimized by the two-dimensional maximum inter-class variance method, and in order to improve the real-time performance of the algorithm, the fast recurrence formula of neural network is derived and given. The PCNN image segmentation method based on genetic algorithm is analyzed. The genetic algorithm improves the loop termination condition and the adaptive setting of model parameters of PCNN image segmentation algorithm, but the PCNN image segmentation algorithm still has the problem of complexity. In order to solve this problem, this paper proposed an IGA-PCNN image segmentation method combining the improved algorithm and PCNN model. Firstly, it used the improved immune genetic algorithm to adaptively obtain the optimal threshold, and then replaced the dynamic threshold in PCNN model with the optimal threshold, and finally used the pulse coupling characteristics of PCNN model to complete the image segmentation. From the coupling characteristics of PCNN, junction close space of image and gray level characteristics, it determined the local gray mean square error of image connection strength coefficient. The feature extraction and object segmentation properties of PCNN come from the spike frequency of neurons, and the number of neurons in PCNN is equal to the number of pixels in the input image. In addition, the spatial and gray value differences of pixels should be considered comprehensively to determine their connection matrix. Digital experiments show that the multi-scale multi-task pulse coupled neural network model can shorten the total training time by 17 h, improve the comprehensive accuracy of the task test data set by 1.04%, and shorten the detection time of each image by 4.8 s compared with the series network model of multiple single tasks. Compared with the traditional PCNN algorithm, it has the advantages of fast visual perception and clear target contour segmentation, and effectively improves the anti-interference performance of the model.
Collapse
Affiliation(s)
- Mingdong Li
- School of Information Engineering, Suzhou University, Suzhou, 234000, China.
| |
Collapse
|
26
|
Wang X, Chen C, Zhu L, Shi K, Peng B, Zhu Y, Mao H, Long H, Ke S, Fu C, Zhu Y, Wan C, Wan Q. Vertically integrated spiking cone photoreceptor arrays for color perception. Nat Commun 2023; 14:3444. [PMID: 37301894 PMCID: PMC10257685 DOI: 10.1038/s41467-023-39143-8] [Citation(s) in RCA: 37] [Impact Index Per Article: 18.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Accepted: 05/31/2023] [Indexed: 06/12/2023] Open
Abstract
The cone photoreceptors in our eyes selectively transduce the natural light into spiking representations, which endows the brain with high energy-efficiency color vision. However, the cone-like device with color-selectivity and spike-encoding capability remains challenging. Here, we propose a metal oxide-based vertically integrated spiking cone photoreceptor array, which can directly transduce persistent lights into spike trains at a certain rate according to the input wavelengths. Such spiking cone photoreceptors have an ultralow power consumption of less than 400 picowatts per spike in visible light, which is very close to biological cones. In this work, lights with three wavelengths were exploited as pseudo-three-primary colors to form 'colorful' images for recognition tasks, and the device with the ability to discriminate mixed colors shows better accuracy. Our results would enable hardware spiking neural networks with biologically plausible visual perception and provide great potential for the development of dynamic vision sensors.
Collapse
Affiliation(s)
- Xiangjing Wang
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Chunsheng Chen
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Li Zhu
- College of Integrated Circuit Science and Engineering, Nanjing University of Posts and Telecommunications, Nanjing, 210003, China
| | - Kailu Shi
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Baocheng Peng
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Yixin Zhu
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Huiwu Mao
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Haotian Long
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Shuo Ke
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Chuanyu Fu
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Ying Zhu
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Changjin Wan
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China.
| | - Qing Wan
- School of Electronic Science and Engineering, Collaborative Innovation Center of Advanced Microstructures, Nanjing University, Nanjing, 210093, China.
- School of Micro Nanoelectronics, Zhejiang University, ZJU-Hangzhou Global Scientific and Technological Innovation Centre, 310027, Hangzhou, PR China.
| |
Collapse
|
27
|
Nath SK, Nandi SK, Das SK, Liang Y, Elliman RG. Thermal transport in metal-NbO x-metal cross-point devices and its effect on threshold switching characteristics. NANOSCALE 2023; 15:7559-7565. [PMID: 37038892 DOI: 10.1039/d3nr00173c] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Volatile threshold switching and current-controlled negative differential resistance (NDR) in metal-oxide-metal (MOM) devices result from thermally driven conductivity changes induced by local Joule heating and are therefore influenced by the thermal properties of the device-structure. In this study, we investigate the effect of the metal electrodes on the threshold switching response of NbOx-based cross-point devices. The electroforming and switching characteristics are shown to be strongly influenced by the thickness and thermal conductivity of the top-electrode due to its effect on heat loss from the NbOx film. Specifically, we demonstrate a 40% reduction in threshold voltage and a 75% reduction in threshold power as the thickness of the top Au electrode is reduced from 125 nm to 25 nm, and a 24% reduction in threshold voltage and 64% reduction in threshold power when the Au electrode is replaced by a Pt electrode of the same thickness of NbOx film, due to its lower thermal conductivity. Lumped element and finite element modelling of the devices show that these improvements are due to a reduction in heat loss to the electrodes, which is dominated by lateral heat flow within the electrode. These results clearly demonstrate the importance of the electrodes in determining the electroforming and threshold switching characteristics of MOM cross point devices.
Collapse
Affiliation(s)
- Shimul Kanti Nath
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National, University, Canberra, ACT 2601, Australia.
- Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, 35 Stirling Highway, Perth 6009, Australia
| | - Sanjoy Kumar Nandi
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National, University, Canberra, ACT 2601, Australia.
| | - Sujan Kumar Das
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National, University, Canberra, ACT 2601, Australia.
| | - Yan Liang
- School of Electronic and Information, Hangzhou Dianzi University, Hangzhou, 310018, China
| | - Robert G Elliman
- Department of Electronic Materials Engineering, Research School of Physics, The Australian National, University, Canberra, ACT 2601, Australia.
| |
Collapse
|
28
|
Wang F, Hu F, Dai M, Zhu S, Sun F, Duan R, Wang C, Han J, Deng W, Chen W, Ye M, Han S, Qiang B, Jin Y, Chua Y, Chi N, Yu S, Nam D, Chae SH, Liu Z, Wang QJ. A two-dimensional mid-infrared optoelectronic retina enabling simultaneous perception and encoding. Nat Commun 2023; 14:1938. [PMID: 37024508 PMCID: PMC10079931 DOI: 10.1038/s41467-023-37623-5] [Citation(s) in RCA: 31] [Impact Index Per Article: 15.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/07/2023] [Accepted: 03/22/2023] [Indexed: 04/08/2023] Open
Abstract
Infrared machine vision system for object perception and recognition is becoming increasingly important in the Internet of Things era. However, the current system suffers from bulkiness and inefficiency as compared to the human retina with the intelligent and compact neural architecture. Here, we present a retina-inspired mid-infrared (MIR) optoelectronic device based on a two-dimensional (2D) heterostructure for simultaneous data perception and encoding. A single device can perceive the illumination intensity of a MIR stimulus signal, while encoding the intensity into a spike train based on a rate encoding algorithm for subsequent neuromorphic computing with the assistance of an all-optical excitation mechanism, a stochastic near-infrared (NIR) sampling terminal. The device features wide dynamic working range, high encoding precision, and flexible adaption ability to the MIR intensity. Moreover, an inference accuracy more than 96% to MIR MNIST data set encoded by the device is achieved using a trained spiking neural network (SNN).
Collapse
Affiliation(s)
- Fakun Wang
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Fangchen Hu
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
- Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University, Shanghai, 200433, China
| | - Mingjin Dai
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Song Zhu
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Fangyuan Sun
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Ruihuan Duan
- School of Materials Science and Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Chongwu Wang
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Jiayue Han
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Wenjie Deng
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Wenduo Chen
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Ming Ye
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Song Han
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Bo Qiang
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Yuhao Jin
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Yunda Chua
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Nan Chi
- Key Laboratory for Information Science of Electromagnetic Waves (MoE), Fudan University, Shanghai, 200433, China
| | - Shaohua Yu
- Peng Cheng Laboratory, Shenzhen, 518055, China
| | - Donguk Nam
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Sang Hoon Chae
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Zheng Liu
- School of Materials Science and Engineering, Nanyang Technological University, Singapore, 639798, Singapore
| | - Qi Jie Wang
- School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore, 639798, Singapore.
- Centre for Disruptive Photonic Technologies, School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore, 637371, Singapore.
| |
Collapse
|
29
|
Han JK, Yun SY, Yu JM, Jeon SB, Choi YK. Artificial Multisensory Neuron with a Single Transistor for Multimodal Perception through Hybrid Visual and Thermal Sensing. ACS APPLIED MATERIALS & INTERFACES 2023; 15:5449-5455. [PMID: 36669163 DOI: 10.1021/acsami.2c19208] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/17/2023]
Abstract
An artificial multisensory device applicable to in-sensor computing is demonstrated with a single-transistor neuron (1T-neuron) for multimodal perception. It simultaneously receives two sensing signals from visual and thermal stimuli. The 1T-neuron transforms these signals into electrical signals in the form of spiking and then fires them for a spiking neural network at the same time. This feature makes it feasible to realize input neurons for multimodal sensing. Visual and thermal sensing is achieved due to the inherent optical and thermal behaviors of the 1T-neuron. To demonstrate a neuromorphic multimodal sensing system with the artificial multisensory 1T-neuron, fingerprint recognition, widely used for biometric security, is implemented. Owing to the simultaneous sensing of heat as well as light, the proposed fingerprint recognition system composed of multisensory 1T-neurons not only identifies a genuine pattern but also judges whether or not it is forged.
Collapse
Affiliation(s)
- Joon-Kyu Han
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon34141, Republic of Korea
| | - Seong-Yun Yun
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon34141, Republic of Korea
| | - Ji-Man Yu
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon34141, Republic of Korea
| | - Seung-Bae Jeon
- Department of Electronic Engineering, Hanbat National University, 125 Dongseo-daero, Yuseong-gu, Daejeon34158, Republic of Korea
| | - Yang-Kyu Choi
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon34141, Republic of Korea
| |
Collapse
|
30
|
Wang W, Gao S, Wang Y, Li Y, Yue W, Niu H, Yin F, Guo Y, Shen G. Advances in Emerging Photonic Memristive and Memristive-Like Devices. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2105577. [PMID: 35945187 PMCID: PMC9534950 DOI: 10.1002/advs.202105577] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/02/2021] [Revised: 06/06/2022] [Indexed: 05/19/2023]
Abstract
Possessing the merits of high efficiency, low consumption, and versatility, emerging photonic memristive and memristive-like devices exhibit an attractive future in constructing novel neuromorphic computing and miniaturized bionic electronic system. Recently, the potential of various emerging materials and structures for photonic memristive and memristive-like devices has attracted tremendous research efforts, generating various novel theories, mechanisms, and applications. Limited by the ambiguity of the mechanism and the reliability of the material, the development and commercialization of such devices are still rare and in their infancy. Therefore, a detailed and systematic review of photonic memristive and memristive-like devices is needed to further promote its development. In this review, the resistive switching mechanisms of photonic memristive and memristive-like devices are first elaborated. Then, a systematic investigation of the active materials, which induce a pivotal influence in the overall performance of photonic memristive and memristive-like devices, is highlighted and evaluated in various indicators. Finally, the recent advanced applications are summarized and discussed. In a word, it is believed that this review provides an extensive impact on many fields of photonic memristive and memristive-like devices, and lay a foundation for academic research and commercial applications.
Collapse
Affiliation(s)
- Wenxiao Wang
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Song Gao
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Yaqi Wang
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Yang Li
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Wenjing Yue
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Hongsen Niu
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Feifei Yin
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Yunjian Guo
- School of Information Science and EngineeringShandong Provincial Key Laboratory of Network Based Intelligent ComputingUniversity of JinanJinan250022China
| | - Guozhen Shen
- School of Integrated Circuits and ElectronicsBeijing Institute of TechnologyBeijing100081China
| |
Collapse
|
31
|
Han JK, Park SC, Yu JM, Ahn JH, Choi YK. A Bioinspired Artificial Gustatory Neuron for a Neuromorphic Based Electronic Tongue. NANO LETTERS 2022; 22:5244-5251. [PMID: 35737524 DOI: 10.1021/acs.nanolett.2c01107] [Citation(s) in RCA: 10] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
A novel biomimicked neuromorphic sensor for an energy efficient and highly scalable electronic tongue (E-tongue) is demonstrated with a metal-oxide-semiconductor field-effect transistor (MOSFET). By mimicking a biological gustatory neuron, the proposed E-tongue can simultaneously detect ion concentrations of chemicals on an extended gate and encode spike signals on the MOSFET, which acts as an input neuron in a spiking neural network (SNN). Such in-sensor neuromorphic functioning can reduce the energy and area consumption of the conventional E-tongue hardware. pH-sensitive and sodium-sensitive artificial gustatory neurons are implemented by using two different sensing materials: Al2O3 for pH sensing and sodium ionophore X for sodium ion sensing. In addition, a sensitivity control function inspired by the biological sensory neuron is demonstrated. After the unit device characterization of the artificial gustatory neuron, a fully hardware-based E-tongue that can classify two distinct liquids is demonstrated to show a practical application of the artificial gustatory neurons.
Collapse
Affiliation(s)
- Joon-Kyu Han
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Sang-Chan Park
- Department of Electronics Engineering, Chungnam National University, 99 Daehak-ro, Yuseong-gu, Daejeon 34134, Republic of Korea
| | - Ji-Man Yu
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| | - Jae-Hyuk Ahn
- Department of Electronics Engineering, Chungnam National University, 99 Daehak-ro, Yuseong-gu, Daejeon 34134, Republic of Korea
| | - Yang-Kyu Choi
- School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Republic of Korea
| |
Collapse
|
32
|
A calibratable sensory neuron based on epitaxial VO 2 for spike-based neuromorphic multisensory system. Nat Commun 2022; 13:3973. [PMID: 35803938 PMCID: PMC9270461 DOI: 10.1038/s41467-022-31747-w] [Citation(s) in RCA: 67] [Impact Index Per Article: 22.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2022] [Accepted: 06/29/2022] [Indexed: 11/08/2022] Open
Abstract
Neuromorphic perception systems inspired by biology have tremendous potential in efficiently processing multi-sensory signals from the physical world, but a highly efficient hardware element capable of sensing and encoding multiple physical signals is still lacking. Here, we report a spike-based neuromorphic perception system consisting of calibratable artificial sensory neurons based on epitaxial VO2, where the high crystalline quality of VO2 leads to significantly improved cycle-to-cycle uniformity. A calibration resistor is introduced to optimize device-to-device consistency, and to adapt the VO2 neuron to different sensors with varied resistance level, a scaling resistor is further incorporated, demonstrating cross-sensory neuromorphic perception component that can encode illuminance, temperature, pressure and curvature signals into spikes. These components are utilized to monitor the curvatures of fingers, thereby achieving hand gesture classification. This study addresses the fundamental cycle-to-cycle and device-to-device variation issues of sensory neurons, therefore promoting the construction of neuromorphic perception systems for e-skin and neurorobotics.
Collapse
|
33
|
Wang J, Zhu Y, Zhu L, Chen C, Wan Q. Emerging Memristive Devices for Brain-Inspired Computing and Artificial Perception. FRONTIERS IN NANOTECHNOLOGY 2022. [DOI: 10.3389/fnano.2022.940825] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Brain-inspired computing is an emerging field that aims at building a compact and massively parallel architecture, to reduce power consumption in conventional Von Neumann Architecture. Recently, memristive devices have gained great attention due to their immense potential in implementing brain-inspired computing and perception. The conductance of a memristor can be modulated by a voltage pulse, enabling emulations of both essential synaptic and neuronal functions, which are considered as the important building blocks for artificial neural networks. As a result, it is critical to review recent developments of memristive devices in terms of neuromorphic computing and perception applications, waiting for new thoughts and breakthroughs. The device structures, operation mechanisms, and materials are introduced sequentially in this review; additionally, late advances in emergent neuromorphic computing and perception based on memristive devices are summed up. Finally, the challenges that memristive devices toward high-performance brain-inspired computing and perception are also briefly discussed. We believe that the advances and challenges will lead to significant advancements in artificial neural networks and intelligent humanoid robots.
Collapse
|
34
|
Han J, Kang M, Jeong J, Cho I, Yu J, Yoon K, Park I, Choi Y. Artificial Olfactory Neuron for an In-Sensor Neuromorphic Nose. ADVANCED SCIENCE (WEINHEIM, BADEN-WURTTEMBERG, GERMANY) 2022; 9:e2106017. [PMID: 35426489 PMCID: PMC9218653 DOI: 10.1002/advs.202106017] [Citation(s) in RCA: 42] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/27/2021] [Revised: 03/10/2022] [Indexed: 06/02/2023]
Abstract
A neuromorphic module of an electronic nose (E-nose) is demonstrated by hybridizing a chemoresistive gas sensor made of a semiconductor metal oxide (SMO) and a single transistor neuron (1T-neuron) made of a metal-oxide-semiconductor field-effect transistor (MOSFET). By mimicking a biological olfactory neuron, it simultaneously detects a gas and encoded spike signals for in-sensor neuromorphic functioning. It identifies an odor source by analyzing the complicated mixed signals using a spiking neural network (SNN). The proposed E-nose does not require conversion circuits, which are essential for processing the sensory signals between the sensor array and processors in the conventional bulky E-nose. In addition, they do not have to include a central processing unit (CPU) and memory, which are required for von Neumann computing. The spike transmission of the biological olfactory system, which is known to be the main factor for reducing power consumption, is realized with the SNN for power savings compared to the conventional E-nose with a deep neural network (DNN). Therefore, the proposed neuromorphic E-nose is promising for application to Internet of Things (IoT), which demands a highly scalable and energy-efficient system. As a practical example, it is employed as an electronic sommelier by classifying different types of wines.
Collapse
Affiliation(s)
- Joon‐Kyu Han
- School of Electrical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| | - Mingu Kang
- Department of Mechanical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| | - Jaeseok Jeong
- Department of Mechanical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| | - Incheol Cho
- Department of Mechanical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| | - Ji‐Man Yu
- School of Electrical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| | - Kuk‐Jin Yoon
- Department of Mechanical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| | - Inkyu Park
- Department of Mechanical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| | - Yang‐Kyu Choi
- School of Electrical EngineeringKorea Advanced Institute of Science and Technology (KAIST)291 Daehak‐ro, Yuseong‐guDaejeon34141Republic of Korea
| |
Collapse
|
35
|
Chen C, He Y, Mao H, Zhu L, Wang X, Zhu Y, Zhu Y, Shi Y, Wan C, Wan Q. A Photoelectric Spiking Neuron for Visual Depth Perception. ADVANCED MATERIALS (DEERFIELD BEACH, FLA.) 2022; 34:e2201895. [PMID: 35305270 DOI: 10.1002/adma.202201895] [Citation(s) in RCA: 43] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/28/2022] [Revised: 03/15/2022] [Indexed: 06/14/2023]
Abstract
The biological visual system encodes optical information into spikes and processes them by the neural network, which enables the perception with high throughput of visual processing with ultralow energy budget. This has inspired a wide spectrum of devices to imitate such neural process, while precise mimicking such procedure is still highly required. Here, a highly bio-realistic photoelectric spiking neuron for visual depth perception is presented. The firing spikes generated by the TaOX memristive spiking encoders have a biologically similar frequency range of 1-200 Hz and sub-micro watts power. Such spiking encoder is integrated with a photodetector and a network of neuromorphic transistors, for information collection and recognition tasks, respectively. The distance-dependent response and eye fatigue of biological visual systems have been mimicked based on such photoelectric spiking neuron. The simulated depth perception shows a recognition improvement by adapting to sights at different distances. The results can advance the technologies in bioinspired or robotic systems that may be endowed with depth perception and power efficiency at the same time.
Collapse
Affiliation(s)
- Chunsheng Chen
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Yongli He
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Huiwu Mao
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Li Zhu
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Xiangjing Wang
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Ying Zhu
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Yixin Zhu
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Yi Shi
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Changjin Wan
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| | - Qing Wan
- School of Electronic Science & Engineering, and Collaborative Innovation Centre of Advanced Microstructures, Nanjing University, Nanjing, 210093, China
| |
Collapse
|
36
|
Li F, Wang R, Song C, Zhao M, Ren H, Wang S, Liang K, Li D, Ma X, Zhu B, Wang H, Hao Y. A Skin-Inspired Artificial Mechanoreceptor for Tactile Enhancement and Integration. ACS NANO 2021; 15:16422-16431. [PMID: 34597014 DOI: 10.1021/acsnano.1c05836] [Citation(s) in RCA: 43] [Impact Index Per Article: 10.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/27/2023]
Abstract
Mechanoreceptors endow humans with the sense of touch by translating the external stimuli into coded spikes, inspiring the rise of artificial mechanoreceptor systems. However, to incorporate slow adaptive receptors-like pressure sensors with artificial neurons remains a challenge. Here we demonstrate an artificial mechanoreceptor by rationally integrating a polypyrrole-based resistive pressure sensor with a volatile NbOx memristor, to mimic the tactile sensation and perception in natural skin, respectively. The artificial mechanoreceptor enables the tactile sensory coding by converting the external mechanical stimuli into strength-modulated electrical spikes. Also, tactile sensation enhancement is achieved by processing the spike frequency characteristics with the pulse coupled neural network. Furthermore, the artificial mechanoreceptor can integrate signals from parallel sensor channels and encode them into unified electrical spikes, resembling the coding of intensity in tactile neural processing. These results provide simple and efficient strategies for constructing future bio-inspired electronic systems.
Collapse
Affiliation(s)
- Fanfan Li
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Advanced Materials and Nanotechnology, Xidian University, Xi'an 710071, China
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou 310024, China
| | - Rui Wang
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Advanced Materials and Nanotechnology, Xidian University, Xi'an 710071, China
| | - Chunyan Song
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou 310024, China
| | - Momo Zhao
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Advanced Materials and Nanotechnology, Xidian University, Xi'an 710071, China
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou 310024, China
| | - Huihui Ren
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou 310024, China
- Zhejiang University, Hangzhou 310027, China
| | - Saisai Wang
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Advanced Materials and Nanotechnology, Xidian University, Xi'an 710071, China
| | - Kun Liang
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou 310024, China
- Zhejiang University, Hangzhou 310027, China
| | - Dingwei Li
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou 310024, China
- Zhejiang University, Hangzhou 310027, China
| | - Xiaohua Ma
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Microelectronics, Xidian University, Xi'an 710071, China
| | - Bowen Zhu
- Key Laboratory of 3D Micro/Nano Fabrication and Characterization of Zhejiang Province, School of Engineering, Westlake University, Hangzhou 310024, China
| | - Hong Wang
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Advanced Materials and Nanotechnology, Xidian University, Xi'an 710071, China
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Microelectronics, Xidian University, Xi'an 710071, China
| | - Yue Hao
- Key Laboratory of Wide Band Gap Semiconductor Technology, School of Microelectronics, Xidian University, Xi'an 710071, China
| |
Collapse
|
37
|
Zrinski I, Minenkov A, Mardare CC, Hassel AW, Mardare AI. Composite Memristors by Nanoscale Modification of Hf/Ta Anodic Oxides. J Phys Chem Lett 2021; 12:8917-8923. [PMID: 34499511 PMCID: PMC8474145 DOI: 10.1021/acs.jpclett.1c02346] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/20/2021] [Accepted: 09/01/2021] [Indexed: 06/13/2023]
Abstract
Composite memristors based on anodic oxidation of Hf superimposed on Ta thin films are studied. A layered structure is obtained by successive sputtering of Ta and Hf thin films. The deposition geometry ensured components' thickness gradient profiles (wedges) aligned in opposite directions. Anodization in citrate buffer electrolyte leads to a nanoscale columnar structuring of Ta2O5 in HfO2 due to the higher electrical resistance of the latter. Following the less resistive path, the ionic current forces Ta oxide to locally grow toward the electrolyte interface according to the Rayleigh-Taylor principle. The obtained composite oxide memristive properties are studied as a function of the Hf/Ta thickness ratio. One pronounced zone prominent for memristive applications is found for ratios between 4 and 5. Here, unipolar and bipolar memristors are found, with remarkable endurance and retention capabilities. This is discussed in the frame of conductive filament formation preferentially along the interfaces between oxides.
Collapse
Affiliation(s)
- Ivana Zrinski
- Institute
of Chemical Technology of Inorganic Materials, Johannes Kepler University Linz, Altenberger Str. 69, 4040 Linz, Austria
| | - Alexey Minenkov
- Christian
Doppler Laboratory for Nanoscale Phase Transformations, Center for
Surface and Nanoanalytics, Johannes Kepler
University Linz, Altenberger
Str. 69, 4040 Linz, Austria
| | - Cezarina Cela Mardare
- Institute
of Chemical Technology of Inorganic Materials, Johannes Kepler University Linz, Altenberger Str. 69, 4040 Linz, Austria
- Danube
Private University, Steiner
Landstrasse 124, 3500 Krems-Stein, Austria
| | - Achim Walter Hassel
- Institute
of Chemical Technology of Inorganic Materials, Johannes Kepler University Linz, Altenberger Str. 69, 4040 Linz, Austria
- Danube
Private University, Steiner
Landstrasse 124, 3500 Krems-Stein, Austria
| | - Andrei Ionut Mardare
- Institute
of Chemical Technology of Inorganic Materials, Johannes Kepler University Linz, Altenberger Str. 69, 4040 Linz, Austria
| |
Collapse
|
38
|
Ying F, Chen S, Pan G, He Z. Artificial Intelligence Pulse Coupled Neural Network Algorithm in the Diagnosis and Treatment of Severe Sepsis Complicated with Acute Kidney Injury under Ultrasound Image. JOURNAL OF HEALTHCARE ENGINEERING 2021; 2021:6761364. [PMID: 34336164 PMCID: PMC8315850 DOI: 10.1155/2021/6761364] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/20/2021] [Revised: 06/24/2021] [Accepted: 07/12/2021] [Indexed: 11/17/2022]
Abstract
The objective of this study was to explore the diagnosis of severe sepsis complicated with acute kidney injury (AKI) by ultrasonic image information based on the artificial intelligence pulse coupled neural network (PCNN) algorithm. In this study, an algorithm of ultrasonic image information enhancement based on the artificial intelligence PCNN was constructed and compared with the histogram equalization algorithm and linear transformation algorithm. After that, it was applied to the ultrasonic image diagnosis of 20 cases of severe sepsis combined with AKI in hospital. The condition of each patient was diagnosed by ultrasound image performance, change of renal resistance index (RRI), ultrasound score, and receiver operator characteristic curve (ROC) analysis. It was found that the histogram distribution of this algorithm was relatively uniform, and the information of each gray level was obviously retained and enhanced, which had the best effect in this algorithm; there was a marked individual difference in the values of RRI. Overall, the values of RRI showed a slight upward trend after admission to the intensive care unit (ICU). The RRI was taken as the dependent variable, time as the fixed-effect model, and patients as the random effect; the parameter value of time was between 0.012 and 0.015, p=0.000 < 0.05. Besides, there was no huge difference in the ultrasonic score among different time measurements (t = 1.348 and p=0.128 > 0.05). The area under the ROC curve of the RRI for the diagnosis of AKI at the 2nd day, 4th day, and 6th day was 0.758, 0.841, and 0.856, respectively, which was all greater than 0.5 (p < 0.05). In conclusion, the proposed algorithm in this study could significantly enhance the amount of information in ultrasound images. In addition, the change of RRI values measured by ultrasound images based on the artificial intelligence PCNN was associated with AKI.
Collapse
Affiliation(s)
- Fu Ying
- Department of Emergency Medicine, Changzhou Cancer Hospital, Changzhou 213000, Jiangsu, China
| | - Shuhua Chen
- Department of Intensive Care Unit, Changzhou Cancer Hospital, Changzhou 213000, Jiangsu, China
| | - Guojun Pan
- Department of Intensive Care Unit, Changzhou Cancer Hospital, Changzhou 213000, Jiangsu, China
| | - Zemin He
- Department of Emergency Medicine, Changzhou Cancer Hospital, Changzhou 213000, Jiangsu, China
| |
Collapse
|