1
|
Cui Y, Zheng X, Shen C, Qian L, Dong H, Liu Q, Chen X, Yang Q, Zhang F, Wang D. Visual-Olfactory Synergistic Perception Based on Dual-Focus Imaging and a Bionic Learning Architecture. ACS Sens 2023; 8:71-79. [PMID: 36574494 DOI: 10.1021/acssensors.2c01721] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
The synergistic interaction of vision and olfaction is critical for both natural and artificial intelligence systems to recognize and adapt to complex environments. However, current bioinspired systems with visual and olfactory sensations are mostly assembled with separate and heterogeneous sensors, inevitably leading to bulky systems and incompatible datasets. Here, we demonstrate on-chip integration of visual and olfactory sensations through a dual-focus imaging approach. By combining lens-based visual imaging and lensless colorimetric imaging, a target object and its odor fingerprint can be captured with a single complementary metal-oxide-semiconductor imager, and the obtained multimodal images are analyzed with a bionic learning architecture for information fusion and perception. To demonstrate the capability of this system, we adapted it to food detection and achieved 100% accuracy in identifying meat freshness and category with a 10 s sampling time. In addition to the highly integrated sensor design, our approach exhibits superior accuracy and efficiency in object recognition, providing a promising approach for robotic sensing and perception.
Collapse
Affiliation(s)
- Yaoxuan Cui
- Intelligent Perception Research Institute, Zhejiang Lab, Hangzhou311100, China
| | - Xubin Zheng
- Intelligent Perception Research Institute, Zhejiang Lab, Hangzhou311100, China
| | - Chen Shen
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou310027, China
| | - Libin Qian
- Intelligent Perception Research Institute, Zhejiang Lab, Hangzhou311100, China
| | - Hao Dong
- Intelligent Perception Research Institute, Zhejiang Lab, Hangzhou311100, China
| | - Qingjun Liu
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou310027, China.,Biosensor National Special Laboratory, Key Laboratory for Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou310027, China
| | - Xing Chen
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou310027, China.,Biosensor National Special Laboratory, Key Laboratory for Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou310027, China
| | - Qing Yang
- Intelligent Perception Research Institute, Zhejiang Lab, Hangzhou311100, China.,State Key Laboratory of Modern Optical Instrumentation, College of Optical Science and Engineering, Joint International Research Laboratory of Photonics, Zhejiang University, Hangzhou310027, China
| | - Fenni Zhang
- College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou310027, China.,Biosensor National Special Laboratory, Key Laboratory for Biomedical Engineering of Education Ministry, Department of Biomedical Engineering, Zhejiang University, Hangzhou310027, China
| | - Di Wang
- Intelligent Perception Research Institute, Zhejiang Lab, Hangzhou311100, China.,College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou310027, China
| |
Collapse
|