Catadioptric planar compound eye with large field of view

Advanced Biomimetic Multispectral Curved Compound Eye Camera for Aerial Multispectral Imaging in a Large Field of View

In this work, we demonstrated a new type of biomimetic multispectral curved compound eye camera (BM3C) inspired by insect compound eyes for aerial multispectral imaging in a large field of view. The proposed system exhibits a maximum field of view (FOV) of 120 degrees and seven-waveband multispectral images ranging from visible to near-infrared wavelengths. Pinhole imaging theory and the image registration method from feature detection are used to reconstruct the multispectral 3D data cube. An airborne imaging experiment is performed by assembling the BM3C on an unmanned aerial vehicle (UAV). As a result, radiation intensity curves of several objects are successfully obtained, and a land type classification is performed using the K-means method based on the aerial image as well. The developed BM3C is proven to have the capability for large FOV aerial multispectral imaging and shows great potential applications for distant detecting based on aerial imaging.

Microlens array camera with variable apertures for single-shot high dynamic range (HDR) imaging

We report a microlens array camera with variable apertures (MACVA) for high dynamic range (HDR) imaging by using microlens arrays with various sizes of apertures. The MACVA comprises variable apertures, microlens arrays, gap spacers, and a CMOS image sensor. The microlenses with variable apertures capture low dynamic range (LDR) images with different f-stops under single-shot exposure. The reconstructed HDR images clearly exhibit expanded dynamic ranges surpassing LDR images as well as high resolution without motion artifacts, comparable to the maximum MTF50 value observed among the LDR images. This compact camera provides, what we believe to be, a new perspective for various machine vision or mobile devices applications.

Toward next-generation endoscopes integrating biomimetic video systems, nonlinear optical microscopy, and deep learning

According to the World Health Organization, the proportion of the world's population over 60 years will approximately double by 2050. This progressive increase in the elderly population will lead to a dramatic growth of age-related diseases, resulting in tremendous pressure on the sustainability of healthcare systems globally. In this context, finding more efficient ways to address cancers, a set of diseases whose incidence is correlated with age, is of utmost importance. Prevention of cancers to decrease morbidity relies on the identification of precursor lesions before the onset of the disease, or at least diagnosis at an early stage. In this article, after briefly discussing some of the most prominent endoscopic approaches for gastric cancer diagnostics, we review relevant progress in three emerging technologies that have significant potential to play pivotal roles in next-generation endoscopy systems: biomimetic vision (with special focus on compound eye cameras), non-linear optical microscopies, and Deep Learning. Such systems are urgently needed to enhance the three major steps required for the successful diagnostics of gastrointestinal cancers: detection, characterization, and confirmation of suspicious lesions. In the final part, we discuss challenges that lie en route to translating these technologies to next-generation endoscopes that could enhance gastrointestinal imaging, and depict a possible configuration of a system capable of (i) biomimetic endoscopic vision enabling easier detection of lesions, (ii) label-free in vivo tissue characterization, and (iii) intelligently automated gastrointestinal cancer diagnostic.

Heterogeneous compound eye camera for dual-scale imaging in a large field of view

Multi-scale imaging with large field of view is pivotal for fast motion detection and target identification. However, existing single camera systems are difficult to achieve snapshot multi-scale imaging with large field of view. To solve this problem, we propose a design method for heterogeneous compound eye, and fabricate a prototype of heterogeneous compound eye camera (HeCECam). This prototype which consists of a heterogeneous compound eye array, an optical relay system and a CMOS sensor, is capable of dual-scale imaging in large field of view (360°×141°). The heterogeneous compound eye array is composed of 31 wide-angle (WA) subeyes and 226 high-definition (HD) subeyes. An optical relay system is introduced to re-image the curved focal surface formed by the heterogeneous compound eye array on a CMOS sensor, resulting in a heterogeneous compound eye image containing dual-scale subimages. To verify the imaging characteristics of this prototype, a series of experiments, such as large field of view imaging, imaging performance, and real-world scene imaging, were conducted. The experiment results show that this prototype can achieve dual-scale imaging in large field of view and has excellent imaging performance. This makes the HeCECam has great potential for UAV navigation, wide-area surveillance, and location tracking, and paves the way for the practical use of bio-inspired compound eye cameras.

Bio-inspired spherical compound eye camera for simultaneous wide-band and large field of view imaging

Natural compound eyes have excellent optical characteristics, namely large field of view, small size, no aberration, and sensitive to motion. Some arthropods have more powerful vision. For example, the Morpho butterfly's compound eyes can perceive the near-infrared and ultraviolet light that the human eye cannot see. This wide-band imaging with a large field of view has great potential in wide-area surveillance, all-weather panoramic imaging, and medical imaging. Hence, a wide-band spherical compound eye camera inspired by the Morpho butterfly's eye was proposed. The wide-band spherical compound eye camera which can achieve a large field of view (360° × 171°) imaging over a wide range of wavelengths from 400nm to 1000nm, mainly consists of three parts: a wide-band spherical compound eye with 234 sub-eyes for light collection, a wide-band optical relay system for light transmission, and a wide-band CMOS image sensor for photoelectric conversion. Our experimental results show that the wide-band spherical compound eye camera not only captures a large field of view without anomalous blurring or aberrations but also perceives near-infrared light that is not recognized by the human eye. These features make it possible for distortion-free panoramic vision and panoramic medical diagnosis.

Continuous zoom compound eye imaging system based on liquid lenses

In this paper, a continuous zoom compound eye imaging system based on liquid lenses is proposed. The main imaging part of the system consists of a liquid compound eye, two liquid lenses and a planar image sensor. By adjusting the liquid injection volumes of the liquid compound eye and liquid lenses, the system can realize continuous zoom imaging without any mechanical movement of imaging components. According to the results of experiments, the paraxial magnification of the target can range from ∼0.019× to ∼0.037× at a fixed working distance. Moreover, the system can realize continuous focusing at a fixed paraxial magnification when the working distance ranges from ∼200mm to ∼300mm. Compared with the traditional artificial compound eye imaging systems, the proposed system increases the adjustability and matches the variable image surfaces of the liquid compound eye to a planar image sensor. The aspherical effects of the liquid compound eye and liquid lenses are also considered in the design of the system. The system is expected to be used for imaging in various scenes, such as continuous zoom panoramic imaging, 3D scanning measurement and so on.

Real-time and ultrahigh accuracy image synthesis algorithm for full field of view imaging system

In this paper, we propose a real time, ultrahigh accuracy and full-field-of-view (RUF) algorithm for full field of view (FOV) imaging system. The proposed algorithm combines rough matching and precise matching method to stitch multiple images with the whole FOV in short time and high imaging quality. In order to verify real-time imaging effect of RUF algorithm, we also fabricate a multi-camera imaging system which includes 19 independent cameras. And the experiment result practically illustrates that full-FOV system can achieve good performances under a near-limiting FOV of 360° × 240° with low distortion, meanwhile, optical resolution reaches up to 95 megapixels. 100% registration-accuracy RUF algorithm for imaging in one second can be widely applied to any optical imaging engineering field with large FOV, such as remote sensing imaging, microscopy imaging, monitoring system engineering fields and so on.

Target orientation detection based on a neural network with a bionic bee-like compound eye

The compound eye of insects has many excellent characteristics. Directional navigation is one of the important features of compound eye, which is able to quickly and accurately determine the orientation of an objects. Therefore, bionic curved compound eye have great potential in detecting the orientation of the target. However, there is a serious non-linear relationship between the orientation of the target and the image obtained by the curved compound eye in wide field of view (FOV), and an effective model has not been established to detect the orientation of target. In this paper, a method for detecting the orientation of the target is proposed, which combines a virtual cylinder target with a neural network. To verify the feasibility of the method, a fiber-optic compound eye that is inspired by the structure of the bee's compound eye and that fully utilizes the transmission characteristics and flexibility of optical fibers is developed. A verification experiment shows that the proposed method is able to realize quantitative detection of orientations using a prototype of the fiber-optic compound eye. The average errors between the ground truth and the predicted values of the horizontal and elevation angles of a target are 0.5951 ^° and 0.6748^°, respectively. This approach has great potential for target tracking, obstacle avoidance by unmanned aerial vehicles, and directional navigation control.

Multispectral curved compound eye camera

In this work, we propose a new type of multispectral imaging system, named multispectral curved compound eye camera (MCCEC). The so called MCCEC consists of three subsystems, a curved micro-lens array integrated with selected narrow-band optical filters, an optical transformation subsystem, and the data processing unit with an image sensor. The novel MCCEC system can achieve multi-spectral imaging at an ultra-large field of view (FOV), and obtain information of multiple spectrum segments at real time. Moreover, the system has the advantages of small size, light weight, and high sensitivity in comparison with conventional multispectral cameras. In current work, we mainly focus on the optical design of the MCCEC based on the overlap of FOV between the neighboring clusters of ommatidia to achieve the multispectral imaging at an ultra-large FOV. The optical layout of the curved micro-lens array, narrow-band filter array and the optical relay system for image plane transformation are carefully designed and optimized. The whole size of the optical system is 93 mm × 42 mm × 42 mm. The simulation results show that a maximum FOV of about 120° can be achieved for seven-waveband multispectral imaging with center wavelengths of 480 nm, 550 nm, 591 nm, 676 nm, 704 nm, 740 nm, and 767 nm. The new designed MCCEC has a great potential as an airborne or satellite-born payload for real time remote sensing and thus paves a new way for the design of compact and light-weight spectral-imaging cameras with an ultra large FOV.

Review of state-of-the-art artificial compound eye imaging systems

The natural compound eye has received much attention in recent years due to its remarkable properties, such as its large field of view (FOV), compact structure, and high sensitivity to moving objects. Many studies have been devoted to mimicking the imaging system of the natural compound eye. The paper gives a review of state-of-the-art artificial compound eye imaging systems. Firstly, we introduce the imaging principle of three types of natural compound eye. Then, we divide current artificial compound eye imaging systems into four categories according to the difference of structural composition. Readers can easily grasp methods to build an artificial compound eye imaging system from the perspective of structural composition. Moreover, we compare the imaging performance of state-of-the-art artificial compound eye imaging systems, which provides a reference for readers to design system parameters of an artificial compound eye imaging system. Next, we present the applications of the artificial compound eye imaging system including imaging with a large FOV, imaging with high resolution, object distance detection, medical imaging, egomotion estimation, and navigation. Finally, an outlook of the artificial compound eye imaging system is highlighted.