Sky light polarization detection with linear polarizer triplet in light field camera inspired by insect vision

Biologically inspired microlens array camera for high-speed and high-sensitivity imaging

Nocturnal and crepuscular fast-eyed insects often exploit multiple optical channels and temporal summation for fast and low-light imaging. Here, we report high-speed and high-sensitive microlens array camera (HS-MAC), inspired by multiple optical channels and temporal summation for insect vision. HS-MAC features cross-talk-free offset microlens arrays on a single rolling shutter CMOS image sensor and performs high-speed and high-sensitivity imaging by using channel fragmentation, temporal summation, and compressive frame reconstruction. The experimental results demonstrate that HS-MAC accurately measures the speed of a color disk rotating at 1950 rpm, recording fast sequences at 9120 fps with low noise equivalent irradiance (0.43 μW/cm2). Besides, HS-MAC visualizes the necking pinch-off of a pool fire flame in dim light conditions below one thousandth of a lux. The compact high-speed low-light camera can offer a distinct route for high-speed and low-light imaging in mobile, surveillance, and biomedical applications.

Bioinspired single-shot polarization photodetector based on four-directional grating arrays capped perovskite single-crystal thin film

Polarization photodetectors (pol-PDs) have widespread applications in geological remote sensing, machine vision, biological medicine, and so on. However, commercial pol-PDs use bulky and complicated optical systems with lenses, polarizers, and mechanical spools, which are complex and cumbersome, and respond slowly. Inspired by the desert ants' compound eyes, we developed a single-shot pol-PD based on four-directional grating arrays capped perovskite single-crystal thin film without other standard polarization optics. Our pol-PD has a high detectivity, two orders of magnitude greater than that of commercial photodetectors, and exhibits high polarization sensitivity. The high performance of our pol-PD is due to the highly crystalline perovskite single-crystal thin film and regular nanograting structure, made by a nanoimprinting crystallization method. Our single-shot pol-PD is a compact on-chip optoelectronic device that demonstrates excellent performance in a wide range of applications including accurate bionic navigation, sharp image restoration in hazy scenes, stress visualization of polymers, and detection of cancerous areas in tissues without histological staining.

Image-registration-based solar meridian detection for accurate and robust polarization navigation

Skylight polarization, inspired by the foraging behavior of insects, has been widely used for navigation for various platforms, such as robots, unmanned aerial vehicles, and others, owing to its stability and non-error-accumulation. Among the characteristics of skylight-polarized patterns, the angle of polarization (AOP) and the degree of polarization (DOP) are two of the most significant characteristics that provide abundant information regarding the position of the sun. In this study, we propose an accurate method for detecting the solar meridian for real-time bioinspired navigation through image registration. This method uses the AOP pattern to detect the solar meridian and eliminates the ambiguity between anti-solar meridian and solar meridian using the DOP pattern, resulting in an accurate heading of the observer. Simulation experiments demonstrated the superior performance of the proposed method compared to the alternative approaches. Field experiments demonstrate that the proposed method achieves real-time, robust, and accurate performance under different weather conditions with a root mean square error of 0.1° under a clear sky, 0.18° under an overcast sky with a thin layer of clouds, and 0.32° under an isolated thick cloud cover. Our findings suggest that the proposed method can be potentially used in skylight polarization for real-time and accurate navigation in GPS-denied environments.

Cloud Detection Method Based on All-Sky Polarization Imaging

Sky cloud detection has a significant application value in the meteorological field. The existing cloud detection methods mainly rely on the color difference between the sky background and the cloud layer in the sky image and are not reliable due to the variable and irregular characteristics of the cloud layer and different weather conditions. This paper proposes a cloud detection method based on all-sky polarization imaging. The core of the algorithm is the "normalized polarization degree difference index" (NPDDI). Instead of relying on the color difference information, this index identifies the difference between degree of polarization (DoPs) of the cloud sky and the clear sky radiation to achieve cloud recognition. The method is not only fast and straightforward in the algorithm, but also can detect the optical thickness of the cloud layer in a qualitative sense. The experimental results show a good cloud detection performance.

All-In-Focus Polarimetric Imaging Based on an Integrated Plenoptic Camera with a Key Electrically Tunable LC Device

In this paper, a prototyped plenoptic camera based on a key electrically tunable liquid-crystal (LC) device for all-in-focus polarimetric imaging is proposed. By using computer numerical control machining and 3D printing, the proposed imaging architecture can be integrated into a hand-held prototyped plenoptic camera so as to greatly improve the applicability for outdoor imaging measurements. Compared with previous square-period liquid-crystal microlens arrays (LCMLA), the utilized hexagonal-period LCMLA has remarkably increased the light utilization rate by ~15%. Experiments demonstrate that the proposed imaging approach can simultaneously realize both the plenoptic and polarimetric imaging without any macroscopic moving parts. With the depth-based rendering method, both the all-in-focus images and the all-in-focus degree of linear polarization (DoLP) images can be obtained efficiently. Due to the large depth-of-field advantage of plenoptic cameras, the proposed camera enables polarimetric imaging in a larger depth range than conventional 2D polarimetric cameras. Currently, the raw light field images with three polarization states including I0 and I60 and I120 can be captured by the proposed imaging architecture, with a switching time of several tens of milliseconds. Some local patterns which are selected as interested target features can be effectively suppressed or obviously enhanced by switching the polarization state mentioned. According to experiments, the visibility in scattering medium can also be apparently improved. It can be expected that the proposed polarimetric imaging approach will exhibit an excellent development potential.

Sensor Modeling and Calibration Method Based on Extinction Ratio Error for Camera-Based Polarization Navigation Sensor

The performance of camera-based polarization sensors largely depends on the estimated model parameters obtained through calibration. Limited by manufacturing processes, the low extinction ratio and inconsistency of the polarizer can reduce the measurement accuracy of the sensor. To account for the challenges, one extinction ratio coefficient was introduced into the calibration model to unify the light intensity of two orthogonal channels. Since the introduced extinction ratio coefficient is associated with degree of polarization (DOP), a new calibration method considering both azimuth of polarization (AOP) error and DOP error for the bionic camera-based polarization sensor was proposed to improve the accuracy of the calibration model parameter estimation. To evaluate the performance of the proposed camera-based polarization calibration model using the new calibration method, both indoor and outdoor calibration experiments were carried out. It was found that the new calibration method for the proposed calibration model could achieve desirable performance in terms of stability and robustness of the calculated AOP and DOP values.

Polarized skylight compass based on a soft-margin support vector machine working in cloudy conditions

The skylight polarization pattern, which is a result of the scattering of unpolarized sunlight by particles in the atmosphere, can be used by many insects for navigation. Inspired by insects, several polarization navigation sensors have been designed and combined with various heading determination methods in recent years. However, up until now, few of these studies have fully considered the influences of different meteorological conditions, which play key roles in navigation accuracy, especially in cloudy weather. Therefore, this study makes a major contribution to the study on bio-inspired heading determination by designing a skylight compass method to suppress cloud disturbances. The proposed method transforms the heading determination problem into a binary classification problem by segmentation, connected component detection, and inversion. Considering the influences of noise and meteorological conditions, the binary classification problem is solved by the soft-margin support vector machine. In addition, to verify this method, a pixelated polarization compass platform is constructed that can take polarization images at four different orientations simultaneously in real time. Finally, field experimental results show that the designed method can more effectively suppress the interference of clouds compared with other methods.

From skylight input to behavioural output: A computational model of the insect polarised light compass

Many insects navigate by integrating the distances and directions travelled on an outward path, allowing direct return to the starting point. Fundamental to the reliability of this process is the use of a neural compass based on external celestial cues. Here we examine how such compass information could be reliably computed by the insect brain, given realistic constraints on the sky polarisation pattern and the insect eye sensor array. By processing the degree of polarisation in different directions for different parts of the sky, our model can directly estimate the solar azimuth and also infer the confidence of the estimate. We introduce a method to correct for tilting of the sensor array, as might be caused by travel over uneven terrain. We also show that the confidence can be used to approximate the change in sun position over time, allowing the compass to remain fixed with respect to 'true north' during long excursions. We demonstrate that the compass is robust to disturbances and can be effectively used as input to an existing neural model of insect path integration. We discuss the plausibility of our model to be mapped to known neural circuits, and to be implemented for robot navigation.

Bio-inspired orientation using the polarization pattern in the sky based on artificial neural networks

Many insects use the pattern of polarized light in the sky as a navigational cue. In this study, we use this sensory ability as a source of inspiration to create a computational orientation model based on an artificial neural network (POL-ANN). After a training phase using numerically generated sky polarization patterns, stable and convergent networks are obtained. We undertook a series of verification tests using four typical but different sky conditions and showed that the post-trained networks were able to make an accurate prediction of the direction of the sun. Comparisons between the proposed models and models based on the convolutional neural network (CNN) structure revealed the merits of the bio-inspired architecture. We further investigated the accuracy of the models based on two different (locust-like, broader; Drosophila-like, narrower) visual fields of the sky. We find that the accuracy of the computations depends on the overhead visual scene, specifically that wider fields of view perform better when information about the overhead polarization pattern is missing.

Polarized skylight-based heading measurements: a bio-inspired approach

Many insects such as desert ants, crickets, locusts, dung beetles, bees and monarch butterflies have been found to extract their navigation cues from the regular pattern of the linearly polarized skylight. These species are equipped with ommatidia in the dorsal rim area of their compound eyes, which are sensitive to the angle of polarization of the skylight. In the polarization-based robotic vision, most of the sensors used so far comprise high-definition CCD or CMOS cameras topped with linear polarizers. Here, we present a 2-pixel polarization-sensitive visual sensor, which was strongly inspired by the dorsal rim area of desert ants' compound eyes, designed to determine the direction of polarization of the skylight. The spectral sensitivity of this minimalistic sensor, which requires no lenses, is in the ultraviolet range. Five different methods of computing the direction of polarization were implemented and tested here. Our own methods, the extended and AntBot method, outperformed the other three, giving a mean angular error of only 0.62° ± 0.40° (median: 0.24°) and 0.69° ± 0.52° (median: 0.39°), respectively (mean ± standard deviation). The results obtained in outdoor field studies show that our celestial compass gives excellent results at a very low computational cost, which makes it highly suitable for autonomous outdoor navigation purposes.

It's Not a Bug, It's a Feature: Functional Materials in Insects

Over the course of their wildly successful proliferation across the earth, the insects as a taxon have evolved enviable adaptations to their diverse habitats, which include adhesives, locomotor systems, hydrophobic surfaces, and sensors and actuators that transduce mechanical, acoustic, optical, thermal, and chemical signals. Insect-inspired designs currently appear in a range of contexts, including antireflective coatings, optical displays, and computing algorithms. However, as over one million distinct and highly specialized species of insects have colonized nearly all habitable regions on the planet, they still provide a largely untapped pool of unique problem-solving strategies. With the intent of providing materials scientists and engineers with a muse for the next generation of bioinspired materials, here, a selection of some of the most spectacular adaptations that insects have evolved is assembled and organized by function. The insects presented display dazzling optical properties as a result of natural photonic crystals, precise hierarchical patterns that span length scales from nanometers to millimeters, and formidable defense mechanisms that deploy an arsenal of chemical weaponry. Successful mimicry of these adaptations may facilitate technological solutions to as wide a range of problems as they solve in the insects that originated them.

Orthogonal vector algorithm to obtain the solar vector using the single-scattering Rayleigh model

Information obtained from a polarization pattern in the sky provides many animals like insects and birds with vital long-distance navigation cues. The solar vector can be derived from the polarization pattern using the single-scattering Rayleigh model. In this paper, an orthogonal vector algorithm, which utilizes the redundancy of the single-scattering Rayleigh model, is proposed. We use the intersection angles between the polarization vectors as the main criteria in our algorithm. The assumption that all polarization vectors can be considered coplanar is used to simplify the three-dimensional (3D) problem with respect to the polarization vectors in our simulation. The surface-normal vector of the plane, which is determined by the polarization vectors after translation, represents the solar vector. Unfortunately, the two-directionality of the polarization vectors makes the resulting solar vector ambiguous. One important result of this study is, however, that this apparent disadvantage has no effect on the complexity of the algorithm. Furthermore, two other universal least-squares algorithms were investigated and compared. A device was then constructed, which consists of five polarized-light sensors as well as a 3D attitude sensor. Both the simulation and experimental data indicate that the orthogonal vector algorithms, if used with a suitable threshold, perform equally well or better than the other two algorithms. Our experimental data reveal that if the intersection angles between the polarization vectors are close to 90°, the solar-vector angle deviations are small. The data also support the assumption of coplanarity. During the 51 min experiment, the mean of the measured solar-vector angle deviations was about 0.242°, as predicted by our theoretical model.

Angle of sky light polarization derived from digital images of the sky under various conditions

Skylight polarization is used for navigation by some birds and insects. Skylight polarization also has potential for human navigation applications. Its advantages include relative immunity from interference and the absence of error accumulation over time. However, there are presently few examples of practical applications for polarization navigation technology. The main reason is its weak robustness during cloudy weather conditions. In this paper, the real-time measurement of the sky light polarization pattern across the sky has been achieved with a wide field of view camera. The images were processed under a new reference coordinate system to clearly display the symmetrical distribution of angle of polarization with respect to the solar meridian. A new algorithm for the extraction of the image axis of symmetry is proposed, in which the real-time azimuth angle between the camera and the solar meridian is accurately calculated. Our experimental results under different weather conditions show that polarization navigation has high accuracy, is strongly robust, and performs well during fog and haze, clouds, and strong sunlight.

Robust sky light polarization detection with an S-wave plate in a light field camera

The sky light polarization navigator has many advantages, such as low cost, no decrease in accuracy with continuous operation, etc. However, current celestial polarization measurement methods often suffer from low performance when the sky is covered by clouds, which reduce the accuracy of navigation. In this paper we introduce a new method and structure based on a handheld light field camera and a radial polarizer, composed of an S-wave plate and a linear polarizer, to detect the sky light polarization pattern across a wide field of view in a single snapshot. Each micro-subimage has a special intensity distribution. After extracting the texture feature of these subimages, stable distribution information of the angle of polarization under a cloudy sky can be obtained. Our experimental results match well with the predicted properties of the theory. Because the polarization pattern is obtained through image processing, rather than traditional methods based on mathematical computation, this method is less sensitive to errors of pixel gray value and thus has better anti-interference performance.