Generalized assorted camera arrays: robust cross-channel registration and applications

Coded aperture snapshot hyperspectral light field tomography

Multidimensional imaging has emerged as a powerful technology capable of simultaneously acquiring spatial, spectral, and depth information about a scene. However, existing approaches often rely on mechanical scanning or multi-modal sensing configurations, leading to prolonged acquisition times and increased system complexity. Coded aperture snapshot spectral imaging (CASSI) has introduced compressed sensing to recover three-dimensional (3D) spatial-spectral datacubes from single snapshot two-dimensional (2D) measurements. Despite its advantages, the reconstruction problem remains severely underdetermined due to the high compression ratio, resulting in limited spatial and spectral reconstruction quality. To overcome this challenge, we developed a novel two-stage cascaded compressed sensing scheme called coded aperture snapshot hyperspectral light field tomography (CASH-LIFT). By appropriately distributing the computation load to each stage, this method utilizes the compressibility of natural scenes in multiple domains, reducing the ill-posed nature of datacube recovery and achieving enhanced spatial resolution, suppressed aliasing artifacts, and improved spectral fidelity. Additionally, leveraging the snapshot 3D imaging capability of LIFT, our approach efficiently records a five-dimensional (5D) plenoptic function in a single snapshot.

Spectral Clustering Super-Resolution Imaging Based on Multispectral Camera Array

Although multispectral and hyperspectral imaging acquisitions are applied in numerous fields, the existing spectral imaging systems suffer from either low temporal or spatial resolution. In this study, a new multispectral imaging system-camera array based multispectral super resolution imaging system (CAMSRIS) is proposed that can simultaneously achieve multispectral imaging with high temporal and spatial resolutions. The proposed registration algorithm is used to align pairs of different peripheral and central view images. A novel, super-resolution, spectral-clustering-based image reconstruction algorithm was developed for the proposed CAMSRIS to improve the spatial resolution of the acquired images and preserve the exact spectral information without introducing false information. The reconstructed results showed that the spatial and spectral quality and operational efficiency of the proposed system are better than those of a multispectral filter array (MSFA) based on different multispectral datasets. The PSNR of the multispectral super-resolution images obtained by the proposed method were respectively higher by 2.03 and 1.93 dB than those of GAP-TV and DeSCI, and the execution time was significantly shortened by approximately 54.55 s and 9820.19 s when the CAMSI dataset was used. The feasibility of the proposed system was verified in practical applications based on different scenes captured by the self-built system.

Snapshot hyperspectral light field tomography

We present snapshot hyperspectral light field tomography (Hyper-LIFT), a highly efficient method in recording a 5D (x, y, spatial coordinates; θ, φ, angular coordinates; λ, wavelength) plenoptic function. Using a Dove prism array and a cylindrical lens array, we simultaneously acquire multi-angled 1D en face projections of the object like those in standard sparse-view computed tomography. We further disperse those projections and measure the spectra in parallel using a 2D image sensor. Within a single snapshot, the resultant system can capture a 5D data cube with 270 × 270 × 4 × 4 × 360 voxels. We demonstrated the performance of Hyper-LIFT in imaging spectral volumetric scenes.

Design of an Integrated Microfluidic Paper-Based Chip and Inspection Machine for the Detection of Mercury in Food with Silver Nanoparticles

For most of the fast screening test papers for detecting Hg2+, the obtained results are qualitative. This study developed an operation for the μPAD and combined it with the chemical colorimetric method. Silver nanoparticle (AgNP) colloids were adopted as the reactive color reagent to combine and react with the Hg standards on the paper-based chip. Then, the RGB values for the color change were used to establish the standard curve (R2 > 0.99). Subsequently, this detection system was employed for the detection tests of actual samples, and the detected RGB values of the samples were substituted back to the formula to calculate the Hg2+ contents in the food. In this study, the Hg2+ content and recovery rate in commercially available packaged water and edible salts were measured. The research results indicate that a swift, economical, and simple detection method for Hg2+ content in food has been successfully developed.

Camera Array for Multi-Spectral Imaging

Recently, many new applications arose for multispectral and hyper-spectral imaging. Besides modern biometric systems for identity verification, also agricultural and medical applications came up, which measure the health condition of plants and humans. Despite the growing demand, the acquisition of multi-spectral data is up to the present complicated. Often, expensive, inflexible, or low resolution acquisition setups are only obtainable for specific professional applications. To overcome these limitations, a novel camera array for multi-spectral imaging is presented in this article for generating consistent multispectral videos. As differing spectral images are acquired at various viewpoints, a geometrically constrained multi-camera sensor layout is introduced, which enables the formulation of novel registration and reconstruction algorithms to globally set up robust models. On average, the novel acquisition approach achieves a gain of 2.5 dB PSNR compared to recently published multi-spectral filter array imaging systems. At the same time, the proposed acquisition system ensures not only a superior spatial, but also a high spectral, and temporal resolution, while filters are flexibly exchangeable by the user depending on the application. Moreover, depth information is generated, so that 3D imaging applications, e.g., for augmented or virtual reality, become possible. The proposed camera array for multi-spectral imaging can be set up using off-the-shelf hardware, which allows for a compact design and employment in, e.g., mobile devices or drones, while being cost-effective.

Hyperspectral Light Field Stereo Matching

In this paper, we describe how scene depth can be extracted using a hyperspectral light field capture (H-LF) system. Our H-LF system consists of a 5 ×6 array of cameras, with each camera sampling a different narrow band in the visible spectrum. There are two parts to extracting scene depth. The first part is our novel cross-spectral pairwise matching technique, which involves a new spectral-invariant feature descriptor and its companion matching metric we call bidirectional weighted normalized cross correlation (BWNCC). The second part, namely, H-LF stereo matching, uses a combination of spectral-dependent correspondence and defocus cues. These two new cost terms are integrated into a Markov Random Field (MRF) for disparity estimation. Experiments on synthetic and real H-LF data show that our approach can produce high-quality disparity maps. We also show that these results can be used to produce the complete plenoptic cube in addition to synthesizing all-focus and defocused color images under different sensor spectral responses.

Segmentation-Based Color Channel Registration for Disparity Estimation of Dual Color-Filtered Aperture Camera

Single-lens-based optical range finding systems were developed as an efficient, compact alternative for conventional stereo camera systems. Among various single-lens-based approaches, a multiple color-filtered aperture (MCA) system can generate disparity information among color channels, as well as normal color information. In this paper, we consider a dual color-filtered aperture (DCA) system as the most minimal version of the MCA system and present a novel inter-color image registration algorithm for disparity estimation. This proposed registration algorithm consists of three steps: (i) color channel independent feature extraction; (ii) feature-based adaptive weight disparity estimation; and (iii) color mapping matrix (CMM)-based cross-channel image registration. Experimental results show that the proposed method can not only generate an accurate disparity map, but also realize high quality cross-channel registration with a disparity prior for DCA-based range finding and color image enhancement.

Robust Matching Cost Function for Stereo Correspondence Using Matching by Tone Mapping and Adaptive Orthogonal Integral Image

Real-world stereo images are inevitably affected by radiometric differences, including variations in exposure, vignetting, lighting, and noise. Stereo images with severe radiometric distortion can have large radiometric differences and include locally nonlinear changes. In this paper, we first introduce an adaptive orthogonal integral image, which is an improved version of an orthogonal integral image. After that, based on matching by tone mapping and the adaptive orthogonal integral image, we propose a robust and accurate matching cost function that can tolerate locally nonlinear intensity distortion. By using the adaptive orthogonal integral image, the proposed matching cost function can adaptively construct different support regions of arbitrary shapes and sizes for different pixels in the reference image, so it can operate robustly within object boundaries. Furthermore, we develop techniques to automatically estimate the values of the parameters of our proposed function. We conduct experiments using the proposed matching cost function and compare it with functions employing the census transform, supporting local binary pattern, and adaptive normalized cross correlation, as well as a mutual information-based matching cost function using different stereo data sets. By using the adaptive orthogonal integral image, the proposed matching cost function reduces the error from 21.51% to 15.73% in the Middlebury data set, and from 15.9% to 10.85% in the Kitti data set, as compared with using the orthogonal integral image. The experimental results indicate that the proposed matching cost function is superior to the state-of-the-art matching cost functions under radiometric variation.