Multispectral Image Out-of-Focus Deblurring Using Interchannel Correlation

Chromatic-aberration-free multispectral optical-resolution photoacoustic microscopy using reflective optics and a supercontinuum light source

A supercontinuum (SC) light source enables multispectral photoacoustic imaging at excitation wavelengths in the visible-to-near-infrared range. However, for such a broad optical wavelength range, chromatic aberration is non-negligible. We developed a multispectral optical-resolution photoacoustic microscopy (MS-OR-PAM) setup with a nanosecond pulsed SC light source and a reflective objective lens to avoid chromatic aberration. Chromatic aberrations generated by reflective and conventional objective lenses were compared, and the images acquired using the reflective objective were not affected by chromatic aberration. Hence, MS-OR-PAM with the reflective objective was used to distinguish red blood cells from melanoma cells via spectral subtraction processing.

Depth estimation using spectrally varying defocus blur

This paper proposes a method to estimate depth from a single multispectral image by using a lens property known as chromatic aberration. Chromatic aberration causes light passing through a lens to be refracted depending on the wavelength. The refraction causes the angle of rays to vary depending on their wavelength and a change in focal length, which leads to a defocus blur for different wavelengths. We propose a theory to recover a continuous depth map from the blur in a single multispectral image that includes chromatic aberration. The proposed method needs only a standard wide-aperture lens, which naturally exhibits chromatic aberration, and a multispectral camera. Moreover, we use a simple yet effective depth-of-field synthesis method to calculate the derivatives and obtain all-in-focus images necessary to approximate spectral derivatives. We verified the effectiveness of the proposed method on various real-world scenes.

High-quality blind defocus deblurring of multispectral images with optics and gradient prior

This paper presents a blind defocus deblurring method that produces high-quality deblurred multispectral images. The high quality is achieved by two means: i) more accurate kernel estimation based on the optics prior by simulating the simple lens imaging, and ii) the gradient-based inter-channel correlation with the reference image generated by the content-adaptive combination of adjacent channels for restoring the latent sharp image. As a result, our method gains the prominence on both effectiveness and efficiency in deblurring defocus multispectral images with very good restoration on the obscure details. The experiments on some multispectral image datasets demonstrate the advantages of our method over state-of-the-art deblurring methods.

Star Image Prediction and Restoration under Dynamic Conditions

The star sensor is widely used in attitude control systems of spacecraft for attitude measurement. However, under high dynamic conditions, frame loss and smearing of the star image may appear and result in decreased accuracy or even failure of the star centroid extraction and attitude determination. To improve the performance of the star sensor under dynamic conditions, a gyroscope-assisted star image prediction method and an improved Richardson-Lucy (RL) algorithm based on the ensemble back-propagation neural network (EBPNN) are proposed. First, for the frame loss problem of the star sensor, considering the distortion of the star sensor lens, a prediction model of the star spot position is obtained by the angular rates of the gyroscope. Second, to restore the smearing star image, the point spread function (PSF) is calculated by the angular velocity of the gyroscope. Then, we use the EBPNN to predict the number of iterations required by the RL algorithm to complete the star image deblurring. Finally, simulation experiments are performed to verify the effectiveness and real-time of the proposed algorithm.

Normalized Total Gradient: A New Measure for Multispectral Image Registration

Image registration is a fundamental issue in multispectral image processing, and is challenged by two main characteristics of multispectral images. First, the regional intensities can be essentially different between band images. Second, the local contrasts of two difference band images are inconsistent or even reversed. Conventional measures can align images with different regional intensity levels, but may fail in the circumstance of severe local intensity variation. In this paper, a new measure called normalized total gradient is proposed for multispectral image registration. The measure is based on the key assumption (observation) that the gradient of the difference between two aligned band images is sparser than that between two misaligned ones. A registration framework, which incorporates image pyramid and global/local optimization, is further introduced for affine transform. Experimental results validate that the proposed method is not only effective for multispectral image registration, but also applicable to general unimodal/multimodal image registration tasks. It performs better than or comparable to the existing methods, both quantitatively and qualitatively.

Spectral bidirectional texture function reconstruction by fusing multiple-color and spectral images

Spectral bidirectional texture function (BTF) is essential for accurate reproduction of material appearance due to its nature of conveying both spatial and spectral information. A practical issue is that the acquisition of raw spectral BTFs is time-consuming. To resolve the limitation, this paper proposes a novel framework for efficient spectral BTF acquisition and reconstruction. The framework acquires red-green-blue (RGB) BTF images and just one spectral image. The full spectral BTFs are reconstructed by fusing the RGB and spectral images based on nonnegative matrix factorization (NMF). Experimental results indicate that the accuracy of spectral reflectance reconstruction is higher than that of existing algorithms. With the reconstructed spectral BTFs, the material appearance can be reproduced with high fidelity under various illumination conditions.

Fast Multispectral Imaging by Spatial Pixel-Binning and Spectral Unmixing

Multispectral imaging system is of wide application in relevant fields for its capability in acquiring spectral information of scenes. Its limitation is that, due to the large number of spectral channels, the imaging process can be quite time-consuming when capturing high-resolution (HR) multispectral images. To resolve this limitation, this paper proposes a fast multispectral imaging framework based on the image sensor pixel-binning and spectral unmixing techniques. The framework comprises a fast imaging stage and a computational reconstruction stage. In the imaging stage, only a few spectral images are acquired in HR, while most spectral images are acquired in low resolution (LR). The LR images are captured by applying pixel binning on the image sensor, such that the exposure time can be greatly reduced. In the reconstruction stage, an optimal number of basis spectra are computed and the signal-dependent noise statistics are estimated. Then the unknown HR images are efficiently reconstructed by solving a closed-form cost function that models the spatial and spectral degradations. The effectiveness of the proposed framework is evaluated using real-scene multispectral images. Experimental results validate that, in general, the method outperforms the state of the arts in terms of reconstruction accuracy, with additional 20× or more improvement in computational efficiency.