Multifocusing and depth estimation using a color shift model-based computational camera

Depth map generation using a single image sensor with phase masks

Conventional stereo matching systems generate a depth map using two or more digital imaging sensors. It is difficult to use the small camera system because of their high costs and bulky sizes. In order to solve this problem, this paper presents a stereo matching system using a single image sensor with phase masks for the phase difference auto-focusing. A novel pattern of phase mask array is proposed to simultaneously acquire two pairs of stereo images. Furthermore, a noise-invariant depth map is generated from the raw format sensor output. The proposed method consists of four steps to compute the depth map: (i) acquisition of stereo images using the proposed mask array, (ii) variational segmentation using merging criteria to simplify the input image, (iii) disparity map generation using the hierarchical block matching for disparity measurement, and (iv) image matting to fill holes to generate the dense depth map. The proposed system can be used in small digital cameras without additional lenses or sensors.

Compressive spectral polarization imaging by a pixelized polarizer and colored patterned detector

A compressive spectral and polarization imager based on a pixelized polarizer and colored patterned detector is presented. The proposed imager captures several dispersed compressive projections with spectral and polarization coding. Stokes parameter images at several wavelengths are reconstructed directly from 2D projections. Employing a pixelized polarizer and colored patterned detector enables compressive sensing over spatial, spectral, and polarization domains, reducing the total number of measurements. Compressive sensing codes are specially designed to enhance the peak signal-to-noise ratio in the reconstructed images. Experiments validate the architecture and reconstruction algorithms.

Snapshot colored compressive spectral imager

Traditional spectral imaging approaches require sensing all the voxels of a scene. Colored mosaic FPA detector-based architectures can acquire sets of the scene's spectral components, but the number of spectral planes depends directly on the number of available filters used on the FPA, which leads to reduced spatiospectral resolutions. Instead of sensing all the voxels of the scene, compressive spectral imaging (CSI) captures coded and dispersed projections of the spatiospectral source. This approach mitigates the resolution issues by exploiting optical phenomena in lenses and other elements, which, in turn, compromise the portability of the devices. This paper presents a compact snapshot colored compressive spectral imager (SCCSI) that exploits the benefits of the colored mosaic FPA detectors and the compression capabilities of CSI sensing techniques. The proposed optical architecture has no moving parts and can capture the spatiospectral information of a scene in a single snapshot by using a dispersive element and a color-patterned detector. The optical and the mathematical models of SCCSI are presented along with a testbed implementation of the system. Simulations and real experiments show the accuracy of SCCSI and compare the reconstructions with those of similar CSI optical architectures, such as the CASSI and SSCSI systems, resulting in improvements of up to 6 dB and 1 dB of PSNR, respectively.

Depth-based defocus map estimation using off-axis apertures

This paper presents a depth-based defocus map estimation method from a single camera with multiple off-axis apertures. The proposed estimation algorithm consists of two steps: (i) object distance estimation using off-axis apertures and (ii) defocus map estimation based on the object distance. The proposed method can accurately estimate the defocus map using object distances that are well-characterized in a color shift model-based computational camera. Experimental results show that the proposed method outperforms the state-of-the-art defocus estimation methods in the sense of both accuracy and the estimation range. The proposed defocus map estimation method is suitable for multifocusing, refocusing, and extended depth of field (EDoF) systems.

Three-dimensional object motion and velocity estimation using a single computational RGB-D camera

In this paper, a three-dimensional (3D) object moving direction and velocity estimation method is presented using a dual off-axis color-filtered aperture (DCA)-based computational camera. Conventional object tracking methods provided only two-dimensional (2D) states of an object in the image for the target representation. The proposed method estimates depth information in the object region from a single DCA camera that transforms 2D spatial information into 3D model parameters of the object. We also present a calibration method of the DCA camera to estimate the entire set of camera parameters for a practical implementation. Experimental results show that the proposed DCA-based color and depth (RGB-D) camera can calculate the 3D object moving direction and velocity of a randomly moving object in a single-camera framework.

Fast image restoration for spatially varying defocus blur of imaging sensor

This paper presents a fast adaptive image restoration method for removing spatially varying out-of-focus blur of a general imaging sensor. After estimating the parameters of space-variant point-spread-function (PSF) using the derivative in each uniformly blurred region, the proposed method performs spatially adaptive image restoration by selecting the optimal restoration filter according to the estimated blur parameters. Each restoration filter is implemented in the form of a combination of multiple FIR filters, which guarantees the fast image restoration without the need of iterative or recursive processing. Experimental results show that the proposed method outperforms existing space-invariant restoration methods in the sense of both objective and subjective performance measures. The proposed algorithm can be employed to a wide area of image restoration applications, such as mobile imaging devices, robot vision, and satellite image processing.

DMD-based implementation of patterned optical filter arrays for compressive spectral imaging

Compressive spectral imaging (CSI) captures multispectral imagery using fewer measurements than those required by traditional Shannon-Nyquist theory-based sensing procedures. CSI systems acquire coded and dispersed random projections of the scene rather than direct measurements of the voxels. To date, the coding procedure in CSI has been realized through the use of block-unblock coded apertures (CAs), commonly implemented as chrome-on-quartz photomasks. These apertures block or permit us to pass the entire spectrum from the scene at given spatial locations, thus modulating the spatial characteristics of the scene. This paper extends the framework of CSI by replacing the traditional block-unblock photomasks by patterned optical filter arrays, referred to as colored coded apertures (CCAs). These, in turn, allow the source to be modulated not only spatially but spectrally as well, entailing more powerful coding strategies. The proposed CCAs are synthesized through linear combinations of low-pass, high-pass, and bandpass filters, paired with binary pattern ensembles realized by a digital micromirror device. The optical forward model of the proposed CSI architecture is presented along with a proof-of-concept implementation, which achieves noticeable improvements in the quality of the reconstruction.

Fast motion deblurring using sensor-aided motion trajectory estimation

This paper presents an image deblurring algorithm to remove motion blur using analysis of motion trajectories and local statistics based on inertial sensors. The proposed method estimates a point-spread-function (PSF) of motion blur by accumulating reweighted projections of the trajectory. A motion blurred image is then adaptively restored using the estimated PSF and spatially varying activity map to reduce both restoration artifacts and noise amplification. Experimental results demonstrate that the proposed method outperforms existing PSF estimation-based motion deconvolution methods in the sense of both objective and subjective performance measures. The proposed algorithm can be employed in various imaging devices because of its efficient implementation without an iterative computational structure.

Colored coded aperture design by concentration of measure in compressive spectral imaging

Compressive spectral imaging (CSI) senses the spatio-spectral information of a scene by measuring 2D coded projections on a focal plane array. A ℓ1-norm-based optimization algorithm is then used to recover the underlying discretized spectral image. The coded aperture snapshot spectral imager (CASSI) is an architecture realizing CSI where the reconstruction image quality relies on the design of a 2D set of binary coded apertures which block-unblock the light from the scene. This paper extends the compressive capabilities of CASSI by replacing the traditional blocking-unblocking coded apertures by a set of colored coded apertures. The colored coded apertures are optimized such that the number of projections is minimized while the quality of reconstruction is maximized. The optimal design of the colored coded apertures aims to better satisfy the restricted isometry property in CASSI. The optimal designs are compared with random colored coded aperture patterns and with the traditional blocking-unblocking coded apertures. Extensive simulations show the improvement in reconstruction PSNR attained by the optimal colored coded apertures designs.

Distance estimation using a single computational camera with dual off-axis color filtered apertures

In this paper, we present a novel computational imaging system using a dual off-axis color filtered aperture (DCA) for distance estimation in a single-camera framework. The DCA consists of two off-axis apertures that are covered by red and cyan color filters. The two apertures generate misaligned color channels in which the amount of misalignment of points in the image plane are a function of the distance from the camera of the corresponding points in the object plane. The primary contribution of this paper is the derivation of a mathematical model of the relationship between the color shifting values and distance of an object from the camera when the camera parameters and the baseline distance between the two apertures in the DCA are given. The proposed computational imaging system can be implemented simply by inserting an appropriately sized DCA into any general optical system. Experimental results show that the DCA camera is able to estimate the distances of objects within a single-camera framework.