Blind identification of multichannel FIR blurs and perfect image restoration

Restoration of fast moving objects

If an object is photographed at motion in front of a static background, the object will be blurred while the background sharp and partially occluded by the object. The goal is to recover the object appearance from such blurred image. We adopt the image formation model for fast moving objects and consider objects undergoing 2D translation and rotation. For this scenario we formulate the estimation of the object shape, appearance, and motion from a single image and known background as a constrained optimization problem with appropriate regularization terms. Both similarities and differences with blind deconvolution are discussed with the latter caused mainly by the coupling of the object appearance and shape in the acquisition model. Necessary conditions for solution uniqueness are derived and a numerical solution based on the alternating direction method of multipliers is presented. The proposed method is evaluated on a new dataset.

Optimal design of multichannel equalizers for the structural similarity index

The optimization of multichannel equalizers is studied for the structural similarity (SSIM) criteria. The closed-form formula is provided for the optimal equalizer when the mean of the source is zero. The formula shows that the equalizer with maximal SSIM index is equal to the one with minimal mean square error (MSE) multiplied by a positive real number, which is shown to be equal to the inverse of the achieved SSIM index. The relation of the maximal SSIM index to the minimal MSE is also established for given blurring filters and fixed length equalizers. An algorithm is also presented to compute the suboptimal equalizer for the general sources. Various numerical examples are given to demonstrate the effectiveness of the results.

Robust multichannel blind deconvolution via fast alternating minimization

Blind deconvolution, which comprises simultaneous blur and image estimations, is a strongly ill-posed problem. It is by now well known that if multiple images of the same scene are acquired, this multichannel (MC) blind deconvolution problem is better posed and allows blur estimation directly from the degraded images. We improve the MC idea by adding robustness to noise and stability in the case of large blurs or if the blur size is vastly overestimated. We formulate blind deconvolution as an l(1) -regularized optimization problem and seek a solution by alternately optimizing with respect to the image and with respect to blurs. Each optimization step is converted to a constrained problem by variable splitting and then is addressed with an augmented Lagrangian method, which permits simple and fast implementation in the Fourier domain. The rapid convergence of the proposed method is illustrated on synthetically blurred data. Applicability is also demonstrated on the deconvolution of real photos taken by a digital camera.

Classification-based adaptive filtering for multiframe blind image restoration

In this paper, the blind restoration of a scene is investigated, when multiple degraded (blurred and noisy) acquisitions are available. An adaptive filtering technique is proposed, where the distorted images are filtered, classified and then fused based upon the classification decisions. Finite normal-density mixture (FNM) models are used to model the filtered outputs at each iteration. For simplicity, fixed number of Gaussian components (classes) is, initially, considered for each degraded frame and the selection of the optimal number of classes is performed according to the global relative entropy criterion. However, there exist cases where dynamically varying FNM models should be considered, where the optimal number of classes is selected according to the Akaike information criterion. The iterative application of classification and fusion, followed by optimal adaptive filtering, converges to a global enhanced representation of the original scene in only a few iterations. The proposed restoration method does not require knowledge of the point-spread-function support size or exact alignment of the acquired frames. Simulation results on synthetic and real data, using both fixed and dynamically varying FNM models, demonstrate its efficiency under both noisy and noise-free conditions.

A new look to multichannel blind image deconvolution

The aim of this paper is to propose a new look to MBID, examine some known approaches, and provide a new MC method for restoring blurred and noisy images. First, the direct image restoration problem is briefly revisited. Then a new method based on inverse filtering for perfect image restoration in the noiseless case is proposed. The noisy case is addressed by introducing a regularization term into the objective function in order to avoid noise amplification. Second, the filter identification problem is considered in the MC context. A new robust solution to estimate the degradation matrix filter is then derived and used in conjunction with a total variation approach to restore the original image. Simulation results and performance evaluations using recent image quality metrics are provided to assess the effectiveness of the proposed methods.

A unified approach to superresolution and multichannel blind deconvolution

This paper presents a new approach to the blind deconvolution and superresolution problem of multiple degraded low-resolution frames of the original scene. We do not assume any prior information about the shape of degradation blurs. The proposed approach consists of building a regularized energy function and minimizing it with respect to the original image and blurs, where regularization is carried out in both the image and blur domains. The image regularization based on variational principles maintains stable performance under severe noise corruption. The blur regularization guarantees consistency of the solution by exploiting differences among the acquired low-resolution images. Several experiments on synthetic and real data illustrate the robustness and utilization of the proposed technique in real applications.

Informative frame classification for endoscopy video

Advances in video technology allow inspection, diagnosis and treatment of the inside of the human body without or with very small scars. Flexible endoscopes are used to inspect the esophagus, stomach, small bowel, colon, and airways, whereas rigid endoscopes are used for a variety of minimal invasive surgeries (i.e., laparoscopy, arthroscopy, endoscopic neurosurgery). These endoscopes come in various sizes, but all have a tiny video camera at the tip. During an endoscopic procedure, the tiny video camera generates a video signal of the interior of the human organ, which is displayed on a monitor for real-time analysis by the physician. However, many out-of-focus frames are present in endoscopy videos because current endoscopes are equipped with a single, wide-angle lens that cannot be focused. We need to distinguish the out-of-focus frames from the in-focus frames to utilize the information of the out-of-focus and/or the in-focus frames for further automatic or semi-automatic computer-aided diagnosis (CAD). This classification can reduce the number of images to be viewed by a physician and to be analyzed by a CAD system. We call an out-of-focus frame a non-informative frame and an in-focus frame an informative frame. The out-of-focus frames have characteristics that are different from those of in-focus frames. In this paper, we propose two new techniques (edge-based and clustering-based) to classify video frames into two classes, informative and non-informative frames. However, because intensive specular reflections reduce the accuracy of the classification we also propose a specular reflection detection technique, and use the detected specular reflection information to increase the accuracy of informative frame classification. Our experimental studies indicate that precision, sensitivity, specificity, and accuracy for the specular reflection detection technique and the two informative frame classification techniques are greater than 90% and 95%, respectively.

Novel cooperative neural fusion algorithms for image restoration and image fusion

To deal with the problem of restoring degraded images with non-Gaussian noise, this paper proposes a novel cooperative neural fusion regularization (CNFR) algorithm for image restoration. Compared with conventional regularization algorithms for image restoration, the proposed CNFR algorithm can relax need of the optimal regularization parameter to be estimated. Furthermore, to enhance the quality of restored images, this paper presents a cooperative neural fusion (CNF) algorithm for image fusion. Compared with existing signal-level image fusion algorithms, the proposed CNF algorithm can greatly reduce the loss of contrast information under blind Gaussian noise environments. The performance analysis shows that the proposed two neural fusion algorithms can converge globally to the robust and optimal image estimate. Simulation results confirm that in different noise environments, the proposed two neural fusion algorithms can obtain a better image estimate than several well known image restoration and image fusion methods.

Multidimensional multichannel FIR deconvolution using Gröbner bases

We present a new method for general multidimensional multichannel deconvolution with finite impulse response (FIR) convolution and deconvolution filters using Gröbner bases. Previous work formulates the problem of multichannel FIR deconvolution as the construction of a left inverse of the convolution matrix, which is solved by numerical linear algebra. However, this approach requires the prior information of the support of deconvolution filters. Using algebraic geometry and Gröbner bases, we find necessary and sufficient conditions for the existence of exact deconvolution FIR filters and propose simple algorithms to find these deconvolution filters. The main contribution of our work is to extend the previous Gröbner basis results on multidimensional multichannel deconvolution for polynomial or causal filters to general FIR filters. The proposed algorithms obtain a set of FIR deconvolution filters with a small number of nonzero coefficients (a desirable feature in the impulsive noise environment) and do not require the prior information of the support. Moreover, we provide a complete characterization of all exact deconvolution FIR filters, from which good FIR deconvolution filters under the additive white noise environment are found. Simulation results show that our approaches achieve good results under different noise settings.

Multiframe blind deconvolution of heavily blurred astronomical images

A multichannel blind deconvolution algorithm that incorporates the maximum-likelihood image restoration by several estimates of the differently blurred point-spread function (PSF) into the Ayers-Dainty iterative algorithm is proposed. The algorithm uses no restrictions on the image and the PSFs except for the assumption that they are positive. The algorithm employs no cost functions, input parameters, a priori probability distributions, or the analytically specified transfer functions. The iterative algorithm permits its application in the presence of different kinds of distortion. The work presents results of digital modeling and the results of processing real telescope data from several satellites. The proof of convergence of the algorithm to the positive estimates of object and the PSFs is given. The convergence of the Ayers-Dainty algorithm with a single processed frame is not obvious in the general case; therefore it is useful to have confidence in its convergence in a multiframe case. The dependence of convergence on the number of processed frames is discussed. Formulas for evaluating the quality of the algorithm performance on each iteration and the rule of stopping its work in accordance with this quality are proposed. A method of building the monotonically converging subsequence of the image estimates of all the images obtained in the iterative process is also proposed.

Reconstructing arbitrarily focused images from two differently focused images using linear filters

We present a novel filtering method for reconstructing an all-in-focus image or an arbitrarily focused image from two images that are focused differently. The method can arbitrarily manipulate the degree of blur of the objects using linear filters without segmentation. The filters are uniquely determined from a linear imaging model in the Fourier domain. An effective and accurate blur estimation method is developed. The simulation results show that the accuracy and computational time of the proposed method are improved compared with the previous iterative method and that the effects of blur estimation error on the quality of the reconstructed image are very small. The method performs well for real images acquired without visible artifacts.

Multichannel blind deconvolution of spatially misaligned images

Existing multichannel blind restoration techniques assume perfect spatial alignment of channels, correct estimation of blur size, and are prone to noise. We developed an alternating minimization scheme based on a maximum a posteriori estimation with a priori distribution of blurs derived from the multichannel framework and a priori distribution of original images defined by the variational integral. This stochastic approach enables us to recover the blurs and the original image from channels severely corrupted by noise. We observe that the exact knowledge of the blur size is not necessary, and we prove that translation misregistration up to a certain extent can be automatically removed in the restoration process.

A soft double regularization approach to parametric blind image deconvolution

This paper proposes a blind image deconvolution scheme based on soft integration of parametric blur structures. Conventional blind image deconvolution methods encounter a difficult dilemma of either imposing stringent and inflexible preconditions on the problem formulation or experiencing poor restoration results due to lack of information. This paper attempts to address this issue by assessing the relevance of parametric blur information, and incorporating the knowledge into the parametric double regularization (PDR) scheme. The PDR method assumes that the actual blur satisfies up to a certain degree of parametric structure, as there are many well-known parametric blurs in practical applications. Further, it can be tailored flexibly to include other blur types if some prior parametric knowledge of the blur is available. A manifold soft parametric modeling technique is proposed to generate the blur manifolds, and estimate the fuzzy blur structure. The PDR scheme involves the development of the meaningful cost function, the estimation of blur support and structure, and the optimization of the cost function. Experimental results show that it is effective in restoring degraded images under different environments.

Multichannel blind iterative image restoration

Blind image deconvolution is required in many applications of microscopy imaging, remote sensing, and astronomical imaging. Unfortunately in a single-channel framework, serious conceptual and numerical problems are often encountered. Very recently, an eigenvector-based method (EVAM) was proposed for a multichannel framework which determines perfectly convolution masks in a noise-free environment if channel disparity, called co-primeness, is satisfied. We propose a novel iterative algorithm based on recent anisotropic denoising techniques of total variation and a Mumford-Shah functional with the EVAM restoration condition included. A linearization scheme of half-quadratic regularization together with a cell-centered finite difference discretization scheme is used in the algorithm and provides a unified approach to the solution of total variation or Mumford-Shah. The algorithm performs well even on very noisy images and does not require an exact estimation of mask orders. We demonstrate capabilities of the algorithm on synthetic data. Finally, the algorithm is applied to defocused images taken with a digital camera and to data from astronomical ground-based observations of the Sun.

Multichannel blind image deconvolution using the Bussgang algorithm: spatial and multiresolution approaches

This work extends the Bussgang blind equalization algorithm to the multichannel case with application to image deconvolution problems. We address the restoration of images with poor spatial correlation as well as strongly correlated (natural) images. The spatial nonlinearity employed in the final estimation step of the Bussgang algorithm is developed according to the minimum mean square error criterion in the case of spatially uncorrelated images. For spatially correlated images, the nonlinearity design is rather conducted using a particular wavelet decomposition that, detecting lines, edges, and higher order structures, carries out a task analogous to those of the (preattentive) stage of the human visual system. Experimental results pertaining to restoration of motion blurred text images, out-of-focus spiky images, and blurred natural images are reported.

On eigenstructure-based direct multichannel blind image restoration

Existing eigenstructure-based direct multichannel blind image restoration techniques include nullspace-based and direct deconvolver estimation techniques. The nullspace-based approach can be formulated as an optimization problem. We show that this formulation implies a new subspace-based approach that uses matrix operations. This new approach has the same advantages as the nullspace-based one but requires less computational complexity. Under some mild conditions, its complexity is equal to that of the FFT. Furthermore, the relation among the nullspace-based approach, the direct deconvolver estimation and the new subspace-based approach is studied.

Perfect blind restoration of images blurred by multiple filters: theory and efficient algorithms

We address the problem of restoring an image from its noisy convolutions with two or more unknown finite impulse response (FIR) filters. We develop theoretical results about the existence and uniqueness of solutions, and show that under some generically true assumptions, both the filters and the image can be determined exactly in the absence of noise, and stably estimated in its presence. We present efficient algorithms to estimate the blur functions and their sizes. These algorithms are of two types, subspace-based and likelihood-based, and are extensions of techniques proposed for the solution of the multichannel blind deconvolution problem in one dimension. We present memory and computation-efficient techniques to handle the very large matrices arising in the two-dimensional (2-D) case. Once the blur functions are determined, they are used in a multichannel deconvolution step to reconstruct the unknown image. The theoretical and practical implications of edge effects, and "weakly exciting" images are examined. Finally, the algorithms are demonstrated on synthetic and real data.