Video deblurring algorithm using accurate blur kernel estimation and residual deconvolution based on a blurred-unblurred frame pair

Dual Prior Learning for Blind and Blended Image Restoration

Unsupervised single image restoration approach, Deep Image Prior (DIP), aims to restore images by learning enough raw image statistic priors from the corrupted observation. However, it is not uncommon that an image is contaminated by the multiple unknown distortions. Thus it is hard to disentangle the clean and the hybrid distortion signals by solely relying on image prior learning to restore the images. To overcome this problem, we propose the Dual Prior Learning (DPL) method by taking both image and distortion priors into account. DPL goes beyond DIP by considering an additional step to explicitly learn the blended distortion prior. Furthermore, to coordinate the learning of two priors and avoid them learning the same knowledge, we exploit unpaired training data to enforce a weakly supervision in an adversarial manner to encourage disentangling two priors. Extensive experiments show the effectiveness and appealing performance of the proposed DPL on restoring images with challenging unknown blended distortions.

Combining Motion Compensation with Spatiotemporal Constraint for Video Deblurring

We propose a video deblurring method by combining motion compensation with spatiotemporal constraint for restoring blurry video caused by camera shake. The proposed method makes effective full use of the spatiotemporal information not only in the blur kernel estimation, but also in the latent sharp frame restoration. Firstly, we estimate a motion vector between the current and the previous blurred frames, and introduce the estimated motion vector for deriving the motion-compensated frame with the previous restored frame. Secondly, we proposed a blur kernel estimation strategy by applying the derived motion-compensated frame to an improved regularization model for improving the quality of the estimated blur kernel and reducing the processing time. Thirdly, we propose a spatiotemporal constraint algorithm that can not only enhance temporal consistency, but also suppress noise and ringing artifacts of the deblurred video through introducing a temporal regularization term. Finally, we extend Fast Total Variation de-convolution (FTVd) for solving the minimization problem of the proposed spatiotemporal constraint energy function. Extensive experiments demonstrate that the proposed method achieve the state-of-the-art results either in subjective vision or objective evaluation.

Temporal Coherence-Based Deblurring Using Non-Uniform Motion Optimization

Non-uniform motion blur due to object movement or camera jitter is a common phenomenon in videos. However, the state-of-the-art video deblurring methods used to deal with this problem can introduce artifacts, and may sometimes fail to handle motion blur due to the movements of the object or the camera. In this paper, we propose a non-uniform motion model to deblur video frames. The proposed method is based on superpixel matching in the video sequence to reconstruct sharp frames from blurry ones. To identify a suitable sharp superpixel to replace a blurry one, we enrich the search space with a non-uniform motion blur kernel, and use a generalized PatchMatch algorithm to handle rotation, scale, and blur differences in the matching step. Instead of using pixel-based or regular patch-based representation, we adopt a superpixel-based representation, and use color and motion to gather similar pixels. Our non-uniform motion blur kernels are estimated from the motion field of these superpixels, and our spatially varying motion model considers spatial and temporal coherence to find sharp superpixels. Experimental results showed that the proposed method can reconstruct sharp video frames from blurred frames caused by complex object and camera movements, and performs better than the state-of-the-art methods.