Single Image Super-Resolution Based on Wiener Filter in Similarity Domain

A Virtual-Sensor Construction Network Based on Physical Imaging for Image Super-Resolution

Image imaging in the real world is based on physical imaging mechanisms. Existing super-resolution methods mainly focus on designing complex network structures to extract and fuse image features more effectively, but ignore the guiding role of physical imaging mechanisms for model design, and cannot mine features from a physical perspective. Inspired by the mechanism of physical imaging, we propose a novel network architecture called Virtual-Sensor Construction network (VSCNet) to simulate the sensor array inside the camera. Specifically, VSCNet first generates different splitting directions to distribute photons to construct virtual sensors, and then performs a multi-stage adaptive fine-tuning operation to fine-tune the number of photons on the virtual sensors to increase the photosensitive area and eliminate photon cross-talk, and finally converts the obtained photon distributions into RGB images. These operations can naturally be regarded as the virtual expansion of the camera's sensor array in the feature space, which makes our VSCNet bridge the physical space and feature space, and uses their complementarity to mine more effective features to improve performance. Extensive experiments on various datasets show that the proposed VSCNet achieves state-of-the-art performance with fewer parameters. Moreover, we perform experiments to validate the connection between the proposed VSCNet and the physical imaging mechanism. The implementation code is available at https://github.com/GZ-T/VSCNet.

Recovering Realistic Details for Magnification-Arbitrary Image Super-Resolution

The emergence of implicit neural representations (INR) has shown the potential to represent images in a continuous form by mapping pixel coordinates to RGB values. Recent work is capable of recovering arbitrary-resolution images from the continuous representations of the input low-resolution (LR) images. However, it can only super-resolve blurry images and lacks the ability to generate perceptual-pleasant details. In this paper, we propose implicit pixel flow (IPF) to model the coordinate dependency between the blurry INR distribution and the sharp real-world distribution. For each pixel near the blurry edges, IPF assigns offsets for the coordinates of the pixel so that the original RGB values can be replaced by the RGB values of a neighboring pixel which are more appropriate to form sharper edges. By modifying the relationship between the INR-domain coordinates and the image-domain pixels via IPF, we convert the original blurry INR distribution to a sharp one. Specifically, we adopt convolutional neural networks to extract continuous flow representations and employ multi-layer perceptrons to build the implicit function for calculating pixel flow. In addition, we propose a new double constraint module to search for more stable and optimal pixel flows during training. To the best of our knowledge, this is the first method to recover perceptually-pleasant details for magnification-arbitrary single image super-resolution. Experimental results on public benchmark datasets demonstrate that we successfully restore shape edges and satisfactory textures from continuous image representations.

Characterization model of silicon dioxide melting based On image analysis

The silicon dioxide is the hardest part to melt among the iron tailing components, the melting behavior of iron tailing can be represented by the melting behavior of silicon dioxide. Estimating the real-time melting rate of silicon dioxide in the time sequence provide guidance for the tailing addition and heat compensation in the process of slag cotton preparation, also indirectly improved the direct fiber forming technology of blast furnace slag. The position of silicon dioxide particles in the high-temperature molten pool during the melting process is changing constantly, using a strong weighted distance centroid algorithm to rack the centroid position of silicon dioxide particles during the melting process, and present the motion trail of centroid of silicon dioxide. In the paper, extracting indexes which represent the edge outline characteristics of silicon dioxide during the melting process of silicon dioxide using Snake active contour algorithm combined with Sobel operator, include shape, perimeter and area. Using the extracted skeleton characteristics, a three-dimensional skeleton generation model is created. From the skeleton data, estimating the volume of silicon dioxide and determine the parameter formula for the actual melting rate of silicon dioxide. The silicon dioxide melting rate at each moment is calculated by numerical simulation. The results of the Hough test circle and the silicon dioxide melting rate are verified. The rationality of the model is further determined.

Deep learning in electron microscopy

Deep learning is transforming most areas of science and technology, including electron microscopy. This review paper offers a practical perspective aimed at developers with limited familiarity. For context, we review popular applications of deep learning in electron microscopy. Following, we discuss hardware and software needed to get started with deep learning and interface with electron microscopes. We then review neural network components, popular architectures, and their optimization. Finally, we discuss future directions of deep learning in electron microscopy.

Reconstruction of Optical Coherence Tomography Images Using Mixed Low Rank Approximation and Second Order Tensor Based Total Variation Method

This paper proposes a mixed low-rank approximation and second-order tensor-based total variation (LRSOTTV) approach for the super-resolution and denoising of retinal optical coherence tomography (OCT) images through effective utilization of nonlocal spatial correlations and local smoothness properties. OCT imaging relies on interferometry, which explains why OCT images suffer from a high level of noise. In addition, data subsampling is conducted during OCT A-scan and B-scan acquisition. Therefore, using effective super-resolution algorithms is necessary for reconstructing high-resolution clean OCT images. In this paper, a low-rank regularization approach is proposed for exploiting nonlocal self-similarity prior to OCT image reconstruction. To benefit from the advantages of the redundancy of multi-slice OCT data, we construct a third-order tensor by extracting the nonlocal similar three-dimensional blocks and grouping them by applying the k-nearest-neighbor method. Next, the nuclear norm is used as a regularization term to shrink the singular values of the constructed tensor in the non-local correlation direction. Further, the regularization approaches of the first-order tensor-based total variation (FOTTV) and SOTTV are proposed for better preservation of retinal layers and suppression of artifacts in OCT images. The alternative direction method of multipliers (ADMM) technique is then used to solve the resulting optimization problem. Our experiments show that integrating SOTTV instead of FOTTV into a low-rank approximation model can achieve noticeably improved results. Our experimental results on the denoising and super-resolution of OCT images demonstrate that the proposed model can provide images whose numerical and visual qualities are higher than those obtained by using state-of-the-art methods.

Heavy-Tailed Self-Similarity Modeling for Single Image Super Resolution

Self-similarity is a prominent characteristic of natural images that can play a major role when it comes to their denoising, restoration or compression. In this paper, we propose a novel probabilistic model that is based on the concept of image patch similarity and applied to the problem of Single Image Super Resolution. Based on this model, we derive a Variational Bayes algorithm, which super resolves low-resolution images, where the assumed distribution for the quantified similarity between two image patches is heavy-tailed. Moreover, we prove mathematically that the proposed algorithm is both an extended and superior version of the probabilistic Non-Local Means (NLM). Its prime advantage remains though, which is that it requires no training. A comparison of the proposed approach with state-of-the-art methods, using various quantitative metrics shows that it is almost on par, for images depicting rural themes and in terms of the Structural Similarity Index (SSIM) with the best performing methods that rely on trained deep learning models. On the other hand, it is clearly inferior to them, for urban themed images and in terms of all metrics, especially for the Mean-Squared-Error (MSE). In addition, qualitative evaluation of the proposed approach is performed using the Perceptual Index metric, which has been introduced to better mimic the human perception of the image quality. This evaluation favors our approach when compared to the best performing method that requires no training, even if they perform equally in qualitative terms, reinforcing the argument that MSE is not always an accurate metric for image quality.

Fast Split Bregman Based Deconvolution Algorithm for Airborne Radar Imaging

Deconvolution methods can be used to improve the azimuth resolution in airborne radar imaging. Due to the sparsity of targets in airborne radar imaging, an L 1 regularization problem usually needs to be solved. Recently, the Split Bregman algorithm (SBA) has been widely used to solve L 1 regularization problems. However, due to the high computational complexity of matrix inversion, the efficiency of the traditional SBA is low, which seriously restricts its real-time performance in airborne radar imaging. To overcome this disadvantage, a fast split Bregman algorithm (FSBA) is proposed in this paper to achieve real-time imaging with an airborne radar. Firstly, under the regularization framework, the problem of azimuth resolution improvement can be converted into an L 1 regularization problem. Then, the L 1 regularization problem can be solved with the proposed FSBA. By utilizing the low displacement rank features of Toeplitz matrix, the proposed FSBA is able to realize fast matrix inversion by using a Gohberg–Semencul (GS) representation. Through simulated and real data processing experiments, we prove that the proposed FSBA significantly improves the resolution, compared with the Wiener filtering (WF), truncated singular value decomposition (TSVD), Tikhonov regularization (REGU), Richardson–Lucy (RL), iterative adaptive approach (IAA) algorithms. The computational advantage of FSBA increases with the increase of echo dimension. Its computational efficiency is 51 times and 77 times of the traditional SBA, respectively, for echoes with dimensions of 218 × 400 and 400 × 400 , optimizing both the image quality and computing time. In addition, for a specific hardware platform, the proposed FSBA can process echo of greater dimensions than traditional SBA. Furthermore, the proposed FSBA causes little performance degradation, when compared with the traditional SBA.

Multi-Branch Deep Residual Network for Single Image Super-Resolution

Recently, algorithms based on the deep neural networks and residual networks have been applied for super-resolution and exhibited excellent performance. In this paper, a multi-branch deep residual network for single image super-resolution (MRSR) is proposed. In the network, we adopt a multi-branch network framework and further optimize the structure of residual network. By using residual blocks and filters reasonably, the model size is greatly expanded while the stable training is also guaranteed. Besides, a perceptual evaluation function, which contains three parts of loss, is proposed. The experiment results show that the evaluation function provides great support for the quality of reconstruction and the competitive performance. The proposed method mainly uses three steps of feature extraction, mapping, and reconstruction to complete the super-resolution reconstruction and shows superior performance than other state-of-the-art super-resolution methods on benchmark datasets.

Optical coherence tomography retinal image reconstruction via nonlocal weighted sparse representation

We present a nonlocal weighted sparse representation (NWSR) method for reconstruction of retinal optical coherence tomography (OCT) images. To reconstruct a high signal-to-noise ratio and high-resolution OCT images, utilization of efficient denoising and interpolation algorithms are necessary, especially when the original data were subsampled during acquisition. However, the OCT images suffer from the presence of a high level of noise, which makes the estimation of sparse representations a difficult task. Thus, the proposed NWSR method merges sparse representations of multiple similar noisy and denoised patches to better estimate a sparse representation for each patch. First, the sparse representation of each patch is independently computed over an overcomplete dictionary, and then a nonlocal weighted sparse coefficient is computed by averaging representations of similar patches. Since the sparsity can reveal relevant information from noisy patches, combining noisy and denoised patches' representations is beneficial to obtain a more robust estimate of the unknown sparse representation. The denoised patches are obtained by applying an off-the-shelf image denoising method and our method provides an efficient way to exploit information from noisy and denoised patches' representations. The experimental results on denoising and interpolation of spectral domain OCT images demonstrated the effectiveness of the proposed NWSR method over existing state-of-the-art methods.