A Blind Stereoscopic Image Quality Evaluator with Segmented Stacked Autoencoders Considering The Whole Visual Perception Route

No-reference stereoscopic image quality assessment based on binocular collaboration

Stereoscopic images typically consist of left and right views along with depth information. Assessing the quality of stereoscopic/3D images (SIQA) is often more complex than that of 2D images due to scene disparities between the left and right views and the intricate process of fusion in binocular vision. To address the problem of quality prediction bias of multi-distortion images, we investigated the visual physiology and the processing of visual information by the primary visual cortex of the human brain and proposed a no-reference stereoscopic image quality evaluation method. The method mainly includes an innovative end-to-end NR-SIQA neural network with a picture patch generation algorithm. The algorithm generates a saliency map by fusing the left and right views and then guides the image cropping in the database based on the saliency map. The proposed models are validated and compared based on publicly available databases. The results show that the model and algorithm together outperform the state-of-the-art NR-SIQA metric in the LIVE 3D database and the WIVC 3D database, and have excellent results in the specific noise metric. The model generalization experiments demonstrate a certain degree of generality of our proposed model.

Information Visualization Based on Visual Transmission and Multimedia Data Fusion

With the rapid development of information technology, the application media of visual identity design has been greatly broadened, and the requirements of dynamic variability and interactivity of vision have become higher and higher. The introduction of data as an element has brought new semantic endowments to visual design. To solve the problems of noisy image reconstruction and many outliers in traditional art design, this paper uses an improved phase correlation algorithm to reconstruct multimedia video images. In addition, based on the visual characteristics of human eyes, a multi-feature fusion viewpoint image quality evaluation algorithm is proposed. The simulation results show that the method adopted in this paper improves the uneven image texture in art design, and its real effect is greatly enhanced.

HFFNet: hierarchical feature fusion network for blind binocular image quality prediction

Compared with monocular images, scene discrepancies between the left- and right-view images impose additional challenges on visual quality predictions in binocular images. Herein, we propose a hierarchical feature fusion network (HFFNet) for blind binocular image quality prediction that handles scene discrepancies and uses multilevel fusion features from the left- and right-view images to reflect distortions in binocular images. Specifically, a feature extraction network based on MobileNetV2 is used to determine the feature layers from distorted binocular images; then, low-level binocular fusion features (or middle-level and high-level binocular fusion features) are obtained by fusing the left and right low-level monocular features (or middle-level and high-level monocular features) using the feature gate module; further, three feature enhancement modules are used to enrich the information of the extracted features at different levels. Finally, the total feature maps obtained from the high-, middle-, and low-level fusion features are applied to a three-input feature fusion module for feature merging. Thus, the proposed HFFNet provides better results, to the best of our knowledge, than existing methods on two benchmark datasets.

Rich Structural Index for Stereoscopic Image Quality Assessment

The human visual system (HVS), affected by viewing distance when perceiving the stereo image information, is of great significance to study of stereoscopic image quality assessment. Many methods of stereoscopic image quality assessment do not have comprehensive consideration for human visual perception characteristics. In accordance with this, we propose a Rich Structural Index (RSI) for Stereoscopic Image objective Quality Assessment (SIQA) method based on multi-scale perception characteristics. To begin with, we put the stereo pair into the image pyramid based on Contrast Sensitivity Function (CSF) to obtain sensitive images of different resolution. Then, we obtain local Luminance and Structural Index (LSI) in a locally adaptive manner on gradient maps which consider the luminance masking and contrast masking. At the same time we use Singular Value Decomposition (SVD) to obtain the Sharpness and Intrinsic Structural Index (SISI) to effectively capture the changes introduced in the image (due to distortion). Meanwhile, considering the disparity edge structures, we use gradient cross-mapping algorithm to obtain Depth Texture Structural Index (DTSI). After that, we apply the standard deviation method for the above results to obtain contrast index of reference and distortion components. Finally, for the loss caused by the randomness of the parameters, we use Support Vector Machine Regression based on Genetic Algorithm (GA-SVR) training to obtain the final quality score. We conducted a comprehensive evaluation with state-of-the-art methods on four open databases. The experimental results show that the proposed method has stable performance and strong competitive advantage.