Towards a Full-Reference Quality Assessment for Color Images Using Directional Statistics

Superpixel-Based PSO Algorithms for Color Image Quantization

Nature-inspired artificial intelligence algorithms have been applied to color image quantization (CIQ) for some time. Among these algorithms, the particle swarm optimization algorithm (PSO-CIQ) and its numerous modifications are important in CIQ. In this article, the usefulness of such a modification, labeled IDE-PSO-CIQ and additionally using the idea of individual difference evolution based on the emotional states of particles, is tested. The superiority of this algorithm over the PSO-CIQ algorithm was demonstrated using a set of quality indices based on pixels, patches, and superpixels. Furthermore, both algorithms studied were applied to superpixel versions of quantized images, creating color palettes in much less time. A heuristic method was proposed to select the number of superpixels, depending on the size of the palette. The effectiveness of the proposed algorithms was experimentally verified on a set of benchmark color images. The results obtained from the computational experiments indicate a multiple reduction in computation time for the superpixel methods while maintaining the high quality of the output quantized images, slightly inferior to that obtained with the pixel methods.

Efficient Color Quantization Using Superpixels

We propose three methods for the color quantization of superpixel images. Prior to the application of each method, the target image is first segmented into a finite number of superpixels by grouping the pixels that are similar in color. The color of a superpixel is given by the arithmetic mean of the colors of all constituent pixels. Following this, the superpixels are quantized using common splitting or clustering methods, such as median cut, k-means, and fuzzy c-means. In this manner, a color palette is generated while the original pixel image undergoes color mapping. The effectiveness of each proposed superpixel method is validated via experimentation using different color images. We compare the proposed methods with state-of-the-art color quantization methods. The results show significantly decreased computation time along with high quality of the quantized images. However, a multi-index evaluation process shows that the image quality is slightly worse than that obtained via pixel methods.

Image Quality Assessment Based on Three Features Fusion in Three Fusion Steps

The objective image quality assessment (IQA) method was developed to replace subjective observer image quality evaluations in various applications. A reliable full reference color IQA method that allows reference and distorted images to be compared in a symmetric way is designed via three fusion steps described in this article. The three fusion steps include luminance channels fusion, similarity maps fusion, and features fusion. A fusion weight coefficient is designed to fuse the luminance channels of input images as an enhancement operator for features. The extracted SR (spectral residual), gradient, and chrominance features, by means of symmetric calculations for the reference and distorted images, are conducted via similarity fusion processing. Then, based on the human visual system (HVS) characteristics of achromatic and chromatic information receiving, a features fusion map represents the weighted sum of three similarity fusion maps. Finally, a deviation pooling strategy is utilized to export the quality score after features fusion. The novel method is called the features fusion similarity index (FFS). Various experiments are carried out based on statistical evaluation criteria to optimize the parameters of FFS, after which the proposed method of FFS is compared with other state-of-the-art IQA methods using large-scale benchmark single distortion databases. The results show that FFS performs with higher consistency with respect to subjective scores in terms of prediction accuracy, e.g., the PLCC can achieve at least 0.9116 accuracy and at most 0.9774 accuracy for four databases. In addition, the average running time of FFS is 0.0657 s—a value representing a higher computational efficiency.

Demystifying MLOps and Presenting a Recipe for the Selection of Open-Source Tools

Nowadays, machine learning projects have become more and more relevant to various real-world use cases. The success of complex Neural Network models depends upon many factors, as the requirement for structured and machine learning-centric project development management arises. Due to the multitude of tools available for different operational phases, responsibilities and requirements become more and more unclear. In this work, Machine Learning Operations (MLOps) technologies and tools for every part of the overall project pipeline, as well as involved roles, are examined and clearly defined. With the focus on the inter-connectivity of specific tools and comparison by well-selected requirements of MLOps, model performance, input data, and system quality metrics are briefly discussed. By identifying aspects of machine learning, which can be reused from project to project, open-source tools which help in specific parts of the pipeline, and possible combinations, an overview of support in MLOps is given. Deep learning has revolutionized the field of Image processing, and building an automated machine learning workflow for object detection is of great interest for many organizations. For this, a simple MLOps workflow for object detection with images is portrayed.

Tensor Oriented No-Reference Light Field Image Quality Assessment

Light field image (LFI) quality assessment is becoming more and more important, which helps to better guide the acquisition, processing and application of immersive media. However, due to the inherent high dimensional characteristics of LFI, the LFI quality assessment turns into a multi-dimensional problem that requires consideration of the quality degradation in both spatial and angular dimensions. Therefore, we propose a novel Tensor oriented No-reference Light Field image Quality evaluator (Tensor-NLFQ) based on tensor theory. Specifically, since the LFI is regarded as a low-rank 4D tensor, the principal components of four oriented sub-aperture view stacks are obtained via Tucker decomposition. Then, the Principal Component Spatial Characteristic (PCSC) is designed to measure the spatial-dimensional quality of LFI considering its global naturalness and local frequency properties. Finally, the Tensor Angular Variation Index (TAVI) is proposed to measure angular consistency quality by analyzing the structural similarity distribution between the first principal component and each view in the view stack. Extensive experimental results on four publicly available LFI quality databases demonstrate that the proposed Tensor-NLFQ model outperforms state-of-the-art 2D, 3D, multi-view, and LFI quality assessment algorithms.

Fast Color Quantization by K-Means Clustering Combined with Image Sampling

Color image quantization has become an important operation often used in tasks of color image processing. There is a need for quantization methods that are fast and at the same time generating high quality quantized images. This paper presents such color quantization method based on downsampling of original image and K-Means clustering on a downsampled image. The nearest neighbor interpolation was used in the downsampling process and Wu’s algorithm was applied for deterministic initialization of K-Means. Comparisons with other methods based on a limited sample of pixels (coreset-based algorithm) showed an advantage of the proposed method. This method significantly accelerated the color quantization without noticeable loss of image quality. The experimental results obtained on 24 color images from the Kodak image dataset demonstrated the advantages of the proposed method. Three quality indices (MSE, DSCSI and HPSI) were used in the assessment process.

SPSIM: A Superpixel-Based Similarity Index for Full-Reference Image Quality Assessment

Full-reference image quality assessment algorithms usually perform comparisons of features extracted from square patches. These patches do not have any visual meanings. On the contrary, a superpixel is a set of image pixels that share similar visual characteristics and is thus perceptually meaningful. Features from superpixels may improve the performance of image quality assessment. Inspired by this, we propose a new superpixel-based similarity index by extracting perceptually meaningful features and revising similarity measures. The proposed method evaluates image quality on the basis of three measurements, namely, superpixel luminance similarity, superpixel chrominance similarity, and pixel gradient similarity. The first two measurements assess the overall visual impression on local images. The third measurement quantifies structural variations. The impact of superpixel-based regional gradient consistency on image quality is also analyzed. Distorted images showing high regional gradient consistency with the corresponding reference images are visually appreciated. Therefore, the three measurements are further revised by incorporating the regional gradient consistency into their computations. A weighting function that indicates superpixel-based texture complexity is utilized in the pooling stage to obtain the final quality score. Experiments on several benchmark databases demonstrate that the proposed method is competitive with the state-of-the-art metrics.

Toward a No-Reference Image Quality Assessment Using Statistics of Perceptual Color Descriptors

Analysis of the statistical properties of natural images has played a vital role in the design of no-reference (NR) image quality assessment (IQA) techniques. In this paper, we propose parametric models describing the general characteristics of chromatic data in natural images. They provide informative cues for quantifying visual discomfort caused by the presence of chromatic image distortions. The established models capture the correlation of chromatic data between spatially adjacent pixels by means of color invariance descriptors. The use of color invariance descriptors is inspired by their relevance to visual perception, since they provide less sensitive descriptions of image scenes against viewing geometry and illumination variations than luminances. In order to approximate the visual quality perception of chromatic distortions, we devise four parametric models derived from invariance descriptors representing independent aspects of color perception: 1) hue; 2) saturation; 3) opponent angle; and 4) spherical angle. The practical utility of the proposed models is examined by deploying them in our new general-purpose NR IQA metric. The metric initially estimates the parameters of the proposed chromatic models from an input image to constitute a collection of quality-aware features (QAF). Thereafter, a machine learning technique is applied to predict visual quality given a set of extracted QAFs. Experimentation performed on large-scale image databases demonstrates that the proposed metric correlates well with the provided subjective ratings of image quality over commonly encountered achromatic and chromatic distortions, indicating that it can be deployed on a wide variety of color image processing problems as a generalized IQA solution.