Understanding gloss perception through the lens of art: Combining perception, image analysis, and painting recipes of 17th century painted grapes

Material perception across different media-comparing perceived attributes in oil paintings and engravings

We investigated the influence of the medium on the perception of depicted objects and materials. Oil paintings and their reproductions in engravings were chosen because they are vastly distinctive media while having completely identical content. A total of 15 pairs were collected, consisting of 88 fragments depicting different materials, including fabric, skin, wood and metal. Besides the original condition, we created three manipulations to understand the effect of colour (a greyscale version) and contrast (equalised histograms towards both painting and engraving). We performed rating experiments on five attributes: three-dimensionality, glossiness, convincingness, smoothness and softness. An average of 25 participants finished each of the 20 online experimental sessions (five attributes X four conditions). Besides clear correlations between the two media, the differences mainly show in their means (different levels of perceived attributes) and standard deviations (perceived range). In most sessions, paintings depict a wider range than engravings. In addition, it was the histogram equalisation (global contrast) that made the most impact on perceived attributes, rather than colour removal. This suggests that engravers compensated for the lack of colour by exploiting the possibilities of local contrast.

Relationship between reflectance and degree of polarization in the VNIR-SWIR: A case study on art paintings with polarimetric reflectance imaging spectroscopy

We study the relationship between reflectance and the degree of linear polarization of radiation that bounces off the surface of an unvarnished oil painting. We design a VNIR-SWIR (400 nm to 2500 nm) polarimetric reflectance imaging spectroscopy setup that deploys unpolarized light and allows us to estimate the Stokes vector at the pixel level. We observe a strong negative correlation between the S0 component of the Stokes vector (which can be used to represent the reflectance) and the degree of linear polarization in the visible interval (average -0.81), while the correlation is weaker and varying in the infrared range (average -0.50 in the NIR range between 780 and 1500 nm, and average -0.87 in the SWIR range between 1500 and 2500 nm). By tackling the problem with multi-resolution image analysis, we observe a dependence of the correlation on the local complexity of the surface. Indeed, we observe a general trend that strengthens the negative correlation for the effect of artificial flattening provoked by low image resolutions.

A framework connecting vision and depiction

We present a framework that connects ideas from the visual arts and visual perception. It adapts two existing frameworks for the analysis of form and content so that it can be used in an educational context for teaching perception through visual arts. The basis is the formal analysis of texture, colour, light, space, and material. This analysis can be conducted both on the medium and the motif, which adds a second level in addition to the formal level. Thirdly, a conte(n/x)t level is discussed which combines a basic notion of semiotics and iconography. We share our experience of implementing pictorial analysis in design and perception education and discuss how the framework is used both in a quantitative and a qualitative fashion. Next to education, the framework provides a basis for further pictorial research.

Effect of material properties on emotion: a virtual reality study

Introduction

Designers know that part of the appreciation of a product comes from the properties of its materials. These materials define the object's appearance and produce emotional reactions that can influence the act of purchase. Although known and observed as important, the affective level of a material remains difficult to assess. While many studies have been conducted regarding material colors, here we focus on two material properties that drive how light is reflected by the object: its metalness and smoothness. In this context, this work aims to study the influence of these properties on the induced emotional response.

Method

We conducted a perceptual user study in virtual reality, allowing participants to visualize and manipulate a neutral object - a mug. We generated 16 material effects by varying it metalness and smoothness characteristics. The emotional reactions produced by the 16 mugs were evaluated on a panel of 29 people using James Russel's circumplex model, for an emotional measurement through two dimensions: arousal (from low to high) and valence (from negative to positive). This scale, used here through VR users' declarative statements allowed us to order their emotional preferences between all the virtual mugs.

Result

Statistical results show significant positive effects of both metalness and smoothness on arousal and valence. Using image processing features, we show that this positive effect is linked to the increasing strength (i.e., sharpness and contrast) of the specular reflections induced by these material properties.

Discussion

The present work is the first to establish this strong relationship between specular reflections induced by material properties and aroused emotions.

Time-of-day perception in paintings

The spectral shape, irradiance, direction, and diffuseness of daylight vary regularly throughout the day. The variations in illumination and their effect on the light reflected from objects may in turn provide visual information as to the time of day. We suggest that artists' color choices for paintings of outdoor scenes might convey this information and that therefore the time of day might be decoded from the colors of paintings. Here we investigate whether human viewers' estimates of the depicted time of day in paintings correlate with their image statistics, specifically chromaticity and luminance variations. We tested time-of-day perception in 17th- to 20th-century Western European paintings via two online rating experiments. In Experiment 1, viewers' ratings from seven time choices varied significantly and largely consistently across paintings but with some ambiguity between morning and evening depictions. Analysis of the relationship between image statistics and ratings revealed correlations with the perceived time of day: higher "morningness" ratings associated with higher brightness, contrast, and saturation and darker yellow/brighter blue hues; "eveningness" with lower brightness, contrast, and saturation and darker blue/brighter yellow hues. Multiple linear regressions of extracted principal components yielded a predictive model that explained 76% of the variance in time-of-day perception. In Experiment 2, viewers rated paintings as morning or evening only; rating distributions differed significantly across paintings, and image statistics predicted people's perceptions. These results suggest that artists used different color palettes and patterns to depict different times of day, and the human visual system holds consistent assumptions about the variation of natural light depicted in paintings.

The role of texture summary statistics in material recognition from drawings and photographs

Material depictions in artwork are useful tools for revealing image features that support material categorization. For example, artistic recipes for drawing specific materials make explicit the critical information leading to recognizable material properties (Di Cicco, Wjintjes, & Pont, 2020) and investigating the recognizability of material renderings as a function of their visual features supports conclusions about the vocabulary of material perception. Here, we examined how the recognition of materials from photographs and drawings was affected by the application of the Portilla-Simoncelli texture synthesis model. This manipulation allowed us to examine how categorization may be affected differently across materials and image formats when only summary statistic information about appearance was retained. Further, we compared human performance to the categorization accuracy obtained from a pretrained deep convolutional neural network to determine if observers' performance was reflected in the network. Although we found some similarities between human and network performance for photographic images, the results obtained from drawings differed substantially. Our results demonstrate that texture statistics play a variable role in material categorization across rendering formats and material categories and that the human perception of material drawings is not effectively captured by deep convolutional neural networks trained for object recognition.

Context effects on the perception of saturation of fruit colors in still-life paintings

Still-life painters, especially of the so-called Golden Age (17th century) in the Netherlands, are famous for their masterful techniques of rendering reality. Their amazing abilities to depict different material properties of fruits and flowers are stunning. But how important are these careful arrangements of different objects for the perception of an individual item? Is the perceived color saturation of a single fruit influenced by its surrounding context? We selected fruits in still-life paintings as stimuli to investigate whether and how perceived saturations of fruits were affected by their original contexts. In our study, we focused especially on effects of five context properties: complementary colors, chromatic and luminance contrast, object overlap, and surround variance. Six fruit varieties depicted in high-quality digital reproductions of 48 classic and eight varieties in 64 more recent, modern still-life paintings were selected. In a single trial, eight images of fruits of the same variety appeared on a neutral gray background; half were single fruit cutouts, and the other half were the same fruits embedded in their circular contexts. Fifteen participants ranked all eight images according to perceived color saturations of the fruits. Saturation ratings showed a high agreement of 77%. Surrounding contexts led to an increase in perceived saturation of central fruits. This effect was mainly driven by object overlap, the presence of the central fruit type also in the context, and surround variance. Chroma contrast between fruits and contexts decreased saturation significantly. No significant context effects were found for complementary colors or luminance contrast. Our results show that in paintings, many of the cues that are usually experimentally isolated occur in interesting combinations and lead to an increase in perceived saturation that makes fruit objects more appealing and convincing.

Material category of visual objects computed from specular image structure

Recognizing materials and their properties visually is vital for successful interactions with our environment, from avoiding slippery floors to handling fragile objects. Yet there is no simple mapping of retinal image intensities to physical properties. Here, we investigated what image information drives material perception by collecting human psychophysical judgements about complex glossy objects. Variations in specular image structure-produced either by manipulating reflectance properties or visual features directly-caused categorical shifts in material appearance, suggesting that specular reflections provide diagnostic information about a wide range of material classes. Perceived material category appeared to mediate cues for surface gloss, providing evidence against a purely feedforward view of neural processing. Our results suggest that the image structure that triggers our perception of surface gloss plays a direct role in visual categorization, and that the perception and neural processing of stimulus properties should be studied in the context of recognition, not in isolation.

Perceived roughness of glossy objects: The influence of Fresnel effects and correlated image statistics

The roughness of a shiny surface determines how sharp the reflected image of the surroundings is, and thus whether the surface appears highly glossy or more or less matte. In a matching experiment, subjects were asked to reproduce the perceived roughness of a given surface (standard) in a comparison stimulus (match), where the standard and the match could differ in both shape and illumination. To compare the effect of the reflection model on the accuracy of the settings, this was done for two different reflectance models (bidirectional reflectance distribution function [BRDF]). The matching errors were smaller, that is, the constancy under shape and illumination changes higher, when Fresnel effects were physically correctly reproduced in the reflectance model (Fresnel-BRDF) than when this was not the case (Ward-BRDF). The subjects' settings in the experiment can be predicted very well by two image statistics, one of which is based on the mean edge strength and the other on a local discrete cosine transform. In particular, these predictions also reflect the empirically observed advantage of the Fresnel-BRDF. These results show that the constancy of perceived roughness across context changes may depend on the BRDF used, with Fresnel effects playing a significant role. The good prediction of subjects' settings using the two image statistics suggests that local brightness variance, which affects both image statistics, can be used as a valid cue for surface roughness.

Perception of material appearance: A comparison between painted and rendered images

Painters are masters in replicating the visual appearance of materials. While the perception of material appearance is not yet fully understood, painters seem to have acquired an implicit understanding of the key visual cues that we need to accurately perceive material properties. In this study, we directly compare the perception of material properties in paintings and in renderings by collecting professional realistic paintings of rendered materials. From both type of images, we collect human judgments of material properties and compute a variety of image features that are known to reflect material properties. Our study reveals that, despite important visual differences between the two types of depiction, material properties in paintings and renderings are perceived very similarly and are linked to the same image features. This suggests that we use similar visual cues independently of the medium and that the presence of such cues is sufficient to provide a good appearance perception of the materials.

If painters give you lemons, squeeze the knowledge out of them. A study on the visual perception of the translucent and juicy appearance of citrus fruits in paintings

Citrus fruits are characterized by a juicy and translucent interior, important properties that drive material recognition and food acceptance. Yet, a thorough understanding of their visual perception is still missing. Using citrus fruits depicted in 17th-century paintings as stimuli, we ran three rating experiments. In Experiment 1, participants rated the perceived similarity in translucency or juiciness of the fruits. In Experiment 2, different groups of participants rated one image feature from a list obtained in a preliminary experiment. In Experiment 3, translucency and juiciness were rated. We constructed two-dimensional perceptual spaces for both material properties and fitted the ratings of the image features into the spaces to interpret them. “Highlights,” “peeled side,” “bumpiness,” and “color saturation” fit the juiciness space best and were high for the highly juicy stimuli. “Peeled side,” “intensity of light gradient,” “highlights,” and “color saturation” were the most salient features of the translucency space, being high for the highly translucent stimuli. The same image features were also indicated in a 17th-century painting manual for material depiction (Beurs, 1692; Beurs, in press). Altogether, we disclosed the expertise of painters with regard to material perception by identifying the image features that trigger a visual impression of juiciness and translucency in citrus fruits.

Soft like velvet and shiny like satin: Perceptual material signatures of fabrics depicted in 17th century paintings

Dutch 17th century painters were masters in depicting materials and their properties in a convincing way. Here, we studied the perception of the material signatures and key image features of different depicted fabrics, like satin and velvet. We also tested whether the perception of fabrics depicted in paintings related to local or global cues, by cropping the stimuli. In Experiment 1, roughness, warmth, softness, heaviness, hairiness, and shininess were rated for the stimuli shown either full figure or cropped. In the full figure, all attributes except shininess were rated higher for velvet, whereas shininess was rated higher for satin. This distinction was less clear in the cropped condition, and some properties were perceived significantly different between the two conditions. In Experiment 2 we tested whether this difference was due to the choice of the cropped area. On the basis of the results of Experiment 1, shininess and softness were rated for multiple crops from each fabric. Most crops from the same fabric differed significantly in shininess, but not in softness perception. Perceived shininess correlated positively with the mean luminance of the crops and the highlights' coverage. Experiment 1 showed that painted velvet and satin triggered distinct perceptions, indicative of robust material signatures of the two fabrics. The results of Experiment 2 suggest that the presence of local image cues affects the perception of optical properties like shininess, but not mechanical properties such as softness.

Spatial Frequency Effective for Increasing Perceived Glossiness by Contrast Enhancement

It has been suggested that luminance edges in retinal images are potential cues for glossiness perception, particularly when the perception relies on low-luminance specular regions. However, a previous study has shown only statistical correlations between luminance edges and perceived glossiness, not their causal relations. Additionally, although specular components should be embedded at various spatial frequencies depending on the micro-roughness on the object surface, it is not well understood what spatial frequencies are essential for glossiness perception on objects with different micro-roughness. To address these issues, we examined the impact of a sub-band contrast enhancement on the perceived glossiness in the two conditions of stimuli: the Full condition where the stimulus had natural specular components and the Dark condition where it had specular components only in dark regions. Object images with various degrees of surface roughness were generated as stimuli, and their contrast was increased in various spatial-frequency sub-bands. The results indicate that the enhancement of the sub-band contrast can significantly increase perceived glossiness as expected. Furthermore, the effectiveness of each spatial frequency band depends on the surface roughness in the Full condition. However, effective spatial frequencies are constant at a middle spatial frequency regardless of the stimulus surface roughness in the Dark condition. These results suggest that, for glossiness perception, our visual system depends on specular-related information embedded in high spatial frequency components but may change the dependency on spatial frequency based on the surface luminance to be judged.

Painterly depiction of material properties

Painters are masters of depiction and have learned to evoke a clear perception of materials and material attributes in a natural, three-dimensional setting, with complex lighting conditions. Furthermore, painters are not constrained by reality, meaning that they could paint materials without exactly following the laws of nature, while still evoking the perception of materials. Paintings have to our knowledge not been studied on a big scale from a material perception perspective. In this article, we studied the perception of painted materials and their attributes by using human annotations to find instances of 15 materials, such as wood, stone, fabric, etc. Participants made perceptual judgments about 30 unique segments of these materials for 10 material attributes, such as glossiness, roughness, hardness, etc. We found that participants were able to perform this task well while being highly consistent. Participants, however, did not consistently agree with each other, and the measure of consistency depended on the material attribute being perceived. Additionally, we found that material perception appears to function independently of the medium of depiction—the results of our principal component analysis agreed well with findings in former studies for photographs and computer renderings.

The influence of Fresnel effects on gloss perception

Glossy surfaces reflect a mirror image of the environment. The perceived gloss depends (a) on the blurriness of this mirror image, which is a function of surface roughness, and (b) the strength of the mirror reflection, which, according to Fresnel's equations, is a function of the material's refractive index and the angle of the incident light. Investigations on gloss perception often used simplified reflection models, e.g., the Ward model (Ward, 1992), which do not correctly account for Fresnel effects. Here, possible perceptual consequences of this simplification are investigated in three experiments, in which the gloss impression produced by a physically more plausible reflection model (Fresnel-bidirectional reflectance distribution function [BRDF]) is compared to the gloss produced by two variants of the Ward model under identical conditions. The results show that it is, in general, not possible to match the gloss impression elicited by a Fresnel-BRDF with a Ward-BRDF. Furthermore, compared with the Ward-BRDF, the gloss impression produced with the Fresnel-BRDF under identical conditions is, in general, stronger, more vivid, and more realistic. Gloss constancy is also improved, i.e., the gloss impression depends less on the type of illumination, the presence and properties of a floor, and surface shape. These differences are especially evident with relatively homogeneous illuminations. The results of a fourth experiment, which tested gloss constancy under changes in illumination and shape with a matching task, confirm an improved gloss constancy with a Fresnel-BRDF. Together, these findings suggest that Fresnel effects are used as a cue in gloss perception.

Factors Influencing the Detection of Spatially-Varying Surface Gloss

In this study, we investigate the ability of human observers to detect spatial inhomogeneities in the glossiness of a surface and how the performance in this task depends on several context factors. We used computer-generated stimuli showing a single object in three-dimensional space whose surface was split into two spatial areas with different microscale smoothness. The context factors were the kind of illumination, the object's shape, the availability of motion information, the degree of edge blurring, the spatial proportions between the two areas of different smoothness, and the general smoothness level. Detection thresholds were determined using a two-alternative forced choice (2AFC) task implemented in a double random staircase procedure, where the subjects had to indicate for each stimulus whether or not the surface appears to have a spatially uniform material. We found evidence that two different cues are used for this task: luminance differences and differences in highlight properties between areas of different microscale smoothness. While the visual system seems to be highly sensitive in detecting gloss differences based on luminance contrast information, detection thresholds were considerably higher when the judgment was mainly based on differences in highlight features, such as their size, intensity, and sharpness.