Reliable Top-Left Light Convention Starts With Early Renaissance: An Extensive Approach Comprising 10k Artworks

Time-of-day perception in paintings

The spectral shape, irradiance, direction, and diffuseness of daylight vary regularly throughout the day. The variations in illumination and their effect on the light reflected from objects may in turn provide visual information as to the time of day. We suggest that artists' color choices for paintings of outdoor scenes might convey this information and that therefore the time of day might be decoded from the colors of paintings. Here we investigate whether human viewers' estimates of the depicted time of day in paintings correlate with their image statistics, specifically chromaticity and luminance variations. We tested time-of-day perception in 17th- to 20th-century Western European paintings via two online rating experiments. In Experiment 1, viewers' ratings from seven time choices varied significantly and largely consistently across paintings but with some ambiguity between morning and evening depictions. Analysis of the relationship between image statistics and ratings revealed correlations with the perceived time of day: higher "morningness" ratings associated with higher brightness, contrast, and saturation and darker yellow/brighter blue hues; "eveningness" with lower brightness, contrast, and saturation and darker blue/brighter yellow hues. Multiple linear regressions of extracted principal components yielded a predictive model that explained 76% of the variance in time-of-day perception. In Experiment 2, viewers rated paintings as morning or evening only; rating distributions differed significantly across paintings, and image statistics predicted people's perceptions. These results suggest that artists used different color palettes and patterns to depict different times of day, and the human visual system holds consistent assumptions about the variation of natural light depicted in paintings.

Materials In Paintings (MIP): An interdisciplinary dataset for perception, art history, and computer vision

In this paper, we capture and explore the painterly depictions of materials to enable the study of depiction and perception of materials through the artists' eye. We annotated a dataset of 19k paintings with 200k+ bounding boxes from which polygon segments were automatically extracted. Each bounding box was assigned a coarse material label (e.g., fabric) and half was also assigned a fine-grained label (e.g., velvety, silky). The dataset in its entirety is available for browsing and downloading at materialsinpaintings.tudelft.nl. We demonstrate the cross-disciplinary utility of our dataset by presenting novel findings across human perception, art history and, computer vision. Our experiments include a demonstration of how painters create convincing depictions using a stylized approach. We further provide an analysis of the spatial and probabilistic distributions of materials depicted in paintings, in which we for example show that strong patterns exists for material presence and location. Furthermore, we demonstrate how paintings could be used to build more robust computer vision classifiers by learning a more perceptually relevant feature representation. Additionally, we demonstrate that training classifiers on paintings could be used to uncover hidden perceptual cues by visualizing the features used by the classifiers. We conclude that our dataset of painterly material depictions is a rich source for gaining insights into the depiction and perception of materials across multiple disciplines and hope that the release of this dataset will drive multidisciplinary research.

Understanding gloss perception through the lens of art: Combining perception, image analysis, and painting recipes of 17th century painted grapes

To understand the key image features that we use to infer the glossiness of materials, we analyzed the pictorial shortcuts used by 17th century painters to imitate the optical phenomenon of specular reflections when depicting grapes. Gloss perception of painted grapes was determined via a rating experiment. We computed the contrast, blurriness, and coverage of the grapes' highlights in the paintings' images, inspired by Marlow and Anderson (2013). The highlights were manually segmented from the images, and next the features contrast, coverage, and blurriness were semiautomatically quantified using self-defined algorithms. Multiple linear regressions of contrast and blurriness resulted in a predictive model that could explain 69% of the variance in gloss perception. No effect was found for coverage. These findings are in agreement with the instructions to render glossiness of grapes contained in a 17th century painting manual (Beurs, 1692/in press), suggesting that painting practice embeds knowledge about key image features that trigger specific material percepts.