A machine learning approach to quantify the specificity of colour-emotion associations and their cultural differences

Using machine learning to predict judgments on Western visual art along content-representational and formal-perceptual attributes

Art research has long aimed to unravel the complex associations between specific attributes, such as color, complexity, and emotional expressiveness, and art judgments, including beauty, creativity, and liking. However, the fundamental distinction between attributes as inherent characteristics or features of the artwork and judgments as subjective evaluations remains an exciting topic. This paper reviews the literature of the last half century, to identify key attributes, and employs machine learning, specifically Gradient Boosted Decision Trees (GBDT), to predict 13 art judgments along 17 attributes. Ratings from 78 art novice participants were collected for 54 Western artworks. Our GBDT models successfully predicted 13 judgments significantly. Notably, judged creativity and disturbing/irritating judgments showed the highest predictability, with the models explaining 31% and 32% of the variance, respectively. The attributes emotional expressiveness, valence, symbolism, as well as complexity emerged as consistent and significant contributors to the models' performance. Content-representational attributes played a more prominent role than formal-perceptual attributes. Moreover, we found in some cases non-linear relationships between attributes and judgments with sudden inclines or declines around medium levels of the rating scales. By uncovering these underlying patterns and dynamics in art judgment behavior, our research provides valuable insights to advance the understanding of aesthetic experiences considering visual art, inform cultural practices, and inspire future research in the field of art appreciation.

Cross-cultural comparison of beauty judgments in visual art using machine learning analysis of art attribute predictors among Japanese and German speakers

In empirical art research, understanding how viewers judge visual artworks as beautiful is often explored through the study of attributes-specific inherent characteristics or artwork features such as color, complexity, and emotional expressiveness. These attributes form the basis for subjective evaluations, including the judgment of beauty. Building on this conceptual framework, our study examines the beauty judgments of 54 Western artworks made by native Japanese and German speakers, utilizing an extreme randomized trees model-a data-driven machine learning approach-to investigate cross-cultural differences in evaluation behavior. Our analysis of 17 attributes revealed that visual harmony, color variety, valence, and complexity significantly influenced beauty judgments across both cultural cohorts. Notably, preferences for complexity diverged significantly: while the native Japanese speakers found simpler artworks as more beautiful, the native German speakers evaluated more complex artworks as more beautiful. Further cultural distinctions were observed: for the native German speakers, emotional expressiveness was a significant factor, whereas for the native Japanese speakers, attributes such as brushwork, color world, and saturation were more impactful. Our findings illuminate the nuanced role that cultural context plays in shaping aesthetic judgments and demonstrate the utility of machine learning in unravelling these complex dynamics. This research not only advances our understanding of how beauty is judged in visual art-considering self-evaluated attributes-across different cultures but also underscores the potential of machine learning to enhance our comprehension of the aesthetic evaluation of visual artworks.

Drawing as a window to emotion with insights from tech-transformed participant images

This study delves into expressing primary emotions anger, happiness, sadness, and fear through drawings. Moving beyond the well-researched color-emotion link, it explores under-examined aspects like spatial concepts and drawing styles. Employing Python and OpenCV for objective analysis, we make a breakthrough by converting subjective perceptions into measurable data through 728 digital images from 182 university students. For the prominent color chosen for each emotion, the majority of participants chose red for anger (73.11%), yellow for happiness (17.8%), blue for sadness (51.1%), and black for fear (40.7%). Happiness led with the highest saturation (68.52%) and brightness (75.44%) percentages, while fear recorded the lowest in both categories (47.33% saturation, 48.78% brightness). Fear, however, topped in color fill percentage (35.49%), with happiness at the lowest (25.14%). Tangible imagery prevailed (71.43-83.52%), with abstract styles peaking in fear representations (28.57%). Facial expressions were a common element (41.76-49.45%). The study achieved an 81.3% predictive accuracy for anger, higher than the 71.3% overall average. Future research can build on these results by improving technological methods to quantify more aspects of drawing content. Investigating a more comprehensive array of emotions and examining factors influencing emotional drawing styles will further our understanding of visual-emotional communication.

A comparative analysis of colour-emotion associations in 16-88-year-old adults from 31 countries

As people age, they tend to spend more time indoors, and the colours in their surroundings may significantly impact their mood and overall well-being. However, there is a lack of empirical evidence to provide informed guidance on colour choices, irrespective of age group. To work towards informed choices, we investigated whether the associations between colours and emotions observed in younger individuals also apply to older adults. We recruited 7393 participants, aged between 16 and 88 years and coming from 31 countries. Each participant associated 12 colour terms with 20 emotion concepts and rated the intensity of each associated emotion. Different age groups exhibited highly similar patterns of colour-emotion associations (average similarity coefficient of .97), with subtle yet meaningful age-related differences. Adolescents associated the greatest number but the least positively biased emotions with colours. Older participants associated a smaller number but more intense and more positive emotions with all colour terms, displaying a positivity effect. Age also predicted arousal and power biases, varying by colour. Findings suggest parallels in colour-emotion associations between younger and older adults, with subtle but significant age-related variations. Future studies should next assess whether colour-emotion associations reflect what people actually feel when exposed to colour.

Mimicking Facial Expressions Facilitates Working Memory for Stimuli in Emotion-Congruent Colours

It is one thing for everyday phrases like "seeing red" to link some emotions with certain colours (e.g., anger with red), but can such links measurably bias information processing? We investigated whether emotional face information (angry/happy/neutral) held in visual working memory (VWM) enhances memory for shapes presented in a conceptually consistent colour (red or green) (Experiment 1). Although emotional information held in VWM appeared not to bias memory for coloured shapes in Experiment 1, exploratory analyses suggested that participants who physically mimicked the face stimuli were better at remembering congruently coloured shapes. Experiment 2 confirmed this finding by asking participants to hold the faces in mind while either mimicking or labelling the emotional expressions of face stimuli. Once again, those who mimicked the expressions were better at remembering shapes with emotion-congruent colours, whereas those who simply labelled them were not. Thus, emotion-colour associations appear powerful enough to guide attention, but-consistent with proposed impacts of "embodied emotion" on cognition-such effects emerged when emotion processing was facilitated through facial mimicry.

The good, the bad, and the red: implicit color-valence associations across cultures

Cultural differences-as well as similarities-have been found in explicit color-emotion associations between Chinese and Western populations. However, implicit associations in a cross-cultural context remain an understudied topic, despite their sensitivity to more implicit knowledge. Moreover, they can be used to study color systems-that is, emotional associations with one color in the context of an opposed one. Therefore, we tested the influence of two different color oppositions on affective stimulus categorization: red versus green and red versus white, in two experiments. In Experiment 1, stimuli comprised positive and negative words, and participants from the West (Austria/Germany), and the East (Mainland China, Macau) were tested in their native languages. The Western group showed a significantly stronger color-valence interaction effect than the Mainland Chinese (but not the Macanese) group for red-green but not for red-white opposition. To explore color-valence interaction effects independently of word stimulus differences between participant groups, we used affective silhouettes instead of words in Experiment 2. Again, the Western group showed a significantly stronger color-valence interaction than the Chinese group in red-green opposition, while effects in red-white opposition did not differ between cultural groups. Our findings complement those from explicit association research in an unexpected manner, where explicit measures showed similarities between cultures (associations for red and green), our results revealed differences and where explicit measures showed differences (associations with white), our results showed similarities, underlining the value of applying comprehensive measures in cross-cultural research on cross-modal associations.

Self-Supervised Color-Concept Association via Image Colorization

The interpretation of colors in visualizations is facilitated when the assignments between colors and concepts in the visualizations match human's expectations, implying that the colors can be interpreted in a semantic manner. However, manually creating a dataset of suitable associations between colors and concepts for use in visualizations is costly, as such associations would have to be collected from humans for a large variety of concepts. To address the challenge of collecting this data, we introduce a method to extract color-concept associations automatically from a set of concept images. While the state-of-the-art method extracts associations from data with supervised learning, we developed a self-supervised method based on colorization that does not require the preparation of ground truth color-concept associations. Our key insight is that a set of images of a concept should be sufficient for learning color-concept associations, since humans also learn to associate colors to concepts mainly from past visual input. Thus, we propose to use an automatic colorization method to extract statistical models of the color-concept associations that appear in concept images. Specifically, we take a colorization model pre-trained on ImageNet and fine-tune it on the set of images associated with a given concept, to predict pixel-wise probability distributions in Lab color space for the images. Then, we convert the predicted probability distributions into color ratings for a given color library and aggregate them for all the images of a concept to obtain the final color-concept associations. We evaluate our method using four different evaluation metrics and via a user study. Experiments show that, although the state-of-the-art method based on supervised learning with user-provided ratings is more effective at capturing relative associations, our self-supervised method obtains overall better results according to metrics like Earth Mover's Distance (EMD) and Entropy Difference (ED), which are closer to human perception of color distributions.

Unifying Effects of Direct and Relational Associations for Visual Communication

People have expectations about how colors map to concepts in visualizations, and they are better at interpreting visualizations that match their expectations. Traditionally, studies on these expectations (inferred mappings) distinguished distinct factors relevant for visualizations of categorical vs. continuous information. Studies on categorical information focused on direct associations (e.g., mangos are associated with yellows) whereas studies on continuous information focused on relational associations (e.g., darker colors map to larger quantities; dark-is-more bias). We unite these two areas within a single framework of assignment inference. Assignment inference is the process by which people infer mappings between perceptual features and concepts represented in encoding systems. Observers infer globally optimal assignments by maximizing the "merit," or "goodness," of each possible assignment. Previous work on assignment inference focused on visualizations of categorical information. We extend this approach to visualizations of continuous data by (a) broadening the notion of merit to include relational associations and (b) developing a method for combining multiple (sometimes conflicting) sources of merit to predict people's inferred mappings. We developed and tested our model on data from experiments in which participants interpreted colormap data visualizations, representing fictitious data about environmental concepts (sunshine, shade, wild fire, ocean water, glacial ice). We found both direct and relational associations contribute independently to inferred mappings. These results can be used to optimize visualization design to facilitate visual communication.

Is Purple Lost in Translation? The Affective Meaning of Purple, Violet, and Lilac Cognates in 16 Languages and 30 Populations

Colour-emotion association data show a universal consistency in colour-emotion associations, apart from emotion associations with PURPLE. Possibly, its heterogeneity was due to different cognates used as basic colour terms between languages. We analysed emotion associations with PURPLE across 30 populations, 28 countries, and 16 languages (4,008 participants in total). Crucially, these languages used the cognates of purple, lilac, or violet to denote the basic PURPLE category. We found small but systematic affective differences between these cognates. They were ordered as purple > lilac > violet on valence, arousal, and power biases. Statistically, the cognate purple was the most strongly biased towards associations with positive emotions, and lilac was biased more strongly than violet. Purple was more biased towards high power emotions than violet, but cognates did not differ on arousal biases. Additionally, affective biases differed by population, suggesting high variability within each cognate. Thus, cognates partly account for inconsistencies in the meaning of PURPLE, without explaining their origins.

Promoting a hand sanitizer by persuasive messages: moving bottle and background color as approach and avoidance cues

In message-based health interventions, peripheral cues such as motion and color capture exogenous attention. These cues may elicit approach and avoidance motivation and the core ingredients of persuasion (argument framing, source of the message, and persuasion knowledge). In two studies, we presented participants with persuasive messages about a hand sanitizer. Messages varied by the framing of the arguments (gain vs. loss) and by the source of the message (healthcare industry vs. public health agency). In Study 1 (N = 137), the forward apparent motion of the hand sanitizer bottle compared to a backward apparent motion increased a positive attitude toward the hand sanitizer, the intention to buy it, and ease of judgment. In Study 2 (N = 280), a small main positive effect of a green background was observed for attractiveness of the hand sanitizer, but only when a green background followed a red one. Green (vs. red) background increased willingness to buy the hand sanitizer. We observed no main effects of argument framing or source of the message. The discussion emphasizes approach and avoidance motivation as a common framework for understanding the respective contribution of peripheral cues and core ingredients of messages to the persuasion process.

Visual communication via the design of food and beverage packaging

A rapidly growing body of empirical research has recently started to emerge highlighting the connotative and/or semiotic meanings that consumers typically associate with specific abstract visual design features, such as colours (either when presented individually or in combination), simple shapes/curvilinearity, and the orientation and relative position of those design elements on product packaging. While certain of our affective responses to such basic visual design features appear almost innate, the majority are likely established via the internalization of the statistical regularities of the food and beverage marketplace (i.e. as a result of associative learning), as in the case of round typeface and sweet-tasting products. Researchers continue to document the wide range of crossmodal correspondences that underpin the links between individual visual packaging design features and specific properties of food and drink products (such as their taste, flavour, or healthfulness), and the ways in which marketers are now capitalizing on such understanding to increase sales. This narrative review highlights the further research that is still needed to establish the connotative or symbolic/semiotic meaning(s) of particular combinations of design features (such as coloured stripes in a specific orientation), as opposed to individual cues in national food markets and also, increasingly, cross-culturally in the case of international brands.

Context Matters: A Theory of Semantic Discriminability for Perceptual Encoding Systems

People's associations between colors and concepts influence their ability to interpret the meanings of colors in information visualizations. Previous work has suggested such effects are limited to concepts that have strong, specific associations with colors. However, although a concept may not be strongly associated with any colors, its mapping can be disambiguated in the context of other concepts in an encoding system. We articulate this view in semantic discriminability theory, a general framework for understanding conditions determining when people can infer meaning from perceptual features. Semantic discriminability is the degree to which observers can infer a unique mapping between visual features and concepts. Semantic discriminability theory posits that the capacity for semantic discriminability for a set of concepts is constrained by the difference between the feature-concept association distributions across the concepts in the set. We define formal properties of this theory and test its implications in two experiments. The results show that the capacity to produce semantically discriminable colors for sets of concepts was indeed constrained by the statistical distance between color-concept association distributions (Experiment 1). Moreover, people could interpret meanings of colors in bar graphs insofar as the colors were semantically discriminable, even for concepts previously considered "non-colorable" (Experiment 2). The results suggest that colors are more robust for visual communication than previously thought.

Personality first in emotion: a deep neural network based on electroencephalogram channel attention for cross-subject emotion recognition

In recent years, more and more researchers have focused on emotion recognition methods based on electroencephalogram (EEG) signals. However, most studies only consider the spatio-temporal characteristics of EEG and the modelling based on this feature, without considering personality factors, let alone studying the potential correlation between different subjects. Considering the particularity of emotions, different individuals may have different subjective responses to the same physical stimulus. Therefore, emotion recognition methods based on EEG signals should tend to be personalized. This paper models the personalized EEG emotion recognition from the macro and micro levels. At the macro level, we use personality characteristics to classify the individuals' personalities from the perspective of 'birds of a feather flock together'. At the micro level, we employ deep learning models to extract the spatio-temporal feature information of EEG. To evaluate the effectiveness of our method, we conduct an EEG emotion recognition experiment on the ASCERTAIN dataset. Our experimental results demonstrate that the recognition accuracy of our proposed method is 72.4% and 75.9% on valence and arousal, respectively, which is 10.2% and 9.1% higher than that of no consideration of personalization.

Colour-emotion associations in individuals with red-green colour blindness

Colours and emotions are associated in languages and traditions. Some of us may convey sadness by saying feeling blue or by wearing black clothes at funerals. The first example is a conceptual experience of colour and the second example is an immediate perceptual experience of colour. To investigate whether one or the other type of experience more strongly drives colour-emotion associations, we tested 64 congenitally red-green colour-blind men and 66 non-colour-blind men. All participants associated 12 colours, presented as terms or patches, with 20 emotion concepts, and rated intensities of the associated emotions. We found that colour-blind and non-colour-blind men associated similar emotions with colours, irrespective of whether colours were conveyed via terms (r = .82) or patches (r = .80). The colour-emotion associations and the emotion intensities were not modulated by participants’ severity of colour blindness. Hinting at some additional, although minor, role of actual colour perception, the consistencies in associations for colour terms and patches were higher in non-colour-blind than colour-blind men. Together, these results suggest that colour-emotion associations in adults do not require immediate perceptual colour experiences, as conceptual experiences are sufficient.

Universal Patterns in Color-Emotion Associations Are Further Shaped by Linguistic and Geographic Proximity

Many of us “see red,” “feel blue,” or “turn green with envy.” Are such color-emotion associations fundamental to our shared cognitive architecture, or are they cultural creations learned through our languages and traditions? To answer these questions, we tested emotional associations of colors in 4,598 participants from 30 nations speaking 22 native languages. Participants associated 20 emotion concepts with 12 color terms. Pattern-similarity analyses revealed universal color-emotion associations (average similarity coefficient r = .88). However, local differences were also apparent. A machine-learning algorithm revealed that nation predicted color-emotion associations above and beyond those observed universally. Similarity was greater when nations were linguistically or geographically close. This study highlights robust universal color-emotion associations, further modulated by linguistic and geographic factors. These results pose further theoretical and empirical questions about the affective properties of color and may inform practice in applied domains, such as well-being and design.

Polarities influence implicit associations between colour and emotion

Colours are linked to emotional concepts. Research on the effect of red in particular has been extensive, and evidence shows that positive as well as negative associations can be salient in different contexts. In this paper, we investigate the impact of the contextual factor of polarity. According to the polarity-correspondence principle, negative and positive category poles are assigned to the binary response categories (here positive vs. negative valence) and the perceptual dimension (green vs. red) in a discrimination task. Response facilitation occurs only where the conceptual category (valence) and the perceptual feature (colour) share the same pole (i.e., where both are plus or both are minus). We asked participants (n = 140) to classify the valence of green and red words within two types of blocks: (a) where all words were of the same colour (monochromatic conditions) providing no opposition in the perceptual dimension, and (b) where red and green words were randomly mixed (mixed-colour conditions). Our results show that red facilitates responses to negative words when the colour green is present (mixed-colour conditions) but not when it is absent (monochromatic conditions). This is in line with the polarity-correspondence principle, but colour-specific valence-affect associations contribute to the found effects.

Blue hues don't bring the blues: questioning conventional notions of color-emotion associations

It is commonly held that yellow is happy and blue is sad, but the reason remains unclear. Part of the problem is that researchers tend to focus on understanding why yellow is happy and blue is sad, but this may be a misleading characterization of color-emotion associations. In this study, we disentangle the contribution of lightness, chroma, and hue in color-happy/sad associations by controlling for lightness and chroma either statistically or colorimetrically. We found that after controlling for lightness and chroma, colors with blue hue were no sadder than colors with yellow hue, and in some cases, colors with blue hue were actually happier. These results can help guide future efforts to understand the nature of color-emotion associations.

Feeling Blue or Seeing Red? Similar Patterns of Emotion Associations With Colour Patches and Colour Terms

For many, colours convey affective meaning. Popular opinion assumes that perception of colour is crucial to influence emotions. However, scientific studies test colour-emotion relationships by presenting colours as patches or terms. When using patches, researchers put great effort into colour presentation. When using terms, researchers have much less control over the colour participants think of. In this between-subjects study, we tested whether emotion associations with colour differ between terms and patches. Participants associated 20 emotion concepts, loading on valence, arousal, and power dimensions, with 12 colours presented as patches (n = 54) or terms (n = 78). We report high similarity in the pattern of associations of specific emotion concepts with terms and patches (r = .82), for all colours except purple (r = .-23). We also observed differences for black, which is associated with more negative emotions and of higher intensity when presented as a term than a patch. Terms and patches differed little in terms of valence, arousal, and power dimensions. Thus, results from studies on colour-emotion relationships using colour terms or patches should be largely comparable. It is possible that emotions are associated with colour concepts rather than particular perceptions or words of colour.