1
|
Schmidt F, Hebart MN, Schmid AC, Fleming RW. Core dimensions of human material perception. Proc Natl Acad Sci U S A 2025; 122:e2417202122. [PMID: 40042912 PMCID: PMC11912425 DOI: 10.1073/pnas.2417202122] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2024] [Accepted: 01/24/2025] [Indexed: 03/19/2025] Open
Abstract
Visually categorizing and comparing materials is crucial for everyday behavior, but what organizational principles underlie our mental representation of materials? Here, we used a large-scale data-driven approach to uncover core latent dimensions of material representations from behavior. First, we created an image dataset of 200 systematically sampled materials and 600 photographs (STUFF dataset, https://osf.io/myutc/). Using these images, we next collected 1.87 million triplet similarity judgments and used a computational model to derive a set of sparse, positive dimensions underlying these judgments. The resulting multidimensional embedding space predicted independent material similarity judgments and the similarity matrix of all images close to the human intersubject consistency. We found that representations of individual images were captured by a combination of 36 material dimensions that were highly reproducible and interpretable, comprising perceptual (e.g., grainy, blue) as well as conceptual (e.g., mineral, viscous) dimensions. These results provide the foundation for a comprehensive understanding of how humans make sense of materials.
Collapse
Affiliation(s)
- Filipp Schmidt
- Experimental Psychology, Justus Liebig University, Giessen35394, Germany
- Center for Mind, Brain and Behavior, Universities of Marburg, Giessen, and Darmstadt, Marburg35032, Germany
| | - Martin N. Hebart
- Center for Mind, Brain and Behavior, Universities of Marburg, Giessen, and Darmstadt, Marburg35032, Germany
- Department of Medicine, Justus Liebig University, Giessen35390, Germany
- Vision and Computational Cognition Group, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig04103, Germany
| | - Alexandra C. Schmid
- Experimental Psychology, Justus Liebig University, Giessen35394, Germany
- Laboratory of Brain and Cognition, National Institute of Mental Health, Bethesda, MD20814
| | - Roland W. Fleming
- Experimental Psychology, Justus Liebig University, Giessen35394, Germany
- Center for Mind, Brain and Behavior, Universities of Marburg, Giessen, and Darmstadt, Marburg35032, Germany
| |
Collapse
|
2
|
Morimoto T, Akbarinia A, Storrs K, Cheeseman JR, Smithson HE, Gegenfurtner KR, Fleming RW. Color and gloss constancy under diverse lighting environments. J Vis 2023; 23:8. [PMID: 37432844 PMCID: PMC10351023 DOI: 10.1167/jov.23.7.8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/13/2023] Open
Abstract
When we look at an object, we simultaneously see how glossy or matte it is, how light or dark, and what color. Yet, at each point on the object's surface, both diffuse and specular reflections are mixed in different proportions, resulting in substantial spatial chromatic and luminance variations. To further complicate matters, this pattern changes radically when the object is viewed under different lighting conditions. The purpose of this study was to simultaneously measure our ability to judge color and gloss using an image set capturing diverse object and illuminant properties. Participants adjusted the hue, lightness, chroma, and specular reflectance of a reference object so that it appeared to be made of the same material as a test object. Critically, the two objects were presented under different lighting environments. We found that hue matches were highly accurate, except for under a chromatically atypical illuminant. Chroma and lightness constancy were generally poor, but these failures correlated well with simple image statistics. Gloss constancy was particularly poor, and these failures were only partially explained by reflection contrast. Importantly, across all measures, participants were highly consistent with one another in their deviations from constancy. Although color and gloss constancy hold well in simple conditions, the variety of lighting and shape in the real world presents significant challenges to our visual system's ability to judge intrinsic material properties.
Collapse
Affiliation(s)
- Takuma Morimoto
- Justus Liebig University Giessen, Giessen, Germany
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | | | - Katherine Storrs
- Justus Liebig University Giessen, Giessen, Germany
- School of Psychology, University of Auckland, New Zealand
| | - Jacob R Cheeseman
- Justus Liebig University Giessen, Giessen, Germany
- Center for Mind, Brain and Behavior (CMBB), Universities of Marburg, Giessen and Darmstadt, Germany
| | - Hannah E Smithson
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | | | - Roland W Fleming
- Justus Liebig University Giessen, Giessen, Germany
- Center for Mind, Brain and Behavior (CMBB), Universities of Marburg, Giessen and Darmstadt, Germany
| |
Collapse
|
3
|
Liao C, Sawayama M, Xiao B. Unsupervised learning reveals interpretable latent representations for translucency perception. PLoS Comput Biol 2023; 19:e1010878. [PMID: 36753520 PMCID: PMC9942964 DOI: 10.1371/journal.pcbi.1010878] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 02/21/2023] [Accepted: 01/18/2023] [Indexed: 02/09/2023] Open
Abstract
Humans constantly assess the appearance of materials to plan actions, such as stepping on icy roads without slipping. Visual inference of materials is important but challenging because a given material can appear dramatically different in various scenes. This problem especially stands out for translucent materials, whose appearance strongly depends on lighting, geometry, and viewpoint. Despite this, humans can still distinguish between different materials, and it remains unsolved how to systematically discover visual features pertinent to material inference from natural images. Here, we develop an unsupervised style-based image generation model to identify perceptually relevant dimensions for translucent material appearances from photographs. We find our model, with its layer-wise latent representation, can synthesize images of diverse and realistic materials. Importantly, without supervision, human-understandable scene attributes, including the object's shape, material, and body color, spontaneously emerge in the model's layer-wise latent space in a scale-specific manner. By embedding an image into the learned latent space, we can manipulate specific layers' latent code to modify the appearance of the object in the image. Specifically, we find that manipulation on the early-layers (coarse spatial scale) transforms the object's shape, while manipulation on the later-layers (fine spatial scale) modifies its body color. The middle-layers of the latent space selectively encode translucency features and manipulation of such layers coherently modifies the translucency appearance, without changing the object's shape or body color. Moreover, we find the middle-layers of the latent space can successfully predict human translucency ratings, suggesting that translucent impressions are established in mid-to-low spatial scale features. This layer-wise latent representation allows us to systematically discover perceptually relevant image features for human translucency perception. Together, our findings reveal that learning the scale-specific statistical structure of natural images might be crucial for humans to efficiently represent material properties across contexts.
Collapse
Affiliation(s)
- Chenxi Liao
- Department of Neuroscience, American University, Washington, D.C., District of Columbia, United States of America
| | - Masataka Sawayama
- Graduate School of Information Science and Technology, The University of Tokyo, Tokyo, Japan
| | - Bei Xiao
- Department of Computer Science, American University, Washington, D.C., District of Columbia, United States of America
| |
Collapse
|
4
|
Adolphe M, Sawayama M, Maurel D, Delmas A, Oudeyer PY, Sauzéon H. An Open-Source Cognitive Test Battery to Assess Human Attention and Memory. Front Psychol 2022; 13:880375. [PMID: 35756204 PMCID: PMC9231481 DOI: 10.3389/fpsyg.2022.880375] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2022] [Accepted: 04/26/2022] [Indexed: 11/13/2022] Open
Abstract
Cognitive test batteries are widely used in diverse research fields, such as cognitive training, cognitive disorder assessment, or brain mechanism understanding. Although they need flexibility according to their usage objectives, most test batteries are not available as open-source software and are not be tuned by researchers in detail. The present study introduces an open-source cognitive test battery to assess attention and memory, using a javascript library, p5.js. Because of the ubiquitous nature of dynamic attention in our daily lives, it is crucial to have tools for its assessment or training. For that purpose, our test battery includes seven cognitive tasks (multiple-objects tracking, enumeration, go/no-go, load-induced blindness, task-switching, working memory, and memorability), common in cognitive science literature. By using the test battery, we conducted an online experiment to collect the benchmark data. Results conducted on 2 separate days showed the high cross-day reliability. Specifically, the task performance did not largely change with the different days. Besides, our test battery captures diverse individual differences and can evaluate them based on the cognitive factors extracted from latent factor analysis. Since we share our source code as open-source software, users can expand and manipulate experimental conditions flexibly. Our test battery is also flexible in terms of the experimental environment, i.e., it is possible to experiment either online or in a laboratory environment.
Collapse
Affiliation(s)
- Maxime Adolphe
- Flowers Team, Inria, Bordeaux, France.,Research and Development Team, Onepoint, Bordeaux, France.,Department of Cognitive Sciences and Ergonomics, Université de Bordeaux, Bordeaux, France
| | | | - Denis Maurel
- Research and Development Team, Onepoint, Bordeaux, France
| | | | - Pierre-Yves Oudeyer
- Flowers Team, Inria, Bordeaux, France.,Microsoft Research Montreal, Montreal, QC, Canada
| | - Hélène Sauzéon
- Flowers Team, Inria, Bordeaux, France.,ACTIVE Team, Université de Bordeaux, INSERM, BPH, U1219, Bordeaux, France
| |
Collapse
|