The Perception of Relations

The role of active perception and naming in sameness comparison

Humans have an exceptional ability to notice relations between different entities and situations and to transfer their relational knowledge across contexts. For example, adults can easily determine whether two objects are the same, regardless of what these objects are. It remains unclear which cognitive resources and strategies underlie human ability to acquire and use generalizable relational concepts. Here, in a set of three experiments (N = 40, N = 40, N = 155; data collected in 2021), we investigate the strategies that human adults use to determine whether two compound items are visually identical. We found that people compare the items by revealing perceptual information in the order that minimizes memory involvement and errors. When participants are prevented from using the perceptual offloading strategy, they switch to a naming strategy-either internally or through explicit verbalization-to maintain accuracy in relational judgment. When the items lack easily nameable features and perceptual offloading is restricted, participants are slower, less accurate, and less efficient in their comparisons. Thus, humans adaptively offload the memory requirements of the relational comparison onto active perceptual interactions with the stimuli and naming. We suggest that cognitive models of visual relational reasoning should consider these perceptual and language-based resources when formalizing mechanisms underlying human relational judgments.

Visual routines for detecting causal interactions are tuned to motion direction

Detecting causal relations structures our perception of events in the world. Here, we determined for visual interactions whether generalized (i.e. feature-invariant) or specialized (i.e. feature-selective) visual routines underlie the perception of causality. To this end, we applied a visual adaptation protocol to assess the adaptability of specific features in classical launching events of simple geometric shapes. We asked observers to report whether they observed a launch or a pass in ambiguous test events (i.e. the overlap between two discs varied from trial to trial). After prolonged exposure to causal launch events (the adaptor) defined by a particular set of features (i.e. a particular motion direction, motion speed, or feature conjunction), observers were less likely to see causal launches in subsequent ambiguous test events than before adaptation. Crucially, adaptation was contingent on the causal impression in launches as demonstrated by a lack of adaptation in non-causal control events. We assessed whether this negative aftereffect transfers to test events with a new set of feature values that were not presented during adaptation. Processing in specialized (as opposed to generalized) visual routines predicts that the transfer of visual adaptation depends on the feature similarity of the adaptor and the test event. We show that the negative aftereffects do not transfer to unadapted launch directions but do transfer to launch events of different speeds. Finally, we used colored discs to assign distinct feature-based identities to the launching and the launched stimulus. We found that the adaptation transferred across colors if the test event had the same motion direction as the adaptor. In summary, visual adaptation allowed us to carve out a visual feature space underlying the perception of causality and revealed specialized visual routines that are tuned to a launch's motion direction.

Actions at a glance: The time course of action, object, and scene recognition in a free recall paradigm

Being able to quickly recognize other people's actions lies at the heart of our ability to efficiently interact with our environment. Action recognition has been suggested to rely on the analysis and integration of information from different perceptual subsystems, e.g., for the processing of objects and scenes. However, stimulus presentation times that are required to extract information about actions, objects, and scenes to our knowledge have not yet been directly compared. To address this gap in the literature, we compared the recognition thresholds for actions, objects, and scenes. First, 30 participants were presented with grayscale images depicting different actions at variable presentation times (33-500 ms) and provided written descriptions of each image. Next, ten naïve raters evaluated these descriptions with respect to the presence and accuracy of information related to actions, objects, scenes, and sensory information. Comparing thresholds across presentation times, we found that recognizing actions required shorter presentation times (from 60 ms onwards) than objects (68 ms) and scenes (84 ms). More specific actions required presentation times of approximately 100 ms. Moreover, thresholds were modulated by action category, with the lowest thresholds for locomotion and the highest thresholds for food-related actions. Together, our data suggest that perceptual evidence for actions, objects, and scenes is gathered in parallel when these are presented in the same scene but accumulates faster for actions that reflect static body posture recognition than for objects and scenes.

What we mean when we say semantic: Toward a multidisciplinary semantic glossary

Tulving characterized semantic memory as a vast repository of meaning that underlies language and many other cognitive processes. This perspective on lexical and conceptual knowledge galvanized a new era of research undertaken by numerous fields, each with their own idiosyncratic methods and terminology. For example, "concept" has different meanings in philosophy, linguistics, and psychology. As such, many fundamental constructs used to delineate semantic theories remain underspecified and/or opaque. Weak construct specificity is among the leading causes of the replication crisis now facing psychology and related fields. Term ambiguity hinders cross-disciplinary communication, falsifiability, and incremental theory-building. Numerous cognitive subdisciplines (e.g., vision, affective neuroscience) have recently addressed these limitations via the development of consensus-based guidelines and definitions. The project to follow represents our effort to produce a multidisciplinary semantic glossary consisting of succinct definitions, background, principled dissenting views, ratings of agreement, and subjective confidence for 17 target constructs (e.g., abstractness, abstraction, concreteness, concept, embodied cognition, event semantics, lexical-semantic, modality, representation, semantic control, semantic feature, simulation, semantic distance, semantic dimension). We discuss potential benefits and pitfalls (e.g., implicit bias, prescriptiveness) of these efforts to specify a common nomenclature that other researchers might index in specifying their own theoretical perspectives (e.g., They said X, but I mean Y).

Paranoid and teleological thinking give rise to distinct social hallucinations in vision

Paranoia (believing others intend harm) and excess teleological thinking (ascribing too much purpose) are non-consensual beliefs about agents. Human vision rapidly detects agents and their intentions. Might paranoia and teleology have roots in visual perception? Using displays that evoke the impression that one disc ('wolf') is chasing another ('sheep'), we find that paranoia and teleology involve perceiving chasing when there is none (studies 1 and 2) - errors we characterize as social hallucinations. When asked to identify the wolf or the sheep (studies 3, 4a, and 4b), we find high-paranoia participants struggled to identify sheep, while high-teleology participants were impaired at identifying wolves - both despite high-confidence. Both types of errors correlated with hallucinatory percepts in the real world. Although paranoia and teleology both involve excess perception of agency, the current results collectively suggest a perceptual distinction between the two, perhaps with clinical import.

Invariant representations in abstract concept grounding - the physical world in grounded cognition

Grounded cognition states that mental representations of concepts consist of experiential aspects. For example, the concept "cup" consists of the sensorimotor experiences from interactions with cups. Typical modalities in which concepts are grounded are: The sensorimotor system (including interoception), emotion, action, language, and social aspects. Here, we argue that this list should be expanded to include physical invariants (unchanging features of physical motion; e.g., gravity, momentum, friction). Research on physical reasoning consistently demonstrates that physical invariants are represented as fundamentally as other grounding substrates, and therefore should qualify. We assess several theories of concept representation (simulation, conceptual metaphor, conceptual spaces, predictive processing) and their positions on physical invariants. We find that the classic grounded cognition theories, simulation and conceptual metaphor theory, have not considered physical invariants, while conceptual spaces and predictive processing have. We conclude that physical invariants should be included into grounded cognition theories, and that the core mechanisms of simulation and conceptual metaphor theory are well suited to do this. Furthermore, conceptual spaces and predictive processing are very promising and should also be integrated with grounded cognition in the future.

Episodic recombination and the role of time in mental travel

Mental time travel is often presented as a singular mechanism, but theoretical and empirical considerations suggest that it is composed of component processes. What are these components? Three hypotheses about the major components of mental time travel are commonly considered: (i) remembering and imagining might, respectively, rely on different processes, (ii) past- and future-directed forms of mental time travel might, respectively, rely on different processes, and (iii) the creation of episodic representations and the determination of their temporal orientation might, respectively, rely on different processes. Here, we flesh out the last of these proposals. First, we argue for 'representational continuism': the view that different forms of mental travel are continuous with regard to their core representational contents. Next, we propose an updated account of episodic recombination (the mechanism generating these episodic contents) and review evidence in its support. On this view, episodic recombination is a natural kind best viewed as a form of compositional computation. Finally, we argue that episodic recombination should be distinguished from mechanisms determining the temporal orientation of episodic representations. Thus, we suggest that mental travel is a singular capacity, while mental time travel has at least two major components: episodic representations and their temporal orientation. This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.

Compositionality in perception: A framework

Perception involves the processing of content or information about the world. In what form is this content represented? I argue that perception is widely compositional. The perceptual system represents many stimulus features (including shape, orientation, and motion) in terms of combinations of other features (such as shape parts, slant and tilt, common and residual motion vectors). But compositionality can take a variety of forms. The ways in which perceptual representations compose are markedly different from the ways in which sentences or thoughts are thought to be composed. I suggest that the thesis that perception is compositional is not itself a concrete hypothesis with specific predictions; rather it affords a productive framework for developing and evaluating specific empirical hypotheses about the form and content of perceptual representations. The question is not just whether perception is compositional, but how. Answering this latter question can provide fundamental insights into perception. This article is categorized under: Philosophy > Representation Philosophy > Foundations of Cognitive Science Psychology > Perception and Psychophysics.

Cognitive development: The origins of structured thought in the mind

Linguistic syntax lets us communicate complex, structured thoughts, like whether a dog chased a man or vice versa. New work shows that seven-month-olds can entertain such structured thoughts even before acquiring their native language, revealing the origins of this sophisticated ability.

The motifs of radical embodied neuroscience

In this paper, I analyse how the emerging scientific framework of radical embodied neuroscience is different from contemporary mainstream cognitive neuroscience. To do so, I propose the notion of motif to enrich the philosophical toolkit of cognitive neuroscience. This notion can be used to characterize the guiding ideas of any given scientific framework in psychology and neuroscience. Motifs are highly unconstrained, open-ended concepts that support equally open-ended families of explanations. Different scientific frameworks-e.g., psychophysics or cognitive neuroscience-provide these motifs to answer the overarching themes of these disciplines, such as the relationship between stimuli and sensations or the proper methods of the sciences of the mind. Some motifs of mainstream cognitive neuroscience are the motif of encoding, the motif of input-output systems, and the motif of algorithms. The two first ones answer the question about the relationship between stimuli, sensations and experience (e.g., stimuli are input and are encoded by brain structures). The latter one answers the question regarding the mechanism of cognition and experience. The three of them are equally unconstrained and open-ended, and they serve as an umbrella for different kinds of explanation-i.e., different positions regarding what counts as a code or as an input. Along with the articulation of the notion of motif, the main aim of this article is to present three motifs for radical embodied neuroscience: the motif of complex stimulation, the motif of organic behaviour and the motif of resonance.

Evidence for a role of synchrony but not common fate in the perception of biological group movements

Extensive research has shown that observers are able to efficiently extract summary information from groups of people. However, little is known about the cues that determine whether multiple people are represented as a social group or as independent individuals. Initial research on this topic has primarily focused on the role of static cues. Here, we instead investigate the role of dynamic cues. In two experiments with male and female human participants, we use EEG frequency tagging to investigate the influence of two fundamental Gestalt principles - synchrony and common fate - on the grouping of biological movements. In Experiment 1, we find that brain responses coupled to four point-light figures walking together are enhanced when they move in sync vs. out of sync, but only when they are presented upright. In contrast, we found no effect of movement direction (i.e., common fate). In Experiment 2, we rule out that synchrony takes precedence over common fate by replicating the null effect of movement direction while keeping synchrony constant. These results suggest that synchrony plays an important role in the processing of biological group movements. In contrast, the role of common fate is less clear and will require further research.

A Phone in a Basket Looks Like a Knife in a Cup: Role-Filler Independence in Visual Processing

When a piece of fruit is in a bowl, and the bowl is on a table, we appreciate not only the individual objects and their features, but also the relations containment and support, which abstract away from the particular objects involved. Independent representation of roles (e.g., containers vs. supporters) and "fillers" of those roles (e.g., bowls vs. cups, tables vs. chairs) is a core principle of language and higher-level reasoning. But does such role-filler independence also arise in automatic visual processing? Here, we show that it does, by exploring a surprising error that such independence can produce. In four experiments, participants saw a stream of images containing different objects arranged in force-dynamic relations-e.g., a phone contained in a basket, a marker resting on a garbage can, or a knife sitting in a cup. Participants had to respond to a single target image (e.g., a phone in a basket) within a stream of distractors presented under time constraints. Surprisingly, even though participants completed this task quickly and accurately, they false-alarmed more often to images matching the target's relational category than to those that did not-even when those images involved completely different objects. In other words, participants searching for a phone in a basket were more likely to mistakenly respond to a knife in a cup than to a marker on a garbage can. Follow-up experiments ruled out strategic responses and also controlled for various confounding image features. We suggest that visual processing represents relations abstractly, in ways that separate roles from fillers.

Simplifying social learning

Social learning is complex, but people often seem to navigate social environments with ease. This ability creates a puzzle for traditional accounts of reinforcement learning (RL) that assume people negotiate a tradeoff between easy-but-simple behavior (model-free learning) and complex-but-difficult behavior (e.g., model-based learning). We offer a theoretical framework for resolving this puzzle: although social environments are complex, people have social expertise that helps them behave flexibly with low cognitive cost. Specifically, by using familiar concepts instead of focusing on novel details, people can turn hard learning problems into simpler ones. This ability highlights social learning as a prototype for studying cognitive simplicity in the face of environmental complexity and identifies a role for conceptual knowledge in everyday reward learning.

The Role of Agentive and Physical Forces in the Neural Representation of Motion Events

How does the brain represent information about motion events in relation to agentive and physical forces? In this study, we investigated the neural activity patterns associated with observing animated actions of agents (e.g., an agent hitting a chair) in comparison to similar movements of inanimate objects that were either shaped solely by the physics of the scene (e.g., gravity causing an object to fall down a hill and hit a chair) or initiated by agents (e.g., a visible agent causing an object to hit a chair). Using an fMRI-based multivariate pattern analysis (MVPA), this design allowed testing where in the brain the neural activity patterns associated with motion events change as a function of, or are invariant to, agentive versus physical forces behind them. A total of 29 human participants (nine male) participated in the study. Cross-decoding revealed a shared neural representation of animate and inanimate motion events that is invariant to agentive or physical forces in regions spanning frontoparietal and posterior temporal cortices. In contrast, the right lateral occipitotemporal cortex showed a higher sensitivity to agentive events, while the left dorsal premotor cortex was more sensitive to information about inanimate object events that were solely shaped by the physics of the scene.

Rotating objects cue spatial attention via the perception of frictive surface contact

We report a new attentional cueing effect, which shows how attention models the physical force of friction. Most objects we see are in frictive contact with a 'floor', such that clockwise rotation causes rightward movement and counterclockwise rotation leftward movement. Is this regularity encoded in spatial orienting responses? In Experiment 1, seeing a clockwise-rotating 'wheel' produced faster responses to subsequent targets appearing on the right vs. left (and vice versa for counterclockwise rotation). Thus, when viewing a lone rotating wheel, we orient attention toward where we predict it will move next, assuming frictive floor contact. But what happens if the rotating wheel is seen touching another visible surface? In Experiment 2, rotational cueing was stronger for wheels touching a visible floor, was abolished for wheels near but not touching another surface, and reversed for wheels touching a ceiling. We conclude that the visual system makes an assumption of frictive floor contact, and rapidly analyzes visual cues to frictive contact with other surfaces, in order to orient attention toward where objects are likely to move next.

Hierarchical organization of social action features along the lateral visual pathway

Recent theoretical work has argued that in addition to the classical ventral (what) and dorsal (where/how) visual streams, there is a third visual stream on the lateral surface of the brain specialized for processing social information. Like visual representations in the ventral and dorsal streams, representations in the lateral stream are thought to be hierarchically organized. However, no prior studies have comprehensively investigated the organization of naturalistic, social visual content in the lateral stream. To address this question, we curated a naturalistic stimulus set of 250 3-s videos of two people engaged in everyday actions. Each clip was richly annotated for its low-level visual features, mid-level scene and object properties, visual social primitives (including the distance between people and the extent to which they were facing), and high-level information about social interactions and affective content. Using a condition-rich fMRI experiment and a within-subject encoding model approach, we found that low-level visual features are represented in early visual cortex (EVC) and middle temporal (MT) area, mid-level visual social features in extrastriate body area (EBA) and lateral occipital complex (LOC), and high-level social interaction information along the superior temporal sulcus (STS). Communicative interactions, in particular, explained unique variance in regions of the STS after accounting for variance explained by all other labeled features. Taken together, these results provide support for representation of increasingly abstract social visual content-consistent with hierarchical organization-along the lateral visual stream and suggest that recognizing communicative actions may be a key computational goal of the lateral visual pathway.

Seeing social interactions

Seeing the interactions between other people is a critical part of our everyday visual experience, but recognizing the social interactions of others is often considered outside the scope of vision and grouped with higher-level social cognition like theory of mind. Recent work, however, has revealed that recognition of social interactions is efficient and automatic, is well modeled by bottom-up computational algorithms, and occurs in visually-selective regions of the brain. We review recent evidence from these three methodologies (behavioral, computational, and neural) that converge to suggest the core of social interaction perception is visual. We propose a computational framework for how this process is carried out in the brain and offer directions for future interdisciplinary investigations of social perception.

Relational visual representations underlie human social interaction recognition

Humans effortlessly recognize social interactions from visual input. Attempts to model this ability have typically relied on generative inverse planning models, which make predictions by inverting a generative model of agents' interactions based on their inferred goals, suggesting humans use a similar process of mental inference to recognize interactions. However, growing behavioral and neuroscience evidence suggests that recognizing social interactions is a visual process, separate from complex mental state inference. Yet despite their success in other domains, visual neural network models have been unable to reproduce human-like interaction recognition. We hypothesize that humans rely on relational visual information in particular, and develop a relational, graph neural network model, SocialGNN. Unlike prior models, SocialGNN accurately predicts human interaction judgments across both animated and natural videos. These results suggest that humans can make complex social interaction judgments without an explicit model of the social and physical world, and that structured, relational visual representations are key to this behavior.

Teenagers' but not young adults' beliefs about intrinsic interpersonal obligations for group members

Previous research has indicated that children perceive social category members as having intrinsic obligations toward each other, which shape their expectations for social interactions. However, it is unclear whether teenagers (aged 13 to 15) and young adults (aged 19 to 21) continue to hold such beliefs, given their increased experience with group dynamics and external social rules. To explore this question, three experiments were conducted with a total of 360 participants (N = 180 for each age group). Experiment 1 examined negative social interactions using different methods in two sub-experiments, while Experiment 2 focused on positive social interactions to examine whether participants viewed social category members as intrinsically obligated to avoid harming each other and to offer assistance. Results revealed that teenagers evaluated within-group harm and non-help as unacceptable, regardless of external rules, whereas they viewed between-group harm and non-help as both acceptable and unacceptable, depending on the presence of external rules. Conversely, young adults considered both within-group and between-group harm/non-help as more acceptable if an external rule permitted such behavior. These findings suggest that teenagers believe that members of a social category are intrinsically obligated to help and not harm each other, whereas young adults believe that individual social interactions are constrained mainly by external rules. That is, teenagers hold stronger beliefs than young adults about intrinsic interpersonal obligations to group members. Thus, in-group moral obligations and external rules contribute differently to the evaluation and interpretation of social interactions at different developmental stages.

Compositionality in visual perception

Quilty-Dunn et al.'s wide-ranging defense of the Language of Thought Hypothesis (LoTH) argues that vision traffics in abstract, structured representational formats. We agree: Vision, like language, is compositional - just as words compose into phrases, many visual representations contain discrete constituents that combine in systematic ways. Here, we amass evidence extending this proposal, and explore its implications for how vision interfaces with the rest of the mind.

Toward biologically plausible artificial vision

Quilty-Dunn et al. argue that deep convolutional neural networks (DCNNs) optimized for image classification exemplify structural disanalogies to human vision. A different kind of artificial vision - found in reinforcement-learning agents navigating artificial three-dimensional environments - can be expected to be more human-like. Recent work suggests that language-like representations substantially improves these agents' performance, lending some indirect support to the language-of-thought hypothesis (LoTH).

Grounding Intuitive Physics in Perceptual Experience

This review article explores the foundation of laypeople's understanding of the physical world rooted in perceptual experience. Beginning with a concise historical overview of the study of intuitive physics, the article presents the hypothesis that laypeople possess accurate internalized representations of physical laws. A key aspect of this hypothesis is the contention that correct representations of physical laws emerge in ecological experimental conditions, where the scenario being examined resembles everyday life experiences. The article critically examines empirical evidence both supporting and challenging this claim, revealing that despite everyday-life-like conditions, fundamental misconceptions often persist. Many of these misconceptions can be attributed to a domain-general heuristic that arises from the overgeneralization of perceptual-motor experiences with physical objects. To conclude, the article delves into ongoing controversies and highlights promising future avenues in the field of intuitive physics, including action-judgment dissociations, insights from developmental psychology, and computational models integrating artificial intelligence.

Social Concepts Simplify Complex Reinforcement Learning

Humans often generalize rewarding experiences across abstract social roles. Theories of reward learning suggest that people generalize through model-based learning, but such learning is cognitively costly. Why do people seem to generalize across social roles with ease? Humans are social experts who easily recognize social roles that reflect familiar semantic concepts (e.g., "helper" or "teacher"). People may associate these roles with model-free reward (e.g., learning that helpers are rewarding), allowing them to generalize easily (e.g., interacting with novel individuals identified as helpers). In four online experiments with U.S. adults (N = 577), we found evidence that social concepts ease complex learning (people generalize more and at faster speed) and that people attach reward directly to abstract roles (they generalize even when roles are unrelated to task structure). These results demonstrate how familiar concepts allow complex behavior to emerge from simple strategies, highlighting social interaction as a prototype for studying cognitive ease in the face of environmental complexity.

The psychophysics of bouncing: Perceptual constraints, physical constraints, animacy, and phenomenal causality

In the present study we broadly explored the perception of physical and animated motion in bouncing-like scenarios through four experiments. In the first experiment, participants were asked to categorize bouncing-like displays as physical bounce, animated motion, or other. Several parameters of the animations were manipulated, that is, the simulated coefficient of restitution, the value of simulated gravitational acceleration, the motion pattern (uniform acceleration/deceleration or constant speed) and the number of bouncing cycles. In the second experiment, a variable delay at the moment of the collision between the bouncing object and the bouncing surface was introduced. Main results show that, although observers appear to have realistic representations of physical constraints like energy conservation and gravitational acceleration/deceleration, the amount of visual information available in the scene has a strong modulation effect on the extent to which they rely on these representations. A coefficient of restitution >1 was a crucial cue to animacy in displays showing three bouncing cycles, but not in displays showing one bouncing cycle. Additionally, bouncing impressions appear to be driven by perceptual constraints that are unrelated to the physical realism of the scene, like preference for simulated gravitational attraction smaller than g and perceived temporal contiguity between the different phases of bouncing. In the third experiment, the visible opaque bouncing surface was removed from the scene, and the results showed that this did not have any substantial effect on the resulting impressions of physical bounce or animated motion, suggesting that the visual system can fill-in the scene with the missing element. The fourth experiment explored visual impressions of causality in bouncing scenarios. At odds with claims of current causal perception theories, results indicate that a passive object can be perceived as the direct cause of the motion behavior of an active object.

Disentangling Abstraction from Statistical Pattern Matching in Human and Machine Learning

The ability to acquire abstract knowledge is a hallmark of human intelligence and is believed by many to be one of the core differences between humans and neural network models. Agents can be endowed with an inductive bias towards abstraction through meta-learning, where they are trained on a distribution of tasks that share some abstract structure that can be learned and applied. However, because neural networks are hard to interpret, it can be difficult to tell whether agents have learned the underlying abstraction, or alternatively statistical patterns that are characteristic of that abstraction. In this work, we compare the performance of humans and agents in a meta-reinforcement learning paradigm in which tasks are generated from abstract rules. We define a novel methodology for building "task metamers" that closely match the statistics of the abstract tasks but use a different underlying generative process, and evaluate performance on both abstract and metamer tasks. We find that humans perform better at abstract tasks than metamer tasks whereas common neural network architectures typically perform worse on the abstract tasks than the matched metamers. This work provides a foundation for characterizing differences between humans and machine learning that can be used in future work towards developing machines with more human-like behavior.

Perceiving animacy from kinematics: visual specification of life-likeness in simple geometric patterns

Since the seminal work of Heider and Simmel, and Michotte's research, many studies have shown that, under appropriate conditions, displays of simple geometric shapes elicit rich and vivid impressions of animacy and intentionality. The main purpose of this review is to emphasize the close relationship between kinematics and perceived animacy by showing which specific motion cues and spatiotemporal patterns automatically trigger visual perceptions of animacy and intentionality. The animacy phenomenon has been demonstrated to be rather fast, automatic, irresistible, and highly stimulus-driven. Moreover, there is growing evidence that animacy attributions, although usually associated with higher-level cognition and long-term memory, may reflect highly specialized visual processes that have evolved to support adaptive behaviors critical for survival. The hypothesis of a life-detector hardwired in the perceptual system is also supported by recent studies in early development and animal cognition, as well as by the issue of the "irresistibility" criterion, i.e., the persistence of animacy perception in adulthood even in the face of conflicting background knowledge. Finally, further support for the hypothesis that animacy is processed in the earliest stages of vision comes from recent experimental evidence on the interaction of animacy with other visual processes, such as visuomotor performance, visual memory, and speed estimation. Summarizing, the ability to detect animacy in all its nuances may be related to the visual system's sensitivity to those changes in kinematics - considered as a multifactorial relational system - that are associated with the presence of living beings, as opposed to the natural, inert behavior of physically constrained, form-invariant objects, or even mutually independent moving agents. This broad predisposition would allow the observer not only to identify the presence of animates and to distinguish them from inanimate, but also to quickly grasp their psychological, emotional, and social characteristics.

What is "Where": Physical Reasoning Informs Object Location

A central puzzle the visual system tries to solve is: "what is where?" While a great deal of research attempts to model object recognition ("what"), a comparatively smaller body of work seeks to model object location ("where"), especially in perceiving everyday objects. How do people locate an object, right now, in front of them? In three experiments collecting over 35,000 judgements on stimuli spanning different levels of realism (line drawings, real images, and crude forms), participants clicked "where" an object is, as if pointing to it. We modeled their responses with eight different methods, including both human response-based models (judgements of physical reasoning, spatial memory, free-response "click anywhere" judgements, and judgements of where people would grab the object), and image-based models (uniform distributions over the image, convex hull, saliency map, and medial axis). Physical reasoning was the best predictor of "where," performing significantly better than even spatial memory and free-response judgements. Our results offer insight into the perception of object locations while also raising interesting questions about the relationship between physical reasoning and visual perception.

Similarity and structured representation in human and nonhuman apes

How we judge the similarity between objects in the world is connected ultimately to how we represent those objects. It has been argued extensively that object representations in humans are 'structured' in nature, meaning that both individual features and the relations between them can influence similarity. In contrast, popular models within comparative psychology assume that nonhuman species appreciate only surface-level, featural similarities. By applying psychological models of structural and featural similarity (from conjunctive feature models to Tversky's Contrast Model) to visual similarity judgements from adult humans, chimpanzees, and gorillas, we demonstrate a cross-species sensitivity to complex structural information, particularly for stimuli that combine colour and shape. These results shed new light on the representational complexity of nonhuman apes, and the fundamental limits of featural coding in explaining object representation and similarity, which emerge strikingly across both human and nonhuman species.

Seeing fast and thinking slow Ned Block Oxford University Press, 2023. 560 pp

A philosopher explores perception and cognition.

Gene Functional Networks from Time Expression Profiles: A Constructive Approach Demonstrated in Chili Pepper (Capsicum annuum L.)

Gene co-expression networks are powerful tools to understand functional interactions between genes. However, large co-expression networks are difficult to interpret and do not guarantee that the relations found will be true for different genotypes. Statistically verified time expression profiles give information about significant changes in expressions through time, and genes with highly correlated time expression profiles, which are annotated in the same biological process, are likely to be functionally connected. A method to obtain robust networks of functionally related genes will be useful to understand the complexity of the transcriptome, leading to biologically relevant insights. We present an algorithm to construct gene functional networks for genes annotated in a given biological process or other aspects of interest. We assume that there are genome-wide time expression profiles for a set of representative genotypes of the species of interest. The method is based on the correlation of time expression profiles, bound by a set of thresholds that assure both, a given false discovery rate, and the discard of correlation outliers. The novelty of the method consists in that a gene expression relation must be repeatedly found in a given set of independent genotypes to be considered valid. This automatically discards relations particular to specific genotypes, assuring a network robustness, which can be set a priori. Additionally, we present an algorithm to find transcription factors candidates for regulating hub genes within a network. The algorithms are demonstrated with data from a large experiment studying gene expression during the development of the fruit in a diverse set of chili pepper genotypes. The algorithm is implemented and demonstrated in a new version of the publicly available R package "Salsa" (version 1.0).

Configural relations in humans and deep convolutional neural networks

Deep convolutional neural networks (DCNNs) have attracted considerable interest as useful devices and as possible windows into understanding perception and cognition in biological systems. In earlier work, we showed that DCNNs differ dramatically from human perceivers in that they have no sensitivity to global object shape. Here, we investigated whether those findings are symptomatic of broader limitations of DCNNs regarding the use of relations. We tested learning and generalization of DCNNs (AlexNet and ResNet-50) for several relations involving objects. One involved classifying two shapes in an otherwise empty field as same or different. Another involved enclosure. Every display contained a closed figure among contour noise fragments and one dot; correct responding depended on whether the dot was inside or outside the figure. The third relation we tested involved a classification that depended on which of two polygons had more sides. One polygon always contained a dot, and correct classification of each display depended on whether the polygon with the dot had a greater number of sides. We used DCNNs that had been trained on the ImageNet database, and we used both restricted and unrestricted transfer learning (connection weights at all layers could change with training). For the same-different experiment, there was little restricted transfer learning (82.2%). Generalization tests showed near chance performance for new shapes. Results for enclosure were at chance for restricted transfer learning and somewhat better for unrestricted (74%). Generalization with two new kinds of shapes showed reduced but above-chance performance (≈66%). Follow-up studies indicated that the networks did not access the enclosure relation in their responses. For the relation of more or fewer sides of polygons, DCNNs showed successful learning with polygons having 3-5 sides under unrestricted transfer learning, but showed chance performance in generalization tests with polygons having 6-10 sides. Experiments with human observers showed learning from relatively few examples of all of the relations tested and complete generalization of relational learning to new stimuli. These results using several different relations suggest that DCNNs have crucial limitations that derive from their lack of computations involving abstraction and relational processing of the sort that are fundamental in human perception.

The best game in town: The reemergence of the language-of-thought hypothesis across the cognitive sciences

Mental representations remain the central posits of psychology after many decades of scrutiny. However, there is no consensus about the representational format(s) of biological cognition. This paper provides a survey of evidence from computational cognitive psychology, perceptual psychology, developmental psychology, comparative psychology, and social psychology, and concludes that one type of format that routinely crops up is the language-of-thought (LoT). We outline six core properties of LoTs: (i) discrete constituents; (ii) role-filler independence; (iii) predicate-argument structure; (iv) logical operators; (v) inferential promiscuity; and (vi) abstract content. These properties cluster together throughout cognitive science. Bayesian computational modeling, compositional features of object perception, complex infant and animal reasoning, and automatic, intuitive cognition in adults all implicate LoT-like structures. Instead of regarding LoT as a relic of the previous century, researchers in cognitive science and philosophy-of-mind must take seriously the explanatory breadth of LoT-based architectures. We grant that the mind may harbor many formats and architectures, including iconic and associative structures as well as deep-neural-network-like architectures. However, as computational/representational approaches to the mind continue to advance, classical compositional symbolic structures - that is, LoTs - only prove more flexible and well-supported over time.

Problems and Mysteries of the Many Languages of Thought

"What is the structure of thought?" is as central a question as any in cognitive science. A classic answer to this question has appealed to a Language of Thought (LoT). We point to emerging research from disparate branches of the field that supports the LoT hypothesis, but also uncovers diversity in LoTs across cognitive systems, stages of development, and species. Our letter formulates open research questions for cognitive science concerning the varieties of rules and representations that underwrite various LoT-based systems and how these variations can help researchers taxonomize cognitive systems.

The evolutionary origins of syntax: Event cognition in nonhuman primates

Languages tend to encode events from the perspective of agents, placing them first and in simpler forms than patients. This agent bias is mirrored by cognition: Agents are more quickly recognized than patients and generally attract more attention. This leads to the hypothesis that key aspects of language structure are fundamentally rooted in a cognition that decomposes events into agents, actions, and patients, privileging agents. Although this type of event representation is almost certainly universal across languages, it remains unclear whether the underlying cognition is uniquely human or more widespread in animals. Here, we review a range of evidence from primates and other animals, which suggests that agent-based event decomposition is phylogenetically older than humans. We propose a research program to test this hypothesis in great apes and human infants, with the goal to resolve one of the major questions in the evolution of language, the origins of syntax.

The spatial distance compression effect is due to social interaction and not mere configuration

In recent years, there has been a surge of interest in perception, evaluation, and memory for social interactions from a third-person perspective. One intriguing finding is a spatial distance compression effect when target dyads are facing each other. Specifically, face-to-face dyads are remembered as being spatially closer than back-to-back dyads. There is a vibrant debate about the mechanism behind this effect, and two hypotheses have been proposed. According to the social interaction hypothesis, face-to-face dyads engage a binding process that represents them as a social unit, which compresses the perceived distance between them. In contrast, the configuration hypothesis holds that the effect is produced by the front-to-front configuration of the two visual targets. In the present research we sought to test these accounts. In Experiment 1 we successfully replicated the distance compression effect with two upright faces that were facing each other, but not with inverted faces. In contrast, we found no distance compression effect with three types of nonsocial stimuli: arrows (Experiment 2a), fans (Experiment 2b), and cars (Experiment 3). In Experiment 4, we replicated this effect with another social stimuli: upright bodies. Taken together, these results provide strong support for the social interaction hypothesis.

"Impossible" Somatosensation and the (Ir)rationality of Perception

Impossible figures represent the world in ways it cannot be. From the work of M. C. Escher to any popular perception textbook, such experiences show how some principles of mental processing can be so entrenched and inflexible as to produce absurd and even incoherent outcomes that could not occur in reality. However, impossible experiences of this sort are mostly limited to visual perception; are there “impossible figures” for other sensory modalities? Here, we import a known magic trick into the laboratory to report and investigate an impossible experience for somatosensation—one that can be physically felt. We show that, even under full-cue conditions with objects that can be freely inspected, subjects can be made to experience a single object alone as feeling heavier than a group of objects that includes the single object as a member—an impossible and phenomenologically striking experience of weight. Moreover, we suggest that this phenomenon—a special case of the size-weight illusion—reflects a kind of “anti-Bayesian” perceptual updating that amplifies a challenge to rational models of perception and cognition.

Physically Implied Surfaces

In addition to seeing objects that are directly in view, we also represent objects that are merely implied (e.g., by occlusion, motion, and other cues). What can imply the presence of an object? Here, we explored (in three preregistered experiments; N = 360 adults) the role of physical interaction in creating impressions of objects that are not actually present. After seeing an actor collide with an invisible wall or step onto an invisible box, participants gave facilitated responses to actual, visible surfaces that appeared where the implied wall or box had been-a Stroop-like pattern of facilitation and interference that suggested automatic inferences about the relevant implied surfaces. Follow-up experiments ruled out confounding geometric cues and anticipatory responses. We suggest that physical interactions can trigger representations of the participating surfaces such that we automatically infer the presence of objects implied only by their physical consequences.