Object recognition is enabled by an experience-dependent appraisal of visual features in the brain's value system

Interactionally Embedded Gestalt Principles of Multimodal Human Communication

Natural human interaction requires us to produce and process many different signals, including speech, hand and head gestures, and facial expressions. These communicative signals, which occur in a variety of temporal relations with each other (e.g., parallel or temporally misaligned), must be rapidly processed as a coherent message by the receiver. In this contribution, we introduce the notion of interactionally embedded, affordance-driven gestalt perception as a framework that can explain how this rapid processing of multimodal signals is achieved as efficiently as it is. We discuss empirical evidence showing how basic principles of gestalt perception can explain some aspects of unimodal phenomena such as verbal language processing and visual scene perception but require additional features to explain multimodal human communication. We propose a framework in which high-level gestalt predictions are continuously updated by incoming sensory input, such as unfolding speech and visual signals. We outline the constituent processes that shape high-level gestalt perception and their role in perceiving relevance and prägnanz. Finally, we provide testable predictions that arise from this multimodal interactionally embedded gestalt-perception framework. This review and framework therefore provide a theoretically motivated account of how we may understand the highly complex, multimodal behaviors inherent in natural social interaction.

Flexibility of emerging face categorization at different levels of abstraction

Categorization of visual stimuli at different levels of abstraction relies on the encoding of relevant diagnostic features present at different spatial scales. We used the Eidolon Factory, an image-manipulation algorithm that introduces random disarray fields across spatial scales, to study how such a process flexibly combines perceptual information to perform successful categorization depending on task demands. Images of animal faces, human faces, and everyday objects were disarrayed coherently (random fields correlated) or incoherently (random fields randomized) to create a family of 50 eidolons per stimulus image with increasing disarray. Participants (N = 243) viewed each family of eidolons in a smooth sequence from maximum disarray to no disarray and performed a category verification task either at the superordinate (any face type) or basic (human face only) levels at two levels of uncertainty: participants in one group used their gut feeling to respond, whereas another group had to be sure of their decision. When participants used their gut feeling to respond, we observed a superordinate-level advantage. When they were sure of their response, we observed a basic-level advantage. Coherently disarrayed sequences impaired target detection compared to incoherently disarrayed sequences for both levels of response certainty. Furthermore, participants' sensitivity in the Any Face condition increased when they observed coherently disarrayed sequences and had to be sure of their response. These results suggest that the visual system does not strictly adhere to feedforward processing but flexibly adjusts to the relevant perceptual information depending on task context.

Bayesian mechanics of perceptual inference and motor control in the brain

The free energy principle (FEP) in the neurosciences stipulates that all viable agents induce and minimize informational free energy in the brain to fit their environmental niche. In this study, we continue our effort to make the FEP a more physically principled formalism by implementing free energy minimization based on the principle of least action. We build a Bayesian mechanics (BM) by casting the formulation reported in the earlier publication (Kim in Neural Comput 30:2616-2659, 2018, https://doi.org/10.1162/neco_a_01115 ) to considering active inference beyond passive perception. The BM is a neural implementation of variational Bayes under the FEP in continuous time. The resulting BM is provided as an effective Hamilton's equation of motion and subject to the control signal arising from the brain's prediction errors at the proprioceptive level. To demonstrate the utility of our approach, we adopt a simple agent-based model and present a concrete numerical illustration of the brain performing recognition dynamics by integrating BM in neural phase space. Furthermore, we recapitulate the major theoretical architectures in the FEP by comparing our approach with the common state-space formulations.