The Binding Problem 2.0: Beyond Perceptual Features

Integrated Information Theory and the Phenomenal Binding Problem: Challenges and Solutions in a Dynamic Framework

Theories of consciousness grounded in neuroscience must explain the phenomenal binding problem, e.g., how micro-units of information are combined to create the macro-scale conscious experience common to human phenomenology. An example is how single 'pixels' of a visual scene are experienced as a single holistic image in the 'mind's eye', rather than as individual, separate, and massively parallel experiences, corresponding perhaps to individual neuron activations, neural ensembles, or foveal saccades, any of which could conceivably deliver identical functionality from an information processing point of view. There are multiple contested candidate solutions to the phenomenal binding problem. This paper explores how the metaphysical infrastructure of Integrated Information Theory (IIT) v4.0 can provide a distinctive solution. The solution-that particular entities aggregable from multiple units ('complexes') define existence-might work in a static picture, but introduces issues in a dynamic system. We ask what happens to our phenomenal self as the main complex moves around a biological neural network. Our account of conscious entities developing through time leads to an apparent dilemma for IIT theorists between non-local entity transitions and contiguous selves: the 'dynamic entity evolution problem'. As well as specifying the dilemma, we describe three ways IIT might dissolve the dilemma before it gains traction. Clarifying IIT's position on the phenomenal binding problem, potentially underpinned with novel empirical or theoretical research, helps researchers understand IIT and assess its plausibility. We see our paper as contributing to IIT's current research emphasis on the shift from static to dynamic analysis.

Foveal vision reduces neural resources in agent-based game learning

Efficient processing of information is crucial for the optimization of neural resources in both biological and artificial visual systems. In this paper, we study the efficiency that may be obtained via the use of a fovea. Using biologically-motivated agents, we study visual information processing, learning, and decision making in a controlled artificial environment, namely the Atari Pong video game. We compare the resources necessary to play Pong between agents with and without a fovea. Our study shows that a fovea can significantly reduce the neural resources, in the form of number of neurons, number of synapses, and number of computations, while at the same time maintaining performance at playing Pong. To our knowledge, this is the first study in which an agent must simultaneously optimize its visual system, along with its decision making and action generation capabilities. That is, the visual system is integral to a complete agent.

A finite set of content-free pointers in visual working memory: magnetoencephalography (MEG) evidence

Human visual working memory (VWM) is known to be capacity-limited, but the nature of this limit continues to be debated. Recent work has proposed that VWM is supported by a finite (~3) set of content-free pointers, acting as stand-ins for individual objects and binding features together. According to this proposal, the pointers do not represent features within themselves, but rather bind features represented elsewhere together. The current study set out to test if neural hallmarks resembling these content-free pointers can be observed with magnetoencephalography (MEG). Based on two VWM delay-match-to-sample experiments (N = 20 each) examining memory for simple and complex objects, we report a sustained response in MEG over right posterior cortex whose magnitude tracks the core hypothesized properties of this content-free pointer system: load-dependent, capacity-limited, and content-free. These results provide novel evidence for a finite set of content-free pointers underlying VWM.

The Visual Integration of Semantic and Spatial Information of Objects in Naturalistic Scenes (VISIONS) database: attentional, conceptual, and perceptual norms

The complex interplay between low- and high-level mechanisms governing our visual system can only be fully understood within ecologically valid naturalistic contexts. For this reason, in recent years, substantial efforts have been devoted to equipping the scientific community with datasets of realistic images normed on semantic or spatial features. Here, we introduce VISIONS, an extensive database of 1136 naturalistic scenes normed on a wide range of perceptual and conceptual norms by 185 English speakers across three levels of granularity: isolated object, whole scene, and object-in-scene. Each naturalistic scene contains a critical object systematically manipulated and normed regarding its semantic consistency (e.g., a toothbrush vs. a flashlight in a bathroom) and spatial position (i.e., left, right). Normative data are also available for low- (i.e., clarity, visual complexity) and high-level (i.e., name agreement, confidence, familiarity, prototypicality, manipulability) features of the critical object and its embedding scene context. Eye-tracking data during a free-viewing task further confirms the experimental validity of our manipulations while theoretically demonstrating that object semantics is acquired in extra-foveal vision and used to guide early overt attention. To our knowledge, VISIONS is the first database exhaustively covering norms about integrating objects in scenes and providing several perceptual and conceptual norms of the two as independently taken. We expect VISIONS to become an invaluable image dataset to examine and answer timely questions above and beyond vision science, where a diversity of perceptual, attentive, mnemonic, or linguistic processes could be explored as they develop, age, or become neuropathological.

Electrophysiological hallmarks for event relations and event roles in working memory

The ability to maintain events (i.e., interactions between/among objects) in working memory is crucial for our everyday cognition, yet the format of this representation is poorly understood. The current ERP study was designed to answer two questions: How is maintaining events (e.g., the tiger hit the lion) neurally different from maintaining item coordinations (e.g., the tiger and the lion)? That is, how is the event relation (present in events but not coordinations) represented? And how is the agent, or initiator of the event encoded differently from the patient, or receiver of the event during maintenance? We used a novel picture-sentence match-across-delay approach in which the working memory representation was "pinged" during the delay, replicated across two ERP experiments with Chinese and English materials. We found that maintenance of events elicited a long-lasting late sustained difference in posterior-occipital electrodes relative to non-events. This effect resembled the negative slow wave reported in previous studies of working memory, suggesting that the maintenance of events in working memory may impose a higher cost compared to coordinations. Although we did not observe significant ERP differences associated with pinging the agent vs. the patient during the delay, we did find that the ping appeared to dampen the ongoing sustained difference, suggesting a shift from sustained activity to activity silent mechanisms. These results suggest a new method by which ERPs can be used to elucidate the format of neural representation for events in working memory.