Solving the binding problem: Assemblies form when neurons enhance their firing rate-they don't need to oscillate or synchronize

Perceptual and attentional impairments of conscious access involve distinct neural mechanisms despite equal task performance

This study investigates failures in conscious access resulting from either weak sensory input (perceptual impairments) or unattended input (attentional impairments). Participants viewed a Kanizsa stimulus with or without an illusory triangle within a rapid serial visual presentation of distractor stimuli. We designed a novel Kanizsa stimulus that contained additional ancillary features of different complexity (local contrast and collinearity) that were independently manipulated. Perceptual performance on the Kanizsa stimulus (presence vs. absence of an illusion) was equated between the perceptual (masking) and attentional (attentional blink) manipulation to circumvent common confounds related to conditional differences in task performance. We trained and tested classifiers on electroencephalogram (EEG) data to reflect the processing of specific stimulus features, with increasing levels of complexity. We show that late stages of processing (~200-250 ms), reflecting the integration of complex stimulus features (collinearity, illusory triangle), were impaired by masking but spared by the attentional blink. In contrast, decoding of local contrast (the spatial arrangement of stimulus features) was observed early in time (~80 ms) and was left largely unaffected by either manipulation. These results replicate previous work showing that feedforward processing is largely preserved under both perceptual and attentional impairments. Crucially, however, under matched levels of performance, only attentional impairments left the processing of more complex visual features relatively intact, likely related to spared lateral and local feedback processes during inattention. These findings reveal distinct neural mechanisms associated with perceptual and attentional impairments and thus contribute to a comprehensive understanding of distinct neural stages leading to conscious access.

A model of thalamo-cortical interaction for incremental binding in mental contour-tracing

Object-basd visual attention marks a key process of mammalian perception. By which mechanisms this process is implemented and how it can be interacted with by means of attentional control is not completely understood yet. Incremental binding is a mechanism required in demanding scenarios of object-based attention and is experimentally well investigated. Attention spreads across a representation of the visual object and labels bound elements by constant up-modulation of neural activity. The speed of incremental binding was found to be dependent on the spatial arrangement of distracting elements in the scene and to be scale invariant giving rise to the growth-cone hypothesis. In this work, we propose a neural dynamical model of incremental binding that provides a mechanistic account for these findings. Through simulations, we investigate the model properties and demonstrate how an attentional spreading mechanism tags neurons that participate in the object binding process. They utilize Gestalt properties and eventually show growth-cone characteristics labeling perceptual items by delayed activity enhancement of neuronal firing rates. We discuss the algorithmic process underlying incremental binding and relate it to our model computations. This theoretical investigation encompasses complexity considerations and finds the model to be not only of explanatory value in terms of neurophysiological evidence, but also to be an efficient implementation of incremental binding striving to establish a normative account. By relating the connectivity motifs of the model to neuroanatomical evidence, we suggest thalamo-cortical interactions to be a likely candidate for the flexible and efficient realization suggested by the model. There, pyramidal cells are proposed to serve as the processors of incremental grouping information. Local bottom-up evidence about stimulus features is integrated via basal dendritic sites. It is combined with an apical signal consisting of contextual grouping information which is gated by attentional task-relevance selection mediated via higher-order thalamic representations.

Thoughtseeds: A Hierarchical and Agentic Framework for Investigating Thought Dynamics in Meditative States

The Thoughtseeds Framework introduces a novel computational approach to modeling thought dynamics in meditative states, conceptualizing thoughtseeds as dynamic attentional agents that integrate information. This hierarchical model, structured as nested Markov blankets, comprises three interconnected levels: (i) knowledge domains as information repositories, (ii) the Thoughtseed Network where thoughtseeds compete, and (iii) meta-cognition regulating awareness. It simulates focused-attention Vipassana meditation via rule-based training informed by empirical neuroscience research on attentional stability and neural dynamics. Four states-breath_control, mind_wandering, meta_awareness, and redirect_breath-emerge organically from thoughtseed interactions, demonstrating self-organizing dynamics. Results indicate that experts sustain control dominance to reinforce focused attention, while novices exhibit frequent, prolonged mind_wandering episodes, reflecting beginner instability. Integrating Global Workspace Theory and the Intrinsic Ignition Framework, the model elucidates how thoughtseeds shape a unitary meditative experience through meta-awareness, balancing epistemic and pragmatic affordances via active inference. Synthesizing computational modeling with phenomenological insights, it provides an embodied perspective on cognitive state emergence and transitions, offering testable predictions about meditation skill development. The framework yields insights into attention regulation, meta-cognitive awareness, and meditation state emergence, establishing a versatile foundation for future research into diverse meditation practices (e.g., Open Monitoring, Non-Dual Awareness), cognitive development across the lifespan, and clinical applications in mindfulness-based interventions for attention disorders, advancing our understanding of the nature of mind and thought.

A neural computational framework for face processing in the human temporal lobe

A key question in cognitive neuroscience is how unified identity representations emerge from visual inputs. Here, we recorded intracranial electroencephalography (iEEG) from the human ventral temporal cortex (VTC) and medial temporal lobe (MTL), as well as single-neuron activity in the MTL, to demonstrate how dense feature-based representations in the VTC are translated into sparse identity-based representations in the MTL. First, we characterized the spatiotemporal neural dynamics of face coding in the VTC and MTL. The VTC, particularly the fusiform gyrus, exhibits robust axis-based feature coding. Remarkably, MTL neurons encode a receptive field within the VTC neural feature space, constructed using VTC neural axes, thereby bridging dense feature and sparse identity representations. We further validated our findings using recordings from a macaque. Lastly, inter-areal interactions between the VTC and MTL provide the physiological basis of this computational framework. Together, we reveal the neurophysiological underpinnings of a computational framework that explains how perceptual information is translated into face identities.

Beyond binding: from modular to natural vision

The classical view of visual cortex organization as a collection of specialized modules processing distinct features like color and motion has profoundly influenced neuroscience for decades. This framework, rooted in historical philosophical distinctions between qualities, gave rise to the 'binding problem': how the brain integrates these separately processed features into coherent percepts. We present converging evidence from electrophysiology, neuroimaging, and lesion studies that challenges this framework. We argue that the binding problem may be an artifact of theoretical assumptions rather than a real computational challenge for the brain. Drawing insights from deep neural networks (DNNs) and recent empirical findings, we propose a framework where the visual cortex represents naturally co-occurring patterns of information rather than processing isolated features that need binding.

Projection Motifs and Wiring Logic of Medial Pulvinar Thalamocortical Axons in the Marmoset Monkey

The medial pulvinar thalamic nucleus (MPu) is an evolutionary novelty of the primate thalamus, prominently expanded in humans. Piecemeal data from studies in various monkey species indicate that MPu axons reach prefrontal, inferior parietal, cingulate, insular, or temporal areas; however, the precise wiring and functional logic of such brain-wide connections remain obscure. In marmoset monkeys (Callithrix jacchus) of both sexes, we visualized the axons originated from specific pulvinar domains by means of biotinylated dextran amine microinjections and compared them across multiple cases. In addition, by injecting retrograde tracers in the cortical areas targeted by the pulvinar axons, we investigated the organization of projection cells within MPu and the existence of long-range branched axons. Specific projection motifs reveal a caudal MPu subnucleus that innervates inferior and ventral temporal areas and a rostral MPu subnucleus that innervates temporal, ventral prefrontal, premotor, inferior posterior parietal, and cingulate areas. We demonstrate numerous MPu neurons that innervate through branched axons prefrontal and parietal or prefrontal and temporal areas; other cells with different projection patterns are closely intermingled with them. Our findings support the notion that MPu is a hub of the brain-wide networks that support complex visual and social cognition, sensory-guided reaching, working memory, and attention. Moreover, the finding of long-range branching MPu axons and dense terminal arborizations suggest that MPu cells may regulate functional connectivity among high-level cortical areas at different spatial scales. Besides, the anatomical "ground truth" provided by our study is relevant for functional imaging and distributed network modeling studies.

Integrated Information Theory and the Phenomenal Binding Problem: Challenges and Solutions in a Dynamic Framework

Theories of consciousness grounded in neuroscience must explain the phenomenal binding problem, e.g., how micro-units of information are combined to create the macro-scale conscious experience common to human phenomenology. An example is how single 'pixels' of a visual scene are experienced as a single holistic image in the 'mind's eye', rather than as individual, separate, and massively parallel experiences, corresponding perhaps to individual neuron activations, neural ensembles, or foveal saccades, any of which could conceivably deliver identical functionality from an information processing point of view. There are multiple contested candidate solutions to the phenomenal binding problem. This paper explores how the metaphysical infrastructure of Integrated Information Theory (IIT) v4.0 can provide a distinctive solution. The solution-that particular entities aggregable from multiple units ('complexes') define existence-might work in a static picture, but introduces issues in a dynamic system. We ask what happens to our phenomenal self as the main complex moves around a biological neural network. Our account of conscious entities developing through time leads to an apparent dilemma for IIT theorists between non-local entity transitions and contiguous selves: the 'dynamic entity evolution problem'. As well as specifying the dilemma, we describe three ways IIT might dissolve the dilemma before it gains traction. Clarifying IIT's position on the phenomenal binding problem, potentially underpinned with novel empirical or theoretical research, helps researchers understand IIT and assess its plausibility. We see our paper as contributing to IIT's current research emphasis on the shift from static to dynamic analysis.

Decoding the rhythmic representation and communication of visual contents

Rhythmic neural activity is considered essential for adaptively modulating responses in the visual system. In this opinion article we posit that visual brain rhythms also serve a key function in the representation and communication of visual contents. Collating a set of recent studies that used multivariate decoding methods on rhythmic brain signals, we highlight such rhythmic content representations in visual perception, imagery, and prediction. We argue that characterizing representations across frequency bands allows researchers to elegantly disentangle content transfer in feedforward and feedback directions. We further propose that alpha dynamics are central to content-specific feedback propagation in the visual system. We conclude that considering rhythmic content codes is pivotal for understanding information coding in vision and beyond.

Using N2pc variability to probe functionality: Linear mixed modelling of trial EEG and behaviour

This paper has two concurrent goals. On one hand, we hope it will serve as a simple primer in the use of linear mixed modelling (LMM) for inferential statistical analysis of multimodal data. We describe how LMM can be easily adopted for the identification of trial-wise relationships between disparate measures and provide a brief cookbook for assessing the suitability of LMM in your analyses. On the other hand, this paper is an empirical report, probing how trial-wise variance in the N2pc, and specifically its sub-component the NT, can be predicted by manual reaction time (RT) and stimuli parameters. Extant work has identified a link between N2pc and RT that has been interpreted as evidence of a direct and causative relationship. However, results have left open the less-interesting possibility that the measures covary as a function of motivation or arousal. Using LMM, we demonstrate that the relationship only emerges when the NT is elicited by targets, not distractors, suggesting a discrete and functional relationship. In other analyses, we find that the target-elicited NT is sensitive to variance in distractor identity even when the distractor cannot itself elicit consistently lateralized brain activity. The NT thus appears closely linked to attentional target processing, supporting the propagation of target-related information to response preparation and execution. At the same time, we find that this component is sensitive to distractor interference, which leaves open the possibility that NT reflects brain activity responsible for the suppression of irrelevant distractor information.

Evolutionary origins of synchronization for integrating information in neurons

The evolution of brain-expressed genes is notably slower than that of genes expressed in other tissues, a phenomenon likely due to high-level functional constraints. One such constraint might be the integration of information by neuron assemblies, enhancing environmental adaptability. This study explores the physiological mechanisms of information integration in neurons through three types of synchronization: chemical, electromagnetic, and quantum. Chemical synchronization involves the diffuse release of neurotransmitters like dopamine and acetylcholine, causing transmission delays of several milliseconds. Electromagnetic synchronization encompasses action potentials, electrical gap junctions, and ephaptic coupling. Electrical gap junctions enable rapid synchronization within cortical GABAergic networks, while ephaptic coupling allows structures like axon bundles to synchronize through extracellular electromagnetic fields, surpassing the speed of chemical processes. Quantum synchronization is hypothesized to involve ion coherence during ion channel passage and the entanglement of photons within the myelin sheath. Unlike the finite-time synchronization seen in chemical and electromagnetic processes, quantum entanglement provides instantaneous non-local coherence states. Neurons might have evolved from slower chemical diffusion to rapid temporal synchronization, with ion passage through gap junctions within cortical GABAergic networks potentially facilitating both fast gamma band synchronization and quantum coherence. This mini-review compiles literature on these three synchronization types, offering new insights into the physiological mechanisms that address the binding problem in neuron assemblies.

Neural Correlates of Visual Feature Binding

We perceive visual objects as unified although different brain areas process different features. An attentional mechanism has been proposed to be involved with feature binding, as evidenced by observations of binding errors (i.e., illusory conjunctions) when attention is diverted. However, the neural underpinnings of this feature binding are not well understood. We examined the neural mechanisms of feature binding by recording EEG during an attentionally demanding discrimination task. Unlike prestimulus alpha oscillatory activity and early ERPs (i.e., the N1 and P1 components), the N1pc, reflecting stimulus-evoked spatial attention, was reduced for errors relative to correct responses and illusory conjunctions. However, the later sustained posterior contralateral negativity, reflecting visual short-term memory, was reduced for illusory conjunctions and errors compared with correct responses. Furthermore, binding errors were associated with distinct posterior lateralized activity during a 200- to 300-msec window. These results implicate a temporal binding window that integrates visual features after stimulus-evoked attention but before encoding into visual short-term memory.

Thalamic spindles and Up states coordinate cortical and hippocampal co-ripples in humans

In the neocortex, ~90 Hz ripples couple to ~12 Hz sleep spindles on the ~1 Hz Down-to-Up state transition during non-rapid eye movement sleep. This conjunction of sleep waves is critical for the consolidation of memories into long-term storage. The widespread co-occurrences of ripples ("co-ripples") may integrate information across the neocortex and hippocampus to facilitate consolidation. While the thalamus synchronizes spindles and Up states in the cortex for memory, it is not known whether it may also organize co-ripples. Using human intracranial recordings during NREM sleep, we investigated whether cortico-cortical co-ripples and hippocampo-cortical co-ripples are either: (1) driven by directly projected thalamic ripples; or (2) coordinated by propagating thalamic spindles or Up states. We found ripples in the anterior and posterior thalamus, with similar characteristics as hippocampal and cortical ripples, including having a center frequency of ~90 Hz and coupling to local spindles on the Down-to-Up state transition. However, thalamic ripples rarely co-occur or phase-lock with cortical or hippocampal ripples. By contrast, spindles and Up states that propagate from the thalamus strongly coordinate co-ripples in the cortex and hippocampus. Thus, thalamo-cortical spindles and Up states, rather than thalamic ripples, may provide input facilitating spatially distributed co-rippling that integrates information for memory consolidation during sleep in humans.

Semi-orthogonal subspaces for value mediate a binding and generalization trade-off

When choosing between options, we must associate their values with the actions needed to select them. We hypothesize that the brain solves this binding problem through neural population subspaces. Here, in macaques performing a choice task, we show that neural populations in five reward-sensitive regions encode the values of offers presented on the left and right in distinct subspaces. This encoding is sufficient to bind offer values to their locations while preserving abstract value information. After offer presentation, all areas encode the value of the first and second offers in orthogonal subspaces; this orthogonalization also affords binding. Our binding-by-subspace hypothesis makes two new predictions confirmed by the data. First, behavioral errors should correlate with spatial, but not temporal, neural misbinding. Second, behavioral errors should increase when offers have low or high values, compared to medium values, even when controlling for value difference. Together, these results support the idea that the brain uses semi-orthogonal subspaces to bind features.

Visual Processing by Hierarchical and Dynamic Multiplexing

The complexity of natural environments requires highly flexible mechanisms for adaptive processing of single and multiple stimuli. Neuronal oscillations could be an ideal candidate for implementing such flexibility in neural systems. Here, we present a framework for structuring attention-guided processing of complex visual scenes in humans, based on multiplexing and phase coding schemes. Importantly, we suggest that the dynamic fluctuations of excitability vary rapidly in terms of magnitude, frequency and wave-form over time, i.e., they are not necessarily sinusoidal or sustained oscillations. Different elements of single objects would be processed within a single cycle (burst) of alpha activity (7-14 Hz), allowing for the formation of coherent object representations while separating multiple objects across multiple cycles. Each element of an object would be processed separately in time-expressed as different gamma band bursts (>30 Hz)-along the alpha phase. Since the processing capacity per alpha cycle is limited, an inverse relationship between object resolution and size of attentional spotlight ensures independence of the proposed mechanism from absolute object complexity. Frequency and wave-shape of those fluctuations would depend on the nature of the object that is processed and on cognitive demands. Multiple objects would further be organized along the phase of slower fluctuations (e.g., theta), potentially driven by saccades. Complex scene processing, involving covert attention and eye movements, would therefore be associated with multiple frequency changes in the alpha and lower frequency range. This framework embraces the idea of a hierarchical organization of visual processing, independent of environmental temporal dynamics.

Binding of cortical functional modules by synchronous high-frequency oscillations

Whether high-frequency phase-locked oscillations facilitate integration ('binding') of information across widespread cortical areas is controversial. Here we show with intracranial electroencephalography that cortico-cortical co-ripples (~100-ms-long ~90 Hz oscillations) increase during reading and semantic decisions, at the times and co-locations when and where binding should occur. Fusiform wordform areas co-ripple with virtually all language areas, maximally from 200 to 400 ms post-word-onset. Semantically specified target words evoke strong co-rippling between wordform, semantic, executive and response areas from 400 to 800 ms, with increased co-rippling between semantic, executive and response areas prior to correct responses. Co-ripples were phase-locked at zero lag over long distances (>12 cm), especially when many areas were co-rippling. General co-activation, indexed by non-oscillatory high gamma, was mainly confined to early latencies in fusiform and earlier visual areas, preceding co-ripples. These findings suggest that widespread synchronous co-ripples may assist the integration of multiple cortical areas for sustained periods during cognition.

High frequency oscillations in human memory and cognition: a neurophysiological substrate of engrams?

Despite advances in understanding the cellular and molecular processes underlying memory and cognition, and recent successful modulation of cognitive performance in brain disorders, the neurophysiological mechanisms remain underexplored. High frequency oscillations beyond the classic electroencephalogram spectrum have emerged as a potential neural correlate of fundamental cognitive processes. High frequency oscillations are detected in the human mesial temporal lobe and neocortical intracranial recordings spanning gamma/epsilon (60-150 Hz), ripple (80-250 Hz) and higher frequency ranges. Separate from other non-oscillatory activities, these brief electrophysiological oscillations of distinct duration, frequency and amplitude are thought to be generated by coordinated spiking of neuronal ensembles within volumes as small as a single cortical column. Although the exact origins, mechanisms and physiological roles in health and disease remain elusive, they have been associated with human memory consolidation and cognitive processing. Recent studies suggest their involvement in encoding and recall of episodic memory with a possible role in the formation and reactivation of memory traces. High frequency oscillations are detected during encoding, throughout maintenance, and right before recall of remembered items, meeting a basic definition for an engram activity. The temporal coordination of high frequency oscillations reactivated across cortical and subcortical neural networks is ideally suited for integrating multimodal memory representations, which can be replayed and consolidated during states of wakefulness and sleep. High frequency oscillations have been shown to reflect coordinated bursts of neuronal assembly firing and offer a promising substrate for tracking and modulation of the hypothetical electrophysiological engram.

Distinct oscillatory patterns differentiate between segregation and integration processes in perceptual grouping

Recently, there has been a resurgence in experimental and conceptual efforts to understand how brain rhythms can serve to organize visual information. Oscillations can provide temporal structure for neuronal processing and form a basis for integrating information across brain areas. Here, we use a bistable paradigm and a data-driven approach to test the hypothesis that oscillatory modulations associate with the integration or segregation of visual elements. Spectral signatures of perception of bound and unbound configurations of visual moving stimuli were studied using magnetoencephalography (MEG) in ambiguous and unambiguous conditions. Using a 2 × 2 design, we were able to isolate correlates from visual integration, either perceptual or stimulus-driven, from attentional and ambiguity-related activity. Two frequency bands were found to be modulated by visual integration: an alpha/beta frequency and a higher frequency gamma-band. Alpha/beta power was increased in several early visual cortical and dorsal visual areas during visual integration, while gamma-band power was surprisingly increased in the extrastriate visual cortex during segregation. This points to an integrative role for alpha/beta activity, likely from top-down signals maintaining a single visual representation. On the other hand, when more representations have to be processed in parallel gamma-band activity is increased, which is at odds with the notion that gamma oscillations are related to perceptual coherence. These modulations were confirmed in intracranial EEG recordings and partially originate from distinct brain areas. Our MEG and stereo-EEG data confirms predictions of binding mechanisms depending on low-frequency activity for long-range integration and for organizing visual processing while refuting a straightforward correlation between gamma-activity and perceptual binding. PRACTITIONER POINTS: Distinct neurophysiological signals underlie competing bistable percepts. Increased alpha/beta activity correlate with visual integration while gamma correlates with segmentation. Ambiguous percepts drive alpha/beta activity in the posterior cingulate cortex.

An integrative view of the role of prefrontal cortex in consciousness

The involvement of the prefrontal cortex (PFC) in consciousness is an ongoing focus of intense investigation. An important question is whether representations of conscious contents and experiences in the PFC are confounded by post-perceptual processes related to cognitive functions. Here, I review recent findings suggesting that neuronal representations of consciously perceived contents-in the absence of post-perceptual processes-can indeed be observed in the PFC. Slower ongoing fluctuations in the electrophysiological state of the PFC seem to control the stability and updates of these prefrontal representations of conscious awareness. In addition to conscious perception, the PFC has been shown to play a critical role in controlling the levels of consciousness as observed during anesthesia, while prefrontal lesions can result in severe loss of perceptual awareness. Together, the convergence of these processes in the PFC suggests its integrative role in consciousness and highlights the complex nature of consciousness itself.

Axonal stimulation affects the linear summation of single-point perception in three Argus II users

Objective.Retinal implants use electrical stimulation to elicit perceived flashes of light ('phosphenes'). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation.Approach.We retrospectively analyzed 3548 phosphene drawings made by three blind participants implanted with an Argus II Retinal Prosthesis. Phosphene shape (characterized by area, perimeter, major and minor axis length) and number of perceived phosphenes were averaged across trials and correlated with the corresponding single-electrode parameters. In addition, the number of phosphenes was correlated with stimulus amplitude and neuroanatomical parameters: electrode-retina and electrode-fovea distance as well as the electrode-electrode distance to ('between-axon') and along axon bundles ('along-axon'). Statistical analyses were conducted using linear regression and partial correlation analysis.Main results.Simple regression revealed that each paired-electrode shape descriptor could be predicted by the sum of the two corresponding single-electrode shape descriptors (p < .001). Multiple regression revealed that paired-electrode phosphene shape was primarily predicted by stimulus amplitude and electrode-fovea distance (p < .05). Interestingly, the number of elicited phosphenes tended to increase with between-axon distance (p < .05), but not with along-axon distance, in two out of three participants.Significance.The shape of phosphenes elicited by paired-electrode stimulation was well predicted by the shape of their corresponding single-electrode phosphenes, suggesting that two-point perception can be expressed as the linear summation of single-point perception. The impact of the between-axon distance on the perceived number of phosphenes provides further evidence in support of the axon map model for epiretinal stimulation. These findings contribute to the growing literature on phosphene perception and have important implications for the design of future retinal prostheses.

Recurrent neural networks that learn multi-step visual routines with reinforcement learning

Many cognitive problems can be decomposed into series of subproblems that are solved sequentially by the brain. When subproblems are solved, relevant intermediate results need to be stored by neurons and propagated to the next subproblem, until the overarching goal has been completed. We will here consider visual tasks, which can be decomposed into sequences of elemental visual operations. Experimental evidence suggests that intermediate results of the elemental operations are stored in working memory as an enhancement of neural activity in the visual cortex. The focus of enhanced activity is then available for subsequent operations to act upon. The main question at stake is how the elemental operations and their sequencing can emerge in neural networks that are trained with only rewards, in a reinforcement learning setting. We here propose a new recurrent neural network architecture that can learn composite visual tasks that require the application of successive elemental operations. Specifically, we selected three tasks for which electrophysiological recordings of monkeys' visual cortex are available. To train the networks, we used RELEARNN, a biologically plausible four-factor Hebbian learning rule, which is local both in time and space. We report that networks learn elemental operations, such as contour grouping and visual search, and execute sequences of operations, solely based on the characteristics of the visual stimuli and the reward structure of a task. After training was completed, the activity of the units of the neural network elicited by behaviorally relevant image items was stronger than that elicited by irrelevant ones, just as has been observed in the visual cortex of monkeys solving the same tasks. Relevant information that needed to be exchanged between subroutines was maintained as a focus of enhanced activity and passed on to the subsequent subroutines. Our results demonstrate how a biologically plausible learning rule can train a recurrent neural network on multistep visual tasks.

Representation of Natural Contours by a Neural Population in Monkey V4

The cortical visual area, V4, has been considered to code contours that contribute to the intermediate-level representation of objects. The neural responses to the complex contour features intrinsic to natural contours are expected to clarify the essence of the representation. To approach the cortical coding of natural contours, we investigated the simultaneous coding of multiple contour features in monkey (Macaca fuscata) V4 neurons and their population-level representation. A substantial number of neurons showed significant tuning for two or more features such as curvature and closure, indicating that a substantial number of V4 neurons simultaneously code multiple contour features. A large portion of the neurons responded vigorously to acutely curved contours that surrounded the center of classical receptive field, suggesting that V4 neurons tend to code prominent features of object contours. The analysis of mutual information (MI) between the neural responses and each contour feature showed that most neurons exhibited similar magnitudes for each type of MI, indicating that many neurons showing the responses depended on multiple contour features. We next examined the population-level representation by using multidimensional scaling analysis. The neural preferences to the multiple contour features and that to natural stimuli compared with silhouette stimuli increased along with the primary and secondary axes, respectively, indicating the contribution of the multiple contour features and surface textures in the population responses. Our analyses suggested that V4 neurons simultaneously code multiple contour features in natural images and represent contour and surface properties in population.

Co-occurring ripple oscillations facilitate neuronal interactions between cortical locations in humans

How the human cortex integrates ("binds") information encoded by spatially distributed neurons remains largely unknown. One hypothesis suggests that synchronous bursts of high-frequency oscillations ("ripples") contribute to binding by facilitating integration of neuronal firing across different cortical locations. While studies have demonstrated that ripples modulate local activity in the cortex, it is not known whether their co-occurrence coordinates neural firing across larger distances. We tested this hypothesis using local field-potentials and single-unit firing from four 96-channel microelectrode arrays in the supragranular cortex of 3 patients. Neurons in co-rippling locations showed increased short-latency co-firing, prediction of each other's firing, and co-participation in neural assemblies. Effects were similar for putative pyramidal and interneurons, during non-rapid eye movement sleep and waking, in temporal and Rolandic cortices, and at distances up to 16 mm (the longest tested). Increased co-prediction during co-ripples was maintained when firing-rate changes were equated, indicating that it was not secondary to non-oscillatory activation. Co-rippling enhanced prediction was strongly modulated by ripple phase, supporting the most common posited mechanism for binding-by-synchrony. Co-ripple enhanced prediction is reciprocal, synergistic with local upstates, and further enhanced when multiple sites co-ripple, supporting re-entrant facilitation. Together, these results support the hypothesis that trans-cortical co-occurring ripples increase the integration of neuronal firing of neurons in different cortical locations and do so in part through phase-modulation rather than unstructured activation.

Measuring uncertainty in human visual segmentation

Segmenting visual stimuli into distinct groups of features and visual objects is central to visual function. Classical psychophysical methods have helped uncover many rules of human perceptual segmentation, and recent progress in machine learning has produced successful algorithms. Yet, the computational logic of human segmentation remains unclear, partially because we lack well-controlled paradigms to measure perceptual segmentation maps and compare models quantitatively. Here we propose a new, integrated approach: given an image, we measure multiple pixel-based same-different judgments and perform model-based reconstruction of the underlying segmentation map. The reconstruction is robust to several experimental manipulations and captures the variability of individual participants. We demonstrate the validity of the approach on human segmentation of natural images and composite textures. We show that image uncertainty affects measured human variability, and it influences how participants weigh different visual features. Because any putative segmentation algorithm can be inserted to perform the reconstruction, our paradigm affords quantitative tests of theories of perception as well as new benchmarks for segmentation algorithms.

Co-occurring ripple oscillations facilitate neuronal interactions between cortical locations in humans

Synchronous bursts of high frequency oscillations ('ripples') are hypothesized to contribute to binding by facilitating integration of neuronal firing across cortical locations. We tested this hypothesis using local field-potentials and single-unit firing from four 96-channel microelectrode arrays in supragranular cortex of 3 patients. Neurons in co-rippling locations showed increased short-latency co-firing, prediction of each-other's firing, and co-participation in neural assemblies. Effects were similar for putative pyramidal and interneurons, during NREM sleep and waking, in temporal and Rolandic cortices, and at distances up to 16mm. Increased co-prediction during co-ripples was maintained when firing-rate changes were equated, and were strongly modulated by ripple phase. Co-ripple enhanced prediction is reciprocal, synergistic with local upstates, and further enhanced when multiple sites co-ripple. Together, these results support the hypothesis that trans-cortical co-ripples increase the integration of neuronal firing of neurons in different cortical locations, and do so in part through phase-modulation rather than unstructured activation.

Toward a computational theory of manifold untangling: from global embedding to local flattening

It has been hypothesized that the ventral stream processing for object recognition is based on a mechanism called cortically local subspace untangling. A mathematical abstraction of object recognition by the visual cortex is how to untangle the manifolds associated with different object categories. Such a manifold untangling problem is closely related to the celebrated kernel trick in metric space. In this paper, we conjecture that there is a more general solution to manifold untangling in the topological space without artificially defining any distance metric. Geometrically, we can either embed a manifold in a higher-dimensional space to promote selectivity or flatten a manifold to promote tolerance. General strategies of both global manifold embedding and local manifold flattening are presented and connected with existing work on the untangling of image, audio, and language data. We also discuss the implications of untangling the manifold into motor control and internal representations.