Information integration without awareness

The search for invertebrate consciousness

There is no agreement on whether any invertebrates are conscious and no agreement on a methodology that could settle the issue. How can the debate move forward? I distinguish three broad types of approach: theory-heavy, theory-neutral and theory-light. Theory-heavy and theory-neutral approaches face serious problems, motivating a middle path: the theory-light approach. At the core of the theory-light approach is a minimal commitment about the relation between phenomenal consciousness and cognition that is compatible with many specific theories of consciousness: the hypothesis that phenomenally conscious perception of a stimulus facilitates, relative to unconscious perception, a cluster of cognitive abilities in relation to that stimulus. This "facilitation hypothesis" can productively guide inquiry into invertebrate consciousness. What is needed? At this stage, not more theory, and not more undirected data gathering. What is needed is a systematic search for consciousness-linked cognitive abilities, their relationships to each other, and their sensitivity to masking.

Temporo-spatial Theory of Consciousness (TTC) - Bridging the gap of neuronal activity and phenomenal states

Consciousness and its neural mechanisms remain a mystery. Current neuroscientific theories focus predominantly on the external input/stimulus and the associated stimulus-related activity during conscious contents. Despite all progress, we encounter two gaps: (i) a gap between spontaneous and stimulus-related activity; (ii) a gap between neuronal and phenomenal features. A novel, different, and unique approach, Temporo-spatial theory of consciousness (TTC) aims to bridge both gaps. The TTC focuses on the brain's spontaneous activity and how its spatial topography and temporal dynamic shape stimulus-related activity and resurface in the corresponding spatial and temporal features of consciousness, i.e., 'common currency'. The TTC introduces four temporo-spatial mechanisms: expansion, globalization, alignment, and nestedness. These are associated with distinct dimensions of consciousness including phenomenal content, access, form/structure, and level/state, respectively. Following up on the first introduction of the TTC in 2017, we review updates, further develop these temporo-spatial mechanisms, and postulate specific neurophenomenal hypotheses. We conclude that the TTC offers a viable approach for (i) linking spontaneous and stimulus-related activity in conscious states; (ii) determining specific neuronal and neurophenomenal mechanisms for the distinct dimensions of consciousness; (iii) an integrative and unifying framework of different neuroscientific theories of consciousness; and (iv) offers novel empirically grounded conceptual assumptions about the biological and ontological nature of consciousness and its relation to the brain.

Regression to the Mean Does Not Explain Away Nonconscious Processing

In studies that use subliminal presentations, participants may become aware of stimuli that are intended to remain subliminal. A common solution to this problem is to analyze the results of the group of participants for whom the stimuli remained subliminal. A recent article (Shanks, 2017) argued that this method leads to a regression to the mean artifact, which may account for many of the observed effects. However, conceptual and statistical characteristics of the original publication lead to overestimation of the influence of the artifact. Using simulations, we demonstrate that this overestimation leads to the mistaken conclusion that regression to the mean accounts for nonconscious effects. We conclude by briefly outlining a new description of the influence of the artifact and how it should be statistically addressed.

Aberrant neural representation of food stimuli in women with acute anorexia nervosa predicts treatment outcome and is improved in weight restored individuals

Anorexia nervosa (AN) has been associated with altered reward processing. We recently reported greater neural response in secondary visual areas when processing visual food stimuli in acutely underweight AN patients (acAN). In order to examine whether the observed alterations are indicative of acute undernutrition or a potential trait marker of AN, we set out to assess neural responses in acAN and in individuals weight-recovered from AN (recAN). FMRI data were collected from a total of 126 female volunteers, 35 acAN, 33 recAN, and 58 age-matched healthy controls (HC) while they viewed streams of food, social and neutral stimuli. A standard general linear model (GLM) was used to interrogate neural responses to the different stimuli in recAN vs. age-matched HC. Moreover, within-subject multivoxel pattern analyses (MVPA) in the two matched samples (acAN/HC and recAN/HC) were used to estimate neural representation of food vs. neutral, and social vs. neutral stimuli. A multiple regression analysis was conducted to test associations between the accuracy of the neural representation and treatment outcome. The GLM revealed no group differences between recAN and HC. The MVPAs showed greater classification accuracy of food stimuli in the posterior fusiform gyrus in acAN but not recAN. Classification accuracy was associated with better treatment outcome. Our findings suggest that the neural representation of food stimuli is altered in secondary visual areas in acAN and normalizes with weight recovery. Possibly this altered representation reflects attentional engagement motivating food intake, which may promote the recovery process.

Refuting the unfolding-argument on the irrelevance of causal structure to consciousness

The unfolding argument (UA) was advanced as a refutation of prominent theories, which posit that phenomenal experience is determined by patterns of neural activation in a recurrent (neural) network (RN) structure. The argument is based on the statement that any input-output function of an RN can be approximated by an "equivalent" feedforward-network (FFN). According to UA, if consciousness depends on causal structure, its presence is unfalsifiable (thus non-scientific), as an equivalent FFN structure is behaviorally indistinguishable with regards to any behavioral test. Here I refute UA by appealing to computational theory and cognitive-neuroscience. I argue that a robust functional equivalence between FFN and RN is not supported by the mathematical work on the Universal Approximator theorem, and is also unlikely to hold, as a conjecture, given data in cognitive neuroscience; I argue that an equivalence of RN and FFN can only apply to static functions between input/output layers and not to the temporal patterns or to the network's reactions to structural perturbations. Finally, I review data indicating that consciousness has functional characteristics, such as a flexible control of behavior, and that cognitive/brain dynamics reveal interacting top-down and bottom-up processes, which are necessary for the mediation of such control processes.

To Grasp the World at a Glance: The Role of Attention in Visual and Semantic Associative Processing

Associative relations among words, concepts and percepts are the core building blocks of high-level cognition. When viewing the world ‘at a glance’, the associative relations between objects in a scene, or between an object and its visual background, are extracted rapidly. The extent to which such relational processing requires attentional capacity, however, has been heavily disputed over the years. In the present manuscript, I review studies investigating scene–object and object–object associative processing. I then present a series of studies in which I assessed the necessity of spatial attention to various types of visual–semantic relations within a scene. Importantly, in all studies, the spatial and temporal aspects of visual attention were tightly controlled in an attempt to minimize unintentional attention shifts from ‘attended’ to ‘unattended’ regions. Pairs of stimuli—either objects, scenes or a scene and an object—were briefly presented on each trial, while participants were asked to detect a pre-defined target category (e.g., an animal, a nonsense shape). Response times (RTs) to the target detection task were registered when visual attention spanned both stimuli in a pair vs. when attention was focused on only one of two stimuli. Among non-prioritized stimuli that were not defined as to-be-detected targets, findings consistently demonstrated rapid associative processing when stimuli were fully attended, i.e., shorter RTs to associated than unassociated pairs. Focusing attention on a single stimulus only, however, largely impaired this relational processing. Notably, prioritized targets continued to affect performance even when positioned at an unattended location, and their associative relations with the attended items were well processed and analyzed. Our findings portray an important dissociation between unattended task-irrelevant and task-relevant items: while the former require spatial attentional resources in order to be linked to stimuli positioned inside the attentional focus, the latter may influence high-level recognition and associative processes via feature-based attentional mechanisms that are largely independent of spatial attention.

Larger capacity for unconscious versus conscious episodic memory

Episodic memory is the memory for experienced events. A peak competence of episodic memory is the mental combination of events to infer commonalities. Inferring commonalities may proceed with and without consciousness of events. Yet what distinguishes conscious from unconscious inference? This question inspired nine experiments that featured strongly and weakly masked cartoon clips presented for unconscious and conscious inference. Each clip featured a scene with a visually impenetrable hiding place. Five animals crossed the scene one-by-one consecutively. One animal trajectory represented one event. The animals moved through the hiding place, where they might linger or not. The participants' task was to observe the animals' entrances and exits to maintain a mental record of which animals hid simultaneously. We manipulated information load to explore capacity limits. Memory of inferences was tested immediately, 3.5 or 6 min following encoding. The participants retrieved inferences well when encoding was conscious. When encoding was unconscious, the participants needed to respond intuitively. Only habitually intuitive decision makers exhibited a significant delayed retrieval of inferences drawn unconsciously. Their unconscious retrieval performance did not drop significantly with increasing information load, while conscious retrieval performance dropped significantly. A working memory network, including hippocampus, was activated during both conscious and unconscious inference and correlated with retrieval success. An episodic retrieval network, including hippocampus, was activated during both conscious and unconscious retrieval of inferences and correlated with retrieval success. Only conscious encoding/retrieval recruited additional brain regions outside these networks. Hence, levels of consciousness influenced the memories' behavioral impact, memory capacity, and the neural representational code.

Measures of explicit and implicit in motor learning: what we know and what we don't

Adaptation tasks are a key tool in characterizing the contribution of explicit and implicit processes to sensorimotor learning. However, different assumptions and ideas underlie methods used to measure these processes, leading to inconsistencies between studies. For instance, it is still unclear explicit and implicit combine additively. Cognitive studies of explicit and implicit processes show how non-additivity and bias in measurement can distort results. We argue that to understand explicit and implicit processes in visuomotor adaptation, we need a stronger characterization of the phenomenology and a richer set of models to test it on.

Interaction Between Conscious and Unconscious Information-Processing of Faces and Words

It is widely acknowledged that holistic processing is a key characteristic of face perception. Although holistic processing implies the automatic integration of face parts, it is unclear whether such processing requires the awareness of face parts. Here, we investigated the interactions between visible face parts and face parts rendered invisible using continuous flash suppression (CFS). In the first experiment with the upper half-face visible and the lower half-face invisible, the results showed that perceived face identity was influenced by the invisible lower half-face, suggesting that integration occurs between the visible and invisible face parts, a variant of the "composite face effect". In the second experiment, we investigated the influence of visible face parts on the processing of invisible face parts, as measured by the time it took for the invisible parts to break out from CFS. The results showed a visible-to-invisible facilitation effect, that the aligned invisible face parts broke through CFS faster than when the visible and invisible face parts were misaligned. Visible eyes had a stronger influence on the invisible nose/mouth than the other way around. Such facilitation of processing from visible to invisible parts was also found when Chinese characters were used as stimuli. These results show that information integration occurs across the consciousness boundary.

Diffusion model-based understanding of subliminal affective priming in continuous flash suppression

Affective states influence our decisions even when processed unconsciously. Continuous flash suppression (CFS) is a new variant of binocular rivalry that can be used to render the prime subliminal. Nonetheless, how prior information from emotional faces suppressed by CFS influences subsequent decision-making remains unclear. Here, we employed a CFS priming task to examine the effect of the two main types of information conveyed by faces, i.e., facial identity and emotion, on the evaluation of target words as positive or negative. The hierarchical diffusion model was used to investigate the underlying mechanisms. A significant interaction effect on response time was observed following the angry face prime but not the happy or neutral face primes. The results of the diffusion model analyses revealed that the priming effects of facial identity were mapped onto the drift rate and erased the 'positive bias' (the processing advantage of positive over negative stimuli). Meanwhile, the positive emotional faces increased the nondecision time in response to negative target words. The model-based analysis implies that both facial identity and emotion are processed under CFS.

Dissociating conscious and unconscious influences on visual detection effects

The scope of unconscious processing is highly debated, with recent studies showing that even high-level functions such as perceptual integration and category-based attention occur unconsciously. For example, upright faces that are suppressed from awareness through interocular suppression break into awareness more quickly than inverted faces. Similarly, verbal object cues boost otherwise invisible objects into awareness. Here, we replicate these findings, but find that they reflect a general difference in detectability not specific to interocular suppression. To dissociate conscious and unconscious influences on visual detection effects, we use an additional discrimination task to rule out conscious processes as a cause for these differences. Results from this detection-discrimination dissociation paradigm reveal that, while face orientation is processed unconsciously, category-based attention requires awareness. These findings provide insights into the function of conscious perception and offer an experimental approach for mapping out the scope and limits of unconscious processing.

Exposure to the thin beauty ideal: Are there subliminal priming effects?

OBJECTIVE

Previous research suggested that exposure to the thin beauty ideal propagated by the media is associated with body dissatisfaction and the development of disordered eating. Given recent suggestions regarding the role of automatic processes, we aimed to enhance our understanding of automatic, unconscious responses to body pictures and the association with the internalization of the thin ideal and the severity of eating disorder symptoms.

METHOD

An affective priming task with body pictures of different weight as primes and a normal-weight body picture as target, which had to be evaluated with regard to attractiveness and desirability, was administered to healthy women with either subliminal prime presentation (Experiment 1) or conscious presentation (Experiment 2).

RESULTS

Subliminal presentation did not affect the evaluation of the normal-weight target, although strength of evaluative shifts was significantly associated with internalization of the thin ideal. In contrast, the conscious presentation of the ultra-thin prime decreased and of the obese prime increased desirability and attractiveness ratings of the target.

DISCUSSION

Prevention strategies focusing on the critical evaluation of the thin ideal are important. Future studies are warranted to enhance our understanding of automatic, unconscious processes in women experiencing eating disorders.

Unconscious preparation: Effects of prime visibility on semantic generalization of task priming

Studies found that subliminal primes can be associated with specific tasks to facilitate task performance, and such learning is highly adaptive and generalizable. Meanwhile, conditioning studies suggest that aversive/reward learning and generalization actually occur at the semantic level. The current study shows that prime-task associations can also be generalized to novel word/neighbour primes from the same semantic category, and this occurs without contingency awareness. Previous studies have counterintuitively suggested that both the learning of task priming and the semantic priming of word neighbours depend on the lack of visibility. Here, we show that semantic generalization indeed depends on reduced visibility, but cannot occur subliminally. The current study shows for the first time that semantic learning and generalization can occur without any emotional or motivational factors, and that semantic priming can occur for arbitrary-linked stimuli in a context completely devoid of semantics.

Windows of Integration Hypothesis Revisited

In the ongoing research of the functions of consciousness, special emphasis has been put on integration of information: the ability to combine different signals into a coherent, unified one. Several theories of consciousness hold that this ability depends on - or at least goes hand in hand with - conscious processing. Yet some empirical findings have suggested otherwise, claiming that integration of information could take place even without awareness. Trying to reconcile this apparent contradiction, the "windows of integration" (WOI) hypothesis claims that conscious access enables signal processing over large integration windows. The hypothesis applies to integration windows defined either temporally, spatially, or semantically. In this review, we explain the hypothesis and re-examine it in light of new studies published since it was suggested. In line with the hypothesis, these studies provide compelling evidence for unconscious integration, but also demonstrate its limits with respect to time, space, and semantic distance. The review further highlights open questions that still need to be pursued to demonstrate the applicability of the WOI hypothesis as a guiding principle for understanding the depth and scope of unconscious processes.

The alleviation effect of knowledge integration on anxiety in parental vaccine decision-making supported by online communities

Purpose

To support vaccine decision-making, a growing number of parents use online communities to obtain informational and emotional support; however, relatively high information heterogeneity and polarization in the online environment make it challenging for parents to make informed vaccine decisions based on the systematic processing of conflicting information. In this context, this study aims to focus on the relationship between parents’ knowledge integration and rational and experiential decision-making and the mediating effect of anxiety on this relationship.

Design/methodology/approach

A theoretical model incorporating the direct and indirect effects of knowledge integration and anxiety on decision-making is proposed and tested through partial least squares structural equation modeling with survey data from 223 parents.

Findings

Knowledge integration negatively affects anxiety. Knowledge integration has a direct positive effect on rational decision-making and an indirect negative effect on experiential decision-making.

Practical implications

These insights into the key role of knowledge integration in parental vaccine decision-making under information heterogeneity and polarization provide support for practical strategies to encourage knowledge integration and alleviate anxiety in online communities.

Originality/value

This study underscores the importance of knowledge integration in vaccine decision-making under information heterogeneity and polarization and reveals distinct mechanisms underlying the effects of knowledge integration on decision-making dominated by rational and experiential modes. The findings also provide insights into the information processing mechanisms underlying the knowledge integration of subjects with insufficient prior knowledge in the non-organizational context.

Can unconscious sequential integration of semantic information occur when the prime Chinese characters are displayed from left to right?

Recent studies have investigated whether conscious awareness is necessary for semantic integration. Although results have varied, simultaneous presentation of words have consistently led to greater semantic integration than sequential presentation in a single location. The current studies were designed to investigate whether the disadvantage of sequential presentation for unconscious semantic integration is specific to unfamiliar word-by-word presentation in one location or extends to the more natural reading conditions of viewing items sequentially from left to right. In Experiment 1, when the first three characters of Chinese idioms were presented simultaneously under masked conditions, performance on a separate two-alternative forced-choice recognition task was at chance level. Despite being unaware of the identity of prime characters, participants were faster to indicate that a subsequent item was a Chinese character when it was congruent with the beginning of the idiom, thus providing evidence of semantic integration. In contrast, when the three (Experiment 2) or two (Experiment 3) prime characters were presented sequentially in time from left to right, there was no evidence of semantic integration. These results indicate that unconscious semantic integration is more limited than previously reported, and may require simultaneous visual presentation.

Predictions from masked motion with and without obstacles

Predicting the future is essential for organisms like Homo sapiens, who live in a dynamic and ever-changing world. Previous research has established that conscious stimuli can lead to non-conscious predictions. Here we examine whether masked stimuli can also induce such predictions. We use masked movement-with and without obstacles-to examine predictions from masked stimuli. In six experiments a moving object was masked using continuous flash suppression (CFS). A few hundred milliseconds after the object had disappeared, a conscious probe appeared in a location that was either consistent with the masked stimulus or not. In Experiments 1-3 the movement was linear, and reaction times (RTs) indicated predictions that were based on direction and speed of movement. In Experiment 4, the masked moving object collided with an obstacle and then disappeared. Predictions in this case should reflect deflection, and indeed reaction times revealed predictions on the deflection route. In Experiments 5 and 6 we introduce an innovative way of using eye-tracking during continuous flash suppression (CFS) and report physiological evidence-in the forms of eye-movements-for masked stimuli induced predictions. We thus conclude that humans can use dynamic masked stimuli to generate active predictions about the future, and use these predictions to guide behavior. We also discuss the possible interpretations of these findings in light of the current scientific discussion regarding the relation between masked presentation, subliminal perception and awareness measurement methods.

Dimensions of Animal Consciousness

How does consciousness vary across the animal kingdom? Are some animals 'more conscious' than others? This article presents a multidimensional framework for understanding interspecies variation in states of consciousness. The framework distinguishes five key dimensions of variation: perceptual richness, evaluative richness, integration at a time, integration across time, and self-consciousness. For each dimension, existing experiments that bear on it are reviewed and future experiments are suggested. By assessing a given species against each dimension, we can construct a consciousness profile for that species. On this framework, there is no single scale along which species can be ranked as more or less conscious. Rather, each species has its own distinctive consciousness profile.

Getting to the heart of it: Multi-method exploration of nonconscious prioritization processes

Understanding the determinants of consciousness is crucial for theories that see it as functionally adaptive, and for explaining how consciousness affects higher-level cognition. The invention of continuous flash suppression (CFS), a long-duration suppression technique, resulted in a proliferation of research into the process of prioritization for consciousness. We developed a new technique, repeated masked suppression (RMS), that facilitates the measurement of long suppression times, but relies on different visual principles. RMS enables a theoretical leap: It allows scientists to examine the central process of prioritization across different suppression methods. In five experiments (n = 282) we collected chronometric RMS and CFS data, finding that the previously reported face inversion effect and the face priority-dimension generalize beyond CFS. Our results validate the use of multi-method designs in the study of prioritization for consciousness. Furthermore, we show how RMS could be used online to reach diverse samples, previously beyond the reach of consciousness science.

All in Good Time: Long-Lasting Postdictive Effects Reveal Discrete Perception

Is consciousness a continuous stream of percepts or is it discrete, occurring only at certain moments in time? This question has puzzled philosophers, psychologists, and neuroscientists for centuries. Both hypotheses have fallen repeatedly in and out of favor. Here, we review recent studies exploring long-lasting postdictive effects and show that the results favor a two-stage discrete model, in which substantial periods of continuous unconscious processing precede discrete conscious percepts. We propose that such a model marries the advantages of both continuous and discrete models and resolves centuries old debates about perception and consciousness.

Dissociation as a disorder of integration - On the footsteps of Pierre Janet

At the end of the 19th century Pierre Janet described dissociation as an altered state of consciousness manifested in disrupted integration of psychological functions. Clinically, such disruption comprises compartmentalization symptoms like amnesia, detachment symptoms like depersonalization/derealization, and structural dissociation of personality with changes in the sense of self. The exact neuronal mechanisms leading to these different symptoms remain unclear. We here suggest to put Janet's original account of dissociation as disrupted integration of psychological functions into a novel context, that is, a neuronal context as related to current brain imaging. This requires a combined theoretical and empirical approach on data supporting such neuronal reframing of Janet. For that, we here review (i) past and (ii) recent psychological and neuronal views on dissociation together with neuroscientific theories of integration, which (iii) are supported and complemented by preliminary fMRI data. We propose three neuronal mechanisms of dynamic integration operating at different levels of the brain's spontaneous activity - temporo-spatial binding on the regional level, temporo-spatial synchronization on the network level, and temporo-spatial globalization on the global level. These neuronal mechanisms, in turn, may be related to different symptomatic manifestation of dissociation operating at different levels, e.g., compartmentalization, detachment, and structural, which, as we suggest, can all be traced to disrupted integration of neuronal and psychological functions as originally envisioned by Janet.

Invisible Adaptation: The Effect of Awareness on the Strength of the Motion Aftereffect

The ability to process information despite the lack of perceptual awareness is one of the most fascinating aspects of the visual system. Such unconscious processing is often investigated using adaptation, where any presence of the former can be traced by its footprint on aftereffects following the latter. We have investigated the mechanisms of the motion aftereffect (MAE) using random dot displays of varying motion coherence as well as crowding to modulate both the physical as well as the perceptual strength of the adaptation stimulus. Perceptual strength was quantitatively measured as the performance in a forced-choice direction-discrimination task. A motion-nulling technique was used to quantitatively measure the strength of the MAE. We show that the strength of the dynamic MAE is independently influenced by both the physical stimulus strength as well as the subjective perceptual strength, with the effect of the former being more prominent than that of the latter. We further show that the MAE still persists under conditions of subthreshold perception. Our results suggest that perceptual awareness can influence the strength of visual processing, but the latter is not fully dependent on the former and can still take place at its partial or even total absence.

The Self and Its Prolonged Intrinsic Neural Timescale in Schizophrenia

Schizophrenia (SCZ) can be characterized as a basic self-disorder that is featured by abnormal temporal integration on phenomenological (experience) and psychological (information processing) levels. Temporal integration on the neuronal level can be measured by the brain's intrinsic neural timescale using the autocorrelation window (ACW) and power-law exponent (PLE). Our goal was to relate intrinsic neural timescales (ACW, PLE), as a proxy of temporal integration on the neuronal level, to temporal integration related to self-disorder on psychological (Enfacement illusion task in electroencephalography) and phenomenological (Examination of Anomalous Self-Experience [EASE]) levels. SCZ participants exhibited prolonged ACW and higher PLE during the self-referential task (Enfacement illusion), but not during the non-self-referential task (auditory oddball). The degree of ACW/PLE change during task relative to rest was significantly reduced in self-referential task in SCZ. A moderation model showed that low and high ACW/PLE exerted differential impact on the relationship of self-disorder (EASE) and negative symptoms (PANSS). In sum, we demonstrate abnormal prolongation in intrinsic neural timescale during self-reference in SCZ including its relation to basic self-disorder and negative symptoms. Our results point to abnormal relation of self and temporal integration at the core of SCZ constituting a "common currency" of neuronal, psychological, and phenomenological levels.

B or 13? Unconscious Top-Down Contextual Effects at the Categorical but Not the Lexical Level

Contextual effects require integration of top-down predictions and bottom-up visual information. Given the widely assumed link between integration and consciousness, we asked whether contextual effects require consciousness. In two experiments (total N = 60), an ambiguous stimulus (which could be read as either B or 13) was presented alongside masked numbers (12 and 14) or letters (A and C). Context biased stimulus classification when it was consciously and unconsciously perceived. However, unconsciously perceived contexts evoked smaller effects. This finding was replicated and generalized into another language in a further experiment (N = 46) using a different set of stimuli, strengthening the claim that symbolic contextual effects can occur without awareness. Moreover, four experiments (total N = 160) suggested that these unconscious effects might be limited to the categorical level (numbers context vs. letters context) and do not extend to the lexical level (words context vs. nonwords context). Taken together, our results suggest that although consciousness may not be necessary for effects that require simple integration or none at all, it is nevertheless required for integration over larger semantic windows.

Bumble bees display cross-modal object recognition between visual and tactile senses

Many animals can associate object shapes with incentives. However, such behavior is possible without storing images of shapes in memory that are accessible to more than one sensory modality. One way to explore whether there are modality-independent internal representations of object shapes is to investigate cross-modal recognition-experiencing an object in one sensory modality and later recognizing it in another. We show that bumble bees trained to discriminate two differently shaped objects (cubes and spheres) using only touch (in darkness) or vision (in light, but barred from touching the objects) could subsequently discriminate those same objects using only the other sensory information. Our experiments demonstrate that bumble bees possess the ability to integrate sensory information in a way that requires modality-independent internal representations.

No evidence for mnemonic modulation of interocularly suppressed visual input

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

The Semantic Integration Between Two Subliminally Perceived Words Simultaneously Presented at Different Locations

In the present study, we showed evidence of an integration between two unconscious semantic representations. In experiment 1, two masked Chinese words of the same or different categories ("orange apple" or "grape hammer") were simultaneously presented in the prime, followed by two Chinese words also of same or different categories in the target. We examined possible prime/target visual feature priming, semantic category priming and motor response priming effects. Moreover, two ISI intervals (53, 163 ms) between the prime and the target words were used to examine the positive and negative priming. The results revealed a negative motor response priming and a positive semantic category priming effect independent of the ISI when the target words were of the same category. Experiment 2 eliminated an alternative interpretation of the effect based on different number of category words changed across the prime and the target. Experiment 3 eliminated a potential confound of unequal numbers of trials for motor congruent and incongruent conditions in Experiment 1. Overall, these results indicated an integration between the meanings of the two subliminally perceived words in the prime. The difference between simultaneous and sequential presentations, and the reason why positive priming was not observed when the interval between the prime and the target was short were discussed in the context of unconscious semantic integration.

Is temporo-spatial dynamics the "common currency" of brain and mind? In Quest of "Spatiotemporal Neuroscience"

Neuroscience has made considerable progress in unraveling the neural correlates of mental phenomena like self, consciousness, and perception. However, the "common currency" shared between neuronal and mental activity, brain and mind, remains yet unclear. In this article, we propose that the dynamics of time and space provides a "common currency" that connects neuronal and mental features. Time and space are here understood in a dynamic context (as in contemporary physics): that is, in terms of the way the brain's spontaneous activity constructs its spatial and temporal relationships, for instance in terms of functional connectivity and different frequencies of fluctuations. Recruiting recent empirical evidence, we show that the different ways in which the spontaneous activity constructs its "inner time and space" are manifested in distinct mental features. Specifically, we demonstrate how spatiotemporal mechanisms like spatiotemporal repertoire, integration, and speed yield mental features like consciousness, self, and time speed perception. The focus on the brain's spatiotemporal mechanisms entails what we describe as "Spatiotemporal Neuroscience". Spatiotemporal Neuroscience conceives neuronal activity in terms of its temporo-spatial dynamics rather than its various functions (e.g., cognitive, affective, social, etc.) as in other branches of neuroscience (as distinguished from Cognitive, Affective, Cultural, Social, etc. Neuroscience). That allows Spatiotemporal Neuroscience to take into view the so-called 'spatio-temporality' of mental features including their non-causal, intrinsic and transformative relationship with neuronal features. In conclusion, Spatiotemporal Neuroscience opens the door to investigate and ultimately reveal the brain's own temporo-spatial dynamics as the hitherto missing "common currency" of neuronal and mental features.

Auditory-visual integration during nonconscious perception

Our study proposes a test of a key assumption of the most prominent model of consciousness - the global workspace (GWS) model (e.g., Baars, 2002, 2005, 2007; Dehaene & Naccache, 2001; Mudrik, Faivre, & Koch, 2014). This assumption is that multimodal integration requires consciousness; however, few studies have explicitly tested if integration can occur between nonconscious information from different modalities. The proposed study examined whether a classic indicator of multimodal integration - the McGurk effect - can be elicited with subliminal auditory-visual speech stimuli. We used a masked speech priming paradigm developed by Kouider and Dupoux (2005) in conjunction with continuous flash suppression (CFS; Tsuchiya & Koch, 2005), a binocular rivalry technique for presenting video stimuli subliminally. Applying these techniques together, we carried out two experiments in which participants categorised auditory syllable targets which were preceded by subliminal auditory-visual (AV) speech primes. Subliminal AV primes were either illusion-inducing (McGurk) or illusion-neutral (Incongruent) combinations of speech stimuli. In Experiment 1, the categorisation of the syllable target ("pa") was facilitated by the same syllable prime when it was part of a McGurk combination (auditory "pa" and visual "ka") but not when part of an Incongruent combination (auditory "pa" and visual "wa"). This dependency on specific AV combinations indicated a nonconscious AV interaction. Experiment 2 presented a different syllable target ("ta") which matched the predicted illusory outcome of the McGurk combination - here, both the McGurk combination (auditory "pa" and visual "ka") and the Incongruent combination (auditory "ta" and visual "ka") failed to facilitate target categorisation. The combined results of both Experiments demonstrate a type of nonconscious multimodal interaction that is distinct from integration - it allows unimodal information that is compatible for integration (i.e., McGurk combinations) to persist and influence later processes, but does not actually combine and alter that information. As the GWS model does not account for non-integrative multimodal interactions, this places some pressure on such models of consciousness.

Imaging object-scene relations processing in visible and invisible natural scenes

Integrating objects with their context is a key step in interpreting complex visual scenes. Here, we used functional Magnetic Resonance Imaging (fMRI) while participants viewed visual scenes depicting a person performing an action with an object that was either congruent or incongruent with the scene. Univariate and multivariate analyses revealed different activity for congruent vs. incongruent scenes in the lateral occipital complex, inferior temporal cortex, parahippocampal cortex, and prefrontal cortex. Importantly, and in contrast to previous studies, these activations could not be explained by task-induced conflict. A secondary goal of this study was to examine whether processing of object-context relations could occur in the absence of awareness. We found no evidence for brain activity differentiating between congruent and incongruent invisible masked scenes, which might reflect a genuine lack of activation, or stem from the limitations of our study. Overall, our results provide novel support for the roles of parahippocampal cortex and frontal areas in conscious processing of object-context relations, which cannot be explained by either low-level differences or task demands. Yet they further suggest that brain activity is decreased by visual masking to the point of becoming undetectable with our fMRI protocol.

Sensorimotor contingency modulates breakthrough of virtual 3D objects during a breaking continuous flash suppression paradigm

To investigate how embodied sensorimotor interactions shape subjective visual experience, we developed a novel combination of Virtual Reality (VR) and Augmented Reality (AR) within an adapted breaking continuous flash suppression (bCFS) paradigm. In a first experiment, participants manipulated novel virtual 3D objects, viewed through a head-mounted display, using three interlocking cogs. This setup allowed us to manipulate the sensorimotor contingencies governing interactions with virtual objects, while characterising the effects on subjective visual experience by measuring breakthrough times from bCFS. We contrasted the effects of the congruency (veridical versus reversed sensorimotor coupling) and contingency (live versus replayed interactions) using a motion discrimination task. The results showed that the contingency but not congruency of sensorimotor coupling affected breakthrough times, with live interactions displaying faster breakthrough times. In a second experiment, we investigated how the contingency of sensorimotor interactions affected object category discrimination within a more naturalistic setting, using a motion tracker that allowed object interactions with increased degrees of freedom. We again found that breakthrough times were faster for live compared to replayed interactions (contingency effect). Together, these data demonstrate that bCFS breakthrough times for unfamiliar 3D virtual objects are modulated by the contingency of the dynamic causal coupling between actions and their visual consequences, in line with theories of perception that emphasise the influence of sensorimotor contingencies on visual experience. The combination of VR/AR and motion tracking technologies with bCFS provides a novel methodology extending the use of binocular suppression paradigms into more dynamic and realistic sensorimotor environments.

Tactile motion biases visual motion perception in binocular rivalry

There is an ongoing debate whether or not multisensory interactions require awareness of the sensory signals. Static visual and tactile stimuli have been shown to influence each other even in the absence of visual awareness. However, it is unclear if this finding generalizes to dynamic contexts. In the present study, we presented visual and tactile motion stimuli and induced fluctuations of visual awareness by means of binocular rivalry: two gratings which drifted in opposite directions were displayed, one to each eye. One visual motion stimulus dominated and reached awareness while the other visual stimulus was suppressed from awareness. Tactile motion stimuli were presented at random time points during the visual stimulation. The motion direction of a tactile stimulus always matched the direction of one of the concurrently presented visual stimuli. The visual gratings were differently tinted, and participants reported the color of the currently seen stimulus. Tactile motion delayed perceptual switches that ended dominance periods of congruently moving visual stimuli compared to switches during visual-only stimulation. In addition, tactile motion fostered the return to dominance of suppressed, congruently moving visual stimuli, but only if the tactile motion started at a late stage of the ongoing visual suppression period. At later stages, perceptual suppression is typically decreasing. These results suggest that visual awareness facilitates but does not gate multisensory interactions between visual and tactile motion signals.

Unconscious Detection of One's Own Image

A key mechanism behind preferential processing of self-related information might be an early and automatic capture of attention. Therefore, the present study tested a hypothesis that one's own face will attract bottom-up attention even without conscious identification. To test this, we used a dot-probe paradigm with electrophysiological recordings, in which participants ( N = 18) viewed masked and unmasked pairs of faces (other, self) presented laterally. Analysis of the sensitivity measure d ' indicated that faces were not consciously identified in the masked condition. A clear N2 posterior-contralateral (N2pc) component (a neural marker of attention shifts) was found in both the masked and unmasked conditions, revealing that one's own face automatically captures attention when processed unconsciously. Therefore, our study (a) demonstrates that self-related information is boosted at an early (preconscious) stage of processing, (b) identifies further features (beyond simple physical ones) that cause automatic attention capture, and (c) provides further evidence for the dissociative nature of attention and consciousness.

The cerebral bases of the bouba-kiki effect

The crossmodal correspondence between some speech sounds and some geometrical shapes, known as the bouba-kiki (BK) effect, constitutes a remarkable exception to the general arbitrariness of the links between word meaning and word sounds. We have analyzed the association of shapes and sounds in order to determine whether it occurs at a perceptual or at a decisional level, and whether it takes place in sensory cortices or in supramodal regions. First, using an Implicit Association Test (IAT), we have shown that the BK effect may occur without participants making any explicit decision relative to sound-shape associations. Second, looking for the brain correlates of implicit BK matching, we have found that intermodal matching influences activations in both auditory and visual sensory cortices. Moreover, we found stronger prefrontal activation to mismatching than to matching stimuli, presumably reflecting a modulation of executive processes by crossmodal correspondence. Thus, through its roots in the physiology of object categorization and crossmodal matching, the BK effect provides a unique insight into some non-linguistic components of word formation.

The circuit architecture of cortical multisensory processing: Distinct functions jointly operating within a common anatomical network

Our perceptual systems continuously process sensory inputs from different modalities and organize these streams of information such that our subjective representation of the outside world is a unified experience. By doing so, they also enable further cognitive processing and behavioral action. While cortical multisensory processing has been extensively investigated in terms of psychophysics and mesoscale neural correlates, an in depth understanding of the underlying circuit-level mechanisms is lacking. Previous studies on circuit-level mechanisms of multisensory processing have predominantly focused on cue integration, i.e. the mechanism by which sensory features from different modalities are combined to yield more reliable stimulus estimates than those obtained by using single sensory modalities. In this review, we expand the framework on the circuit-level mechanisms of cortical multisensory processing by highlighting that multisensory processing is a family of functions - rather than a single operation - which involves not only the integration but also the segregation of modalities. In addition, multisensory processing not only depends on stimulus features, but also on cognitive resources, such as attention and memory, as well as behavioral context, to determine the behavioral outcome. We focus on rodent models as a powerful instrument to study the circuit-level bases of multisensory processes, because they enable combining cell-type-specific recording and interventional techniques with complex behavioral paradigms. We conclude that distinct multisensory processes share overlapping anatomical substrates, are implemented by diverse neuronal micro-circuitries that operate in parallel, and are flexibly recruited based on factors such as stimulus features and behavioral constraints.

Can Computers Become Conscious and Overcome Humans?

The idea of machines overcoming humans can be intrinsically related to conscious machines. Surpassing humans would mean replicating, reaching and exceeding key distinctive properties of human beings, for example, high-level cognition associated with conscious perception. However, can computers be compared with humans? Can computers become conscious? Can computers outstrip human capabilities? These are paradoxical and controversial questions, particularly because there are many hidden assumptions and misconceptions about the understanding of the brain. In this sense, it is necessary to first explore these assumptions and then suggest how the specific information processing of brains would be replicated by machines. Therefore, this article will discuss a subset of human capabilities and the connection with conscious behavior, secondly, a prototype theory of consciousness will be explored and machines will be classified according to this framework. Finally, this analysis will show the paradoxical conclusion that trying to achieve conscious machines to beat humans implies that computers will never completely exceed human capabilities, or if the computer were to do it, the machine should not be considered a computer anymore.

The relationship between attention and consciousness: an expanded taxonomy and implications for 'no-report' paradigms

Tensions between global neuronal workspace theory and recurrent processing theory have sparked much debate in the field of consciousness research. Here, we focus on one of the key distinctions between these theories: the proposed relationship between attention and consciousness. By reviewing recent empirical evidence, we argue that both theories contain key insights and that certain aspects of each theory can be reconciled into a novel framework that may help guide future research. Alternative theories are also considered, including attended intermediate-level representations theory, integrated information theory and higher order thought theory. With the aim of offering a fresh and nuanced perspective to current theoretical debates, an updated taxonomy of conscious and non-conscious states is proposed. This framework maps a wider spectrum of conscious states by incorporating contemporary views from cognitive neuroscience regarding the variety of attentional mechanisms that are known to interact with sensory processing. Whether certain types of attention are necessary for phenomenal and access consciousness is considered and incorporated into this extended taxonomy. To navigate this expanded space, we review recent 'no-report' paradigms and address several methodological misunderstandings in order to pave a clear path forward for identifying the neural basis of perceptual awareness.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Invisible Flashes Alter Perceived Sound Location

Information integration across the senses is fundamental for effective interactions with our environment. The extent to which signals from different senses can interact in the absence of awareness is controversial. Combining the spatial ventriloquist illusion and dynamic continuous flash suppression (dCFS), we investigated in a series of two experiments whether visual signals that observers do not consciously perceive can influence spatial perception of sounds. Importantly, dCFS obliterated visual awareness only on a fraction of trials allowing us to compare spatial ventriloquism for physically identical flashes that were judged as visible or invisible. Our results show a stronger ventriloquist effect for visible than invisible flashes. Critically, a robust ventriloquist effect emerged also for invisible flashes even when participants were at chance when locating the flash. Collectively, our findings demonstrate that signals that we are not aware of in one sensory modality can alter spatial perception of signals in another sensory modality.

Designing Brains for Pain: Human to Mollusc

There is compelling evidence that the "what it feels like" subjective experience of sensory stimuli arises in the cerebral cortex in both humans as well as mammalian experimental animal models. Humans are alone in their ability to verbally communicate their experience of the external environment. In other species, sensory awareness is extrapolated on the basis of behavioral indicators. For instance, cephalopods have been claimed to be sentient on the basis of their complex behavior and anecdotal reports of human-like intelligence. We have interrogated the findings of avoidance learning behavioral paradigms and classical brain lesion studies and conclude that there is no evidence for cephalopods feeling pain. This analysis highlighted the questionable nature of anthropometric assumptions about sensory experience with increased phylogenetic distance from humans. We contend that understanding whether invertebrates such as molluscs are sentient should first begin with defining the computational processes and neural circuitries underpinning subjective awareness. Using fundamental design principles, we advance the notion that subjective awareness is dependent on observer neural networks (networks that in some sense introspect the neural processing generating neural representations of sensory stimuli). This introspective process allows the observer network to create an internal model that predicts the neural processing taking place in the network being surveyed. Predictions arising from the internal model form the basis of a rudimentary form of awareness. We develop an algorithm built on parallel observer networks that generates multiple levels of sensory awareness. A network of cortical regions in the human brain has the appropriate functional properties and neural interconnectivity that is consistent with the predicted circuitry of the algorithm generating pain awareness. By contrast, the cephalopod brain lacks the necessary neural circuitry to implement such an algorithm. In conclusion, we find no compelling behavioral, functional, or neuroanatomical evidence to indicate that cephalopods feel pain.

The Complex Interplay Between Multisensory Integration and Perceptual Awareness

The integration of information has been considered a hallmark of human consciousness, as it requires information being globally available via widespread neural interactions. Yet the complex interdependencies between multisensory integration and perceptual awareness, or consciousness, remain to be defined. While perceptual awareness has traditionally been studied in a single sense, in recent years we have witnessed a surge of interest in the role of multisensory integration in perceptual awareness. Based on a recent IMRF symposium on multisensory awareness, this review discusses three key questions from conceptual, methodological and experimental perspectives: (1) What do we study when we study multisensory awareness? (2) What is the relationship between multisensory integration and perceptual awareness? (3) Which experimental approaches are most promising to characterize multisensory awareness? We hope that this review paper will provoke lively discussions, novel experiments, and conceptual considerations to advance our understanding of the multifaceted interplay between multisensory integration and consciousness.