The human visual system differentially represents subjectively and objectively invisible stimuli

Brain Representation in Conscious and Unconscious Vision

The development of robust frameworks to understand how the human brain represents conscious and unconscious perceptual contents is paramount to make progress in the neuroscience of consciousness. Recent functional MRI studies using multi-voxel pattern classification analyses showed that unconscious contents could be decoded from brain activity patterns. However, decoding does not imply a full understanding of neural representations. Here we re-analysed data from a high-precision fMRI study coupled with representational similarity analysis based on convolutional neural network models to provide a detailed information-based approach to neural representations of both unconscious and conscious perceptual content. The results showed that computer vision model representations strongly predicted brain responses in ventral visual cortex and in fronto-parietal regions to both conscious and unconscious contents. Moreover, this pattern of results generalised when the models were trained and tested with different participants. Remarkably, these observations results held even when the analysis was restricted to observers that showed null perceptual sensitivity. In light of the highly distributed brain representation of unconscious information, we suggest that the functional role of fronto-parietal cortex in conscious perception is unlikely to be related to the broadcasting of information, as proposed by the global neuronal workspace theory, and may instead relate to the generation of meta-representations as proposed by higher-order theories.

Decoding dynamic faces and scenes without awareness under dis-continuous flash suppression

In the perceptual sciences, there is an ongoing debate about the depth of unconscious processing. Here, we address this issue by implementing three improvements with regards to paradigm, stimuli and analyses to explore the neural correlates of unconscious face processing. Our results demonstrated that conscious faces elicited broader univariate activations than conscious scenes. Such results were absent when faces/scenes were suppressed and invisible (n = 43). However, further whole-brain multivariate classification revealed that both static and dynamic invisible faces could be distinguished from scenes in the occipital-temporal region. ROI analysis showed that bilateral FFA and OFA could differentiate dynamic invisible faces from dynamic invisible scenes. These findings suggest that interocularly suppressed faces are still processed in-depth in the ventral visual stream. Therefore, our study highlights the importance of optimizing stimulus signal, experimental paradigm, and analysis to extract unconscious signals in the brain.

Consciousness Under the Spotlight: The Problem of Measuring Subjective Experience

The study of consciousness is considered by many one of the most difficult contemporary scientific endeavors and confronts several methodological and theoretical challenges. A central issue that makes the study of consciousness so challenging is that, while the rest of science is concerned with problems that can be verified from a "third person" view (i.e., objectively), the study of consciousness deals with the phenomenon of subjective experience, only accessible from a "first person" view. In the present article, we review early (starting during the late 19th century) and later efforts on measuring consciousness and its absence, focusing on the two main approaches used by researchers within the field: objective (i.e., performance based) and subjective (i.e., report based) measures of awareness. In addition, we compare the advantages and disadvantages of both types of awareness measures, evaluate them according to different methodological considerations, and discuss, among other issues, the possibility of comparing them by transforming them to a common sensitivity measure (d'). Finally, we explore several new approaches-such as Bayesian models to support the absence of awareness or new machine-learning based decoding models-as well as future challenges-such as measuring the qualia, the qualitative contents of awareness-in consciousness research.

Registered Report Stage II: Decoding the category information from evoked potentials to visible and invisible visual objects

Previous studies that use decoding methods and EEG to investigate the neural representation of the category information of visual objects focused mainly on consciously processed visual objects. It remains unclear whether the category information of unconsciously processed visual objects can be decoded and whether the decoding performance is different for consciously and unconsciously processed visual objects. The present study compared the neural decoding of the animacy category of visible and invisible visual objects via EEG and decoding methods. The results revealed that the animacy of visible visual objects could be decoded above the chance level by the P200, N300, and N400, but not by the early N/P100. However, the animacy of invisible visual objects could not be decoded above the chance level by neither early nor late ERP components. The decoding accuracy was greater for visible visual objects than that for invisible visual objects for the P200, N300 and N400. These results suggested that access to animacy category information for visual objects requires conscious processing.

Awareness is needed for contextual effects in ambiguous object recognition

Despite its centrality to human experience, the functional role of conscious awareness is not yet known. One hypothesis suggests that consciousness is necessary for allowing high-level information to refine low-level processing in a "top-down" manner. To test this hypothesis, in this work we examined whether consciousness is needed for integrating contextual information with sensory information during visual object recognition, a case of top-down processing that is automatic and ubiquitous to our daily visual experience. In three experiments, 137 participants were asked to determine the identity of an ambiguous object presented to them. Crucially, a scene biasing the interpretation of the object towards one option over another (e.g., a picture of a tree when the object could equally be perceived as a fish or a leaf) was presented either before, after, or alongside the ambiguous object. In all three experiments, the scene biased perception of the ambiguous object when it was consciously perceived, but not when it was processed unconsciously. The results therefore suggest that conscious awareness may be needed for top-down contextual processes.

How (not) to demonstrate unconscious priming: Overcoming issues with post-hoc data selection, low power, and frequentist statistics

One widely used scientific approach to studying consciousness involves contrasting conscious operations with unconscious ones. However, challenges in establishing the absence of conscious awareness have led to debates about the extent and existence of unconscious processes. We collected experimental data on unconscious semantic priming, manipulating prime presentation duration to highlight the critical role of the analysis approach in attributing priming effects to unconscious processing. We demonstrate that common practices like post-hoc data selection, low statistical power, and frequentist statistical testing can erroneously support claims of unconscious priming. Conversely, adopting best practices like direct performance-awareness contrasts, Bayesian tests, and increased statistical power can prevent such erroneous conclusions. Many past experiments, including our own, fail to meet these standards, casting doubt on previous claims about unconscious processing. Implementing these robust practices will enhance our understanding of unconscious processing and shed light on the functions and neural mechanisms of consciousness.

Progressing, not regressing: A possible solution to the problem of regression to the mean in unconscious processing studies

How convincing is current evidence for unconscious processing? Recently, a major criticism suggested that some, if not much, of this evidence might be explained by a mere statistical phenomenon: regression to the mean (RttM). Excluding participants based on an awareness assessment is a common practice in studies of unconscious processing, and this post hoc data selection might lead to false effects that are driven by RttM for aware participants wrongfully classified as unaware. Here, we examined this criticism using both simulations and data from 12 studies probing unconscious processing (35 effects overall). In line with the original criticism, we confirmed that the reliability of awareness measures in the field is concerningly low. Yet, using simulations, we showed that reliability measures might be unsuitable for estimating error in awareness measures. Furthermore, we examined other solutions for assessing whether an effect is genuine or reflects RttM; all suffered from substantial limitations, such as a lack of specificity to unconscious processing, lack of power, or unjustified assumptions. Accordingly, we suggest a new nonparametric solution, which enjoys high specificity and relatively high power. Together, this work emphasizes the need to account for measurement error in awareness measures and evaluate its consequences for unconscious processing effects. It further suggests a way to meet the important challenge posed by RttM, in an attempt to establish a reliable and robust corpus of knowledge in studying unconscious processing.

Spatial perspective and identity in visual awareness of the bodily self-other distinction

Spatial perspective and identity of visual bodily stimuli are two key cues for the self-other distinction. However, how they emerge into visual awareness is largely unknown. Here, self- or other-hands presented in first- or third-person perspective were compared in a breaking-Continuous Flash Suppression paradigm (Experiment 1) measuring the time the stimuli need to access visual awareness, and in a Binocular Rivalry paradigm (Experiment 2), measuring predominance in perceptual awareness. Results showed that, irrespectively of identity, first-person perspective speeded up the access, whereas the third-person one increased the dominance. We suggest that the effect of first-person perspective represents an unconscious prioritization of an egocentric body coding important for visuomotor control. On the other hand, the effect of third-person perspective indicates a conscious advantage of an allocentric body representation fundamental for detecting the presence of another intentional agent. Summarizing, the emergence of self-other distinction into visual awareness would strongly depend on the interplay between spatial perspectives, with an inverse prioritization before and after conscious perception. On the other hand, identity features might rely on post-perceptual processes.

Time Courses of Attended and Ignored Object Representations

Selective attention prioritizes information that is relevant to behavioral goals. Previous studies have shown that attended visual information is processed and represented more efficiently, but distracting visual information is not fully suppressed, and may also continue to be represented in the brain. In natural vision, to-be-attended and to-be-ignored objects may be present simultaneously in the scene. Understanding precisely how each is represented in the visual system, and how these neural representations evolve over time, remains a key goal in cognitive neuroscience. In this study, we recorded EEG while participants performed a cued object-based attention task that involved attending to target objects and ignoring simultaneously presented and spatially overlapping distractor objects. We performed support vector machine classification on the stimulus-evoked EEG data to separately track the temporal dynamics of target and distractor representations. We found that (1) both target and distractor objects were decodable during the early phase of object processing (∼100 msec to ∼200 msec after target onset), and (2) the representations of both objects were sustained over time, remaining decodable above chance until ∼1000-msec latency. However, (3) the distractor object information faded significantly beginning after about 300-msec latency. These findings provide information about the fate of attended and ignored visual information in complex scene perception.

Illusory object recognition is either perceptual or cognitive in origin depending on decision confidence

We occasionally misinterpret ambiguous sensory input or report a stimulus when none is presented. It is unknown whether such errors have a sensory origin and reflect true perceptual illusions, or whether they have a more cognitive origin (e.g., are due to guessing), or both. When participants performed an error-prone and challenging face/house discrimination task, multivariate electroencephalography (EEG) analyses revealed that during decision errors (e.g., mistaking a face for a house), sensory stages of visual information processing initially represent the presented stimulus category. Crucially however, when participants were confident in their erroneous decision, so when the illusion was strongest, this neural representation flipped later in time and reflected the incorrectly reported percept. This flip in neural pattern was absent for decisions that were made with low confidence. This work demonstrates that decision confidence arbitrates between perceptual decision errors, which reflect true illusions of perception, and cognitive decision errors, which do not.

Efficient Video Watermarking Algorithm Based on Convolutional Neural Networks with Entropy-Based Information Mapper

This paper presents a method for the transparent, robust, and highly capacitive watermarking of video signals using an information mapper. The proposed architecture is based on the use of deep neural networks to embed the watermark in the luminance channel in the YUV color space. An information mapper was used to enable the transformation of a multi-bit binary signature of varying capacitance reflecting the entropy measure of the system into a watermark embedded in the signal frame. To confirm the effectiveness of the method, tests were carried out for video frames with a resolution of 256 × 256 pixels, with a watermark capacity of 4 to 16,384 bits. Transparency metrics (SSIM and PSNR) and a robustness metric-the bit error rate (BER)-were used to assess the performance of the algorithms.

An academic survey on theoretical foundations, common assumptions and the current state of consciousness science

We report the results of an academic survey into the theoretical and methodological foundations, common assumptions, and the current state of the field of consciousness research. The survey consisted of 22 questions and was distributed on two different occasions of the annual meeting of the Association of the Scientific Study of Consciousness (2018 and 2019). We examined responses from 166 consciousness researchers with different backgrounds (e.g. philosophy, neuroscience, psychology, and computer science) and at various stages of their careers (e.g. junior/senior faculty and graduate/undergraduate students). The results reveal that there remains considerable discussion and debate between the surveyed researchers about the definition of consciousness and the way it should be studied. To highlight a few observations, a majority of respondents believe that machines could have consciousness, that consciousness is a gradual phenomenon in the animal kingdom, and that unconscious processing is extensive, encompassing both low-level and high-level cognitive functions. Further, we show which theories of consciousness are currently considered most promising by respondents and how supposedly different theories cluster together, which dependent measures are considered best to index the presence or absence of consciousness, and which neural measures are thought to be the most likely signatures of consciousness. These findings provide us with a snapshot of the current views of researchers in the field and may therefore help prioritize research and theoretical approaches to foster progress.

Neuroscientific Evidence for Processing Without Awareness

The extent to which we are affected by perceptual input of which we are unaware is widely debated. By measuring neural responses to sensory stimulation, neuroscientific data could complement behavioral results with valuable evidence. Here we review neuroscientific findings of processing of high-level information, as well as interactions with attention and memory. Although the results are mixed, we find initial support for processing object categories and words, possibly to the semantic level, as well as emotional expressions. Robust neural evidence for face individuation and integration of sentences or scenes is lacking. Attention affects the processing of stimuli that are not consciously perceived, and such stimuli may exogenously but not endogenously capture attention when relevant, and be maintained in memory over time. Sources of inconsistency in the literature include variability in control for awareness as well as individual differences, calling for future studies that adopt stricter measures of awareness and probe multiple processes within subjects.

Unconscious Visual Working Memory: A critical review and Bayesian meta-analysis

The relationship between consciousness and working memory (WM) has been recently debated both at the theoretical and methodological levels (Persuh et al., 2018; Velichkovsky, 2017). While there is behavioral and neural evidence that argues for the existence of unconscious WM, several methodological concerns have been raised, rendering this issue highly controversial. To address the robustness of the previous findings, here we adopt a meta-analytic approach to estimate the effect size and heterogeneity of the previously reported unconscious WM results, also including unpublished results. We used meta-regression to isolate relevant experimental variables, in particular, consciousness manipulation and the WM paradigm to identify the source of the heterogeneity in the reported effect size of the unconscious WM. Our meta-analysis supports the existence of the unconscious WM effect and critically reveals several experimental variables that contribute to relevant heterogeneity. Our analysis clarifies several theoretical and methodological issues. We recommend that future studies explicitly operationalize the definition of consciousness, standardize the methodology and systematically explore the role of critical variables for the unconscious WM effect.

Unconscious priming shares a common resource pool with the manipulation subsystem

Background

Working memory can be subdivided into two relatively independent subordinate systems, the maintenance subsystem and the manipulation subsystem. Although the two subsystems are quite heterogeneous, research thus far has not adequately distinguished the resource pools of the two subsystems. Additionally, previous research on the relationship between working memory and unconscious priming is paradoxical. Different subsystems leading to different effects on unconscious priming might be the reason for the paradoxical research. Therefore, the current article aimed to distinguish the resource pools among two working-memory subsystems and to investigate the relationship between the two subsystems and unconscious priming.

Methods

To address these issues, a maintenance dual-task and a manipulation dual-task program were developed. Each participant had to separately perform the two dual tasks in a balanced order. In each dual task, participants first completed a masked priming task accompanied by working-memory load. As a control, participants completed a prime identification test to confirm that the processing of the masked prime was at the unconscious level. The maintenance dual task comprised sandwich masking trials accompanied by Sternberg trials, while the manipulation dual task comprised sandwich masking trials accompanied by N-back trials.

Results

The results of the prime identification test indicated that the participants could not consciously perceive the masked prime of both dual tasks. The results of the working-memory task of both dual tasks indicated that the load manipulation was successful for both dual tasks. Most importantly, the results of the masking task of both dual tasks showed that an increase in working-memory load decreased the magnitude of unconscious priming in the manipulation dual task, whereas an increase in working-memory load did not decrease unconscious priming in the maintenance dual task. These observations demonstrate that the manipulation subsystem, rather than the maintenance subsystem, interferes with unconscious priming. Together with previous research, we propose a two-pool attention resource model to explain the modulation of working memory on unconscious priming by dissociating the executive resource pool of the manipulation system from the retention resource pool of the maintenance system. Thus, the current work confirms and extends the extant literature about the dependence of unconscious processing on attention resources by suggesting that unconscious priming shares a common resource pool with the manipulation subsystem.

The Pervasive Problem of Post Hoc Data Selection in Studies on Unconscious Processing

Studies on unconscious mental processes typically require that participants are unaware of some information (e.g., a visual stimulus). An important methodological question in this field of research is how to deal with data from participants who become aware of the critical stimulus according to some measure of awareness. While it has previously been argued that the post hoc selection of participants dependent on an awareness measure may often result in regression-to-the-mean artifacts (Shanks, 2017), a recent article (Sklar et al., 2021) challenged this conclusion claiming that the consideration of this statistical artifact might lead to unjustified rejections of true unconscious influences. In this reply, we explain this pervasive statistical problem with a basic and concrete example, show that Sklar et al. fundamentally mischaracterize it, and then refute the argument that the influence of the artifact has previously been overestimated. We conclude that, without safeguards, the method of post hoc data selection should never be employed in studies on unconscious processing.

Informative neural representations of unseen contents during higher-order processing in human brains and deep artificial networks

A framework to pinpoint the scope of unconscious processing is critical to improve models of visual consciousness. Previous research observed brain signatures of unconscious processing in visual cortex, but these were not reliably identified. Further, whether unconscious contents are represented in high-level stages of the ventral visual stream and linked parieto-frontal areas remains unknown. Using a within-subject, high-precision functional magnetic resonance imaging approach, we show that unconscious contents can be decoded from multi-voxel patterns that are highly distributed alongside the ventral visual pathway and also involving parieto-frontal substrates. Classifiers trained with multi-voxel patterns of conscious items generalized to predict the unconscious counterparts, indicating that their neural representations overlap. These findings suggest revisions to models of consciousness such as the neuronal global workspace. We then provide a computational simulation of visual processing/representation without perceptual sensitivity by using deep neural networks performing a similar visual task. The work provides a framework for pinpointing the representation of unconscious knowledge across different task domains.

Visual adaptation and 7T fMRI reveal facial identity processing in the human brain under shallow interocular suppression

Face identity is represented at a high level of the visual hierarchy. Whether the human brain can process facial identity information in the absence of visual awareness remains unclear. In this study, we investigated potential face identity representation through face-identity adaptation with the adapting faces interocularly suppressed by Continuous Flash Suppression (CFS) noise, a modified binocular rivalry paradigm. The strength of interocular suppression was manipulated by varying the contrast of CFS noise. While obeservers reported the face images subjectively unperceived and the face identity objectively unrecognizable, a significant face identity aftereffect was observed under low but not high contrast CFS noise. In addition, the identity of face images under shallow interocular suppression can be decoded from multi-voxel patterns in the right fusiform face area (FFA) obtained with high-resolution 7T fMRI. Thus the comined evidence from visual adaptation and 7T fMRI suggest that face identity can be represented in the human brain without explicit perceptual recognition. The processing of interocularly suppressed faces could occur at different levels depending on how "deep" the information is suppressed.

Apical amplification-a cellular mechanism of conscious perception?

We present a theoretical view of the cellular foundations for network-level processes involved in producing our conscious experience. Inputs to apical synapses in layer 1 of a large subset of neocortical cells are summed at an integration zone near the top of their apical trunk. These inputs come from diverse sources and provide a context within which the transmission of information abstracted from sensory input to their basal and perisomatic synapses can be amplified when relevant. We argue that apical amplification enables conscious perceptual experience and makes it more flexible, and thus more adaptive, by being sensitive to context. Apical amplification provides a possible mechanism for recurrent processing theory that avoids strong loops. It makes the broadcasting hypothesized by global neuronal workspace theories feasible while preserving the distinct contributions of the individual cells receiving the broadcast. It also provides mechanisms that contribute to the holistic aspects of integrated information theory. As apical amplification is highly dependent on cholinergic, aminergic, and other neuromodulators, it relates the specific contents of conscious experience to global mental states and to fluctuations in arousal when awake. We conclude that apical dendrites provide a cellular mechanism for the context-sensitive selective amplification that is a cardinal prerequisite of conscious perception.

The Challenge of Inferring Unconscious Mental Processes

Studies of unconscious mental processes often compare a performance measure (e.g., some assessment of perception or memory) with a measure of awareness (e.g., a verbal report or forced-choice response) of the critical cue or contingency taken either concurrently or separately. The resulting patterns of bivariate data across participants lend themselves to several analytic approaches for inferring the existence of unconscious mental processes, but it is rare for researchers to consider the underlying generative processes that might cause these patterns. We show that bivariate data are generally insufficient to discriminate single-process models, with a unitary latent process determining both performance and awareness, from dual-process models, comprising distinct latent processes for performance and awareness. Future research attempting to isolate and investigate unconscious processes will need to employ richer types of data and analyses.