Evidence that instrumental conditioning requires conscious awareness in humans

Agency

Agency is action aimed at goals selected by an agent. A deterministic world view leaves scant room for agency. To reconcile the arguments, we represent action as nested control systems, ranging from clearly deterministic to clearly volitional. Negative feedback minimizes deviations from setpoints (goals). Goals are determined by higher modules in a hierarchy of systems, ranging from gamma-efferent spindles through reflexes to operant responses; these last, and the larger system that contains them, called the Self, comprise volitional agents. When operants become habitual they descend to closed teleonomic systems-automaticity. Change in emotional states, and unpredicted changes in the context-raise them back to full volitional systems. At the highest level is the Self-the brain's model of the agent. When aroused out of open teleonomic functioning, it must reconsider means and ends. It does so by simulating action plans, using the same neural systems it uses to effect them. The simulated stimuli and responses are conscious, approximating their perceptions as experienced in real time; this verisimilitude gives them their hedonic value. Positive feedback plays a key role in these complex adaptive systems, as it focuses and holds attention on the most salient percepts and goals, permitting the self-organization of action plans. The Self is not a separate entity, but a colloquy of command modules wearing the avatar of the agent. This system is put into correspondence with Grossberg's Adaptive Resonance Theory. Free will and determinism emerge not as binary opposites, but the modulating inputs to a spectrum of systems.

Stimulus awareness is necessary for both instrumental learning and instrumental responding to previously learned stimuli

Instrumental conditioning is a crucial part of adaptive behaviour, allowing agents to selectively interact with stimuli in their environment. Recent evidence suggests that instrumental conditioning cannot proceed without stimulus awareness. However, whether accurate unconscious instrumental responding can emerge from consciously acquired knowledge of the stimulus-action-outcome contingencies is unknown. We studied this question using instrumental trace conditioning, where participants learned to make approach/avoid decisions in two within-subject modes: conscious (stimuli in plain view) and unconscious (visually masked). Both tasks were followed by an unconscious-only instrumental performance task. We show that even when the contingencies are reliably learned in the conscious mode, participants fail to act upon them in the unconscious responding task. We also replicate the previous results that no instrumental learning occurs in the unconscious mode. Consequently, the absence of stimulus awareness not only precludes instrumental conditioning, but also precludes any kind of instrumental responding to already known stimuli. This suggests that instrumental behaviour is entirely supported by conscious awareness of the world, and corroborates the proposals that consciousness may be necessary for adaptive behaviours requiring selective action.

Unconscious Manipulation of Conceptual Representations with Decoded Neurofeedback Impacts Search Behavior

The necessity of conscious awareness in human learning has been a long-standing topic in psychology and neuroscience. Previous research on non-conscious associative learning is limited by the low signal-to-noise ratio of the subliminal stimulus, and the evidence remains controversial, including failures to replicate. Using functional MRI decoded neurofeedback, we guided participants from both sexes to generate neural patterns akin to those observed when visually perceiving real-world entities (e.g., dogs). Importantly, participants remained unaware of the actual content represented by these patterns. We utilized an associative DecNef approach to imbue perceptual meaning (e.g., dogs) into Japanese hiragana characters that held no inherent meaning for our participants, bypassing a conscious link between the characters and the dogs concept. Despite their lack of awareness regarding the neurofeedback objective, participants successfully learned to activate the target perceptual representations in the bilateral fusiform. The behavioral significance of our training was evaluated in a visual search task. DecNef and control participants searched for dogs or scissors targets that were pre-cued by the hiragana used during DecNef training or by a control hiragana. The DecNef hiragana did not prime search for its associated target but, strikingly, participants were impaired at searching for the targeted perceptual category. Hence, conscious awareness may function to support higher-order associative learning. Meanwhile, lower-level forms of re-learning, modification, or plasticity in existing neural representations can occur unconsciously, with behavioral consequences outside the original training context. The work also provides an account of DecNef effects in terms of neural representational drift.

Implicit bias training can remove bias from subliminal stimuli, restoring choice divergence: A proof-of-concept study

Subliminal information can influence our conscious life. Subliminal stimuli can influence cognitive tasks, while endogenous subliminal neural information can sway decisions before volition. Are decisions inextricably biased towards subliminal information? Or can they diverge away from subliminal biases via training? We report that implicit bias training can remove biases from subliminal sensory primes. We first show that subliminal stimuli biased an imagery-content decision task. Participants (n = 17) had to choose one of two different patterns to subsequently imagine. Subliminal primes significantly biased decisions towards imagining the primed option. Then, we trained participants (n = 7) to choose the non-primed option, via post choice feedback. This training was successful despite participants being unaware of the purpose or structure of the reward schedule. This implicit bias training persisted up to one week later. Our proof-of-concept study indicates that decisions might not always have to be biased towards non-conscious information, but instead can diverge from subliminal primes through training.

Food reinforcement architecture: A framework for impulsive and compulsive overeating and food abuse

Few reward-based theories address key drivers of susceptibility to food cues and consumption beyond fullness. Decision-making and habit formation are governed by reinforcement-based learning processes that, when overstimulated, can drive unregulated hedonically motivated overeating. Here, a model food reinforcement architecture is proposed that uses fundamental concepts in reinforcement and decision-making to identify maladaptive eating habits that can lead to obesity. This model is unique in that it identifies metabolic drivers of reward and incorporates neuroscience, computational decision-making, and psychology to map overeating and obesity. Food reinforcement architecture identifies two paths to overeating: a propensity for hedonic targeting of food cues contributing to impulsive overeating and lack of satiation that contributes to compulsive overeating. A combination of those paths will result in a conscious and subconscious drive to overeat independent of negative consequences, leading to food abuse and/or obesity. Use of this model to identify aberrant reinforcement learning processes and decision-making systems that can serve as markers of overeating risk may provide an opportunity for early intervention in obesity.

Habits, Goals, and Behavioral Signs of Cognitive Perseveration on Wisconsin Card-Sorting Tasks

Wisconsin card-sorting tasks provide unique opportunities to study cognitive flexibility and its limitations, which express themselves behaviorally as perseverative errors (PE). PE refer to those behavioral errors on Wisconsin card-sorting tasks that are committed when cognitive rules are maintained even though recently received outcomes demand to switch to other rules (i.e., cognitive perseveration). We explored error-suppression effects (ESE) across three Wisconsin card-sorting studies. ESE refer to the phenomenon that PE are reduced on repetitive trials compared to non-repetitive trials. We replicated ESE in all three Wisconsin card-sorting studies. Study 1 revealed that non-associative accounts of ESE, in particular the idea that cognitive inhibition may account for them, are not tenable. Study 2 suggested that models of instrumental learning are among the most promising associative accounts of ESE. Instrumental learning comprises goal-directed control and the formation of corresponding associative memories over and above the formation of habitual memories according to dual-process models of instrumental learning. Study 3 showed that cognitive, rather than motor, representations of responses should be conceptualized as elements entering goal-directed instrumental memories. Collectively, the results imply that ESE on Wisconsin card-sorting tasks are not only a highly replicable phenomenon, but they also indicate that ESE provide an opportunity to study cognitive mechanisms of goal-directed instrumental control. Based on the reported data, we present a novel theory of cognitive perseveration (i.e., the 'goal-directed instrumental control' GIC model), which is outlined in the Concluding Discussion.

Unconscious knowledge of rewards guides instrumental behaviors via conscious judgments

The demonstration that unconscious learning supports instrumental behaviors (i.e., choosing the stimuli that lead to rewards) is central for the tenet that unconscious cognition sustains human adaptation. Recent studies, using reliable subliminal conditioning paradigms and improved awareness measurements have found evidence against unconscious knowledge sustaining accurate instrumental responses. The present preregistered study proposes a paradigm, in which unconscious processing is stimulated not by subliminally exposing the predictive (conditioned) stimuli, but by employing predictive regularities that are complex and difficult to detect consciously. Participants (N = 211) were exposed to letter strings that, unknown to them, were built from two complex artificial grammars: a "rewarded" or a "non-rewarded" grammar. On each trial, participants memorized a string, and subsequently had to discriminate the memorized string from a distractor. Correct discriminations were rewarded only when the identified string followed the rewarded grammar, but not when it followed the non-rewarded grammar. In a subsequent test phase, participants were presented with new strings from the rewarded and from the unrewarded grammar. Their task was now to directly choose the strings from the rewarded grammar, in order to collect more rewards. A trial-by-trial awareness measure revealed that participants accurately choose novel strings from the rewarded grammar when they had no conscious knowledge of the grammar. The awareness measure also showed that participants were accurate only when the unconsciously learned grammar led to conscious judgments. The present study shows that unconscious knowledge can guide instrumental responses, but only to the extent it supports conscious judgments.

The effect of brief mindfulness training on the micro-structure of human free-operant responding: Mindfulness affects stimulus-driven responding

BACKGROUND AND OBJECTIVES

The current study examines the extent to which mindfulness impacts on operant conditioning processes, and explores the suggestion that mindfulness training serves to make humans more sensitive to the current reinforcement contingencies with which they are presented. In particular, the effect of mindfulness on the micro-structure of human schedule performance was explored. It was expected that mindfulness might impact bout-initiation responding to a greater degree than within-bout responding, premised on the assumption that bout-initiation responses are habitual and not under conscious control, but within-bout responses are goal-directed and conscious.

METHODS

Nonclinical participants experienced one of three brief (15min) interventions: focused attention breathing exercise (mindfulness), an unfocused attention breathing exercises, or no intervention. They then responded on a multiple random ratio (RR) random interval (RI) schedule.

RESULTS

In the no intervention and unfocused attention groups, overall and within-bout response rates were higher on the RR than the RI schedule, but bout-initiation rates were the same on the two schedules. However, for the mindfulness groups all forms of responding were higher for the RR than the RI schedule. Previous work has noted that habitual, and/or unconscious or fringe-conscious events, are impacted by mindfulness training.

LIMITATIONS

A nonclinical sample may limit generality.

CONCLUSIONS

The current pattern of results suggests that this is also true in schedule-controlled performance, and offers an insight into the manner in which mindfulness alongside conditioning-based interventions, to bring all responses under conscious control.

Perceptual and semantic same-different processing under subliminal conditions

Although other types of subliminal integrative processing are widely refuted by recent studies, subliminal same-different processing (SSDP) remains unchallenged to this day. Using shapes, categorical images, and Chinese characters as stimuli, the current study assessed whether SSDP can occur on a perceptual and semantic basis. Although some significant results were found, the effects are much weaker than previous studies, with Bayes factors suggesting that these effects are not reliable. It is therefore concluded that substantiating claims of SSDP requires more reliable evidence than currently available.

Feasibility of unconscious instrumental conditioning: A registered replication

The extent to which high-level, complex functions can proceed unconsciously has been a topic of considerable debate. While unconscious processing has been demonstrated for a range of low-level processes, from feature integration to simple forms of conditioning and learning, theoretical contributions suggest that increasing complexity requires conscious access. Here, we focus our attention on instrumental conditioning, which has been previously shown to proceed without stimulus awareness. Yet, instrumental conditioning also involves integrating information over a large temporal scale and distinct modalities in order to deploy selective action, constituting a process of substantial complexity. With this in mind, we revisit the question of feasibility of instrumental conditioning in the unconscious domain. Firstly, we address the theoretical and practical considerations relevant to unconscious learning in general. Secondly, we aim to replicate the first study to show instrumental conditioning in the absence of stimulus awareness (Pessiglione et al., 2008), following the original design and supplementing the original crucial analyses with a Bayesian approach (Experiment 1). We found that apparent unconscious learning took place when replicating the original methods directly and according to the tests of awareness used. However, we could not establish that the full sample was unaware in a separate awareness check. We therefore attempted to replicate the effect yet again with improved methods to address the issues related to sensitivity and immediacy (Experiment 2), including an individual threshold-setting task and a trial-by-trial awareness check permitting exclusion of individual aware trials. Here, we found evidence for absence of unconscious learning. This result provides evidence that instrumental conditioning did not occur without stimulus awareness in this paradigm, supporting the view that complex forms of learning may rely on conscious access. Our results provides support for the proposal that perceptual consciousness may be necessary for complex, flexible processes, especially where selective action and behavioural adaptation are required.

Unconscious integration: Current evidence for integrative processing under subliminal conditions

Integrative processing is traditionally believed to be dependent on consciousness. While earlier studies within the last decade reported many types of integration under subliminal conditions (i.e. without perceptual awareness), these findings are widely challenged recently. This review evaluates the current evidence for 10 types of subliminal integration that are widely studied: arithmetic processing, object-context integration, multi-word processing, same-different processing, multisensory integration and 5 different types of associative learning. Potential methodological issues concerning awareness measures are also taken into account. It is concluded that while there is currently no reliable evidence for subliminal integration, this does not necessarily refute 'unconscious' integration defined through non-subliminal (e.g. implicit) approaches.

Awareness is required for autonomic performance monitoring in instrumental learning: Evidence from cardiac activity

Performance monitoring is a vital aspect of successful learning and decision-making. Performance errors are reflected in the autonomic nervous system, indicating the need for behavioral adjustment. As part of this response, errors cause a pronounced deceleration in heart rate, compared to correct decisions, and precede explicit awareness of stimulus-response outcome contingencies. However, it is unknown whether those signals are present and able to inform instrumental learning without stimulus awareness, where explicit performance monitoring is disabled. With mixed evidence for unconscious instrumental learning, determining the presence or absence of autonomic signatures of performance monitoring can shed light on its feasibility. Here, we employed an unconscious instrumental conditioning task, where successful learning is evidenced by increased approach responses to visually masked rewarding stimuli, and avoidance of punishing stimuli. An electrocardiogram (ECG) assessed cardiac activity throughout the learning process. Natural fluctuations of awareness under masking permitted us to contrast learning and cardiac deceleration for trials with, versus without, conscious stimulus awareness. Our results demonstrate that on trials where participants did not consciously perceive the stimulus, there was no differentiation in cardiac response between rewarding and punishing feedback, indicating an absence of performance monitoring. In contrast, consciously perceived stimuli elicited the expected error-related deceleration. This result suggests that, in unconscious instrumental learning, the brain might be unable to acquire knowledge of stimulus values to guide correct instrumental choices. This evidence provides support for the notion that consciousness might be required for flexible adaptive behavior, and that this may be mediated through bodily signals.

Cardiac Cycle Affects the Asymmetric Value Updating in Instrumental Reward Learning

This study aimed to investigate whether instrumental reward learning is affected by the cardiac cycle. To this end, we examined the effects of the cardiac cycle (systole or diastole) on the computational processes underlying the participants' choices in the instrumental learning task. In the instrumental learning task, participants were required to select one of two discriminative stimuli (neutral visual stimuli) and immediately receive reward/punishment feedback depending on the probability assigned to the chosen stimuli. To manipulate the cardiac cycle, the presentation of discriminative stimuli was timed to coincide with either cardiac systole or diastole. We fitted the participants' choices in the task with reinforcement learning (RL) models and estimated parameters involving instrumental learning (i.e., learning rate and inverse temperature) separately in the systole and diastole trials. Model-based analysis revealed that the learning rate for positive prediction errors was higher than that for negative prediction errors in the systole trials; however, learning rates did not differ between positive and negative prediction errors in the diastole trials. These results demonstrate that the natural fluctuation of cardiac afferent signals can affect asymmetric value updating in instrumental reward learning.

Animal sentience

'Sentience' sometimes refers to the capacity for any type of subjective experience, and sometimes to the capacity to have subjective experiences with a positive or negative valence, such as pain or pleasure. We review recent controversies regarding sentience in fish and invertebrates and consider the deep methodological challenge posed by these cases. We then present two ways of responding to the challenge. In a policy-making context, precautionary thinking can help us treat animals appropriately despite continuing uncertainty about their sentience. In a scientific context, we can draw inspiration from the science of human consciousness to disentangle conscious and unconscious perception (especially vision) in animals. Developing better ways to disentangle conscious and unconscious affect is a key priority for future research.

What Is It Like to Be a Bass? Red Herrings, Fish Pain and the Study of Animal Sentience

Debates around fishes' ability to feel pain concern sentience: do reactions to tissue damage indicate evaluative consciousness (conscious affect), or mere nociception? Thanks to Braithwaite's discovery of trout nociceptors, and concerns that current practices could compromise welfare in countless fish, this issue's importance is beyond dispute. However, nociceptors are merely necessary, not sufficient, for true pain, and many measures held to indicate sentience have the same problem. The question of whether fish feel pain - or indeed anything at all - therefore stimulates sometimes polarized debate. Here, we try to bridge the divide. After reviewing key consciousness concepts, we identify "red herring" measures that should not be used to infer sentience because also present in non-sentient organisms, notably those lacking nervous systems, like plants and protozoa (P); spines disconnected from brains (S); decerebrate mammals and birds (D); and humans in unaware states (U). These "S.P.U.D. subjects" can show approach/withdrawal; react with apparent emotion; change their reactivity with food deprivation or analgesia; discriminate between stimuli; display Pavlovian learning, including some forms of trace conditioning; and even learn simple instrumental responses. Consequently, none of these responses are good indicators of sentience. Potentially more valid are aspects of working memory, operant conditioning, the self-report of state, and forms of higher order cognition. We suggest new experiments on humans to test these hypotheses, as well as modifications to tests for "mental time travel" and self-awareness (e.g., mirror self-recognition) that could allow these to now probe sentience (since currently they reflect perceptual rather than evaluative, affective aspects of consciousness). Because "bullet-proof" neurological and behavioral indicators of sentience are thus still lacking, agnosticism about fish sentience remains widespread. To end, we address how to balance such doubts with welfare protection, discussing concerns raised by key skeptics in this debate. Overall, we celebrate the rigorous evidential standards required by those unconvinced that fish are sentient; laud the compassion and ethical rigor shown by those advocating for welfare protections; and seek to show how precautionary principles still support protecting fish from physical harm.

From non-conscious processing to conscious events: a minimalist approach

The minimalist approach that we develop here is a framework that allows to appreciate how non-conscious processing and conscious contents shape human cognition, broadly defined. It is composed of three simple principles. First, cognitive processes are inherently non-conscious, while their inputs and (interim) outputs may be consciously experienced. Second, non-conscious processes and elements of the cognitive architecture prioritize information for conscious experiences. Third, conscious events are composed of series of conscious contents and non-conscious processes, with increased duration leading to more opportunity for processing. The narrowness of conscious experiences is conceptualized here as a solution to the problem of channeling the plethora of non-conscious processes into action and communication processes that are largely serial. The framework highlights the importance of prioritization for consciousness, and we provide an illustrative review of three main factors that shape prioritization-stimulus strength, motivational relevance and mental accessibility. We further discuss when and how this framework (i) is compatible with previous theories, (ii) enables new understandings of established findings and models, and (iii) generates new predictions and understandings.

Utility of Smartphone-based real-time report (Ecological Momentary Assessment) in the assessment and monitoring of awake bruxism: A multiple-week interval study in a Portuguese population of university students

PURPOSE

The Ecological Momentary Assessment (EMA) concept was the basis for developing a smartphone application for the on-time report of awake bruxism (AB) activities. This study aims to monitor AB report over time in a population of healthy young adults.

METHODS

A population of young adults recruited among dental students in good general health was recruited. All answered to a questionnaire, including the Oral Behavior Checklist (OBC-1). They were then monitored with a smartphone application for AB report during seven consecutive days and completed three further observation periods (EMA-1, 2 and 3) at one-month intervals. After the third period (EMA-3), participants answered again the OBC questionnaire (OBC-2). Changes over time were described, and Pearson Correlation test was performed to assess the correlation between EMA and OBC items reports. A significance level of p = .01 was set.

RESULTS

Thirty-one University students completed the study protocol. Answers to the OBC showed an increase in the prevalence of self-reported bracing and teeth clenching from the first to second report (38.7%-54.8% and 77.4%-90.3%, respectively). A slight increase in the 'relaxed' condition (62.5%-69.0%) was observed with EMA-based smartphone application over time. No correlation between OBC items and EMA was detected between OBC-1 and EMA-1. A moderate positive correlation in bracing report (+0.509, p = .01) and weak positive correlation in teeth contact report (+0.380, p = .05) were found between OBC-2 and EMA-3.

CONCLUSION

Using a smartphone-based approach to AB report may be helpful to monitor AB over time and increase an individual's awareness to recognise actions such as bracing and teeth contact concerning the single-time report.

A Traditional Scientific Perspective on the Integrated Information Theory of Consciousness

This paper assesses two different theories for explaining consciousness, a phenomenon that is widely considered amenable to scientific investigation despite its puzzling subjective aspects. I focus on Integrated Information Theory (IIT), which says that consciousness is integrated information (as ϕMax) and says even simple systems with interacting parts possess some consciousness. First, I evaluate IIT on its own merits. Second, I compare it to a more traditionally derived theory called Neurobiological Naturalism (NN), which says consciousness is an evolved, emergent feature of complex brains. Comparing these theories is informative because it reveals strengths and weaknesses of each, thereby suggesting better ways to study consciousness in the future. IIT's strengths are the reasonable axioms at its core; its strong logic and mathematical formalism; its creative "experience-first" approach to studying consciousness; the way it avoids the mind-body ("hard") problem; its consistency with evolutionary theory; and its many scientifically testable predictions. The potential weakness of IIT is that it contains stretches of logic-based reasoning that were not checked against hard evidence when the theory was being constructed, whereas scientific arguments require such supporting evidence to keep the reasoning on course. This is less of a concern for the other theory, NN, because it incorporated evidence much earlier in its construction process. NN is a less mature theory than IIT, less formalized and quantitative, and less well tested. However, it has identified its own neural correlates of consciousness (NCC) and offers a roadmap through which these NNCs may answer the questions of consciousness using the hypothesize-test-hypothesize-test steps of the scientific method.

Unconscious preparation: Effects of prime visibility on semantic generalization of task priming

Studies found that subliminal primes can be associated with specific tasks to facilitate task performance, and such learning is highly adaptive and generalizable. Meanwhile, conditioning studies suggest that aversive/reward learning and generalization actually occur at the semantic level. The current study shows that prime-task associations can also be generalized to novel word/neighbour primes from the same semantic category, and this occurs without contingency awareness. Previous studies have counterintuitively suggested that both the learning of task priming and the semantic priming of word neighbours depend on the lack of visibility. Here, we show that semantic generalization indeed depends on reduced visibility, but cannot occur subliminally. The current study shows for the first time that semantic learning and generalization can occur without any emotional or motivational factors, and that semantic priming can occur for arbitrary-linked stimuli in a context completely devoid of semantics.