Neurorestorative effects of cerebellar transcranial direct current stimulation on social prediction of adolescents and young adults with congenital cerebellar malformations

BACKGROUND

Converging evidence points to impairments of the predictive function exerted by the cerebellum as one of the causes of the social cognition deficits observed in patients with cerebellar disorders.

OBJECTIVE

We tested the neurorestorative effects of cerebellar transcranial direct current stimulation (ctDCS) on the use of contextual expectations to interpret actions occurring in ambiguous sensory sceneries in a sample of adolescents and young adults with congenital, non-progressive cerebellar malformation (CM).

METHODS

We administered an action prediction task in which, in an implicit-learning phase, the probability of co-occurrence between actions and contextual elements was manipulated to form either strongly or moderately informative expectations. Subsequently, in a testing phase, we probed the use of these contextual expectations for predicting ambiguous (i.e., temporally occluded) actions. In a sham-controlled, within-subject design, participants received anodic or sham ctDCS during the task.

RESULTS

Anodic ctDCS, compared to sham, improved patients' ability to use contextual expectations to predict the unfolding of actions embedded in moderately, but not strongly, informative contexts.

CONCLUSIONS

These findings corroborate the role of the cerebellum in using previously learned contextual associations to predict social events and document the efficacy of ctDCS to boost social prediction in patients with congenital cerebellar malformation. The study encourages the further exploration of ctDCS as a neurorestorative tool for the neurorehabilitation of social cognition abilities in neurological, neuropsychiatric, and neurodevelopmental disorders featured by macro- or micro-structural alterations of the cerebellum.

Intact or impaired? The understanding of give-and-take interactions in children with autism spectrum disorder

BACKGROUND

Understanding and predicting others' behavior in a dynamic and rapidly changing world is a fundamental aspect of social interactions. However, it remains unclear as to whether children with autism spectrum disorder (ASD) could understand and predict goal-directed social actions.

AIMS

To investigate the understanding of give-and-take interactions in children with ASD with the use of eye tracking.

METHODS AND PROCEDURES

Experiment 1 and 2 investigated the understanding of giving and taking respectively in 5-to 8-year-old Chinese children with ASD and typically developing children by using the eye-tracking technology.

OUTCOMES AND RESULTS

We found that children with ASD could predict actions, but they were less proficient in processing give-and-take interactions. Moreover, children with ASD showed impaired understanding of giving but not taking.

CONCLUSIONS

These results indicate that the basic mechanisms of action prediction are intact in children with ASD whereas there may be deficits in the top-down social processing of the giving gesture.

Predictive motor activation: Modulated by expectancy or predictability?

Predicting actions is a fundamental ability that helps us to comprehend what is happening in our environment and to interact with others. The motor system was previously identified as source of action predictions. Yet, which aspect of the statistical likelihood of upcoming actions the motor system is sensitive to remains an open question. This EEG study investigated how regularities in observed actions are reflected in the motor system and utilized to predict upcoming actions. Prior to measuring EEG, participants watched videos of action sequences with different transitional probabilities. After training, participants' brain activity over motor areas was measured using EEG while watching videos of action sequences with the same statistical structure. Focusing on the mu and beta frequency bands we tested whether activity of the motor system reflects the statistical likelihood of upcoming actions. We also explored two distinct aspects of the statistical structure that capture different prediction processes, expectancy and predictability. Expectancy describes participants' expectation of the most likely action, whereas predictability represents all possible actions and their relative probabilities. Results revealed that mu and beta oscillations play different roles during action prediction. While the mu rhythm reflected anticipatory activity without any link to the statistical structure, the beta rhythm was related to the expectancy of an action. Our findings support theories proposing that the motor system underlies action prediction, and they extend such theories by showing that multiple forms of statistical information are extracted when observing action sequences. This information is integrated in the prediction generated by the neural motor system of which action is going to happen next.

Evidence of motor resonance in stroke patients with severe upper limb function impairments

For the past fifteen years, observation of actions has proved to be effective in the motor rehabilitation of stroke. Despite this, no evidence has ever been provided that this practice is able to activate the efferent motor system of a limb unable to perform the observed action due to stroke. In fact, transcranial magnetic stimulation cannot easily be used in these patients, and the fMRI evidence is inconclusive. This creates a logical problem, as the effectiveness of action observation in functional recovery is attributed to its ability to evoke action simulation, up to sub-threshold muscle activation (i.e., motor resonance), in healthy individuals. To provide the necessary proof-of-concept, patients with severe upper limb function impairments and matched control participants were submitted to a verified action prediction paradigm. They were asked to watch videos showing gripping movements towards a graspable or an ungraspable object, and to press a button the instant the agent touched the object. The presence of more accurate responses for the graspable object trials is considered an indirect evidence of motor resonance. Participants were required to perform the task in two sessions which differed in the hand used to respond. Despite the serious difficulty of movement, 8 out of 18 patients were able to perform the task with their impaired hand. We found that the responses given by the paretic hand showed a modulation of the action prediction time no different from that showed by the non-paretic hand, which, in turn, did not differ from that showed by the matched control participants. The present proof-of-concept study shows that action observation involves the efferent motor system even when the hand used to respond is unable to perform the observed action due to a cortical lesion, providing the missing evidence to support the already established use of Action Observation Training (AOT) in motor rehabilitation of stroke.

The Involvement of the Posterior Cerebellum in Reconstructing and Predicting Social Action Sequences

Recent advances in social neuroscience have highlighted the critical role of the cerebellum and especially the posterior cerebellar Crus in social mentalizing (i.e., theory of mind). Research in the past 5 years has provided growing evidence supporting the view that the posterior cerebellum builds internal action models of our social interactions to predict how other people's actions will be executed, and what our most likely responses to these actions will be. This paper presents an overview of a series of fMRI experiments on novel tasks involving a combination of (a) the learning or generation of chronological sequences of social actions either in an explicit or implicit manner, which (b) require social mentalizing on another person's mental state such as goals, beliefs, and implied traits. Together, the results strongly confirm the central role of the posterior cerebellar Crus in identifying and automatizing action sequencing during social mentalizing, and in predicting future action sequences based on social mentalizing inferences about others. This research program has important implications: It provides for the first time (a) fruitful starting points for diagnosing and investigating social sequencing dysfunctions in a variety of mental disorders which have also been related to cerebellar dysfunctions, (b) provides the necessary tools for testing whether non-invasive neurostimulation targeting the posterior cerebellum has a causal effect on social functioning, and (c) whether these stimulation techniques and training programs guided by novel cerebellar social sequencing insights, can be exploited to increase posterior cerebellar plasticity in order to alleviate social impairments in mental disorders.

Constrained Choice: Children's and Adults' Attribution of Choice to a Humanoid Robot

Young children, like adults, understand that human agents can flexibly choose different actions in different contexts, and they evaluate these agents based on such choices. However, little is known about children's tendencies to attribute the capacity to choose to robots, despite increased contact with robotic agents. In this paper, we compare 5- to 7-year-old children's and adults' attributions of free choice to a robot and to a human child by using a series of tasks measuring agency attribution, action prediction, and choice attribution. In morally neutral scenarios, children ascribed similar levels of free choice to the robot and the human, while adults were more likely to ascribe free choice to the human. For morally relevant scenarios, however, both age groups considered the robot's actions to be more constrained than the human's actions. These findings demonstrate that children and adults hold a nuanced understanding of free choice that is sensitive to both the agent type and constraints within a given scenario.

Social Thinking is for Doing: The Posterior Cerebellum Supports Prediction of Social Actions Based on Personality Traits

Can we predict the future by reading others’ minds? This study explores whether attributing others’ personality traits facilitates predictions about their future actions and the temporal order of these future actions. Prior evidence demonstrated that the posterior cerebellar crus is involved in identifying the temporal sequence of social actions and the person’s traits they imply. Based on this, we hypothesized that this area might also be recruited in the reverse process; that is, knowledge of another person’s personality traits supports predictions of temporal sequences of others’ actions. In this study, participants were informed about the trait of a person and then had to select actions that were consistent with this information and arrange them in the most likely temporal order. As hypothesized, the posterior cerebellar crus 1 and crus 2 were strongly activated when compared to a control task which involved only the selection of actions (without temporal ordering) or which depicted non-social objects and their characteristics. Our findings highlight the important function of the posterior cerebellar crus in the prediction of social action sequences in social understanding.

Predictive action perception from explicit intention information in autism

Social difficulties in autism spectrum disorder (ASD) may originate from a reduced top-down modulation of sensory information that prevents the spontaneous attribution of intentions to observed behaviour. However, although people with autism are able to explicitly reason about others' mental states, the effect of abstract intention information on perceptual processes has remained untested. ASD participants (n = 23) and a neurotypical (NT) control group (n = 23) observed a hand either reaching for an object or withdrawing from it. Prior to action onset, the participant either instructed the actor to "Take it" or "Leave it", or heard the actor state "I'll take it" or "I'll leave it", which provided an explicit intention that was equally likely to be congruent or incongruent with the subsequent action. The hand disappeared before completion of the action, and participants reported the last seen position of the tip of the index finger by touching the screen. NT participants exhibited a predictive bias in response to action direction (reaches perceived nearer the object, withdrawals perceived farther away), and in response to prior knowledge of the actor's intentions (nearer the object after "Take it", farther away after "Leave it"). However, ASD participants exhibited a predictive perceptual bias only in response to the explicit intentions, but not in response to the motion of the action itself. Perception in ASD is not immune from top-down modulation. However, the information must be explicitly presented independently from the stimulus itself, and not inferred from cues inherent in the stimulus.

Action predictability is reflected in beta power attenuation and predictive eye movements in adolescents with and without autism

Most theoretical accounts of autism posit difficulties in predicting others' actions, and this difficulty has been proposed to be at the root of autistic individuals' social communication differences. Empirical results are mixed, however, with autistic individuals showing reduced action prediction in some studies but not in others. It has recently been proposed that this effect might be observed primarily when observed actions are less predictable, but this idea has yet to be tested. To assess the influence of predictability on neural and behavioural action prediction, the current study employed an action observation paradigm with multi-step actions that become gradually more predictable. Autistic and non-autistic adolescents showed similar patterns of motor system activation during observation, as seen in attenuated mu and beta power compared to baseline, with beta power further modulated by predictability in both groups. Bayesian statistics confirmed that action predictability influenced beta power similarly in both groups. The groups also made similar behavioural predictions, as seen in three eye-movement measures. We found no evidence that autistic adolescents responded differently than non-autistic adolescents to the predictability of an observed action. These findings show that autistic adolescents do spontaneously predict others' actions, both neurally and behaviourally, which calls into question the role of action prediction as a key mechanism underlying autism.

Real-time multiple spatiotemporal action localization and prediction approach using deep learning

Detecting the locations of multiple actions in videos and classifying them in real-time are challenging problems termed "action localization and prediction" problem. Convolutional neural networks (ConvNets) have achieved great success for action localization and prediction in still images. A major advance occurred when the AlexNet architecture was introduced in the ImageNet competition. ConvNets have since achieved state-of-the-art performances across a wide variety of machine vision tasks, including object detection, image segmentation, image classification, facial recognition, human pose estimation, and tracking. However, few works exist that address action localization and prediction in videos. The current action localization research primarily focuses on the classification of temporally trimmed videos in which only one action occurs per frame. Moreover, nearly all the current approaches work only offline and are too slow to be useful in real-world environments. In this work, we propose a fast and accurate deep-learning approach to perform real-time action localization and prediction. The proposed approach uses convolutional neural networks to localize multiple actions and predict their classes in real time. This approach starts by using appearance and motion detection networks (known as "you only look once" (YOLO) networks) to localize and classify actions from RGB frames and optical flow frames using a two-stream model. We then propose a fusion step that increases the localization accuracy of the proposed approach. Moreover, we generate an action tube based on frame level detection. The frame by frame processing introduces an early action detection and prediction with top performance in terms of detection speed and precision. The experimental results demonstrate this superiority of our proposed approach in terms of both processing time and accuracy compared to recent offline and online action localization and prediction approaches on the challenging UCF-101-24 and J-HMDB-21 benchmarks.

Translating visual information into action predictions: Statistical learning in action and nonaction contexts

Humans are sensitive to the statistical regularities in action sequences carried out by others. In the present eyetracking study, we investigated whether this sensitivity can support the prediction of upcoming actions when observing unfamiliar action sequences. In two between-subjects conditions, we examined whether observers would be more sensitive to statistical regularities in sequences performed by a human agent versus self-propelled 'ghost' events. Secondly, we investigated whether regularities are learned better when they are associated with contingent effects. Both implicit and explicit measures of learning were compared between agent and ghost conditions. Implicit learning was measured via predictive eye movements to upcoming actions or events, and explicit learning was measured via both uninstructed reproduction of the action sequences and verbal reports of the regularities. The findings revealed that participants, regardless of condition, readily learned the regularities and made correct predictive eye movements to upcoming events during online observation. However, different patterns of explicit-learning outcomes emerged following observation: Participants were most likely to re-create the sequence regularities and to verbally report them when they had observed an actor create a contingent effect. These results suggest that the shift from implicit predictions to explicit knowledge of what has been learned is facilitated when observers perceive another agent's actions and when these actions cause effects. These findings are discussed with respect to the potential role of the motor system in modulating how statistical regularities are learned and used to modify behavior.

Oxytocin promotes action prediction

The neuropeptide oxytocin (OT) has been suggested to facilitate social cognition and behavior. As predicting others' behavior is at the core of human social-cognitive abilities and is indispensable for successful social interaction, we hypothesized that OT would increase action prediction. To test this hypothesis, 61 male and female healthy participants self-administered OT or placebo intranasally and their anticipatory eye-movements were recorded using eye-tracking techniques. We found that the ability to predict others' future actions was enhanced following OT treatment. This effect was mediated by the time to the first anticipatory eye-movement suggesting that improved action prediction might operate by increased attention to social cues. These findings provide direct evidence for the role of OT in promoting perception and processing of social cues.

You cannot speak and listen at the same time: a probabilistic model of turn-taking

Turn-taking is a preverbal skill whose mastering constitutes an important precondition for many social interactions and joint actions. However, the cognitive mechanisms supporting turn-taking abilities are still poorly understood. Here, we propose a computational analysis of turn-taking in terms of two general mechanisms supporting joint actions: action prediction (e.g., recognizing the interlocutor's message and predicting the end of turn) and signaling (e.g., modifying one's own speech to make it more predictable and discriminable). We test the hypothesis that in a simulated conversational scenario dyads using these two mechanisms can recognize the utterances of their co-actors faster, which in turn permits them to give and take turns more efficiently. Furthermore, we discuss how turn-taking dynamics depend on the fact that agents cannot simultaneously use their internal models for both action (or messages) prediction and production, as these have different requirements-or, in other words, they cannot speak and listen at the same time with the same level of accuracy. Our results provide a computational-level characterization of turn-taking in terms of cognitive mechanisms of action prediction and signaling that are shared across various interaction and joint action domains.

Toddlers' action prediction: Statistical learning of continuous action sequences

The current eye-tracking study investigated whether toddlers use statistical information to make anticipatory eye movements while observing continuous action sequences. In two conditions, 19-month-old participants watched either a person performing an action sequence (Agent condition) or a self-propelled visual event sequence (Ghost condition). Both sequences featured a statistical structure in which certain action pairs occurred with deterministic transitional probabilities. Toddlers learned the transitional probabilities between the action steps of the deterministic action pairs and made predictive fixations to the location of the next action in the Agent condition but not in the Ghost condition. These findings suggest that young toddlers gain unique information from the statistical structure contained within action sequences and are able to successfully predict upcoming action steps based on this acquired knowledge. Furthermore, predictive gaze behavior was correlated with reproduction of sequential actions following exposure to statistical regularities. This study extends previous developmental work by showing that statistical learning can guide the emergence of anticipatory eye movements during observation of continuous action sequences.

Reaction time profiles of adults' action prediction reveal two mindreading systems

Human beings are able to quickly step into others' shoes to predict peoples' actions. There is little consensus over how this cognitive feat might be accomplished. We tested the hypotheses that an efficient, but inflexible, mindreading system gives rise to appropriate reaction time facilitation in a standard unexpected transfer task, but not in a task involving an identity component. We created a new behavioural paradigm where adults had to quickly select whether an actor would reach, or not reach, for an object based on the actor's false belief about the object's location. By manipulating the type of object we compared participants' responding behaviour when they did and did not have to take the actor's perspective into account. While the overall accuracy reflected a high level of flexible belief reasoning across both tasks, the pattern of response times across conditions revealed a limit in the processing scope of an efficient mindreading system. Thus, we show, for the first time, that there are indeed different profiles of reaction times for object-location scenarios and for object-identity scenarios. The results elevate growing evidence that adult humans have not one, but two mindreading systems for dealing with mental states that underlie action.

Predictive action tracking without motor experience in 8-month-old infants

•

Can infants predictively track the kinematics of actions outside their motor repertoire?

•

Pre-walking infants predictively tracked upright, but not inverted stepping actions.

•

Sensorimotor cortex was activated more when infants observed upright stepping actions.

•

Motor experience is not necessary for predictive tracking of action kinematics.

A popular idea in cognitive neuroscience is that to predict others’ actions, observers need to map those actions onto their own motor repertoire. If this is true, infants with a relatively limited motor repertoire should be unable to predict actions with which they have no previous motor experience. We investigated this idea by presenting pre-walking infants with videos of upright and inverted stepping actions that were briefly occluded from view, followed by either a correct (time-coherent) or an incorrect (time-incoherent) continuation of the action (Experiment 1). Pre-walking infants looked significantly longer to the still frame after the incorrect compared to the correct continuations of the upright, but not the inverted stepping actions. This demonstrates that motor experience is not necessary for predictive tracking of action kinematics. In a follow-up study (Experiment 2), we investigated sensorimotor cortex activation as a neural indication of predictive action tracking in another group of pre-walking infants. Infants showed significantly more sensorimotor cortex activation during the occlusion of the upright stepping actions that the infants in Experiment 1 could predictively track, than during the occlusion of the inverted stepping actions that the infants in Experiment 1 could not predictively track. Taken together, these findings are inconsistent with the idea that motor experience is necessary for the predictive tracking of action kinematics, and suggest that infants may be able to use their extensive experience with observing others’ actions to generate real-time action predictions.

Social constraints from an observer's perspective: Coordinated actions make an agent's position more predictable

Action prediction, a crucial ability to support social activities, is sensitive to the individual goals of expected actions. This article reports a novel finding that the predictions of observed actions for a temporarily invisible agent are influenced, and even enhanced, when this agent has a joint/collective goal to implement coordinated actions with others (i.e., with coordination information). Specifically, we manipulated the coordination information by presenting two chasers and one common target to perform coordinated or individual chases, and subjects were required to predict the expected action (i.e., position) for one chaser after it became momentarily invisible. To control for possible low-level physical properties, we also established some intense paired controls for each type of chase, such as backward replay (Experiment 1), making the chasing target invisible (Experiment 2) and a direct manipulation of the goal-directedness of one chaser's movements to disrupt coordination information (Experiment 3). The results show that the prediction error for invisible chasers depends on whether the second chaser is coordinated with the first, and this effect vanishes when the chasers behaves with exactly the same motions, but without coordination information between them; furthermore, this influence results in enhancing the performance of action prediction. These findings extend the influential factors of action prediction to the level of observed coordination information, implying that the functional characteristic of mutual constraints of coordinated actions can be utilized by vision.

Attending to what matters: flexibility in adults' and infants' action perception

Action perception is selective in that observers attend to and encode certain dimensions of action over others. But how flexible is action perception in its selection of perceptual information? One possibility is that observers consistently attend to particular dimensions of action over others across different contexts. Another possibility, tested here, is that observers flexibly vary their attention to different dimensions of action based on the context in which action occurs. We investigated 9.5-month-old infants' and adults' ability to attend to drop height under varying contexts-aiming to drop an object into a narrow container versus a wide container. We predicted differential attention to increases in aiming height for the narrow container versus the wide container because an increase in aiming height has a differential effect on success (i.e., getting the object into the container) depending on the width of the container. Both adults and infants showed an asymmetry in their attention to aiming height as a function of context; in the wide container condition increases and decreases in aiming height were equally detectable, whereas in the narrow container condition observers more readily discriminated increases over decreases in aiming height. These results indicate that action perception is both selective and flexible according to context, aiding in action prediction and infants' social-cognitive development more broadly.

Prediction processes during multiple object tracking (MOT): involvement of dorsal and ventral premotor cortices

BACKGROUND

The multiple object tracking (MOT) paradigm is a cognitive task that requires parallel tracking of several identical, moving objects following nongoal-directed, arbitrary motion trajectories.

AIMS

The current study aimed to investigate the employment of prediction processes during MOT. As an indicator for the involvement of prediction processes, we targeted the human premotor cortex (PM). The PM has been repeatedly implicated to serve the internal modeling of future actions and action effects, as well as purely perceptual events, by means of predictive feedforward functions.

MATERIALS AND METHODS

Using functional magnetic resonance imaging (fMRI), BOLD activations recorded during MOT were contrasted with those recorded during the execution of a cognitive control task that used an identical stimulus display and demanded similar attentional load. A particular effort was made to identify and exclude previously found activation in the PM-adjacent frontal eye fields (FEF).

RESULTS

We replicated prior results, revealing occipitotemporal, parietal, and frontal areas to be engaged in MOT.

DISCUSSION

The activation in frontal areas is interpreted to originate from dorsal and ventral premotor cortices. The results are discussed in light of our assumption that MOT engages prediction processes.

CONCLUSION

We propose that our results provide first clues that MOT does not only involve visuospatial perception and attention processes, but prediction processes as well.

Dynamic simulation and static matching for action prediction: evidence from body part priming

Accurately predicting other people's actions may involve two processes: internal real-time simulation (dynamic updating) and matching recently perceived action images (static matching). Using a priming of body parts, this study aimed to differentiate the two processes. Specifically, participants played a motion-controlled video game with either their arms or legs. They then observed arm movements of a point-light actor, which were briefly occluded from view, followed by a static test pose. Participants judged whether this test pose depicted a coherent continuation of the previously seen action (i.e., "action prediction task"). Evidence of dynamic updating was obtained after compatible effector priming (i.e., arms), whereas incompatible effector priming (i.e., legs) indicated static matching. Together, the results support action prediction as engaging two distinct processes, dynamic simulation and static matching, and indicate that their relative contributions depend on contextual factors like compatibility of body parts involved in performed and observed action.