Cortical and subcortical responses to biological motion

Examining Biological Motion as a Potential Factor in E-Scooter Conspicuity and Safety

BackgroundE-scooter injuries have risen in recent years and riders report a relatively high prevalence of accidents. Collisions with motor vehicles pose a high risk to e-scooter users. E-scooter riders move fast relative to runners but lack movement of limbs that present aspects of biological motion to drivers, which may diminish conspicuity.MethodTwo experiments measured participants' detection of point light representations beneath masking visual noise. Study 1 presented a runner, e-scooter rider, and rectangular object. Study 2 modified the e-scooter stimuli to remove motion sway and added alternative e-scooter presentations, one with moving lights consistent with biological motion and the other with the same motion in reverse, inconsistent with biological motion.ResultsStudy 1 found a main effect of figure type, with the runner resulting in superior detection, recognition, and response time compared to the e-scooter rider, which performed better than the object. Study 2 found better perception performance for the runner, including better detection compared to the reverse motion e-scooter.ConclusionFindings suggest that reduced biological motion produced by e-scooter users slightly worsens and slows their detection by other road users and indicates an advantage for the perception of human body configurations. Any inclusion of apparent motion to improve detection, especially near the ground, should be consistent with biological motion.ApplicationVisual display alterations (e.g., lighting) to introduce apparent motion that mimics biological movements or is consistent with biological motion may potentially confer a detection advantage over other movement patterns.

Semantic audio-visual congruence modulates visual sensitivity to biological motion across awareness levels

Whether cross-modal interaction requires conscious awareness of multisensory information or whether it can occur in the absence of awareness, is still an open question. Here, we investigated if sounds can enhance detection sensitivity of semantically matching visual stimuli at varying levels of visual awareness. We presented biological motion stimuli of human actions (walking, rowing, sawing) during dynamic continuous flash suppression (CFS) to 80 participants and measured the effect of co-occurring, semantically matching or non-matching action sounds on visual sensitivity (d'). By individually thresholding stimulus contrast, we distinguished participants who detected motion either above or at chance level. Participants who reliably detected visual motion above chance showed higher sensitivity to upright versus inverted biological motion across all experimental conditions. In contrast, participants detecting visual motion at chance level, i.e. during successful suppression, demonstrated this upright advantage exclusively during trials with semantically congruent sounds. Across the whole sample, the impact of sounds on visual sensitivity increased as participants' visual detection performance decreased, revealing a systematic trade-off between auditory and visual processing. Our findings suggest that semantic congruence between auditory and visual information can selectively modulate biological motion perception when visual awareness is minimal or absent, while more robust visual signals enable perception of biological motion independent of auditory input. Thus, semantically congruent sounds may impact visual representations as a function of the level of visual awareness.

Sensorimotor simulation and distributed processing of biological motion: Insights from healthy and paraplegic adults

The processing of biological motion (BM), particularly the local motion cues tracing the movements of crucial joints, is vital for social interaction and human survival. While numerous studies have focused on the brain mechanisms underlying BM processing, the contribution of sensorimotor simulation at peripheral effectors remains unclear. In this study, we examined healthy adults and paraplegic spinal cord injury participants to investigate this issue. For healthy adults, both intact BM stimuli and local BM cues without global configuration induced a temporal dilation effect when sitting (sensorimotor simulation accessible), but not when standing (sensorimotor simulation temporarily hindered). In contrast, for participants with permanently hindered sensorimotor simulation, the temporal dilation effect was observed only with intact BM stimuli but not with local BM cues, indicating a robust reliance on sensorimotor simulation during the processing of local BM cues and a selective compensation based on global configuration cues for the permanent loss of sensorimotor simulation. These findings highlight the role of embodied cognition in the distributed processing of biological motion and suggest the importance of selective compensation under damaged sensorimotor circuits.

Shared and Unique Neural Codes for Biological Motion Perception in Humans and Macaque Monkeys

Throughout evolution, living organisms have honed the ability to swiftly recognize biological motion (BM) across species. However, how the brain processes within- and cross-species BM, and the evolutionary progression of these processes, remain unclear. To investigate these questions, the current study examined brain activity in the lateral temporal areas of humans and monkeys as they passively observed upright and inverted human and macaque BM stimuli. In humans, the middle temporal area (hMT+) responded to both human and macaque BM stimuli, while the right posterior superior temporal sulcus (hpSTS) exhibited selective responses to human BM stimuli. This selectivity is evidenced by an increased feedforward connection from hMT+ to hpSTS during the processing of human BM stimuli. In monkeys, the MT region processed BM stimuli from both species, but no subregion in the STS anterior to MT is specific to conspecific BM stimuli. A comparison of these findings suggests that upstream brain regions (i.e., MT) may retain homologous functions across species, while downstream brain regions (i.e., STS) may have undergone differentiation and specialization throughout evolution. These results provide insights into the commonalities and differences in the specialized visual pathway engaged in processing within- and cross-species BMs, as well as their functional divergence during evolution.

Estimations of biological motion walking direction are affected by observer and walker genders

A facing-towards bias is commonly reported when observers are asked to judge the motion directions of others. However, it remains unclear just how accurately observers are able to estimate the motion direction of others and whether the gender of the observer and the walker affects the direction estimation. Here, we asked male and female participants to estimate the direction of a point-light walker (PLW) in three experiments. The gender of PLWs was neutral (Experiment 1, 96 participants), clearly male or female (Experiment 2, 72 participants), or more subtlety male or female (Experiment 3, 98 participants). We found that female PLWs showed a stronger reference repulsion bias (RRB) than male PLWs. That is, for female PLWs, the estimates of facing directions were biased away from the boundaries of facing-towards, facing-away, and lateral (left/right)-motion directions. Interestingly, RRBs differed depending on whether the observer was male or female. When the PLW gender difference was clear, the RRB was stronger for female participants than male participants; when the PLW gender difference was reduced, the trend disappeared or was reversed. Finally, the perceived PLW direction was biased towards the previously seen PLW direction, showing serial dependence that was not affected by the PLW and observer genders. In conclusion, the current study shows that observers can accurately estimate PLW directions, but that judgments are curiously affected by both the observers' and PLWs' genders.

Detecting biological motion signals in human and monkey superior colliculus: a subcortical-cortical pathway for biological motion perception

Most vertebrates, including humans, are highly adept at detecting and encoding biological motion, even when it is portrayed by just a few point lights attached to the head and major joints. However, the function of subcortical regions in biological motion perception has been scarcely explored. Here, we investigate the role of the superior colliculus in local biological motion processing. Using high-field (3 T) and ultra-high-field (7 T) functional magnetic resonance imaging, we record the neural responses of the superior colliculus to scrambled point-light walkers (with local kinematics retained) in both humans and male macaque monkeys. Results show that the superior colliculus, especially the superficial layers, selectively responds to local biological motion. Furthermore, dynamic causal modeling analysis reveals a subcortical-cortical functional pathway that transmits local biological motion signals from the superior colliculus via the middle temporal visual complex to the posterior superior temporal sulcus in the human brain. These findings suggest the existence of a cross-species mechanism in the superior colliculus that facilitates the detection of local biological motion at the early stage of the visual processing stream.

A common and specialized neural code for social attention triggered by eye gaze and biological motion

Humans appear to be endowed with the ability to readily share attention with interactive partners through the utilization of social direction cues, such as eye gaze and biological motion (BM). Here, we investigated the specialized brain mechanism underlying this fundamental social attention ability by incorporating different types of social (i.e., BM, gaze) and non-social (arrow) cues and combining functional magnetic resonance imaging (fMRI) with a modified central cueing paradigm. Using multi-voxel pattern analysis (MVPA), we found that although gaze- and BM-mediated attentional orienting could be decoded from neural activity in a wide range of brain areas, only the right anterior and posterior superior temporal sulcus (aSTS and pSTS) could specifically decode attentional orienting triggered by social but not non-social cues. Critically, cross-category MVPA further revealed that social attention could be decoded across BM and gaze cues in the right STS and the right superior temporal gyrus (STG). However, these regions could not decode attentional orienting across social and non-social cues. These findings together provide evidence for the existence of a specialized social attention module in the human brain, with the right STS/STG being the critical neural site dedicated to social attention.

I like the way you move: how animate motion affects visual attention in early human infancy

The ability to detect animates (as compared with inanimates) rapidly is advantageous for human survival. Due to its relevance, not only the adult human brain has evolved specific neural mechanisms to discriminate animates, but it has been proposed that selection finely tuned the human visual attention system to prioritize visual cues that signal the presence of living things. Among them, animate motion-i.e., the motion of animate entities -, is one of the most powerful cues that triggers humans' attention. From a developmental point of view, whether such specialization is inborn or acquired through experience is a fascinating research topic. This mini-review aims to summarize and discuss recent behavioral and electrophysiological research that suggests that animate motion has an attentional advantage in the first year of life starting from birth. Specifically, the rationale underlying this paper concerns how attention deployment is affected by animate motion conveyed both by the movement of a single dot and, also, when the single dot is embedded in a complex array, named biological motion. Overall, it will highlight the importance of both inborn predispositions to pay attention preferentially to animate motion, mainly supported by subcortical structures, and the exposure to certain experiences, shortly after birth, to drive the cortical attentional visual system to become the way it is in adults.

Atypical local and global biological motion perception in children with attention deficit hyperactivity disorder

Perceiving biological motion (BM) is crucial for human survival and social interaction. Many studies have reported impaired BM perception in autism spectrum disorder, which is characterised by deficits in social interaction. Children with attention deficit hyperactivity disorder (ADHD) often exhibit similar difficulties in social interaction. However, few studies have investigated BM perception in children with ADHD. Here, we compared differences in the ability to process local kinematic and global configurational cues, two fundamental abilities of BM perception, between typically developing and ADHD children. We further investigated the relationship between BM perception and social interaction skills measured using the Social Responsiveness Scale and examined the contributions of latent factors (e.g. sex, age, attention, and intelligence) to BM perception. The results revealed that children with ADHD exhibited atypical BM perception. Local and global BM processing showed distinct features. Local BM processing ability was related to social interaction skills, whereas global BM processing ability significantly improved with age. Critically, general BM perception (i.e. both local and global BM processing) may be affected by sustained attentional ability in children with ADHD. This relationship was primarily mediated by reasoning intelligence. These findings elucidate atypical BM perception in ADHD and the latent factors related to BM perception. Moreover, this study provides new evidence that BM perception is a hallmark of social cognition and advances our understanding of the potential roles of local and global processing in BM perception and social cognitive disorders.

Natural scenes reveal diverse representations of 2D and 3D body pose in the human brain

Human pose, defined as the spatial relationships between body parts, carries instrumental information supporting the understanding of motion and action of a person. A substantial body of previous work has identified cortical areas responsive to images of bodies and different body parts. However, the neural basis underlying the visual perception of body part relationships has received less attention. To broaden our understanding of body perception, we analyzed high-resolution fMRI responses to a wide range of poses from over 4,000 complex natural scenes. Using ground-truth annotations and an application of three-dimensional (3D) pose reconstruction algorithms, we compared similarity patterns of cortical activity with similarity patterns built from human pose models with different levels of depth availability and viewpoint dependency. Targeting the challenge of explaining variance in complex natural image responses with interpretable models, we achieved statistically significant correlations between pose models and cortical activity patterns (though performance levels are substantially lower than the noise ceiling). We found that the 3D view-independent pose model, compared with two-dimensional models, better captures the activation from distinct cortical areas, including the right posterior superior temporal sulcus (pSTS). These areas, together with other pose-selective regions in the LOTC, form a broader, distributed cortical network with greater view-tolerance in more anterior patches. We interpret these findings in light of the computational complexity of natural body images, the wide range of visual tasks supported by pose structures, and possible shared principles for view-invariant processing between articulated objects and ordinary, rigid objects.

Life motion signals bias the perception of apparent motion direction

Walking direction conveyed by biological motion (BM) cues, which humans are highly sensitive to since birth, can elicit involuntary shifts of attention to enhance the detection of static targets. Here, we demonstrated that such intrinsic sensitivity to walking direction could also modulate the direction perception of simultaneously presented dynamic stimuli. We showed that the perceived direction of apparent motion was biased towards the walking direction even though observers had been informed in advance that the walking direction of BM did not predict the apparent motion direction. In particular, rightward BM cues had an advantage over leftward BM cues in altering the perception of motion direction. Intriguingly, this perceptual bias disappeared when BM cues were shown inverted, or when the critical biological characteristics were removed from the cues. Critically, both the perceptual direction bias and the rightward advantage persisted even when only local BM cues were presented without any global configuration. Furthermore, the rightward advantage was found to be specific to social cues (i.e., BM), as it vanished when non-social cues (i.e., arrows) were utilized. Taken together, these findings support the existence of a specific processing mechanism for life motion signals and shed new light on their influences in a dynamic environment.

Eye pupil signals life motion perception

The ability to readily detect and recognize biological motion (BM) is fundamental to survival and interpersonal communication. However, perception of BM is strongly disrupted when it is shown upside down. This well-known inversion effect is proposed to be caused by a life motion detection mechanism highly tuned to gravity-compatible motion cues. In the current study, we assessed the inversion effect in BM perception using a no-report pupillometry. We found that the pupil size was significantly enlarged when observers viewed upright BMs (gravity-compatible) compared with the inverted counterparts (gravity-incompatible). Importantly, such an effect critically depended on the dynamic biological characteristics, and could be extended to local feet motion signals. These findings demonstrate that the eye pupil can signal gravity-dependent life motion perception. More importantly, with the convenience, objectivity, and noninvasiveness of pupillometry, the current study paves the way for the potential application of pupillary responses in detecting the deficiency of life motion perception in individuals with socio-cognitive disorders.

Audiovisual correspondence facilitates the visual search for biological motion

Hearing synchronous sounds may facilitate the visual search for the concurrently changed visual targets. Evidence for this audiovisual attentional facilitation effect mainly comes from studies using artificial stimuli with relatively simple temporal dynamics, indicating a stimulus-driven mechanism whereby synchronous audiovisual cues create a salient object to capture attention. Here, we investigated the crossmodal attentional facilitation effect on biological motion (BM), a natural, biologically significant stimulus with complex and unique dynamic profiles. We found that listening to temporally congruent sounds, compared with incongruent sounds, enhanced the visual search for BM targets. More intriguingly, such a facilitation effect requires the presence of distinctive local motion cues (especially the accelerations in feet movement) independent of the global BM configuration, suggesting a crossmodal mechanism triggered by specific biological features to enhance the salience of BM signals. These findings provide novel insights into how audiovisual integration boosts attention to biologically relevant motion stimuli and extend the function of a proposed life detection system driven by local kinematics of BM to multisensory life motion perception.

Distributed and hierarchical neural encoding of multidimensional biological motion attributes in the human brain

The human visual system can efficiently extract distinct physical, biological, and social attributes (e.g. facing direction, gender, and emotional state) from biological motion (BM), but how these attributes are encoded in the brain remains largely unknown. In the current study, we used functional magnetic resonance imaging to investigate this issue when participants viewed multidimensional BM stimuli. Using multiple regression representational similarity analysis, we identified distributed brain areas, respectively, related to the processing of facing direction, gender, and emotional state conveyed by BM. These brain areas are governed by a hierarchical structure in which the respective neural encoding of facing direction, gender, and emotional state is modulated by each other in descending order. We further revealed that a portion of the brain areas identified in representational similarity analysis was specific to the neural encoding of each attribute and correlated with the corresponding behavioral results. These findings unravel the brain networks for encoding BM attributes in consideration of their interactions, and highlight that the processing of multidimensional BM attributes is recurrently interactive.

A 5-emotions stimuli set for emotion perception research with full-body dance movements

Ekman famously contended that there are different channels of emotional expression (face, voice, body), and that emotion recognition ability confers an adaptive advantage to the individual. Yet, still today, much emotion perception research is focussed on emotion recognition from the face, and few validated emotionally expressive full-body stimuli sets are available. Based on research on emotional speech perception, we created a new, highly controlled full-body stimuli set. We used the same-sequence approach, and not emotional actions (e.g., jumping of joy, recoiling in fear): One professional dancer danced 30 sequences of (dance) movements five times each, expressing joy, anger, fear, sadness or a neutral state, one at each repetition. We outline the creation of a total of 150, 6-s-long such video stimuli, that show the dancer as a white silhouette on a black background. Ratings from 90 participants (emotion recognition, aesthetic judgment) showed that intended emotion was recognized above chance (chance: 20%; joy: 45%, anger: 48%, fear: 37%, sadness: 50%, neutral state: 51%), and that aesthetic judgment was sensitive to the intended emotion (beauty ratings: joy > anger > fear > neutral state, and sad > fear > neutral state). The stimuli set, normative values and code are available for download.

Cortical encoding of rhythmic kinematic structures in biological motion

Biological motion (BM) perception is of great survival value to human beings. The critical characteristics of BM information lie in kinematic cues containing rhythmic structures. However, how rhythmic kinematic structures of BM are dynamically represented in the brain and contribute to visual BM processing remains largely unknown. Here, we probed this issue in three experiments using electroencephalogram (EEG). We found that neural oscillations of observers entrained to the hierarchical kinematic structures of the BM sequences (i.e., step-cycle and gait-cycle for point-light walkers). Notably, only the cortical tracking of the higher-level rhythmic structure (i.e., gait-cycle) exhibited a BM processing specificity, manifested by enhanced neural responses to upright over inverted BM stimuli. This effect could be extended to different motion types and tasks, with its strength positively correlated with the perceptual sensitivity to BM stimuli at the right temporal brain region dedicated to visual BM processing. Modeling results further suggest that the neural encoding of spatiotemporally integrative kinematic cues, in particular the opponent motions of bilateral limbs, drives the selective cortical tracking of BM information. These findings underscore the existence of a cortical mechanism that encodes periodic kinematic features of body movements, which underlies the dynamic construction of visual BM perception.

Life Detection From Biological Motion

Life motion, the active movements of people and other animals, contains a wealth of information that is potentially accessible to the visual system of an observer. Biological-motion point-light displays have been widely used to study both the information contained in life motion stimuli and the visual mechanisms that make use of it. Biological motion conveys motion-mediated dynamic shape, which in turn can be used for identification and recognition of the agent, but it also contains local visual invariants that humans and other animals use as a general detection system that signals the presence of other agents in the visual environment. Here, we review recent research on behavioral, neurophysiological, and genetic aspects of this life-detection system and discuss its functional significance in the light of earlier hypotheses.

The McNorm library: creating and validating a new library of emotionally expressive whole body dance movements

The ability to exchange affective cues with others plays a key role in our ability to create and maintain meaningful social relationships. We express our emotions through a variety of socially salient cues, including facial expressions, the voice, and body movement. While significant advances have been made in our understanding of verbal and facial communication, to date, understanding of the role played by human body movement in our social interactions remains incomplete. To this end, here we describe the creation and validation of a new set of emotionally expressive whole-body dance movement stimuli, named the Motion Capture Norming (McNorm) Library, which was designed to reconcile a number of limitations associated with previous movement stimuli. This library comprises a series of point-light representations of a dancer's movements, which were performed to communicate to observers neutrality, happiness, sadness, anger, and fear. Based on results from two validation experiments, participants could reliably discriminate the intended emotion expressed in the clips in this stimulus set, with accuracy rates up to 60% (chance = 20%). We further explored the impact of dance experience and trait empathy on emotion recognition and found that neither significantly impacted emotion discrimination. As all materials for presenting and analysing this movement library are openly available, we hope this resource will aid other researchers in further exploration of affective communication expressed by human bodily movement.

Using EEG movement tagging to isolate brain responses coupled to biological movements

Detecting biological motion is essential for adaptive social behavior. Previous research has revealed the brain processes underlying this ability. However, brain activity during biological motion perception captures a multitude of processes. As a result, it is often unclear which processes reflect movement processing and which processes reflect secondary processes that build on movement processing. To address this issue, we developed a new approach to measure brain responses directly coupled to observed movements. Specifically, we showed 30 male and female adults a point-light walker moving at a pace of 2.4 Hz and used EEG frequency tagging to measure the brain response coupled to that pace ('movement tagging'). The results revealed a reliable response at the walking frequency that was reduced by two manipulations known to disrupt biological motion perception: phase scrambling and inversion. Interestingly, we also identified a brain response at half the walking frequency (i.e., 1.2 Hz), corresponding to the rate at which the individual dots completed a cycle. In contrast to the 2.4 Hz response, the response at 1.2 Hz was increased for scrambled (vs. unscrambled) walkers. These results show that frequency tagging can be used to capture the visual processing of biological movements and can dissociate between global (2.4 Hz) and local (1.2 Hz) processes involved in biological motion perception, at different frequencies of the brain signal.

Fetal blockade of nicotinic acetylcholine transmission causes autism-like impairment of biological motion preference in the neonatal chick

Several environmental chemicals are suspected risk factors for autism spectrum disorder (ASD), including valproic acid (VPA) and pesticides acting on nicotinic acetylcholine receptors (nAChRs), if administered during pregnancy. However, their target processes in fetal neuro-development are unknown. We report that the injection of VPA into the fetus impaired imprinting to an artificial object in neonatal chicks, while a predisposed preference for biological motion (BM) remained intact. Blockade of nAChRs acted oppositely, sparing imprinting and impairing BM preference. Beside ketamine and tubocurarine, significant effects of imidacloprid (a neonicotinoid insecticide) appeared at a dose ≤1 ppm. In accord with the behavioral dissociations, VPA enhanced histone acetylation in the primary cell culture of fetal telencephalon, whereas ketamine did not. VPA reduced the brain weight and the ratio of NeuN-positive cells (matured neurons) in the telencephalon of hatchlings, whereas ketamine/tubocurarine did not. Despite the distinct underlying mechanisms, both VPA and nAChR blockade similarly impaired imprinting to biological image composed of point-light animations. Furthermore, both impairments were abolished by postnatal bumetanide treatment, suggesting a common pathology underlying the social attachment malformation. Neurotransmission via nAChR is thus critical for the early social bond formation, which is hindered by ambient neonicotinoids through impaired visual predispositions for animate objects.

A two-stage framework for neural processing of biological motion

It remains to be understood how biological motion is hierarchically computed, from discrimination of local biological motion animacy to global dynamic body perception. Here, we addressed this functional separation of the correlates of the perception of local biological motion from perception of global motion of a body. We hypothesized that local biological motion processing can be isolated, by using a single dot motion perceptual decision paradigm featuring the biomechanical details of local realistic motion of a single joint. To ensure that we were indeed tackling processing of biological motion properties we used a discrimination instead of detection task. We discovered using representational similarity analysis that two key early dorsal and two ventral stream regions (visual motion selective hMT+ and V3A, extrastriate body area EBA and a region within fusiform gyrus FFG) showed robust and separable signals related to encoding of local biological motion and global motion-mediated shape. These signals reflected two independent processing stages, as revealed by representational similarity analysis and deconvolution of fMRI responses to each motion pattern. This study showed that higher level pSTS encodes both classes of biological motion in a similar way, revealing a higher-level integrative stage, reflecting scale independent biological motion perception. Our results reveal a two-stage framework for neural computation of biological motion, with an independent contribution of dorsal and ventral regions for the initial stage.

Decoding point-light displays and fully visible hand grasping actions within the action observation network

Action observation typically recruits visual areas and dorsal and ventral sectors of the parietal and premotor cortex. This network has been collectively termed as extended action observation network (eAON). Within this network, the elaboration of kinematic aspects of biological motion is crucial. Previous studies investigated these aspects by presenting subjects with point-light displays (PLDs) videos of whole-body movements, showing the recruitment of some of the eAON areas. However, studies focused on cortical activation during observation of PLDs grasping actions are lacking. In the present functional magnetic resonance imaging (fMRI) study, we assessed the activation of eAON in healthy participants during the observation of both PLDs and fully visible hand grasping actions, excluding confounding effects due to low-level visual features, motion, and context. Results showed that the observation of PLDs grasping stimuli elicited a bilateral activation of the eAON. Region of interest analyses performed on visual and sensorimotor areas showed no significant differences in signal intensity between PLDs and fully visible experimental conditions, indicating that both conditions evoked a similar motor resonance mechanism. Multivoxel pattern analysis (MVPA) revealed significant decoding of PLDs and fully visible grasping observation conditions in occipital, parietal, and premotor areas belonging to eAON. Data show that kinematic features conveyed by PLDs stimuli are sufficient to elicit a complete action representation, suggesting that these features can be disentangled within the eAON from the features usually characterizing fully visible actions. PLDs stimuli could be useful in assessing which areas are recruited, when only kinematic cues are available, for action recognition, imitation, and motor learning.

Optimizing EEG Source Reconstruction with Concurrent fMRI-Derived Spatial Priors

Reconstructing EEG sources involves a complex pipeline, with the inverse problem being the most challenging. Multiple inversion algorithms are being continuously developed, aiming to tackle the non-uniqueness of this problem, which has been shown to be partially circumvented by including prior information in the inverse models. Despite a few efforts, there are still current and persistent controversies regarding the inversion algorithm of choice and the optimal set of spatial priors to be included in the inversion models. The use of simultaneous EEG-fMRI data is one approach to tackle this problem. The spatial resolution of fMRI makes fMRI derived spatial priors very convenient for EEG reconstruction, however, only task activation maps and resting-state networks (RSNs) have been explored so far, overlooking the recent, but already accepted, notion that brain networks exhibit dynamic functional connectivity fluctuations. The lack of a systematic comparison between different source reconstruction algorithms, considering potentially more brain-informative priors such as fMRI, motivates the search for better reconstruction models. Using simultaneous EEG-fMRI data, here we compared four different inversion algorithms (minimum norm, MN; low resolution electromagnetic tomography, LORETA; empirical Bayes beamformer, EBB; and multiple sparse priors, MSP) under a Bayesian framework (as implemented in SPM), each with three different sets of priors consisting of: (1) those specific to the algorithm; (2) those specific to the algorithm plus fMRI task activation maps and RSNs; and (3) those specific to the algorithm plus fMRI task activation maps and RSNs and network modules of task-related dFC states estimated from the dFC fluctuations. The quality of the reconstructed EEG sources was quantified in terms of model-based metrics, namely the expectation of the posterior probability P(model|data) and variance explained of the inversion models, and the overlap/proportion of brain regions known to be involved in the visual perception tasks that the participants were submitted to, and RSN templates, with/within EEG source components. Model-based metrics suggested that model parsimony is preferred, with the combination MSP and priors specific to this algorithm exhibiting the best performance. However, optimal overlap/proportion values were found using EBB and priors specific to this algorithm and fMRI task activation maps and RSNs or MSP and considering all the priors (algorithm priors, fMRI task activation maps and RSNs and dFC state modules), respectively, indicating that fMRI spatial priors, including dFC state modules, might contain useful information to recover EEG source components reflecting neuronal activity of interest. Our main results show that providing fMRI spatial derived priors that reflect the dynamics of the brain might be useful to map neuronal activity more accurately from EEG-fMRI. Furthermore, this work paves the way towards a more informative selection of the optimal EEG source reconstruction approach, which may be critical in future studies.

Quantitative Assessment of the Impact of Geometric Distortions and Their Correction on fMRI Data Analyses

Functional magnetic resonance imaging (fMRI) data is typically collected with gradient-echo echo-planar imaging (GE-EPI) sequences, which are particularly prone to the susceptibility artifact as a result of B₀ field inhomogeneity. The component derived from in-plane spin dephasing induces pixel intensity variations and, more critically, geometric distortions. Despite the physical mechanisms underlying the susceptibility artifact being well established, a systematic investigation on the impact of the associated geometric distortions, and the direct comparison of different approaches to tackle them, on fMRI data analyses is missing. Here, we compared two different distortion correction approaches, by acquiring additional: (1) EPI data with reversed phase encoding direction (TOPUP), and (2) standard (and undistorted) GE data at two different echo times (GRE). We first characterized the geometric distortions and the correction approaches based on the estimated ΔB₀ field offset and voxel shift maps, and then conducted three types of analyses on the distorted and corrected fMRI data: (1) registration into structural data, (2) identification of resting-state networks (RSNs), and (3) mapping of task-related brain regions of interest. GRE estimated the largest voxel shifts and more positively impacted the quality of the analyses, in terms of the (significantly lower) cost function of the registration, the (higher) spatial overlap between the RSNs and appropriate templates, and the (significantly higher) sensitivity of the task-related mapping based on the Z-score values of the associated activation maps, although also evident when considering TOPUP. fMRI data should thus be corrected for geometric distortions, with the choice of the approach having a modest, albeit positive, impact on the fMRI analyses.

Spatiotemporal dynamics of responses to biological motion in the human brain

We sought to understand the spatiotemporal characteristics of biological motion perception. We presented observers with biological motion walkers that differed in terms of form coherence or kinematics (i.e., the presence or absence of natural acceleration). Participants were asked to discriminate the facing direction of the stimuli while their magnetoencephalographic responses were concurrently imaged. We found that two univariate response components can be observed around ~200 msec and ~650 msec post-stimulus onset, each engaging lateral-occipital and parietal cortex prior to temporal and frontal cortex. Moreover, while univariate responses show biological motion form-specificity only after 300 msec, multivariate patterns specific to form can be well discriminated from those for local cues as early as 100 msec after stimulus onset. By finally examining the representational similarity of fMRI and MEG patterned responses, we show that early responses to biological motion are most likely sourced to occipital cortex while later responses likely originate from extrastriate body areas.

Dynamic dot displays reveal material motion network in the human brain

There is growing research interest in the neural mechanisms underlying the recognition of material categories and properties. This research field, however, is relatively more recent and limited compared to investigations of the neural mechanisms underlying object and scene category recognition. Motion is particularly important for the perception of non-rigid materials, but the neural basis of non-rigid material motion remains unexplored. Using fMRI, we investigated which brain regions respond preferentially to material motion versus other types of motion. We introduce a new database of stimuli - dynamic dot materials - that are animations of moving dots that induce vivid percepts of various materials in motion, e.g. flapping cloth, liquid waves, wobbling jelly. Control stimuli were scrambled versions of these same animations and rigid three-dimensional rotating dots. Results showed that isolating material motion properties with dynamic dots (in contrast with other kinds of motion) activates a network of cortical regions in both ventral and dorsal visual pathways, including areas normally associated with the processing of surface properties and shape, and extending to somatosensory and premotor cortices. We suggest that such a widespread preference for material motion is due to strong associations between stimulus properties. For example viewing dots moving in a specific pattern not only elicits percepts of material motion; one perceives a flexible, non-rigid shape, identifies the object as a cloth flapping in the wind, infers the object's weight under gravity, and anticipates how it would feel to reach out and touch the material. These results are a first important step in mapping out the cortical architecture and dynamics in material-related motion processing.

Early Prediction of Cognitive Deficit in Very Preterm Infants Using Brain Structural Connectome With Transfer Learning Enhanced Deep Convolutional Neural Networks

Up to 40% of very preterm infants (≤32 weeks’ gestational age) were identified with a cognitive deficit at 2 years of age. Yet, accurate clinical diagnosis of cognitive deficit cannot be made until early childhood around 3–5 years of age. Recently, brain structural connectome that was constructed by advanced diffusion tensor imaging (DTI) technique has been playing an important role in understanding human cognitive functions. However, available annotated neuroimaging datasets with clinical and outcome information are usually limited and expensive to enlarge in the very preterm infants’ studies. These challenges hinder the development of neonatal prognostic tools for early prediction of cognitive deficit in very preterm infants. In this study, we considered the brain structural connectome as a 2D image and applied established deep convolutional neural networks to learn the spatial and topological information of the brain connectome. Furthermore, the transfer learning technique was utilized to mitigate the issue of insufficient training data. As such, we developed a transfer learning enhanced convolutional neural network (TL-CNN) model for early prediction of cognitive assessment at 2 years of age in very preterm infants using brain structural connectome. A total of 110 very preterm infants were enrolled in this work. Brain structural connectome was constructed using DTI images scanned at term-equivalent age. Bayley III cognitive assessments were conducted at 2 years of corrected age. We applied the proposed model to both cognitive deficit classification and continuous cognitive score prediction tasks. The results demonstrated that TL-CNN achieved improved performance compared to multiple peer models. Finally, we identified the brain regions most discriminative to the cognitive deficit. The results suggest that deep learning models may facilitate early prediction of later neurodevelopmental outcomes in very preterm infants at term-equivalent age.

The two-process theory of biological motion processing

Perception, identification, and understanding of others' actions from motion information are vital for our survival in the social world. A breakthrough in the understanding of action perception was the discovery that our visual system is sensitive to human action from the sparse motion input of only a dozen point lights, a phenomenon known as biological motion (BM) processing. Previous psychological and computational models cannot fully explain the emerging evidence for the existence of BM processing during early ontogeny. Here, we propose a two-process model of the mechanisms underlying BM processing. We hypothesize that the first system, the 'Step Detector,' rapidly processes the local foot motion and feet-below-the-body information that is specific to vertebrates, is less dependent on postnatal learning, and involves subcortical networks. The second system, the 'Bodily Action Evaluator,' slowly processes the fine global structure-from-motion, is specific to conspecific, and dependent on gradual learning processed in cortical networks. This proposed model provides new insight into research on the development of BM processing.

Biological motion perception is differentially predicted by Autistic trait domains

We tested the relationship between biological motion perception and the Autism-Spectrum Quotient. In three experiments, we indexed observers' performance on a classic left-right discrimination task in which participants were asked to report the facing direction of walkers containing solely structural or kinematics information, a motion discrimination task in which participants were asked to indicate the apparent motion of a (non-biological) random-dot stimulus, and a novel naturalness discrimination task. In the naturalness discrimination task, we systematically manipulated the degree of natural acceleration contained in the stimulus by parametrically morphing between a fully veridical stimulus and one where acceleration was removed. Participants were asked to discriminate the more natural stimulus (i.e., acceleration-containing stimulus) from the constant velocity stimulus. Although we found no reliable associations between overall AQ scores nor subdomain scores with performance on the direction-related tasks, we found a robust association between performance on the biological motion naturalness task and attention switching domain scores. Our findings suggest that understanding the relationship between the Autism Spectrum and perception is a far more intricate problem than previously suggested. While it has been shown that the AQ can be used as a proxy to tap into perceptual endophenotypes in Autism, the eventual diagnostic value of the perceptual task depends on the task's consideration of biological content and demands.

MinE-RFE: determine the optimal subset from RFE by minimizing the subset-accuracy–defined energy

Recursive feature elimination (RFE), as one of the most popular feature selection algorithms, has been extensively applied to bioinformatics. During the training, a group of candidate subsets are generated by iteratively eliminating the least important features from the original features. However, how to determine the optimal subset from them still remains ambiguous. Among most current studies, either overall accuracy or subset size (SS) is used to select the most predictive features. Using which one or both and how they affect the prediction performance are still open questions. In this study, we proposed MinE-RFE, a novel RFE-based feature selection approach by sufficiently considering the effect of both factors. Subset decision problem was reflected into subset-accuracy space and became an energy-minimization problem. We also provided a mathematical description of the relationship between the overall accuracy and SS using Gaussian Mixture Models together with spline fitting. Besides, we comprehensively reviewed a variety of state-of-the-art applications in bioinformatics using RFE. We compared their approaches of deciding the final subset from all the candidate subsets with MinE-RFE on diverse bioinformatics data sets. Additionally, we also compared MinE-RFE with some well-used feature selection algorithms. The comparative results demonstrate that the proposed approach exhibits the best performance among all the approaches. To facilitate the use of MinE-RFE, we further established a user-friendly web server with the implementation of the proposed approach, which is accessible at http://qgking.wicp.net/MinE/. We expect this web server will be a useful tool for research community.

The connectivity signature of co-speech gesture integration: The superior temporal sulcus modulates connectivity between areas related to visual gesture and auditory speech processing

Humans integrate information communicated by speech and gestures. Functional magnetic resonance imaging (fMRI) studies suggest that the posterior superior temporal sulcus (STS) and adjacent gyri are relevant for multisensory integration. However, a connectivity model representing this essential combinatory process is still missing. Here, we used dynamic causal modeling for fMRI to analyze the effective connectivity pattern between middle temporal gyrus (MTG), occipital cortex (OC) and STS associated with auditory verbal, visual gesture-related, and integrative processing, respectively, to unveil the neural mechanisms underlying integration of intrinsically meaningful gestures (e.g., "Thumbs-up gesture") and corresponding speech. 20 participants were presented videos of an actor either performing intrinsic meaningful gestures in the context of German or Russian sentences, or speaking a German sentence without gesture, while performing a content judgment task. The connectivity analyses resulted in a winning model that included bidirectional intrinsic connectivity between all areas. Furthermore, the model included modulations of both connections to the STS (OC→STS; MTG→STS), and non-linear modulatory effects of the STS on bidirectional connections between MTG and OC. Coupling strength in the occipital pathway (OC→STS) correlated with gesture related advantages in task performance, whereas the temporal pathway (MTG→STS) correlated with performance in the speech only condition. Coupling between MTG and OC correlated negatively with subsequent memory performance for sentences of the Gesture-German condition. Our model provides a first step towards a better understanding of speech-gesture integration on network level. It corroborates the importance of the STS during audio-visual integration by showing that this region inhibits direct auditory-visual coupling.

Attraction of posture and motion-trajectory elements of conspecific biological motion in medaka fish

Visual recognition of conspecifics is necessary for a wide range of social behaviours in many animals. Medaka (Japanese rice fish), a commonly used model organism, are known to be attracted by the biological motion of conspecifics. However, biological motion is a composite of both body-shape motion and entire-field motion trajectory (i.e., posture or motion-trajectory elements, respectively), and it has not been revealed which element mediates the attractiveness. Here, we show that either posture or motion-trajectory elements alone can attract medaka. We decomposed biological motion of the medaka into the two elements and synthesized visual stimuli that contain both, either, or none of the two elements. We found that medaka were attracted by visual stimuli that contain at least one of the two elements. In the context of other known static visual information regarding the medaka, the potential multiplicity of information regarding conspecific recognition has further accumulated. Our strategy of decomposing biological motion into these partial elements is applicable to other animals, and further studies using this technique will enhance the basic understanding of visual recognition of conspecifics.