Multisensory components of rapid motor responses to fingertip loading

Linking tonic and phasic pupil responses to P300 amplitude in an emotional face-word Stroop task

The locus coeruleus-norepinephrine (LC-NE) system, which regulates arousal levels, is important for cognitive control, including emotional conflict resolution. Additionally, the LC-NE system is implicated in P300 generation. If the P300 is mediated by the LC-NE system, and considering the established correlations between LC activity and pupil dilation, P300 amplitude should correlate with task-evoked (phasic) pupil dilation on a trial-by-trial basis. However, prior studies, predominantly utilizing oddball-type paradigms, have not demonstrated correlations between concurrently recorded task-evoked pupil dilation and P300 responses. Using a recently developed emotional face-word Stroop task that links pupil dilation to the LC-NE system, here, we examined both intra- and inter-individual correlations between task-evoked pupil dilation and P300 amplitude. We found that lower accuracy, slower reaction times, and larger task-evoked pupil dilation were obtained in the incongruent compared to the congruent condition. Furthermore, we observed intra-individual correlations between task-evoked pupil dilation and P300 amplitude, with larger pupil dilation correlating with a greater P300 amplitude. In contrast, pupil dilation did not exhibit consistent correlations with N450 and N170 amplitudes. Baseline (tonic) pupil size also showed correlations with P300 and N170 amplitudes, with smaller pupil size corresponding to larger amplitude. Moreover, inter-individual differences in task-evoked pupil dilation between the congruent and incongruent conditions correlated with differences in reaction time and P300 amplitude, though these effects only approached significance. To summarize, our study provides evidence for a connection between task-evoked pupil dilation and P300 amplitude at the single-trial level, suggesting the involvement of the LC-NE system in P300 generation.

Fast grip force adaptation to friction relies on localized fingerpad strains

During object manipulation, humans adjust the grip force to friction, such that slippery objects are squeezed more firmly than sticky ones. This essential mechanism to keep a stable grasp relies on feedback from tactile afferents innervating the fingertips, that are sensitive to local skin strains. To test if this feedback originates from the skin-object interface, we asked participants to perform a grip-lift task with an instrumented object able to monitor skin strains at the contact through transparent plates of different frictions. We observed that, following an unbeknown change in plate across trials, participants adapted their grip force to friction. After switching from high to low friction, we found a significant increase in strain inside the contact arising ~100 ms before the modulation of grip force, suggesting that differences in strain patterns before lift-off are used by the nervous system to quickly adjust the force to the frictional properties of manipulated objects.

Open-Source Instrumented Object to Study Dexterous Object Manipulation

Humans use tactile feedback to perform skillful manipulation. When tactile sensory feedback is unavailable, for instance, if the fingers are anesthetized, dexterity is severely impaired. Imaging the deformation of the finger pad skin when in contact with a transparent plate provides information about the tactile feedback received by the central nervous system. Indeed, skin deformations are transduced into neural signals by the mechanoreceptors of the finger pad skin. Understanding how this feedback is used for active object manipulation would improve our understanding of human dexterity. In this paper, we present a new device for imaging the skin of the finger pad of one finger during manipulation performed with a precision grip. The device's mass (300 g) makes it easy to use during unconstrained dexterous manipulation. Using this device, we reproduced the experiment performed in Delhaye et al. (2021) We extracted the strains aligned with the object's movement, i.e., the vertical strains in the ulnar and radial parts of the fingerpad, to see how correlated they were with the grip force (GF) adaptation. Interestingly, parts of our results differed from those in Delhaye et al. (2021) due to weight and inertia differences between the devices, with average GF across participants differing significantly. Our results highlight a large variability in the behavior of the skin across participants, with generally low correlations between strain and GF adjustments, suggesting that skin deformations are not the primary driver of GF adaptation in this manipulation scenario.

Time-on-task effects on human pupillary and saccadic metrics after theta burst transcranial magnetic stimulation over the frontal eye field

Pupil size undergoes constant changes primarily influenced by ambient luminance. These changes are referred to as the pupillary light reflex (PLR), where the pupil transiently constricts in response to light. PLR kinematics provides valuable insights into autonomic nervous system function and have significant clinical applications. Recent research indicates that attention plays a role in modulating the PLR, and the circuit involving the frontal eye field (FEF) and superior colliculus is causally involved in controlling this pupillary modulation. However, there is limited research exploring the role of the human FEF in these pupillary responses, and its impact on PLR metrics remains unexplored. Additionally, although the protocol of continuous theta-burst stimulation (cTBS) is well-established, the period of disruption after cTBS is yet to be examined in pupillary responses. Our study aimed to investigate the effects of FEF cTBS on pupillary and saccadic metrics in relation to time spent performing a task (referred to as time-on-task). We presented a bright stimulus to induce the PLR in visual- and memory-delay saccade tasks following cTBS over the right FEF or vertex. FEF cTBS, compared to vertex cTBS, resulted in decreased baseline pupil size, peak constriction velocities, and amplitude. Furthermore, the time-on-task effects on baseline pupil size, peak amplitude, and peak time differed between the two stimulation conditions. In contrast, the time-on-task effects on saccadic metrics were less pronounced between the two conditions. In summary, our study provides the first evidence that FEF cTBS affects human PLR metrics and that these effects are modulated by time-on-task.

Human reaching control in dynamic environments

Closed-loop models of movement control have attracted growing interest in how the nervous system transforms sensory information into motor commands, and several brain structures have been identified as neural substrates for these computational operations. Recently, several studies have focused on how these models need to be updated when environmental parameters change. Current evidence suggests that when the task changes, rapid control updates enable flexible modifications of current actions and online decisions. At the same time, when movement dynamics change, humans use different strategies based on a combination of adaptation and modulation of controller sensitivity to exogenous perturbations (robust control). This review proposes a unified framework to capture these results based on online estimation of model parameters with dynamic updates in control. The reviewed studies also identify the time scales of associated behavioral mechanisms to guide future research on the neural basis of movement control.

Continuous evaluation of cost-to-go for flexible reaching control and online decisions

Humans consider the parameters linked to movement goal during reaching to adjust their control strategy online. Indeed, rapid changes in target structure or disturbances interfering with their initial plan elicit rapid changes in behavior. Here, we hypothesize that these changes could result from the continuous use of a decision variable combining motor and cognitive components. We combine an optimal feedback controller with a real-time evaluation of the expected cost-to-go, which considers target- and movement-related costs, in a common theoretical framework. This model reproduces human behaviors in presence of changes in the target structure occurring during movement and of online decisions to flexibly change target following external perturbations. It also predicts that the time taken to decide to select a novel goal after a perturbation depends on the amplitude of the disturbance and on the rewards of the different options, which is a direct result of the continuous monitoring of the cost-to-go. We show that this result was present in our previously collected dataset. Together our developments point towards a continuous evaluation of the cost-to-go during reaching to update control online and make efficient decisions about movement goal.

Efficient tactile encoding of object slippage

When grasping objects, we rely on our sense of touch to adjust our grip and react against external perturbations. Less than 200 ms after an unexpected event, the sensorimotor system is able to process tactile information to deduce the frictional strength of the contact and to react accordingly. Given that roughly 1,300 afferents innervate the fingertips, it is unclear how the nervous system can process such a large influx of data in a sufficiently short time span. In this study, we measured the deformation of the skin during the initial stages of incipient sliding for a wide range of frictional conditions. We show that the dominant patterns of deformation are sufficient to estimate the distance between the frictional force and the frictional strength of the contact. From these stereotypical patterns, a classifier can predict if an object is about to slide during the initial stages of incipient slip. The prediction is robust to the actual value of the interfacial friction, showing sensory invariance. These results suggest the existence of a possible compact set of bases that we call Eigenstrains. These Eigenstrains are a potential mechanism to rapidly decode the margin from full slip from the tactile information contained in the deformation of the skin. Our findings suggest that only 6 of these Eigenstrains are necessary to classify whether the object is firmly stuck to the fingers or is close to slipping away. These findings give clues about the tactile regulation of grasp and the insights are directly applicable to the design of robotic grippers and prosthetics that rapidly react to external perturbations.

Mapping the Integration of Sensory Information across Fingers in Human Sensorimotor Cortex

The integration of somatosensory signals across fingers is essential for dexterous object manipulation. Previous experiments suggest that this integration occurs in neural populations in the primary somatosensory cortex (S1). However, the integration process has not been fully characterized, as previous studies have mainly used 2-finger stimulation paradigms. Here, we addressed this gap by stimulating all 31 single- and multifinger combinations. We measured population-wide activity patterns evoked during finger stimulation in human S1 and primary motor cortex (M1) using 7T fMRI in female and male participants. Using multivariate fMRI analyses, we found clear evidence of unique nonlinear interactions between fingers. In Brodmann area (BA) 3b, interactions predominantly occurred between pairs of neighboring fingers. In BA 2, however, we found equally strong interactions between spatially distant fingers, as well as interactions between finger triplets and quadruplets. We additionally observed strong interactions in the hand area of M1. In both M1 and S1, these nonlinear interactions did not reflect a general suppression of overall activity, suggesting instead that the interactions we observed reflect rich, nonlinear integration of sensory inputs from the fingers. We suggest that this nonlinear finger integration allows for a highly flexible mapping from finger sensory inputs to motor responses that facilitates dexterous object manipulation.SIGNIFICANCE STATEMENT Processing of somatosensory information in primary somatosensory cortex (S1) is essential for dexterous object manipulation. To successfully handle an object, the sensorimotor system needs to detect complex patterns of haptic information, which requires the nonlinear integration of sensory inputs across multiple fingers. Using multivariate fMRI analyses, we characterized brain activity patterns evoked by stimulating all single- and multifinger combinations. We report that progressively stronger multifinger interactions emerge in posterior S1 and in the primary motor cortex (M1), with interactions arising between inputs from neighboring and spatially distant fingers. Our results suggest that S1 and M1 provide the neural substrate necessary to support a flexible mapping from sensory inputs to motor responses of the hand.

A review of the neurobiomechanical processes underlying secure gripping in object manipulation

O'SHEA, H. and S. J. Redmond. A review of the neurobiomechanical processes underlying secure gripping in object manipulation. NEUROSCI BIOBEHAV REV 286-300, 2021. Humans display skilful control over the objects they manipulate, so much so that biomimetic systems have yet to emulate this remarkable behaviour. Two key control processes are assumed to facilitate such dexterity: predictive cognitive-motor processes that guide manipulation procedures by anticipating action outcomes; and reactive sensorimotor processes that provide important error-based information for movement adaptation. Notwithstanding increased interdisciplinary research interest in object manipulation behaviour, the complexity of the perceptual-sensorimotor-cognitive processes involved and the theoretical divide regarding the fundamentality of control mean that the essential mechanisms underlying manipulative action remain undetermined. In this paper, following a detailed discussion of the theoretical and empirical bases for understanding human dexterous movement, we emphasise the role of tactile-related sensory events in secure object handling, and consider the contribution of certain biophysical and biomechanical phenomena. We aim to provide an integrated account of the current state-of-art in skilled human-object interaction that bridges the literature in neuroscience, cognitive psychology, and biophysics. We also propose novel directions for future research exploration in this area.

Effects of pupillary light and darkness reflex on the generation of pro- And anti-saccades

Saccades are often directed toward a stimulus that provides useful information for observers to navigate the visual world. The quality of visual signals of a stimulus is influenced by global luminance, and the pupil constricts or dilates after a luminance increase or decrease, respectively, to optimize visual signals for further information processing. Although luminance level changes regularly in the real environment, saccades are mostly studied in the luminance-unchanged setup. Whether pupillary responses triggered by global luminance changes modulate saccadic behavior are yet to be explored. Through varying background luminance level in an interleaved pro- and anti-saccade paradigm, we investigated the modulation of pupillary luminance responses on the generation of reflexive and voluntary saccades. Subjects were instructed to either automatically look at the peripheral stimulus (pro-saccade) or to suppress the automatic response and voluntarily look in the opposite direction from the stimulus (anti-saccade). Level of background luminance was increased (light), decreased (dark), or unchanged (control) during the instructed fixation period. Saccade reaction time distributions of correct pro- and anti-saccades in the light and dark conditions were differed significantly from those in the control condition. Moreover, the luminance condition modulated saccade kinematics, showing reduced performances in the light condition than in the control condition, particularly in pro-saccades. Modeling results further suggested that both pupil diameter and pupil size derivative significantly modulated saccade behavior, though effect sizes were small and mainly mediated by intersubject differences. Together, our results demonstrated the influence of pupillary luminance responses on the generation of pro- and anti-saccades.

Effects of aging on rapid grip force responses during bimanual manipulation of an active object

Rapid grip force responses to unexpected pulling loads on the fingertips are deteriorated in older adults due to, in part, age-related declines in somatosensory function. Such reports are limited to one-hand conditions despite the higher frequency of using two hands together in daily living activities of older adults. Unexpected perturbations during bimanual movements elicit goal-oriented and cortically-meditated bilateral rapid motor responses. Since aging is associated with declined somatosensory and cognitive functions, we hypothesized that bilateral rapid motor responses differ between young and older adults, such that older adults exert stronger grip forces following perturbation and the unperturbed hand is more involved in stabilizing the object in older adults. We tested our hypothesis by comparing the rapid grip force responses of both hands in young and older adults. A total of 13 right-handed young individuals (24.2 ± 4.0 years old, 5 men) and 13 right-handed older individuals (68.7 ± 7.1 years old, 5 men) were recruited. Tactile detection threshold, fingertip friction, and the rapid grip force responses of both hands triggered by unpredicted pulling loads during grip-lift movements were assessed. Older adults had higher tactile detection thresholds and lower fingertip friction compared to young adults. Regardless of age, rapid motor responses were found in both the perturbed (right) hand and the indirectly perturbed (left) hand at 73 ms and 135 ms after the perturbation, respectively, while magnitudes of the responses depended on perturbation magnitudes. Higher values in maximum grip force and maximum grip force rate were found in older adults as compared to young adults. In older adults, the indirectly perturbed (left) hand was more involved in stabilizing the object as compared to young healthy adults. The current study suggests that age-related changes in the peripheral and central nervous systems contribute to the greater involvement of the indirectly perturbed hand in older adults.

Voluntary modification of rapid tactile-motor responses during reaching differs from its visuomotor counterpart

People commonly hold and manipulate a variety of objects in everyday life, and these objects have different physical properties. To successfully control this wide range of objects, people must associate new patterns of tactile stimuli with appropriate motor outputs. We performed a series of experiments investigating the extent to which people can voluntarily modify tactile-motor associations in the context of a rapid tactile-motor response guiding the hand to a moving target (previously described in Pruszynski JA, Johansson RS, Flanagan JR. Curr Biol 26: 788-792, 2016) by using an anti-reach paradigm in which participants were instructed to move their hands in the opposite direction of a target jump. We compared performance to that observed when people make visually guided reaches to a moving target (cf. Day BL, Lyon IN. Exp Brain Res 130: 159-168, 2000; Pisella L, Grea H, Tilikete C, Vighetto A, Desmurget M, Rode G, Boisson D, Rossetti Y. Nat Neurosci 3: 729-736, 2000). When participants had visual feedback, motor responses during the anti-reach task showed early automatic responses toward the moving target before later modification to move in the instructed direction. When the same participants had only tactile feedback, however, they were able to suppress this early phase of the motor response, which occurs <100 ms after the target jump. Our results indicate that while the tactile motor and visual motor systems both support rapid responses that appear similar under some conditions, the circuits underlying responses show sharp distinctions in terms of their malleability.NEW & NOTEWORTHY When people reach toward a visual target that moves suddenly, they automatically correct their reach to follow the object; even when explicitly instructed not to follow a moving visual target, people exhibit an initial incorrect movement before moving in the correct direction. We show that when people use tactile feedback, they do not show an initial incorrect response, even though early muscle activity still occurs.

Feedback Adaptation to Unpredictable Force Fields in 250 ms

Motor learning and adaptation are important functions of the nervous system. Classical studies have characterized how humans adapt to changes in the environment during tasks such as reaching, and have documented improvements in behavior across movements. However, little is known about how quickly the nervous system adapts to such disturbances. In particular, recent work has suggested that adaptation could be sufficiently fast to alter the control strategies of an ongoing movement. To further address the possibility that learning occurred within a single movement, we designed a series of human reaching experiments to extract from muscles recordings the latency of feedback adaptation. Our results confirmed that participants adapted their feedback responses to unanticipated force fields applied randomly. In addition, our analyses revealed that the feedback response was specifically and finely tuned to the ongoing perturbation not only across trials with the same force field, but also across different kinds of force fields. Finally, changes in muscle activity consistent with feedback adaptation occurred in ∼250 ms following reach onset. The adaptation that we observed across trials presented in a random context was similar to the one observed when the force fields could be anticipated, suggesting that these two adaptive processes may be closely linked to each other. In such case, our measurement of 250 ms may correspond to the latency of motor adaptation in the nervous system.

Stretching the skin immediately enhances perceived stiffness and gradually enhances the predictive control of grip force

When manipulating objects, we use kinesthetic and tactile information to form an internal representation of their mechanical properties for cognitive perception and for preventing their slippage using predictive control of grip force. A major challenge in understanding the dissociable contributions of tactile and kinesthetic information to perception and action is the natural coupling between them. Unlike previous studies that addressed this question either by focusing on impaired sensory processing in patients or using local anesthesia, we used a behavioral study with a programmable mechatronic device that stretches the skin of the fingertips to address this issue in the intact sensorimotor system. We found that artificial skin-stretch increases the predictive grip force modulation in anticipation of the load force. Moreover, the stretch causes an immediate illusion of touching a harder object that does not depend on the gradual development of the predictive modulation of grip force.

Sensory information from a slipping object elicits a rapid and automatic shoulder response

Humans have the remarkable ability to hold, grasp, and manipulate objects. Previous work has reported rapid and coordinated reactions in hand and shoulder muscles in response to external perturbations to the arm during object manipulation; however, little is known about how somatosensory feedback of an object slipping in the hand influences responses of the arm. We built a handheld device to stimulate the sensation of slipping at all five fingertips. The device was integrated into an exoskeleton robot that supported it against gravity. The setup allowed us to decouple somatosensory stimulation in the fingers from forces applied to the arm, two variables that are highly interdependent in real-world scenarios. Fourteen participants performed three experiments in which we measured their arm feedback responses during slip stimulation. Slip stimulations were applied horizontally in one of two directions, and participants were instructed to either follow the slip direction or move the arm in the opposite direction. Participants showed shoulder muscle responses within ∼67 ms of slip onset when following the direction of slip but significantly slower responses when instructed to move in the opposite direction. Shoulder responses were modulated by the speed but not the distance of the slip. Finally, when slip stimulation was combined with mechanical perturbations to the arm, we found that sensory information from the fingertips significantly modulated the shoulder feedback responses. Overall, the results demonstrate the existence of a rapid feedback system that stabilizes handheld objects.NEW & NOTEWORTHY We tested whether the sensation of an object slipping from the fingers modulates shoulder feedback responses. We found rapid shoulder feedback responses when participants were instructed to follow the slip direction with the arm. Shoulder responses following mechanical joint perturbations were also potentiated when combined with slipping. These results demonstrate the existence of fast and automatic feedback responses in the arm in reaction to sensory input to the fingertips that maintain grip on handheld objects.

Reaching trajectories unravel modality-dependent temporal dynamics of the automatic process in the Simon task: a model-based approach

The Simon effect represents a phenomenon in which the location of the stimuli affects the speed and accuracy of the response, despite being irrelevant for the task demands. This is believed to be due to an automatic activation of a response corresponding to the location of the stimuli, which conflicts with the controlled decision process based on relevant stimuli features. Previously, differences in the nature of the Simon effect (i.e., the pattern of change of the effect across the distribution of response times) between visual and somatosensory stimuli were reported. We hypothesize that the temporal dynamics of visual and somatosensory automatic and controlled processes vary, thus driving the reported behavioral differences. While most studies have used response times to study the underlying mechanisms involved, in this study we had participants reach out to touch the targets and recorded their arm movements using a motion capture system. Importantly, the participants started their movements before a final decision was made. In this way, we could analyze the movements to gain insights into the competition between the automatic and controlled processes. We used this technique to describe the results in terms of a model assuming automatic activation due to location-based evidence, followed by inhibition. We found that for the somatosensory Simon effect, the decay of the automatic process is significantly slower than for the visual Simon effect, suggesting quantitative differences in this automatic process between the visual and somatosensory modalities.

State Estimation for Early Feedback Responses in Reaching: Intramodal or Multimodal?

Humans are highly skilled in controlling their reaching movements, making fast and task-dependent movement corrections to unforeseen perturbations. To guide these corrections, the neural control system requires a continuous, instantaneous estimate of the current state of the arm and body in the world. According to Optimal Feedback Control theory, this estimate is multimodal and constructed based on the integration of forward motor predictions and sensory feedback, such as proprioceptive, visual and vestibular information, modulated by context, and shaped by past experience. But how can a multimodal estimate drive fast movement corrections, given that the involved sensory modalities have different processing delays, different coordinate representations, and different noise levels? We develop the hypothesis that the earliest online movement corrections are based on multiple single modality state estimates rather than one combined multimodal estimate. We review studies that have investigated online multimodal integration for reach control and offer suggestions for experiments to test for the existence of intramodal state estimates. If proven true, the framework of Optimal Feedback Control needs to be extended with a stage of intramodal state estimation, serving to drive short-latency movement corrections.