Vision of the hand and environmental context in human prehension

Robot Arm Reaching Based on Inner Rehearsal

Robot arm motion control is a fundamental aspect of robot capabilities, with arm reaching ability serving as the foundation for complex arm manipulation tasks. However, traditional inverse kinematics-based methods for robot arm reaching struggle to cope with the increasing complexity and diversity of robot environments, as they heavily rely on the accuracy of physical models. In this paper, we introduce an innovative approach to robot arm motion control, inspired by the cognitive mechanism of inner rehearsal observed in humans. The core concept revolves around the robot's ability to predict or evaluate the outcomes of motion commands before execution. This approach enhances the learning efficiency of models and reduces the mechanical wear on robots caused by excessive physical executions. We conduct experiments using the Baxter robot in simulation and the humanoid robot PKU-HR6.0 II in a real environment to demonstrate the effectiveness and efficiency of our proposed approach for robot arm reaching across different platforms. The internal models converge quickly and the average error distance between the target and the end-effector on the two platforms is reduced by 80% and 38%, respectively.

Visual uncertainty unveils the distinct role of haptic cues in multisensory grasping

Human multisensory grasping movements (i.e., seeing and feeling a handheld object while grasping it with the contralateral hand) are superior to movements guided by each separate modality. This multisensory advantage might be driven by the integration of vision with either the haptic position only or with both position and size cues. To contrast these two hypotheses, we manipulated visual uncertainty (central vs. peripheral vision) and the availability of haptic cues during multisensory grasping. We showed a multisensory benefit irrespective of the degree of visual uncertainty suggesting that the integration process involved in multisensory grasping can be flexibly modulated by the contribution of each modality. Increasing visual uncertainty revealed the role of the distinct haptic cues. The haptic position cue was sufficient to promote multisensory benefits evidenced by faster actions with smaller grip apertures, whereas the haptic size was fundamental in fine-tuning the grip aperture scaling. These results support the hypothesis that, in multisensory grasping, vision is integrated with all haptic cues, with the haptic position cue playing the key part. Our findings highlight the important role of non-visual sensory inputs in sensorimotor control and hint at the potential contributions of the haptic modality in developing and maintaining visuomotor functions.Significance statementThe longstanding view that vision is considered the primary sense we rely on to guide grasping movements relegates the equally important haptic inputs, such as touch and proprioception, to a secondary role. Here we show that by increasing visual uncertainty during visuo-haptic grasping, the central nervous system exploits distinct haptic inputs about the object position and size to optimize grasping performance. Specifically, we demonstrate that haptic inputs about the object position are fundamental to support vision in enhancing grasping performance, whereas haptic size inputs can further refine hand shaping. Our results provide strong evidence that non-visual inputs serve an important, previously under-appreciated, functional role in grasping.

Infant Reaching in the First Year of Life: A Scoping Review of Typical Development and Examples of Atypical Development

AIMS

Our objective was to identify the most common variables used for infant reaching assessment, describe values of these variables across the first year of life, and identify methodological considerations and knowledge gaps for future research.

METHODS

Studies were included if they met the following criteria: (1) assessed infant reaching in any position, (2) included a sample of infants with typical development: healthy, full-term, with no known impairments, (3) infants were under one year old at the first data collection, and (4) counted successful reaches resulting in object contact.

RESULTS

We identified 6 commonly assessed kinematic reaching variables: frequency, duration, movement units, peak velocity, average velocity, and straightness index. Methodological inconsistencies limit our ability to interpret values of these variables across studies.

CONCLUSIONS

Eliminating inconsistencies in study design and data analysis methods is the next step to defining a normative reference standard for reaching development. Establishing a normative reference standard for reaching in the first year of life will be important for assessment of typical and atypical reaching development.

Influence of the Inter-Trial Interval, Movement Observation, and Hand Dominance on the Previous Trial Effect

Previous studies have shown that current movement is influenced by the previous movement, which is known as the previous trial effect. In this study, we investigated the influence of the inter-trial interval, movement observation, and hand dominance on the previous trial effect of the non-target discrete movement. Right-handed healthy humans abducted the index finger in response to a start cue, and this task was repeated with constant inter-trial intervals. The absolute difference in the reaction time (RT) between the previous and current trials increased as the inter-trial interval increased. The absolute difference in RT reflects the reproducibility of the time taken for the motor execution between two consecutive trials. Thus, the finding supported the view that there is a carryover of movement information from one trial to the next, and that the underlying reproducibility of the RT between the two consecutive trials decays over time. This carryover of movement information is presumably conveyed by implicit short-term memory, which also decays within a short period of time. The correlation coefficient of the RT between the previous and current trials decreased with an increase in the inter-trial interval, indicating that the common responsiveness of two consecutive trials weakens over time. The absolute difference was smaller when the response was performed while observing finger movement, indicating that a carryover of the visual information to the next trial enhances the reproducibility of the motor execution process between consecutive trials. Hand dominance did not influence the absolute difference or correlation coefficient, indicating that the central process mediating previous trial effect of hand movement is not greatly lateralized.

Influence of visual feedback, hand dominance and sex on individuated finger movements

The ability to perform individual finger movements, highly important in daily activities, involves visual monitoring and proprioception. We investigated the influence of vision on the spatial and temporal control of independent finger movements, for the dominant and non-dominant hand and in relation to sex. Twenty-six healthy middle-aged to old adults (M age = 61 years; range 46–79 years; females n = 13) participated. Participants performed cyclic flexion–extension movements at the metacarpophalangeal joint of one finger at a time while keeping the other fingers as still as possible. Movements were recorded using 3D optoelectronic motion technique (120 Hz). The movement trajectory distance; speed peaks (movement smoothness); Individuation Index (II; the degree a finger can move in isolation from the other fingers) and Stationarity Index (SI; how still a finger remains while the other fingers move) were extracted. The main findings were: (1) vision only improved the II and SI marginally; (2) longer trajectories were evident in the no-vision condition for the fingers of the dominant hand in the female group; (3) longer trajectories were specifically evident for the middle and ring fingers within the female group; (4) females had marginally higher II and SI compared with males; and (5) females had fewer speed peaks than males, particularly for the ring finger. Our results suggest that visual monitoring of finger movements marginally improves performance of our non-manipulative finger movement task. A consistent finding was that females showed greater independent finger control compared with males.

Grasping performance depends upon the richness of hand feedback

Although visual feedback of the hand allows fast and accurate grasping actions, little is known about whether the nature of feedback of the hand affects performance. We investigated kinematics during precision grasping (with the index finger and thumb) when participants received different levels of hand feedback, with or without visual feedback of the target. Specifically, we compared performance when participants saw (1) no hand feedback; (2) only the two critical points on the index finger and thumb tips; (3) 21 points on all digit tips and hand joints; (4) 21 points connected by a "skeleton", or (5) full feedback of the hand wearing a glove. When less hand feedback was available, participants took longer to execute the movement because they allowed more time to slow the reach and close the hand. When target feedback was unavailable, participants took longer to plan the movement and reached with higher velocity. We were particularly interested in investigating maximum grip aperture (MGA), which can reflect the margin of error that participants allow to compensate for uncertainty. A trend suggested that MGA was smallest when ample feedback was available (skeleton and full hand feedback, regardless of target feedback) and when only essential information about hand and target was provided (2-point hand feedback + target feedback) but increased when non-essential points were included (21-point feedback). These results suggest that visual feedback of the hand affects grasping performance and that, while more feedback is usually beneficial, this is not necessarily always the case.

A summary statistical representation influences perceptions but not visually or memory-guided grasping

A statistical summary representation (SSR) is a phenomenon wherein a target property (e.g., size) is encoded based on the average of the stimulus-set to which it belongs. An SSR has been demonstrated in obligatory judgment tasks; however, to our knowledge no work has examined whether it influences grasps to 3D targets. Here, participants completed a method of adjustment task, and visually and memory-guided grasps in conditions wherein differently sized 3D targets (widths: 20, 30 and 40 mm; height and depth = 10 mm) were presented with equal frequency (i.e., control) and when the smallest (i.e., 20-mm: small-target) and largest (i.e., 40-mm: large-target) targets were presented five times as often as the other targets in the stimulus-set. In the method of adjustment task, responses for the small- and large-target weighting conditions were smaller and larger than the control condition, respectively. In other words, an SSR biased perceptions in the direction of the most frequently presented target in the stimulus-set - a result consistent with the view that perceptions are supported by relative visual information laid down by the ventral visual pathway. In contrast, grip apertures were refractory to target-weighting and was a finding independent of the presence (i.e., visually guided) or absence (i.e., memory-guided) of visual feedback. Furthermore, two one-sided tests showed that peak grip apertures for the different target weighting conditions were within an equivalence boundary. Accordingly, an SSR does not influence 3D grasps and is a finding adding to a growing literature reporting that actions are supported by the absolute visuomotor networks of the dorsal visual pathway.

Virtual image of a hand displaced in space influences action performance of the real hand

The rubber hand illusion (RHI) demonstrates that under some circumstances a fake hand can be regarded as part of one’s body; the RHI and related phenomena have been used to explore the flexibility of the body schema. Recent work has shown that a sense of embodiment may be generated by virtual reality (VR). In a series of experiments, we used VR to assess the effects of the displacement of the virtual image of subjects’ hands on action. Specifically, we tested whether spatial and temporal parameters of action change when participants perform a reaching movement towards the location of their virtual hand, the position of which was distorted on some trials. In different experiments, participants were sometimes provided with incorrect visual feedback regarding the position of the to-be-touched hand (Experiment 1), were deprived of visual feedback regarding the position of the reaching hand when acting (Experiment 2) or reached with the hand, the apparent position of which had been manipulated (Experiment 3). The effect was greatest when participants reached towards (Experiment 1) or with (Experiment 3) the displaced hand when the hand was visible during the reaching, but not when the vision of the hand was removed during the action (Experiment 2). Taken together, these data suggest that visual images of one’s hand presented in VR influence the body schema and action performance.

Some binocular advantages for planning reach, but not grasp, components of prehension

Proficient (fast, accurate, precise) hand actions for reaching-to-grasp 3D objects are known to benefit significantly from the use of binocular vision compared to one eye alone. We examined whether these binocular advantages derive from increased reliability in encoding the goal object’s properties for feedforward planning of prehension movements or from enhanced feedback mediating their online control. Adult participants reached for, precision grasped and lifted cylindrical table-top objects (two sizes, 2 distances) using binocular vision or only their dominant/sighting eye or their non-dominant eye to program and fully execute their movements or using each of the three viewing conditions only to plan their reach-to-grasp during a 1 s preview, with vision occluded just before movement onset. Various kinematic measures of reaching and grasping proficiency, including corrective error rates, were quantified and compared by view, feedback and object type. Some significant benefits of binocular over monocular vision when they were just available for pre-movement planning were retained for the reach regardless of target distance, including higher peak velocities, straighter paths and shorter low velocity approach times, although these latter were contaminated by more velocity corrections and by poorer coordination with object contact. By contrast, virtually all binocular advantages for grasping, including improvements in peak grip aperture scaling, the accuracy and precision of digit placements at object contact and shorter grip application times preceding the lift, were eliminated with no feedback available, outcomes that were influenced by the object’s size. We argue that vergence cues can improve the reliability of binocular internal representations of object distance for the feedforward programming of hand transport, whereas the major benefits of binocular vision for enhancing grasping performance derive exclusively from its continuous presence online.

Grasping movements toward seen and handheld objects

Grasping movements are typically performed toward visually sensed objects. However, planning and execution of grasping movements can be supported also by haptic information when we grasp objects held in the other hand. In the present study we investigated this sensorimotor integration process by comparing grasping movements towards objects sensed through visual, haptic or visuo-haptic signals. When movements were based on haptic information only, hand preshaping was initiated earlier, the digits closed on the object more slowly, and the final phase was more cautious compared to movements based on only visual information. Importantly, the simultaneous availability of vision and haptics led to faster movements and to an overall decrease of the grip aperture. Our findings also show that each modality contributes to a different extent in different phases of the movement, with haptics being more crucial in the initial phases and vision being more important for the final on-line control. Thus, vision and haptics can be flexibly combined to optimize the execution of grasping movement.

A margin for error in grasping: hand pre-shaping takes into account task-dependent changes in the probability of errors

Ideal grasping movements should maintain an appropriate probability of success, while controlling movement-related costs, in the presence of varying visual (and motor) uncertainty. It is often assumed that the probability of errors is managed by adjusting a margin for error in hand opening (e.g., opening the hand wider with increased visual uncertainty). This idea is intuitive, but non-trivial. It implies not only that the brain can estimate the amount of uncertainty, but also that it can compute how different possible alterations to the movement will affect the probability of errors—which we term the ‘probability landscape’. Previous work suggests the amount of uncertainty is factored into grasping movements. Our aim was to determine whether grasping movements are also sensitive to the probability landscape. Subjects completed three different grasping tasks, with naturally different probability landscapes, such that appropriate margin-for-error responses to increased uncertainty were qualitatively different (opening the hand wider, the same amount, or less wide). We increased visual uncertainty by blurring vision, and by covering one eye. Movements were performed without visual feedback to isolate uncertainty in the brain’s initial estimate of object properties. Changes to hand opening in response to increased visual uncertainty closely resembled those predicted by the margin-for-error account, suggesting that grasping is sensitive to the probability landscape associated with different tasks. Our findings therefore support the intuitive idea that grasping movements employ a true margin-for-error mechanism, which exerts active control over the probability of errors across changing circumstances.

How removing visual information affects grasping movements

Our interaction with objects is facilitated by the availability of visual feedback. Here, we investigate how and when visual feedback affects the way we grasp an object. Based on the main views on grasping (reach-and-grasp and double-pointing views), we designed four experiments to test: (1) whether the availability of visual feedback influences the digits independently, and (2) whether the absence of visual feedback affects the initial part of the movement. Our results show that occluding (part of) the hand's movement path influences the movement trajectory from the beginning. Thus, people consider the available feedback when planning their movements. The influence of the visual feedback depends on which digit is occluded, but its effect is not restricted to the occluded digit. Our findings indicate that the control mechanisms are more complex than those suggested by current views on grasping.

The Role of Sight of the Hand in the Development of Prehension in Childhood

In two studies, children between 5 and 10 years of age were asked to reach to grasp an object without sight of the hand during the movement. The oldest children and adults were faster when they could see the hand and increased maximum grip aperture when they could not see the hand. The 10-year-olds were less able to integrate grasp and lift than adults when they could see their hands. Children aged 5 and 6 showed no increase in movement time when they could not see the hand and did not adapt maximum grip aperture to lack of sight. These effects remained when children were encouraged to reach for and lift the target as quickly as possible. The results indicate that younger children did not give preference to vision in the control of prehension, while older children used visual feedback to improve efficiency. Dependence on sight of the hand for the control of prehension does not simply decrease with age, but it may be integrated into an anticipatory control strategy where it contributes to the efficiency of control.

Evidence for the representation of movement kinematics in the discharge of F5 mirror neurons during the observation of transitive and intransitive actions

Mirror neurons (MirNs) are sensorimotor neurons that fire both when an animal performs a goal-directed action and when the same animal observes another agent performing the same or a similar transitive action. It has been claimed that the observation of intransitive actions does not activate MirNs in a monkey’s brain. Prompted by recent evidence indicating that the discharge of MirNs is modulated also by non-object-directed actions, we investigated thoroughly the efficacy of intransitive actions to trigger MirNs’ discharge. Using representational similarity analysis, we also studied whether the elements constituting the visual scene presented to the monkey during the observation of actions (both transitive and intransitive) are represented in the discharge of MirNs. For this purpose, the moving hand was modeled by its kinematics and the object by features of its geometry. We found that MirNs respond to the observation of both transitive and intransitive actions and that the discharge differences evoked by the observation of object- and non-object-directed actions are correlated more with the kinematic differences of these actions than with the differences of the objects’ features. These findings support the view that observed action kinematics contribute to action mirroring.

NEW & NOTEWORTHY Mirror neurons in the monkey brain are thought to respond exclusively to the observation of object-directed actions. Here, we show that mirror neurons also respond to the observation of intransitive actions and that the kinematics of the observed movements are represented in their discharge. This finding supports the view that mirror neurons provide also a kinematics-based representation of actions.

When perception trips action! The increase in the perceived size of both hand and target matters in reaching and grasping movements

Reaching and grasping movements rely on visual information regarding the target characteristics (e.g. size) and the hand position during the action execution. Changes in the visual representation of the body (e.g. increase in the perceived size of the hand) can modify action performance, but it is still unclear how these modifications interact with changes in the external environment. We investigated this topic by manipulating the perceived size of both hand and target objects and the degree of visual feedback available during the movement execution. Ten young adults were asked to reach and grasp geometrical objects in four different conditions: (i) with normal vision with the light on, (ii) with normal vision in the dark, (iii) using magnifying lenses in the light and (iv) using magnifying lenses in the dark. In contrast with previous works, our results show that movement execution is longer in magnified vision compared to normal when the action is executed in the light, but the grasping component was not affected by changes in size in this condition. On the contrary, when the visual feedback of the hand was removed and participants performed the action in the dark, movements were faster and the distances across fingers larger in the magnified than normal vision. This pattern of data suggests that grasping movements adapt rapidly and compensate for changes in vision when this process depends on the degree of visual feedback and/or environmental cues available. In the debate regarding the dissociation between action and perception, our data suggest that action may overcome changes in perception when visual feedback is available, but perception may trick action in situations of reduced visual information.

Handedness and Reach-to-Place Kinematics in Adults: Left-Handers Are Not Reversed Right-Handers

The primary goal of this study was to examine the relations between limb control and handedness in adults. Participants were categorized as left or right handed for analyses using the Edinburgh Handedness Inventory. Three-dimensional recordings were made of each arm on two reach-to-place tasks: adults reached to a ball and placed it into the opening of a toy (fitting task), or reached to a Cheerio inside a cup, which they placed on a designated mark after each trial (cup task). We hypothesized that limb control and handedness were related, and we predicted that we would observe side differences favoring the dominant limb based on the dynamic dominance hypothesis of motor lateralization. Specifically, we predicted that the dominant limb would be straighter and smoother on both tasks compared with the nondominant limb (i.e., right arm in right-handers and left arm in left-handers). Our results only partially supported these predictions for right-handers, but not for left-handers. When differences between hands were observed, the right hand was favored regardless of handedness group. Our findings suggest that left-handers are not reversed right-handers when compared on interlimb kinematics for reach-to-place tasks, and reaffirm that task selection is critical when evaluating manual asymmetries.

The development of motor behavior

This article reviews research on the development of motor behavior from a developmental systems perspective. We focus on infancy when basic action systems are acquired. Posture provides a stable base for locomotion, manual actions, and facial actions. Experience facilitates improvements in motor behavior and infants accumulate immense amounts of experience with all of their basic action systems. At every point in development, perception guides motor behavior by providing feedback about the results of just prior movements and information about what to do next. Reciprocally, the development of motor behavior provides fodder for perception. More generally, motor development brings about new opportunities for acquiring knowledge about the world, and burgeoning motor skills can instigate cascades of developmental changes in perceptual, cognitive, and social domains. WIREs Cogn Sci 2017, 8:e1430. doi: 10.1002/wcs.1430 For further resources related to this article, please visit the WIREs website.

Visual information about object size and object position are retained differently in the visual brain: Evidence from grasping studies

Many experiments have examined how the visual information used for action control is represented in our brain, and whether or not visually-guided and memory-guided hand movements rely on dissociable visual representations that are processed in different brain areas (dorsal vs. ventral). However, little is known about how these representations decay over longer time periods and whether or not different visual properties are retained in a similar fashion. In three experiments we investigated how information about object size and object position affect grasping as visual memory demands increase. We found that position information decayed rapidly with increasing delays between viewing the object and initiating subsequent actions - impacting both the accuracy of the transport component (lower end-point accuracy) and the grasp component (larger grip apertures) of the movement. In contrast, grip apertures and fingertip forces remained well-adjusted to target size in conditions in which positional information was either irrelevant or provided, regardless of delay, indicating that object size is encoded in a more stable manner than object position. The findings provide evidence that different grasp-relevant properties are encoded differently by the visual system. Furthermore, we argue that caution is required when making inferences about object size representations based on alterations in the grip component as these variations are confounded with the accuracy with which object position is represented. Instead fingertip forces seem to provide a reliable and confound-free measure to assess internal size estimations in conditions of increased visual uncertainty.

Sensor fusion and computer vision for context-aware control of a multi degree-of-freedom prosthesis

OBJECTIVE

Myoelectric activity volitionally generated by the user is often used for controlling hand prostheses in order to replicate the synergistic actions of muscles in healthy humans during grasping. Muscle synergies in healthy humans are based on the integration of visual perception, heuristics and proprioception. Here, we demonstrate how sensor fusion that combines artificial vision and proprioceptive information with the high-level processing characteristics of biological systems can be effectively used in transradial prosthesis control.

APPROACH

We developed a novel context- and user-aware prosthesis (CASP) controller integrating computer vision and inertial sensing with myoelectric activity in order to achieve semi-autonomous and reactive control of a prosthetic hand. The presented method semi-automatically provides simultaneous and proportional control of multiple degrees-of-freedom (DOFs), thus decreasing overall physical effort while retaining full user control. The system was compared against the major commercial state-of-the art myoelectric control system in ten able-bodied and one amputee subject. All subjects used transradial prosthesis with an active wrist to grasp objects typically associated with activities of daily living.

MAIN RESULTS

The CASP significantly outperformed the myoelectric interface when controlling all of the prosthesis DOF. However, when tested with less complex prosthetic system (smaller number of DOF), the CASP was slower but resulted with reaching motions that contained less compensatory movements. Another important finding is that the CASP system required minimal user adaptation and training.

SIGNIFICANCE

The CASP constitutes a substantial improvement for the control of multi-DOF prostheses. The application of the CASP will have a significant impact when translated to real-life scenarious, particularly with respect to improving the usability and acceptance of highly complex systems (e.g., full prosthetic arms) by amputees.

Lack of depth constancy for grasping movements in both virtual and real environments

Recent studies on visuomotor processes using virtual setups have suggested that actions are affected by similar biases as perceptual tasks. In particular, a strong lack of depth constancy is revealed, resembling biases in perceptual estimates of relative depth. With this study we aim to understand whether these findings are mostly caused by a lack of metric accuracy of the visuomotor system or by the limited cues provided by the use of virtual reality. We addressed this issue by comparing grasping movements towards a spherical object located at four distances (420, 450, 480, and 510 mm) performed in three conditions: 1) virtual, in which the target was a virtual object defined by binocular cues, 2) glow-in-the-dark, in which the object was painted with luminous paint but no other cue was provided, and 3) full-cue, in which the movement was performed with the lights on and all the environmental information was available. Results revealed a striking effect of object distance on grip aperture equally in all three conditions. Specifically, grip aperture gradually decreased with increase in object distance, proving a consistent lack of depth constancy. These findings clearly demonstrate that systematic biases in grasping actions are not induced by the use of virtual environments and that action and perception may involve the same visual information, which does not engage a metric reconstruction of the scene.

Use of early phase online vision for grip configuration is modulated according to movement duration in prehension

Our previous study (Hum Mov Sci 25:349-371, 2006) investigated whether and how online vision in the early phase of movement influences the control of reach-to-grasp movements (movement duration: approximately 1000 ms). We used liquid-crystal shutter goggles to manipulate the duration of available online vision during the movement and specified that online vision during the early phase influences grasping movements. The current study examined the effect of online early phase vision on the grip configuration according to the movement duration and compared it between two different movement durations (approximately 500 and 1000 ms). We found that non-availability of early phase online vision affected the grip configuration (i.e., inducing a larger peak grip aperture) even in the shorter movement duration. The influential period for online vision for grasping control shifts to an earlier time when movement time is shorter (i.e., from approximately 214 to 106 ms after movement onset), indicating a flexible mechanism for grip configuration according to the movement duration and the available online vision.

Hand preference status and reach kinematics in infants

Infants show age-related improvements in reach straightness and smoothness over the first years of life as well as a decrease in average movement speed. This period of changing kinematics overlaps the emergence of handedness. We examined whether infant hand preference status is related to the development of motor control in 53 infants ranging from 11 to 14 months old. Hand preference status was assessed from reaching to a set of 5 objects presented individually at the infant's midline; infants were classified into 'right preference' or 'no preference' groups. Three-dimensional (3-D) recordings were made of each arm for reaches under two distinct conditions: pick up a ball and fit it into the opening of a toy (grasp-to-place task) or pick up a Cheerio® and consume it (grasp-to-eat task). Contrary to expectations, there was no effect of hand preference status on reach smoothness or straightness for either task. On the grasp-to-eat task only, average speed of the left hand differed as a function of hand preference status. Infants in the no preference group exhibited higher left hand average speeds than infants in the right preference group. Our results suggest that while behavioral differences in the use of the two hands may be present in some infants, these differences do not appear to be systematically linked to biases in motor control of the arms early in development.

Coordination deficits during trunk-assisted reach-to-grasp movements in Parkinson's disease

The present study investigated how Parkinson's disease (PD) affects temporal coordination among the trunk, arm, and fingers during trunk-assisted reach-to-grasp movements. Seated participants with PD and healthy controls made prehensile movements. During the reach to the object, the involvement of the trunk was altered based on the instruction; the trunk was not involved, moved forward (flexion), or moved backward (extension) in the sagittal plane. Each of the trunk movements was combined with an extension or flexion motion of the arm during the reach. For the transport component, the individuals with PD substantially delayed the onset of trunk motion relative to that of arm motion in conditions where the trunk was moved in the direction opposite from the arm reaching toward the object. At the same time, variability of intervals between the onsets and intervals between the velocity peaks of the trunk and wrist movements was increased. The magnitudes of the variability measures were significantly correlated with the severity of PD. Regarding the grasp component, the individuals with PD delayed the onset of finger movements during reaching. These results imply that PD impairs temporal coordination between the axial and distal body segments during goal-directed skilled actions. When there is a directional discrepancy between the trunk and wrist motions, individuals with PD appear to prioritize wrist motion that is tied to the task goal over the trunk motion. An increase in disease severity magnifies the coordination deficits.

Bimanual coordination in children: manipulation of object distance

The performance of many everyday activities requires the coordination of the two upper limbs to achieve the goal of the task. Although bimanual performance has been studied in detail in adults, few studies have examined how children coordinate the movements of the two hands during symmetric and asymmetric bimanual prehension. With the current study, we asked younger (4-6 years, n = 14) and older (7-10 years, n = 16) children to complete a discrete bimanual task. Specifically, they reached to grasp cylinders located at near and far positions in either unimanual or bimanual condition. During bimanual symmetric conditions, participants performed movements with both hands toward two objects located at the same distance (both near or both far), while in the bimanual asymmetric conditions, they reached for objects at different distances. Results of the kinematic analyses indicated that the young children consistently experienced the "two target" effect, whereby bimanual movements were executed more slowly than unimanual movements to the same distance. Older children employed a hybrid strategy, exhibiting slower movements in bimanual congruent conditions, but larger non-dominant apertures in bimanual incongruent conditions. This hybrid strategy was hypothesized to stem from developmental changes occurring in the integration of sensory information around 8 years of age. While older children exhibited temporal and spatial coordination patterns that were similar to patterns reported in adults, large relative timing differences at the start and end of bimanual movements and considerably weaker spatial coupling were seen in the younger children.

Early infant's use of visual feedback in voluntary reaching for a spatial target

Capacity of using visual feedback by infants at the age of reaching onset has been controversial. In this investigation we assessed movement kinematics in the task of reaching for a toy in 5-month-olds, comparing movements performed with the preferred arm under full vision versus visual occlusion. That comparison was made in consecutive periods of visual occlusion. Analysis of results revealed that visual occlusion led to decreased straightness of arm displacement toward the toy as compared to full vision. Longer periods of occlusion did not augment that effect. These results offer preliminary evidence for use of visual feedback early in infants' reaching development. Reconciliation of previous and current findings is made by proposing a hybrid mode of feedback processing for manual control reweighting the roles of vision and proprioception as a function of availability of environmental information.

Activity in ventral premotor cortex is modulated by vision of own hand in action

Parietal and premotor cortices of the macaque monkey contain distinct populations of neurons which, in addition to their motor discharge, are also activated by visual stimulation. Among these visuomotor neurons, a population of grasping neurons located in the anterior intraparietal area (AIP) shows discharge modulation when the own hand is visible during object grasping. Given the dense connections between AIP and inferior frontal regions, we aimed at investigating whether two hand-related frontal areas, ventral premotor area F5 and primary motor cortex (area F1), contain neurons with similar properties. Two macaques were involved in a grasping task executed in various light/dark conditions in which the to-be-grasped object was kept visible by a dim retro-illumination. Approximately 62% of F5 and 55% of F1 motor neurons showed light/dark modulations. To better isolate the effect of hand-related visual input, we introduced two further conditions characterized by kinematic features similar to the dark condition. The scene was briefly illuminated (i) during hand preshaping (pre-touch flash, PT-flash) and (ii) at hand-object contact (touch flash, T-flash). Approximately 48% of F5 and 44% of F1 motor neurons showed a flash-related modulation. Considering flash-modulated neurons in the two flash conditions, ∼40% from F5 and ∼52% from F1 showed stronger activity in PT- than T-flash (PT-flash-dominant), whereas ∼60% from F5 and ∼48% from F1 showed stronger activity in T- than PT-flash (T-flash-dominant). Furthermore, F5, but not F1, flash-dominant neurons were characterized by a higher peak and mean discharge in the preferred flash condition as compared to light and dark conditions. Still considering F5, the distribution of the time of peak discharge was similar in light and preferred flash conditions. This study shows that the frontal cortex contains neurons, previously classified as motor neurons, which are sensitive to the observation of meaningful phases of the own grasping action. We conclude by discussing the possible functional role of these populations.

Control of aperture closure initiation during trunk-assisted reach-to-grasp movements

The present study investigated how the involvement and direction of trunk movement during reach-to-grasp movements affect the coordination between the transport and grasping components. Seated young adults made prehensile movements in which the involvement of the trunk was varied; the trunk was not involved, moved forward (flexion), or moved backward (extension) in the sagittal plane during the reach to the object. Each of the trunk movements was combined with an extension or flexion motion of the arm during the reach. Regarding the relationship between the trunk and arm motion for arm transport, the onset of wrist motion relative to that of the trunk was delayed to a greater extent for the trunk extension than for the trunk flexion. The variability of the time period from the peak of wrist velocity to the peak of trunk velocity was also significantly greater for trunk extension compared to trunk flexion. These findings indicate that trunk flexion was better integrated into the control of wrist transport than trunk extension. In terms of the temporal relationship between wrist transport and grip aperture, the relationship between the time of peak wrist velocity and the time of peak grip aperture did not change or become less steady across conditions. Therefore, the stability of temporal coordination between wrist transport and grip aperture was maintained despite the variation of the pattern of intersegmental coordination between the arm and the trunk during arm transport. The transport-aperture coordination was further assessed in terms of the control law according to which the initiation of aperture closure during the reach occurs when the hand crosses a hand-to-target distance threshold for grasp initiation, which is a function of peak aperture, wrist velocity and acceleration, trunk velocity and acceleration, and trunk-to-target distance at the time of aperture closure initiation. The participants increased the hand-to-target distance threshold for grasp initiation in the conditions where the trunk was involved compared to the conditions where the trunk was not involved. An increase also occurred when the trunk was extended compared to when it was flexed. The increased distance threshold implies an increase in the hand-to-target distance-related safety margin for grasping when the trunk is involved, especially when it is extended. These results suggest that the CNS significantly utilizes the parameters of trunk movement together with movement parameters related to the arm and the hand for controlling grasp initiation.

Visual information and object size in infant reaching

It has been suggested that the onset of successful reaching in infants is mediated by the onset of an ability to use sight of the hand to make corrective reaches. However, removing vision of the hand in infants younger than 6 months has not been shown to have an effect on reaching onset or kinematics. We investigated the use of vision of the hand by testing 6-, 9-, and 12-month-old infants reaching for objects in the light and in the dark. We found that infants reached faster in the dark at 6 months, and faster in the light at 1 year. Parallel effects were observed in the movement times. Consistent effects of altering target object size on average speed were seen at 12 months. The data support the hypothesis that vision is used by older infants around 6 months-of-age, and that reach and grasp planning differentiate with object size at about 9 and 12 months-of-age. At younger ages reaches are corrected on the basis of proprioceptive information and sight of the target object.

Large perspective changes yield perception of metric shape that allows accurate feedforward reaches-to-grasp and it persists after the optic flow has stopped!

Lee et al. (Percept Psychophys 70:1032-1046, 2008a) investigated whether visual perception of metric shape could be calibrated when used to guide feedforward reaches-to-grasp. It could not. Seated participants viewed target objects (elliptical cylinders) in normal lighting using stereo vision and free head movements that allowed small (approximately 10 degrees) perspective changes. The authors concluded that poor perception of metric shape was the reason reaches-to-grasp should be visually guided online. However, Bingham and Lind (Percept Psychophys 70:524-540, 2008) showed that large perspective changes (> or =45 degrees) yield good perception of metric shape. So, now we repeated the Lee et al.'s study with the addition of information from large perspective changes. The results were accurate feedforward reaches-to-grasp reflecting accurate perception of both metric shape and metric size. Large perspective changes occur when one locomotes into a workspace in which reaches-to-grasp are subsequently performed. Does the resulting perception of metric shape persist after the large perspective changes have ceased? Experiments 2 and 3 tested reaches-to-grasp with delays (Exp. 2, 5-s delay; Exp. 3, approximately 16-s delay) and multiple objects to be grasped after a single viewing. Perception of metric shape and metric size persisted yielding accurate reaches-to-grasp. We advocate the study of nested actions using a dynamic approach to perception/action.

Learning arm/hand coordination with an altered visual input

The focus of this study was to test a novel tool for the analysis of motor coordination with an altered visual input. The altered visual input was created using special glasses that presented the view as recorded by a video camera placed at various positions around the subject. The camera was positioned at a frontal (F), lateral (L), or top (T) position with respect to the subject. We studied the differences between the arm-end (wrist) trajectories while grasping an object between altered vision (F, L, and T conditions) and normal vision (N) in ten subjects. The outcome measures from the analysis were the trajectory errors, the movement parameters, and the time of execution. We found substantial trajectory errors and an increased execution time at the baseline of the study. We also found that trajectory errors decreased in all conditions after three days of practice with the altered vision in the F condition only for 20 minutes per day, suggesting that recalibration of the visual systems occurred relatively quickly. These results indicate that this recalibration occurs via movement training in an altered condition. The results also suggest that recalibration is more difficult to achieve for altered vision in the F and L conditions compared to the T condition. This study has direct implications on the design of new rehabilitation systems.

Dissociating networks of imitation

The investigation of imitation, which consists of observation and later reproduction of voluntary actions, promises insights into the complex processes of human actions. Although several aspects concerning the component neural processes necessary for action execution are known, our current understanding of the neural networks underlying these remains sparse. The present study applies independent component analysis (ICA) to functional magnetic resonance imaging (fMRI) data acquired during imitation of abstract gestures and object-related actions. This enables identification of neural networks underlying the production of these imitations. The explorative approach of ICA is complemented by an analysis of time courses from the maxima of each component. Four independent networks were active during delayed imitation. These can be assigned to the aspects of (1) action perception, (2) motor preparation and action execution, (3) encoding and retrieval into and from working memory, as well as (4) the dynamic integration of object affordances into the action. At least two of these networks participate in action preparation, one contains areas involved with motor working memory and one includes areas which are connected to the true action execution. The fourth network only shows activity shortly before an object-related action is imitated. This indicates a late integration of object affordances into the movement as the time course of activity in this network pertains to action rather than perception of the object.

Temporal Coordination during Bimanual Reach-to-Grasp Movements: The Role of Vision

The performance of bimanual movements involving separate objects presents an obvious challenge to the visuo-motor system: Visual feedback can only be obtained from one target at a time. To overcome this challenge overt shifts in visual attention may occur so that visual feedback from both movements may be used directly (Bingham, Hughes, & Mon-Williams, 2008; Riek, Tresilian, Mon-Williams, Coppard, & Carson, 2003). Alternatively, visual feedback from both movements may be obtained in the absence of eye movements, presumably by covert shifts in attention (Diedrichsen, Nambisan, Kennerley, & Ivry, 2004). Given that the quality of information falls with increasing distance from the fixated point, can we obtain the level of information required to accurately guide each hand for precision grasping of separate objects without moving our eyes to fixate each target separately? The purpose of the current study was to examine how the temporal coordination between the upper limbs is affected by the quality of visual information available during the performance of a bimanual task. A total of 11 participants performed congruent and incongruent movements towards near and/or far objects. Movements were performed in natural, fixate-centre, fixate-left, and fixate-right vision conditions. Analyses revealed that the transport phase of incongruent movements was similar across vision conditions for the temporal aspects of both the transport and grasp, whereas the spatial aspects of grasp formation were influenced by the quality of visual feedback. We suggest that bimanual coordination of the temporal aspects of reach-to-grasp movements are not influenced solely by overt shifts in visual attention but instead are influenced by a combination of factors in a task-constrained way.

The influence of visual feedback and register changes on sign language production: A kinematic study with deaf signers

Speakers monitor their speech output by listening to their own voice. However, signers do not look directly at their hands and cannot see their own face. We investigated the importance of a visual perceptual loop for sign language monitoring by examining whether changes in visual input alter sign production. Deaf signers produced American Sign Language (ASL) signs within a carrier phrase under five conditions: blindfolded, wearing tunnel-vision goggles, normal (citation) signing, shouting, and informal signing. Three-dimensional movement trajectories were obtained using an Optotrak Certus system. Informally produced signs were shorter with less vertical movement. Shouted signs were displaced forward and to the right and were produced within a larger volume of signing space, with greater velocity, greater distance traveled, and a longer duration. Tunnel vision caused signers to produce less movement within the vertical dimension of signing space, but blind and citation signing did not differ significantly on any measure, except duration. Thus, signers do not "sign louder" when they cannot see themselves, but they do alter their sign production when vision is restricted. We hypothesize that visual feedback serves primarily to fine-tune the size of signing space rather than as input to a comprehension-based monitor.

Prediction of object contact during grasping

The maximum grip aperture (MGA) during prehension is linearly related to the size of objects to be grasped and is adapted to the haptically sensed object size when there is a discrepancy between visual and haptic information. We have investigated what information is used to drive this adaptation process and how the onset of fingertip forces on the object is triggered. Subjects performed a reach-to-grasp task, where the object seen and the object grasped physically never were the same. We measured the movements of the index finger and the thumb and the contact forces between each fingertip and the object. The subjects' adaptation of the MGA was unrelated both to different fingertip velocities at the moment of object contact, or the fingertip forces. Instead, the 'timing' of contact between the fingers and the object was most consistently influenced by introducing a size discrepancy. Specifically, if the object was larger than expected, the moment of contact occurred earlier, and if the object was decreased in size, then the contact occurred later. During adaptation, these timing differences were markedly reduced. Also, the motor command for applying forces on the object seemed to be released in anticipation of the predicted moment of contact. We therefore conclude that the CNS dynamically predicts when contact between the fingertips and objects occur and that aperture adaptation is primarily driven by timing prediction errors.

Temporal integration limits of stereovision in reaching and grasping

Even though there have been extensive investigations of the temporal integration limits of binocular vision in perceptual tasks, relatively little is known about temporal integration limits during the completion of visuomotor tasks. To assess the temporal integration limits of binocular disparity within the action domain, accuracy of reach kinematics in a reaching and grasping task under continuous binocular and monocular viewing conditions were compared with those obtained under alternating monocular viewing conditions with interocular delays ranging from 14 to 58 ms. Even the shortest of the interocular delays resulted in larger grip apertures than those in the continuous monocular and binocular viewing conditions. The short temporal integration interval of stereovision obtained in this study cannot be accounted for by differential visual feedback in the binocular and interocular delay conditions, nor is it likely to be a consequence of visual disruption due to the interocular delays. Our findings suggest that the visuomotor system has little tolerance to interocular delay.