Discrimination of line orientation in humans and monkeys

Tactile adaptation to orientation produces a robust tilt aftereffect and exhibits crossmodal transfer when tested in vision

Orientation processing is one of the most fundamental functions in both visual and somatosensory perception. Converging findings suggest that orientation processing in both modalities is closely linked: somatosensory neurons share a similar orientation organisation as visual neurons, and the visual cortex has been found to be heavily involved in tactile orientation perception. Hence, we hypothesized that somatosensation would exhibit a similar orientation adaptation effect, and this adaptation effect would be transferable between the two modalities, considering the above-mentioned connection. The tilt aftereffect (TAE) is a demonstration of orientation adaptation and is used widely in behavioural experiments to investigate orientation mechanisms in vision. By testing the classic TAE paradigm in both tactile and crossmodal orientation tasks between vision and touch, we were able to show that tactile perception of orientation shows a very robust TAE, similar to its visual counterpart. We further show that orientation adaptation in touch transfers to produce a TAE when tested in vision, but not vice versa. Additionally, when examining the test sequence following adaptation for serial effects, we observed another asymmetry between the two conditions where the visual test sequence displayed a repulsive intramodal serial dependence effect while the tactile test sequence exhibited an attractive serial dependence. These findings provide concrete evidence that vision and touch engage a similar orientation processing mechanism. However, the asymmetry in the crossmodal transfer of TAE and serial dependence points to a non-reciprocal connection between the two modalities, providing further insights into the underlying processing mechanism.

V1 neurons encode the perceptual compensation of false torsion arising from Listing's law

We try to deploy the retinal fovea to optimally scrutinize an object of interest by directing our eyes to it. The horizontal and vertical components of eye positions acquired by goal-directed saccades are determined by the object's location. However, the eccentric eye positions also involve a torsional component, which according to Donder's law is fully determined by the two-dimensional (2D) eye position acquired. According to von Helmholtz, knowledge of the amount of torsion provided by Listing's law, an extension of Donder's law, alleviates the perceptual interpretation of the image tilt that changes with 2D eye position, a view supported by psychophysical experiments he pioneered. We address the question of where and how Listing's law is implemented in the visual system and we show that neurons in monkey area V1 use knowledge of eye torsion to compensate the image tilt associated with specific eye positions as set by Listing's law.

Comparison of Object Recognition Behavior in Human and Monkey

Although the rhesus monkey is used widely as an animal model of human visual processing, it is not known whether invariant visual object recognition behavior is quantitatively comparable across monkeys and humans. To address this question, we systematically compared the core object recognition behavior of two monkeys with that of human subjects. To test true object recognition behavior (rather than image matching), we generated several thousand naturalistic synthetic images of 24 basic-level objects with high variation in viewing parameters and image background. Monkeys were trained to perform binary object recognition tasks on a match-to-sample paradigm. Data from 605 human subjects performing the same tasks on Mechanical Turk were aggregated to characterize "pooled human" object recognition behavior, as well as 33 separate Mechanical Turk subjects to characterize individual human subject behavior. Our results show that monkeys learn each new object in a few days, after which they not only match mean human performance but show a pattern of object confusion that is highly correlated with pooled human confusion patterns and is statistically indistinguishable from individual human subjects. Importantly, this shared human and monkey pattern of 3D object confusion is not shared with low-level visual representations (pixels, V1+; models of the retina and primary visual cortex) but is shared with a state-of-the-art computer vision feature representation. Together, these results are consistent with the hypothesis that rhesus monkeys and humans share a common neural shape representation that directly supports object perception.

SIGNIFICANCE STATEMENT

To date, several mammalian species have shown promise as animal models for studying the neural mechanisms underlying high-level visual processing in humans. In light of this diversity, making tight comparisons between nonhuman and human primates is particularly critical in determining the best use of nonhuman primates to further the goal of the field of translating knowledge gained from animal models to humans. To the best of our knowledge, this study is the first systematic attempt at comparing a high-level visual behavior of humans and macaque monkeys.

The influence of spatiotemporal structure of noisy stimuli in decision making

Decision making is a process of utmost importance in our daily lives, the study of which has been receiving notable attention for decades. Nevertheless, the neural mechanisms underlying decision making are still not fully understood. Computational modeling has revealed itself as a valuable asset to address some of the fundamental questions. Biophysically plausible models, in particular, are useful in bridging the different levels of description that experimental studies provide, from the neural spiking activity recorded at the cellular level to the performance reported at the behavioral level. In this article, we have reviewed some of the recent progress made in the understanding of the neural mechanisms that underlie decision making. We have performed a critical evaluation of the available results and address, from a computational perspective, aspects of both experimentation and modeling that so far have eluded comprehension. To guide the discussion, we have selected a central theme which revolves around the following question: how does the spatiotemporal structure of sensory stimuli affect the perceptual decision-making process? This question is a timely one as several issues that still remain unresolved stem from this central theme. These include: (i) the role of spatiotemporal input fluctuations in perceptual decision making, (ii) how to extend the current results and models derived from two-alternative choice studies to scenarios with multiple competing evidences, and (iii) to establish whether different types of spatiotemporal input fluctuations affect decision-making outcomes in distinctive ways. And although we have restricted our discussion mostly to visual decisions, our main conclusions are arguably generalizable; hence, their possible extension to other sensory modalities is one of the points in our discussion.

What limits tool use in nonhuman primates? Insights from tufted capuchin monkeys (Sapajus spp.) and chimpanzees (Pan troglodytes) aligning three-dimensional objects to a surface

Perceptuomotor functions that support using hand tools can be examined in other manipulation tasks, such as alignment of objects to surfaces. We examined tufted capuchin monkeys' and chimpanzees' performance at aligning objects to surfaces while managing one or two spatial relations to do so. We presented six subjects of each species with a single stick to place into a groove, two sticks of equal length to place into two grooves, or two sticks joined as a T to place into a T-shaped groove. Tufted capuchins and chimpanzees performed equivalently on these tasks, aligning the straight stick to within 22.5° of parallel to the groove in approximately half of their attempts to place it, and taking more attempts to place the T stick than two straight sticks. The findings provide strong evidence that tufted capuchins and chimpanzees do not reliably align even one prominent axial feature of an object to a surface, and that managing two concurrent allocentric spatial relations in an alignment problem is significantly more challenging to them than managing two sequential relations. In contrast, humans from 2 years of age display very different perceptuomotor abilities in a similar task: they align sticks to a groove reliably on each attempt, and they readily manage two allocentric spatial relations concurrently. Limitations in aligning objects and in managing two or more relations at a time significantly constrain how nonhuman primates can use hand tools.

A fast and simple population code for orientation in primate V1

Orientation tuning has been a classic model for understanding single-neuron computation in the neocortex. However, little is known about how orientation can be read out from the activity of neural populations, in particular in alert animals. Our study is a first step toward that goal. We recorded from up to 20 well isolated single neurons in the primary visual cortex of alert macaques simultaneously and applied a simple, neurally plausible decoder to read out the population code. We focus on two questions: First, what are the time course and the timescale at which orientation can be read out from the population response? Second, how complex does the decoding mechanism in a downstream neuron have to be to reliably discriminate between visual stimuli with different orientations? We show that the neural ensembles in primary visual cortex of awake macaques represent orientation in a way that facilitates a fast and simple readout mechanism: With an average latency of 30-80 ms, the population code can be read out instantaneously with a short integration time of only tens of milliseconds, and neither stimulus contrast nor correlations need to be taken into account to compute the optimal synaptic weight pattern. Our study shows that-similar to the case of single-neuron computation-the representation of orientation in the spike patterns of neural populations can serve as an exemplary case for understanding the computations performed by neural ensembles underlying visual processing during behavior.

Conversion of sensory signals into perceptual decisions

A fundamental problem in neurobiology is to understand how brain circuits represent sensory information and how such representations give rise to perception, memory and decision-making. We demonstrate that a sensory stimulus engages multiple areas of the cerebral cortex, including primary sensory, prefrontal, premotor and motor cortices. As information transverses the cortical circuits it shows progressively more relation to perception, memory and decision reports. In particular, we show how somatosensory areas on the parietal lobe generate a parameterized representation of a tactile stimulus. This representation is maintained in working memory by prefrontal and premotor areas of the frontal lobe. The presentation of a second stimulus, that monkeys are trained to compare with the first, generates decision-related activity reflecting which stimulus had the higher frequency. Importantly, decision-related activity is observed across several cortical circuits including prefrontal, premotor and parietal cortices. Sensory information is encoded by neuronal populations with opposite tuning, and suggests that a simple subtraction operation could be the underlying mechanism by which past and present sensory information is compared to generate perceptual decisions.

Effects of a robot-assisted training of grasp and pronation/supination in chronic stroke: a pilot study

Background

Rehabilitation of hand function is challenging, and only few studies have investigated robot-assisted rehabilitation focusing on distal joints of the upper limb. This paper investigates the feasibility of using the HapticKnob, a table-top end-effector device, for robot-assisted rehabilitation of grasping and forearm pronation/supination, two important functions for activities of daily living involving the hand, and which are often impaired in chronic stroke patients. It evaluates the effectiveness of this device for improving hand function and the transfer of improvement to arm function.

Methods

A single group of fifteen chronic stroke patients with impaired arm and hand functions (Fugl-Meyer motor assessment scale (FM) 10-45/66) participated in a 6-week 3-hours/week rehabilitation program with the HapticKnob. Outcome measures consisted primarily of the FM and Motricity Index (MI) and their respective subsections related to distal and proximal arm function, and were assessed at the beginning, end of treatment and in a 6-weeks follow-up.

Results

Thirteen subjects successfully completed robot-assisted therapy, with significantly improved hand and arm motor functions, demonstrated by an average 3.00 points increase on the FM and 4.55 on the MI at the completion of the therapy (4.85 FM and 6.84 MI six weeks post-therapy). Improvements were observed both in distal and proximal components of the clinical scales at the completion of the study (2.00 FM wrist/hand, 2.55 FM shoulder/elbow, 2.23 MI hand and 4.23 MI shoulder/elbow). In addition, improvements in hand function were observed, as measured by the Motor Assessment Scale, grip force, and a decrease in arm muscle spasticity. These results were confirmed by motion data collected by the robot.

Conclusions

The results of this study show the feasibility of this robot-assisted therapy with patients presenting a large range of impairment levels. A significant homogeneous improvement in both hand and arm function was observed, which was maintained 6 weeks after end of the therapy.

Assessing neural activity related to decision-making through flexible odds ratio curves and their derivatives

It is well established that neural activity is stochastically modulated over time. Therefore, direct comparisons across experimental conditions and determination of change points or maximum firing rates are not straightforward. This study sought to compare temporal firing probability curves that may vary across groups defined by different experimental conditions. Odds-ratio (OR) curves were used as a measure of comparison, and the main goal was to provide a global test to detect significant differences of such curves through the study of their derivatives. An algorithm is proposed that enables ORs based on generalized additive models, including factor-by-curve-type interactions to be flexibly estimated. Bootstrap methods were used to draw inferences from the derivatives curves, and binning techniques were applied to speed up computation in the estimation and testing processes. A simulation study was conducted to assess the validity of these bootstrap-based tests. This methodology was applied to study premotor ventral cortex neural activity associated with decision-making. The proposed statistical procedures proved very useful in revealing the neural activity correlates of decision-making in a visual discrimination task.

Reassessing optimal neural population codes with neurometric functions

Cortical circuits perform the computations underlying rapid perceptual decisions within a few dozen milliseconds with each neuron emitting only a few spikes. Under these conditions, the theoretical analysis of neural population codes is challenging, as the most commonly used theoretical tool--Fisher information--can lead to erroneous conclusions about the optimality of different coding schemes. Here we revisit the effect of tuning function width and correlation structure on neural population codes based on ideal observer analysis in both a discrimination and a reconstruction task. We show that the optimal tuning function width and the optimal correlation structure in both paradigms strongly depend on the available decoding time in a very similar way. In contrast, population codes optimized for Fisher information do not depend on decoding time and are severely suboptimal when only few spikes are available. In addition, we use the neurometric functions of the ideal observer in the classification task to investigate the differential coding properties of these Fisher-optimal codes for fine and coarse discrimination. We find that the discrimination error for these codes does not decrease to zero with increasing population size, even in simple coarse discrimination tasks. Our results suggest that quite different population codes may be optimal for rapid decoding in cortical computations than those inferred from the optimization of Fisher information.

Orientation tuning in the visual cortex of 3-month-old human infants

Sensitivity to orientation is critical for making a whole and complete picture of the world. We measured the orientation tuning of mechanisms in the visual cortex of typically developing 3-month-olds and adults using a nonlinear analysis of the two-input steady-state Visually Evoked Potential (VEP). Two gratings, one a fixed test and the other a variable orientation masker were tagged with distinct temporal frequencies and the corresponding evoked responses were measured at the harmonics of the test and masker frequencies and at a frequency equal to the sum of the two stimulus frequencies. The magnitude of the sum frequency component depended strongly on the relative orientation of the test and masker in both infants and adults. The VEP tuning bandwidths of the 3-month-olds measured at the sum frequency were similar to those of adults, suggesting that behavioral immaturities in functions such as orientation discrimination and contour integration may result from other immaturities in long-range lateral projections or feedback mechanisms.

Characterizing perceptual performance at multiple discrimination precisions in external noise

Existing observer models developed for studies with the external noise paradigm are strictly applicable only to target detection or identification/discrimination of orthogonal target(s). We elaborated the perceptual template model (PTM) to account for contrast thresholds in identifying nonorthogonal targets. Full contrast psychometric functions were measured in an orientation identification task with four orientation differences across a wide range of external noise levels. We showed that observer performance can be modeled by the elaborated PTM with two templates that correspond to the two stimulus categories. Sampling efficiencies of the human observers were also estimated. The elaborated PTM provides a theoretical framework for characterizing joint feature and contrast sensitivity of human observers.

A role for the ventral premotor cortex beyond performance monitoring

Depending on the circumstances, decision making requires either comparing current sensory information with that showed recently or with that recovered from long-term memory (LTM). In both cases, to learn from past decisions and adapt future ones, memories and outcomes have to be available after the report of a decision. The ventral premotor cortex (PMv) is a good candidate for integrating memory traces and outcomes because it is involved in working-memory, decision-making, and encoding the outcomes. To test this hypothesis we recorded the extracellular unit activity while monkeys performed 2 variants of a visual discrimination task. In one task, the decision was based on the comparison of the orientation of a current stimulus with that of another stimulus recently shown. In the other task, the monkeys had to compare the current orientation of the stimulus with the correct one retrieved from LTM. Here, we report that when the task required retrieval of the stimulus and its use in the following trials, the neurons continue encoding this internal representation together with the outcomes after the monkey has emitted the motor response. However, this codification did not occur when the stimulus was shown recently and updated every trial. These results suggest that the PMv activity represents the information needed to evaluate the consequences of a decision. We interpret these results as evidence that the PMv plays a role in evaluating the outcomes that can serve to learn and thus adapt future decision to environmental demands.

Neural correlates of decisions and their outcomes in the ventral premotor cortex

Selection of the appropriate action in a changing environment involves a chain of events that goes from perception through decision to action and evaluation of the outcomes. What and where in the brain are the correlates of these events? The ventral premotor cortex (PMv) is a candidate because (1) it is involved in sensory transformations for visually guided actions and in perceptual decisions, and (2) it is connected with sensory, motor, and high-level cognitive areas related to performance monitoring. Therefore, we hypothesized that it would be the site for representing sensory perception for action and for evaluating the decision consequences. Trained monkeys were required to discriminate the orientation of two lines showed in sequence and separated by a delay. Monkeys compared the orientation of the second line with the memory trace of the first and communicated whether the second was to the left or to the right of the first. Here we show that the activity of PMv neurons reflected (1) the first stimuli and its memory trace during the delay and comparison periods, (2) its comparison with the second stimuli, including the strength of the evidence, and (3) the result of the discrimination (choice). After the monkeys reported the choice, there were neurons that only encoded the choices, others only the outcomes, and others the choices and outcomes together. The representation of task cues, decision variables, and their outcomes suggest a role of PMv as part of a supervisory network involved in shaping future behavior and in learning.

Application of penalized splines in analyzing neuronal data

Neuron experiments produce high-dimensional data structures. Therefore, application of smoothing techniques in the analysis of neuronal data from electrophysiological experiments has received considerable attention of late. We investigate the use of penalized splines in the analysis of neuronal data. This is first illustrated when interested in the temporal trend of a single neuron. An approach to investigate the maximal firing rate, based on the penalizedspline model is proposed. Determination of the time of maximal firing rate is based on non-linear optimization of the objective function with the corresponding confidence intervals constructed based on the first-order derivative function. To distinguish between the curves from different experimental conditions in a moment-by-moment sense, bias adjusted simulation-based simultaneous confidence bands leading to global inference in the time domain are constructed. The bands are an extension of the approach proposed by Ruppert et al. (2003). These methods are in a second step extended towards the analysis of a population of neurons via a marginal or population-averaged model.

Concept of uprightness in baboons: assessment with pictures of realistic scenes

How nonhuman primates process pictures of natural scenes or objects remains a matter of debates. This issue was addressed in the current research by questioning the processing of the canonical orientation of pictures in baboons. Two adult guinea baboons were trained to use an interactive key (IK) on a touch-screen to change the orientation of target pictures showing humans or quadruped mammals until upright. In experiment 1, both baboons successfully learned to use the IK when that key induced a 90 degrees rightward rotation of the picture, but post-training transfer of performance did not occur to novel pictures of natural scenes due to potential motor biases. In Experiment 2, a touch on IK randomly displayed the pictures in any of the four cardinal orientations. Baboons successfully learned the task, but transfer to novel pictures could only be demonstrated after they had been exposed to 360-480 pictures in that condition. Experiment 3 confirmed positive transfers to novel pictures, and showed that both the figure and background information controlled the behavior. Our research on baboons therefore demonstrates the development and use of an "upright" concept, and indicates that picture processing modes strongly depend on the subject's past experience with naturalistic pictorial stimuli.

Neural correlates of memory retrieval in the prefrontal cortex

Working memory includes short-term representations of information that were recently experienced or retrieved from long-term representations of sensory stimuli. Evidence is presented here that working memory activates the same dorsolateral prefrontal cortex neurons that: (a) maintained recently perceived visual stimuli; and (b) retrieved visual stimuli from long-term memory (LTM). Single neuron activity was recorded in the dorsolateral prefrontal cortex while trained monkeys discriminated between two orientated lines shown sequentially, separated by a fixed interstimulus interval. This visual task required the monkey to compare the orientation of the second line with the memory trace of the first and to decide the relative orientation of the second. When the behavioural task required the monkey to maintain in working memory a first stimulus that continually changed from trial to trial, the discharge in these cells was related to the parameters--the orientation--of the memorized item. Then, what the monkey had to recall from memory was manipulated by switching to another task in which the first stimulus was not shown, and had to be retrieved from LTM. The discharge rates of the same neurons also varied depending on the parameters of the memorized stimuli, and their response was progressively delayed as the monkey performed the task. These results suggest that working memory activates dorsolateral prefrontal cortex neurons that maintain parametrical visual information in short-term and LTM, and that the contents of working memory cannot be limited to what has recently happened in the sensory environment.

Bootstrap-based methods for testing factor-by-curve interactions in generalized additive models: assessing prefrontal cortex neural activity related to decision-making

In many situations the effect of a continuous covariate on response varies across groups defined by levels of a categorical variable. This paper addresses generalized additive models incorporating the so-called factor-by-curve interaction. A local scoring algorithm based on local linear kernel smoothers was used to estimate the model. Two different types of bootstrap-based procedures are proposed for testing interaction terms, namely, the likelihood ratio test, and a procedure based on an estimate of the interaction terms. Given the high computational cost involved, binning techniques were used to speed up computation in the estimation and testing processes. A simulation study was conducted to assess the validity of these bootstrap-based tests. This methodology was applied to studying prefrontal cortex neural activity associated with decision-making in monkeys. The proposed statistical procedure proved very useful in revealing the neural activity correlates of decision-making strategies adopted by monkeys in accordance with different behavioural tasks.

Categorical perception of orientation in monkeys

Structural and functional substrates of orientation processing in monkeys have been clarified. However, orientation perception in monkeys has not been fully studied. In this study, the cognitive mechanism that controls monkeys' perception of orientation was evaluated. After the monkeys were trained to discriminate between a cardinal and an oblique orientation (e.g., 0 degrees and 30 degrees), their perceptual mechanisms underlying orientation discrimination were tested by using six orientations, ranging from 0 degrees to 150 degrees, including ones used in the discrimination training. Generalization tests showed that the monkeys who were trained with cardinal orientations (e.g., 0 degrees) as positive stimuli generalized their responses to the other cardinal orientation (e.g., 90 degrees). Similarly, the monkeys who were trained with oblique orientations (e.g., 30 degrees) as positive stimuli generalized their responses to all other oblique orientations (e.g., 60 degrees, 120 degrees, and 150 degrees). These findings indicated that the monkeys abstracted the quality of the cardinal/oblique category from the physical features of orientation stimuli although they were not trained to do so. Such an abstraction also suggested a discrepancy between a continuously and orderly arranged cortical map and a discontinuously categorized perception of orientation. The present findings provide insight into the learning-correlated plasticity of cortical orientation preference.

Improvement of visual acuity by spatial cueing: a comparative study in human and non-human primates

This study investigated the influence of spatial cueing (valid/invalid/no cue) on visual discrimination in human and non-human primates. We employed a spatial resolution task which required the accurate discrimination of the orientation of a Landolt "C" ring. The C appeared as single target in specific retinal locations while subjects maintained fixation of a central fixation point. The minimal discernable size of the "C" (=acuity threshold) was determined as a function of cue condition, retinal eccentricity (3 degrees -15 degrees ), and stimulus onset asynchrony (SOA) (200-1100 ms). For both species, we found consistent benefits from spatial cueing with differences in absolute thresholds ranging from 6% to 25%. These differences increased with retinal eccentricity and decreased with longer SOAs. Further experiments performed with humans only, showed that the effect of spatial cueing on visual discrimination is independent of spatial uncertainty, i.e. the number of possible target locations (2 versus 4), but fades with longer target presentation times. From our results we draw the following conclusions. (i) Since sensory noise and spatial uncertainty was small in our tasks, spatial shifts of attention involve signal enhancement in both, human and non-human primates. (ii) The similarity of the results obtained for humans and macaque monkeys indicates that the latter may serve as a suitable model system in studies trying to tackle the neural underpinnings of attentional control. (iii) In order to elicit robust effects on visual discrimination by spatial shifts of attention, a paradigm comprising short SOAs (approximately 200 ms) and target presentation times (approximately 150 ms), and retinal eccentricities larger than approximately 9 degrees seems most promising.

Perceptual learning modulates sensory evoked response during vowel segregation

With practice, people become better at discriminating two similar stimuli, such as two sounds. The neural mechanisms that underlie this type of learning have been of interest to researchers investigating neural plasticity associated with learning and recovery of function following stroke. We utilized event related potentials (ERP) to study the neural substrates underlying auditory discrimination learning. Stimuli were five steady-state American English vowels. On each trial, participants were presented with a pair of vowels created by summing together the digital waveforms of two different vowels. Listeners were instructed to identify both vowels in the pair. ERPs were recorded during two sessions separated by 1 week. Half of the participants practised the discrimination task during the intervening week while the other half served as controls and did not receive any training. Trained listeners showed greater improvement in accuracy than untrained participants. In both groups, vowels generated N1 and P2 waves at the fronto-central and temporal scalp regions. The behavioral effects of training were paralleled by decreased N1 and P2 latencies as well as enhanced P2 amplitude in the trained compared with untrained listeners. The effects of training on sensory evoked responses are consistent with the proposal that perceptual learning is associated with changes in sensory cortices.

Flutter discrimination: neural codes, perception, memory and decision making

Recent studies combining psychophysical and neurophysiological experiments in behaving monkeys have provided new insights into how several cortical areas integrate efforts to solve a vibrotactile discrimination task. In particular, these studies have addressed how neural codes are related to perception, working memory and decision making in this model. The primary somatosensory cortex drives higher cortical areas where past and current sensory information are combined, such that a comparison of the two evolves into a behavioural decision. These and other observations in visual tasks indicate that decisions emerge from highly-distributed processes in which the details of a scheduled motor plan are gradually specified by sensory information.