Limited translation invariance of human visual pattern recognition

Eye movements disrupt EEG alpha-band coding of behaviorally relevant and irrelevant spatial locations held in working memory

Oscillations in the alpha frequency band (∼8-12 Hz) of the human electroencephalogram play an important role in supporting selective attention to visual items and maintaining their spatial locations in working memory (WM). Recent findings suggest that spatial information maintained in alpha is modulated by interruptions to continuous visual input, such that attention shifts, eye closure, and backward masking of the encoded item cause reconstructed representations of remembered locations to become degraded. Here, we investigated how another common visual disruption-eye movements-modulates reconstructions of behaviorally relevant and irrelevant item locations held in WM. Participants completed a delayed estimation task, where they encoded and recalled either the location or color of an object after a brief retention period. During retention, participants either fixated at the center or executed a sequence of eye movements. Electroencephalography (EEG) was recorded at the scalp and eye position was monitored with an eye tracker. Inverted encoding modeling (IEM) was applied to reconstruct location-selective responses across multiple frequency bands during encoding and retention. Location-selective responses were successfully reconstructed from alpha activity during retention where participants fixated at the center, but these reconstructions were disrupted during eye movements. Recall performance decreased during eye-movements conditions but remained largely intact, and further analyses revealed that under specific task conditions, it was possible to reconstruct retained location information from lower frequency bands (1-4 Hz) during eye movements. These results suggest that eye movements disrupt maintained spatial information in alpha in a manner consistent with other acute interruptions to continuous visual input, but this information may be represented in other frequency bands.NEW & NOTEWORTHY Neural oscillations in the alpha frequency band support selective attention to visual items and maintenance of their spatial locations in human working memory. Here, we investigate how eye movements disrupt representations of item locations held in working memory. Although it was not possible to recover item locations from alpha during eye movements, retained location information could be recovered from select lower frequency bands. This suggests that during eye movements, stored spatial information may be represented in other frequencies.

Visual stream connectivity predicts assessments of image quality

Despite extensive study of early vision, new and unexpected mechanisms continue to be identified. We introduce a novel formal treatment of the psychophysics of image similarity, derived directly from straightforward connectivity patterns in early visual pathways. The resulting differential geometry formulation is shown to provide accurate and explanatory accounts of human perceptual similarity judgments. The direct formal predictions are then shown to be further improved via simple regression on human behavioral reports, which in turn are used to construct more elaborate hypothesized neural connectivity patterns. It is shown that the predictive approaches introduced here outperform a standard successful published measure of perceived image fidelity; moreover, the approach provides clear explanatory principles of these similarity findings.

Convolutional Neural Networks as a Model of the Visual System: Past, Present, and Future

Convolutional neural networks (CNNs) were inspired by early findings in the study of biological vision. They have since become successful tools in computer vision and state-of-the-art models of both neural activity and behavior on visual tasks. This review highlights what, in the context of CNNs, it means to be a good model in computational neuroscience and the various ways models can provide insight. Specifically, it covers the origins of CNNs and the methods by which we validate them as models of biological vision. It then goes on to elaborate on what we can learn about biological vision by understanding and experimenting on CNNs and discusses emerging opportunities for the use of CNNs in vision research beyond basic object recognition.

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements

Visual translation tolerance refers to our capacity to recognize objects over a wide range of different retinal locations. Although translation is perhaps the simplest spatial transform that the visual system needs to cope with, the extent to which the human visual system can identify objects at previously unseen locations is unclear, with some studies reporting near complete invariance over 10 degrees and other reporting zero invariance at 4 degrees of visual angle. Similarly, there is confusion regarding the extent of translation tolerance in computational models of vision, as well as the degree of match between human and model performance. Here, we report a series of eye-tracking studies (total N = 70) demonstrating that novel objects trained at one retinal location can be recognized at high accuracy rates following translations up to 18 degrees. We also show that standard deep convolutional neural networks (DCNNs) support our findings when pretrained to classify another set of stimuli across a range of locations, or when a global average pooling (GAP) layer is added to produce larger receptive fields. Our findings provide a strong constraint for theories of human vision and help explain inconsistent findings previously reported with convolutional neural networks (CNNs).

Scale and translation-invariance for novel objects in human vision

Though the range of invariance in recognition of novel objects is a basic aspect of human vision, its characterization has remained surprisingly elusive. Here we report tolerance to scale and position changes in one-shot learning by measuring recognition accuracy of Korean letters presented in a flash to non-Korean subjects who had no previous experience with Korean letters. We found that humans have significant scale-invariance after only a single exposure to a novel object. The range of translation-invariance is limited, depending on the size and position of presented objects. To understand the underlying brain computation associated with the invariance properties, we compared experimental data with computational modeling results. Our results suggest that to explain invariant recognition of objects by humans, neural network models should explicitly incorporate built-in scale-invariance, by encoding different scale channels as well as eccentricity-dependent representations captured by neurons' receptive field sizes and sampling density that change with eccentricity. Our psychophysical experiments and related simulations strongly suggest that the human visual system uses a computational strategy that differs in some key aspects from current deep learning architectures, being more data efficient and relying more critically on eye-movements.

Alpha-Band Activity Reveals Spontaneous Representations of Spatial Position in Visual Working Memory

An emerging view suggests that spatial position is an integral component of working memory (WM), such that non-spatial features are bound to locations regardless of whether space is relevant [1, 2]. For instance, past work has shown that stimulus position is spontaneously remembered when non-spatial features are stored. Item recognition is enhanced when memoranda appear at the same location where they were encoded [3-5], and accessing non-spatial information elicits shifts of spatial attention to the original position of the stimulus [6, 7]. However, these findings do not establish that a persistent, active representation of stimulus position is maintained in WM because similar effects have also been documented following storage in long-term memory [8, 9]. Here we show that the spatial position of the memorandum is actively coded by persistent neural activity during a non-spatial WM task. We used a spatial encoding model in conjunction with electroencephalogram (EEG) measurements of oscillatory alpha-band (8-12 Hz) activity to track active representations of spatial position. The position of the stimulus varied trial to trial but was wholly irrelevant to the tasks. We nevertheless observed active neural representations of the original stimulus position that persisted throughout the retention interval. Further experiments established that these spatial representations are dependent on the volitional storage of non-spatial features rather than being a lingering effect of sensory energy or initial encoding demands. These findings provide strong evidence that online spatial representations are spontaneously maintained in WM-regardless of task relevance-during the storage of non-spatial features.

Visual memory performance for color depends on spatiotemporal context

Performance on visual short-term memory for features has been known to depend on stimulus complexity, spatial layout, and feature context. However, with few exceptions, memory capacity has been measured for abruptly appearing, single-instance displays. In everyday life, objects often have a spatiotemporal history as they or the observer move around. In three experiments, we investigated the effect of spatiotemporal history on explicit memory for color. Observers saw a memory display emerge from behind a wall, after which it disappeared again. The test display then emerged from either the same side as the memory display or the opposite side. In the first two experiments, memory improved for intermediate set sizes when the test display emerged in the same way as the memory display. A third experiment then showed that the benefit was tied to the original motion trajectory and not to the display object per se. The results indicate that memory for color is embedded in a richer episodic context that includes the spatiotemporal history of the display.

The positional-specificity effect reveals a passive-trace contribution to visual short-term memory

The positional-specificity effect refers to enhanced performance in visual short-term memory (VSTM) when the recognition probe is presented at the same location as had been the sample, even though location is irrelevant to the match/nonmatch decision. We investigated the mechanisms underlying this effect with behavioral and fMRI studies of object change-detection performance. To test whether the positional-specificity effect is a direct consequence of active storage in VSTM, we varied memory load, reasoning that it should be observed for all objects presented in a sub-span array of items. The results, however, indicated that although robust with a memory load of 1, the positional-specificity effect was restricted to the second of two sequentially presented sample stimuli in a load-of-2 experiment. An additional behavioral experiment showed that this disruption wasn’t due to the increased load per se, because actively processing a second object – in the absence of a storage requirement – also eliminated the effect. These behavioral findings suggest that, during tests of object memory, position-related information is not actively stored in VSTM, but may be retained in a passive tag that marks the most recent site of selection. The fMRI data were consistent with this interpretation, failing to find location-specific bias in sustained delay-period activity, but revealing an enhanced response to recognition probes that matched the location of that trial’s sample stimulus.

Learning not to feel: reshaping the resolution of tactile perception

We asked whether biased feedback during training could cause human subjects to lose perceptual acuity in a vibrotactile frequency discrimination task. Prior to training, we determined each subject's vibration frequency discrimination capacity on one fingertip, the Just Noticeable Difference (JND). Subjects then received 850 trials in which they performed a same/different judgment on two vibrations presented to that fingertip. They gained points whenever their judgment matched the computer-generated feedback on that trial. Feedback, however, was biased: the probability per trial of “same” feedback was drawn from a normal distribution with standard deviation twice as wide as the subject's JND. After training, the JND was significantly widened: stimulus pairs previously perceived as different were now perceived as the same. The widening of the JND extended to the untrained hand, indicating that the decrease in resolution originated in non-topographic brain regions. In sum, the acuity of subjects' sensory-perceptual systems shifted in order to match the feedback received during training.

Does visual subordinate-level categorisation engage the functionally defined fusiform face area?

Functional magnetic resonance imaging was used to compare brain activation associated with basic-level (e.g. bird) and subordinate-level (e.g. eagle) processing for both visual and semantic judgements. We localised the putative face area for 11 subjects, who also performed visual matching judgements for pictures and aurally presented words. The middle fusiform and occipital gyri were recruited for subordinate minus basic visual judgements, reflecting additional perceptual processing. When the face area was localised individually for each subject, analyses in the middle fusiform gyri revealed that subordinate-level processing activated the individuals face area. We propose that what is unique about the way faces engage this region is the focal spatial distribution of the activation rather than the recruitment of the face per se. Eight subjects also performed semantic judgements on aurally presented basic- and subordinate-level words. The parahippocampal gyri were more activated for subordinate-level than basic-level semantic judgements. Finally, the left posterior inferior temporal gyrus was activated for subordinate-level judgements, both visual and semantic, as well as during passive viewing of faces.

The sensory components of high-capacity iconic memory and visual working memory

Early visual memory can be split into two primary components: a high-capacity, short-lived iconic memory followed by a limited-capacity visual working memory that can last many seconds. Whereas a large number of studies have investigated visual working memory for low-level sensory features, much research on iconic memory has used more “high-level” alphanumeric stimuli such as letters or numbers. These two forms of memory are typically examined separately, despite an intrinsic overlap in their characteristics. Here, we used a purely sensory paradigm to examine visual short-term memory for 10 homogeneous items of three different visual features (color, orientation and motion) across a range of durations from 0 to 6 s. We found that the amount of information stored in iconic memory is smaller for motion than for color or orientation. Performance declined exponentially with longer storage durations and reached chance levels after ∼2 s. Further experiments showed that performance for the 10 items at 1 s was contingent on unperturbed attentional resources. In addition, for orientation stimuli, performance was contingent on the location of stimuli in the visual field, especially for short cue delays. Overall, our results suggest a smooth transition between an automatic, high-capacity, feature-specific sensory-iconic memory, and an effortful “lower-capacity” visual working memory.

Peripheral vision and pattern recognition: a review

We summarize the various strands of research on peripheral vision and relate them to theories of form perception. After a historical overview, we describe quantifications of the cortical magnification hypothesis, including an extension of Schwartz's cortical mapping function. The merits of this concept are considered across a wide range of psychophysical tasks, followed by a discussion of its limitations and the need for non-spatial scaling. We also review the eccentricity dependence of other low-level functions including reaction time, temporal resolution, and spatial summation, as well as perimetric methods. A central topic is then the recognition of characters in peripheral vision, both at low and high levels of contrast, and the impact of surrounding contours known as crowding. We demonstrate how Bouma's law, specifying the critical distance for the onset of crowding, can be stated in terms of the retinocortical mapping. The recognition of more complex stimuli, like textures, faces, and scenes, reveals a substantial impact of mid-level vision and cognitive factors. We further consider eccentricity-dependent limitations of learning, both at the level of perceptual learning and pattern category learning. Generic limitations of extrafoveal vision are observed for the latter in categorization tasks involving multiple stimulus classes. Finally, models of peripheral form vision are discussed. We report that peripheral vision is limited with regard to pattern categorization by a distinctly lower representational complexity and processing speed. Taken together, the limitations of cognitive processing in peripheral vision appear to be as significant as those imposed on low-level functions and by way of crowding.

High-level visual object representations are constrained by position

It is widely assumed that high-level visual object representations are position-independent (or invariant). While there is sensitivity to position in high-level object-selective cortex, position and object identity are thought to be encoded independently in the population response such that position information is available across objects and object information is available across positions. Contrary to this view, we show, with both behavior and neuroimaging, that visual object representations are position-dependent (tied to limited portions of the visual field). Behaviorally, we show that the effect of priming an object was greatly reduced with any change in position (within- or between-hemifields), indicating nonoverlapping representations of the same object across different positions. Furthermore, using neuroimaging, we show that object-selective cortex is not only highly sensitive to object position but also the ability to differentiate objects based on its response is greatly reduced across different positions, consistent with the observed behavior and the receptive field properties observed in macaque object-selective neurons. Thus, even at the population level, the object information available in response of object-selective cortex is constrained by position. We conclude that even high-level visual object representations are position-dependent.

(In) sensitivity to spatial distortion in natural scenes

The perception of object structure in the natural environment is remarkably stable under large variation in image size and projection, especially given our insensitivity to spatial position outside the fovea. Sensitivity to periodic spatial distortions that were introduced into one quadrant of gray-scale natural images was measured in a 4AFC task. Observers were able to detect the presence of distortions in unfamiliar images even though they did not significantly affect the amplitude spectrum. Sensitivity depended on the spatial period of the distortion and on the image structure at the location of the distortion. The results suggest that the detection of distortion involves decisions made in the late stages of image perception and is based on an expectation of the typical structure of natural scenes.

Spatially global representations in human primary visual cortex during working memory maintenance

Recent studies suggest that visual features are stored in working memory (WM) via sensory recruitment or sustained stimulus-specific patterns of activity in cortical regions that encode memoranda. One important question concerns the spatial extent of sensory recruitment. One possibility is that sensory recruitment is restricted to neurons that are retinotopically mapped to the positions occupied by the remembered items. Alternatively, specific feature values could be represented via a spatially global recruitment of neurons that encode the remembered feature, regardless of the retinotopic position of the remembered stimulus. Here, we evaluated these alternatives by requiring subjects to remember the orientation of a grating presented in the left or right visual field. Functional magnetic resonance imaging and multivoxel pattern analysis were then used to examine feature-specific activations in early visual regions during memory maintenance. Activation patterns that discriminated the remembered feature were found in regions of contralateral visual cortex that corresponded to the retinotopic position of the remembered item, as well as in ipsilateral regions that were not retinotopically mapped to the position of the stored stimulus. These results suggest that visual details are held in WM through a spatially global recruitment of early sensory cortex. This spatially global recruitment may enhance memory precision by facilitating robust population coding of the stored information.

Does learned shape selectivity in inferior temporal cortex automatically generalize across retinal position?

Biological visual systems have the remarkable ability to recognize objects despite confounding factors such as object position, size, pose, and lighting. In primates, this ability likely results from neuronal responses at the highest stage of the ventral visual stream [inferior temporal cortex (IT)] that signal object identity while tolerating these factors. However, for even the apparently simplest IT tolerance ("invariance"), tolerance to object position on the retina, little is known about how this feat is achieved. One possibility is that IT position tolerance is innate in that discriminatory power for newly learned objects automatically generalizes across position. Alternatively, visual experience plays a role in developing position tolerance. To test these ideas, we trained adult monkeys in a difficult object discrimination task in which their visual experience with novel objects was restricted to a single retinal position. After training, we recorded the spiking activity of an unbiased population of IT neurons and found that it contained significantly greater selectivity among the newly learned objects at the experienced position compared with a carefully matched, non-experienced position. Interleaved testing with other objects shows that this difference cannot be attributed to a bias in spatial attention or neuronal sampling. We conclude from these results that, at least under some conditions, full transfer of IT neuronal selectivity across retinal position is not automatic. This finding raises the possibility that visual experience plays a role in building neuronal tolerance in the ventral visual stream and the recognition abilities it supports.

Unsupervised natural experience rapidly alters invariant object representation in visual cortex

Object recognition is challenging because each object produces myriad retinal images. Responses of neurons from the inferior temporal cortex (IT) are selective to different objects, yet tolerant ("invariant") to changes in object position, scale, and pose. How does the brain construct this neuronal tolerance? We report a form of neuronal learning that suggests the underlying solution. Targeted alteration of the natural temporal contiguity of visual experience caused specific changes in IT position tolerance. This unsupervised temporal slowness learning (UTL) was substantial, increased with experience, and was significant in single IT neurons after just 1 hour. Together with previous theoretical work and human object perception experiments, we speculate that UTL may reflect the mechanism by which the visual stream builds and maintains tolerant object representations.

Retinotopy of the face aftereffect

Physiological results for the size of face-specific units in inferotemporal cortex (IT) support an extraordinarily large range of possible sizes--from 2.5 degrees to 30 degrees or more. We use a behavioral test of face-specific aftereffects to measure the face analysis regions and find a coarse retinotopy consistent with receptive fields of intermediate size (10 degrees -12 degrees at 3 degrees eccentricity). In the first experiment, observers were adapted to a single face at 3 degrees from fixation. A test (a morph of the face and its anti-face) was then presented at different locations around fixation and subjects classified it as face or anti-face. The face aftereffect (FAE) was not constant at all test locations--it dropped to half its maximum value for tests 5 degrees from the adapting location. Simultaneous adaptation to both a face and its anti-face, placed at opposite locations across fixation, produced two separate regions of opposite aftereffects. However, with four stimuli, faces alternating with anti-faces equally spaced around fixation, the FAE was greatly reduced at all locations, implying a fairly coarse localization of the aftereffect. In the second experiment, observers adapted to a face and its anti-face presented either simultaneously or in alternation. Results showed that the simultaneous presentation of a face and its anti-face leads to stronger FAEs than sequential presentation, suggesting that face processing has a dynamic nature and its region of analysis is sharpened when there is more than one face in the scene. In the final experiment, a face and two anti-face flankers with different spatial offsets were presented during adaptation and the FAE was measured at the face location. Results showed that FAE at the face location was inhibited more as the distance of anti-face flankers to the face stimulus was reduced. This confirms the spatial extent of face analysis regions in a test with a fixed number of stimuli where only distance varied.

Category learning induces position invariance of pattern recognition across the visual field

Human object recognition is considered to be largely invariant to translation across the visual field. However, the origin of this invariance to positional changes has remained elusive, since numerous studies found that the ability to discriminate between visual patterns develops in a largely location-specific manner, with only a limited transfer to novel visual field positions. In order to reconcile these contradicting observations, we traced the acquisition of categories of unfamiliar grey-level patterns within an interleaved learning and testing paradigm that involved either the same or different retinal locations. Our results show that position invariance is an emergent property of category learning. Pattern categories acquired over several hours at a fixed location in either the peripheral or central visual field gradually become accessible at new locations without any position-specific feedback. Furthermore, categories of novel patterns presented in the left hemifield are distinctly faster learnt and better generalized to other locations than those learnt in the right hemifield. Our results suggest that during learning initially position-specific representations of categories based on spatial pattern structure become encoded in a relational, position-invariant format. Such representational shifts may provide a generic mechanism to achieve perceptual invariance in object recognition.

What Is the Optimal Architecture for Visual Information Routing?

Analyzing the design of networks for visual information routing is an underconstrained problem due to insufficient anatomical and physiological data. We propose here optimality criteria for the design of routing networks. For a very general architecture, we derive the number of routing layers and the fanout that minimize the required neural circuitry. The optimal fanout l is independent of network size, while the number k of layers scales logarithmically (with a prefactor below 1), with the number n of visual resolution units to be routed independently. The results are found to agree with data of the primate visual system.

'Breaking' position-invariant object recognition

While it is often assumed that objects can be recognized irrespective of where they fall on the retina, little is known about the mechanisms underlying this ability. By exposing human subjects to an altered world where some objects systematically changed identity during the transient blindness that accompanies eye movements, we induced predictable object confusions across retinal positions, effectively 'breaking' position invariance. Thus, position invariance is not a rigid property of vision but is constantly adapting to the statistics of the environment.

Remembering "what" brings along "where" in visual working memory

Does a behavioral and anatomical division exist between spatial and object working memory? In this article, we explore this question by testing human participants in simple visual working memory tasks. We compared a condition in which there was no location change with conditions in which absolute location change and absolute plus relative location change were manipulated. The results showed that object memory was influenced by memory for relative but not for absolute location information. Furthermore, we demonstrated that relative space can be specified by a salient surrounding box or by distractor objects with no touching surfaces. Verbal memory was not influenced by any type of spatial information. Taken together, these results indicate that memory for "where" influences memory for "what." We propose that there is an asymmetry in memory according to which object memory always contains location information.

Visual shape recognition with contour propagation

A neural architecture is presented that encodes the visual space inside and outside of a shape. The contours of a shape are propagated across an excitable neuronal map and fed through a set of orientation columns, thus creating a pattern which can be viewed as a vector field. This vector field is then burned as synaptic, directional connections into a propagation map, which will serve as a "shape map". The shape map identifies its own, preferred input when it is translated, deformed, scaled and fragmented, and discriminates other shapes very distinctively. Encoding visual space is much more efficient for shape recognition than determining contour geometry only.

The interaction of shape- and location-based priming in object categorisation: Evidence for a hybrid “what+where” representation stage

The relationship between part shape and location is not well elucidated in current theories of object recognition. Here we investigated the role of shape and location of object parts on recognition, using a classification priming paradigm with novel 3D objects. In Experiment 1, the relative displacement of two parts comprising the prime gradually reduced the priming effect. In Experiment 2, presenting single-part primes in locations progressively different from those in the composite target had no effect on priming. In Experiment 3, manipulating the relative position of composite prime and target strongly affected priming. Finally, in Experiment 4 the relative displacement of single-part primes and composite targets did influence response time. Together, these findings are best interpreted in terms of a hybrid theory, according to which conjunctions of shape and location are explicitly represented at some stage of visual object processing.

Investigations into the organization of information in sensory cortex

One might take the exploration of sensory cortex in the first decades of the last century as the opening chapter of modern neuroscience. The combined approaches of (i) measuring effects of restricted ablation on functional capacities, both in the clinic and the laboratory, together with (ii) anatomical investigations of cortical lamination, arealization, and connectivity, and (iii) the early physiological probing of sensory representations, led to a fundamental body of knowledge that remains relevant to this day. In our time, there can be little doubt that its organization as a mosaic of columnar modules is the pervasive functional property of mammalian sensory cortex [Brain 120 (1997) 701]. If one accepts the assertion that columns and maps must improve the functioning of the brain (why else would they be the very hallmark of neocortex?), then the inevitable question is: exactly what advantages do they permit? In this review of our recent presentation at the workshop on Homeostasis, plasticity and learning at the Institut Henri Poincaré, we will outline a systematic approach to investigating the role of modular, map-like cortical organization in the processing of sensory information. We survey current evidence concerning the functional significance of cortical maps and modules, arguing that sensory cortex is involved not solely in the online processing of afferent data, but also in the storage and retrieval of information. We also show that the topographic framework of primary sensory cortex renders the encoding of sensory information efficient, fast and reliable.

Anterior inferotemporal neurons of monkeys engaged in object recognition can be highly sensitive to object retinal position

Visual object recognition is computationally difficult because changes in an object's position, distance, pose, or setting may cause it to produce a different retinal image on each encounter. To robustly recognize objects, the primate brain must have mechanisms to compensate for these variations. Although these mechanisms are poorly understood, it is thought that they elaborate neuronal representations in the inferotemporal cortex that are sensitive to object form but substantially invariant to other image variations. This study examines this hypothesis for image variation resulting from changes in object position. We studied the effect of small differences (+/-1.5 degrees ) in the retinal position of small (0.6 degrees wide) visual forms on both the behavior of monkeys trained to identify those forms and the responses of 146 anterior IT (AIT) neurons collected during that behavior. Behavioral accuracy and speed were largely unaffected by these small changes in position. Consistent with previous studies, many AIT responses were highly selective for the forms. However, AIT responses showed far greater sensitivity to retinal position than predicted from their reported receptive field (RF) sizes. The median AIT neuron showed a approximately 60% response decrease between positions within +/-1.5 degrees of the center of gaze, and 52% of neurons were unresponsive to one or more of these positions. Consistent with previous studies, each neuron's rank order of target preferences was largely unaffected across position changes. Although we have not yet determined the conditions necessary to observe this marked position sensitivity in AIT responses, we rule out effects of spatial-frequency content, eye movements, and failures to include the RF center. To reconcile this observation with previous studies, we hypothesize that either AIT position sensitivity strongly depends on object size or that position sensitivity is sharpened by extensive visual experience at fixed retinal positions or by the presence of flanking distractors.

The topography of tactile working memory

To investigate the contribution of topographically organized brain areas to tactile working memory, we asked human subjects to compare the frequency of two vibrations presented to the same fingertip or to different fingertips. The vibrations ranged from 14 to 24 Hz and were separated by a retention interval of variable length. For intervals <1 sec, subjects were accurate when both vibrations were delivered to the same fingertip but were less accurate when the two vibrations were delivered to different fingertips. For 1 or 2 sec intervals, subjects performed equally well when comparing vibrations delivered either to the same finger or to corresponding fingers on opposite hands, but they performed poorly when the vibrations were delivered to distant fingers on either hand. These results suggest that working memory resides within a topographic framework. As a further test, we performed an experiment in which the two comparison vibrations were presented to the same fingertip but an interference vibration was presented during the retention interval. The interpolated vibration disrupted accuracy most when delivered to the same finger as the comparison vibrations and had progressively less effect when delivered to more distant fingers. We conclude that topographically organized regions of somatosensory cortex contribute to tactile working memory, possibly by holding the memory trace across the retention interval. One stimulus can be accurately compared with the memory of a previous stimulus if they engage overlapping representations, but activation of the common cortical territory by an interpolated stimulus can disrupt the memory trace.

Imperfect invariance to object translation in the discrimination of complex shapes

The positional specificity of short-term visual memory for a variety of 3-D shapes was investigated in a series of 'same'/'different' discrimination experiments, with computer-rendered stimuli displayed either at the same or at different locations in the visual field. For animal-like shapes, we found complete translation invariance, regardless of the interstimulus similarity, and irrespective of direction and size of the displacement (experiments 1 and 2). Invariance to translation was obtained also with animal-like stimuli that had been 'scrambled' by randomizing the relative locations of their parts (experiment 3). The invariance broke down when the stimuli were made to differ in their composition, but not in the shapes of the corresponding parts (experiments 4 and 5). We interpret this pattern of findings in the context of several current theories of recognition, focusing in particular on the issue of the representation of object structure.

The topography of tactile learning in humans

The spatial distribution of learned information within a sensory system can shed light on the brain mechanisms of sensory-perceptual learning. It has been argued that tactile memories are stored within a somatotopic framework in monkeys and rats but within a widely distributed network in humans. We have performed experiments to reexamine the spread of tactile learning across the fingertips. In all experiments, subjects were trained to use one fingertip to discriminate between two stimuli. Experiment 1 required identification of vibration frequency, experiment 2 punctate pressure, and experiment 3 surface roughness. After learning to identify the stimuli reliably, subjects were tested with the trained fingertip, its first and second neighbors on the same hand, and the three corresponding fingertips on the opposite hand. As expected, for all stimulus types, subjects showed retention of learning with the trained fingertip. However, the transfer beyond the trained fingertip varied according to the stimulus type. For vibration, learning did not transfer to other fingertips. For both pressure and roughness stimuli, there was limited transfer, dictated by topographic distance; subjects performed well with the first neighbor of the trained finger and with the finger symmetrically opposite the trained one. These results indicate that tactile learning is organized within a somatotopic framework, reconciling the findings in humans with those in other species. The differential distribution of tactile memory according to stimulus type suggests that the information is stored in stimulus-specific somatosensory cortical fields, each characterized by a unique receptive field organization, feature selectivity, and callosal connectivity.

Learning to see: experience and attention in primary visual cortex

The response properties of neurons in primary sensory cortices remain malleable throughout life. The existence of such plasticity, and the characteristics of a form of implicit learning known as perceptual learning, suggest that changes in primary sensory cortex may mediate learning. We explored whether modification of the functional properties of primary visual cortex (V1) accompanies perceptual learning. Basic receptive field properties, such as location, size and orientation selectivity, were unaffected by perceptual training, and visual topography (as measured by magnification factor) was indistinguishable between trained and untrained animals. On the other hand, the influence of contextual stimuli placed outside the receptive field showed a change consistent with the trained discrimination. Furthermore, this property showed task dependence, only being manifest when the animal was performing the trained discrimination.

Ipsilateral and contralateral transfer of tactile learning

We examined the spatial organization of perceptual learning in a cortex-dependent task. Rats learned a tactile task using four whiskers on one side of the snout, all others being clipped. These trained whiskers were then clipped and prosthetic whiskers were attached. Subsequent performance was found to be determined by the location of the prosthetic whiskers. There was partial transfer of learning to neighbouring whisker positions. In addition, there was partial transfer of learning to whisker positions on the other side of the snout, but only if the prosthetic whiskers were symmetrically opposite the trained whiskers. These findings suggest that neural changes underlying perceptual learning are distributed according to the topographic organization of the sensory cortical map.

Display symmetry affects positional specificity in same-different judgment of pairs of novel visual patterns

Deciding whether a novel visual pattern is the same as or different from a previously seen reference is easier if both stimuli are presented to the same rather than to different locations in the field of view (Foster & Kahn (1985). Biological Cybernetics, 51, 305-312; Dill & Fahle (1998). Perception and Psychophysics, 60, 65-81). We investigated whether pattern symmetry interacts with the effect of translation. Patterns were small dot-clouds which could be mirror-symmetric or asymmetric. Translations were displacements of the visual pattern symmetrically across the fovea, either left-right or above-below. We found that same-different discriminations were worse (less accurate and slower) for translated patterns, to an extent which in general was not influenced by pattern symmetry, or pattern orientation, or direction of displacement. However, if the displaced pattern was a mirror image of the original one (along the trajectory of the displacement), then performance was largely invariant to translation. Both positional specificity and its reduction in symmetric displays may be explained by location-specific pre-processing of the visual input.

The role of visual field position in pattern-discrimination learning

Invariance of object recognition to translation in the visual field is a fundamental property of human pattern vision. In three experiments we investigated this capability by training subjects to distinguish between random checkerboard stimuli. We show that the improvement of discrimination performance does not transfer across the visual field if learning is restricted to a particular location in the retinal image. Accuracy after retinal translation shows no sign of decay over time and remains at the same level it had at the beginning of the training. It is suggested that in two-dimensional translation invariance-as in three-dimensional rotation invariance-the human visual system is relying on memory-intensive rather than computation-intensive processes. Multiple position- and stimulus-specific learning events may be required before recognition is independent of retinal location.

Task difficulty and the specificity of perceptual learning

Practising simple visual tasks leads to a dramatic improvement in performing them. This learning is specific to the stimuli used for training. We show here that the degree of specificity depends on the difficulty of the training conditions. We find that the pattern of specificities maps onto the pattern of receptive field selectivities along the visual pathway. With easy conditions, learning generalizes across orientation and retinal position, matching the spatial generalization of higher visual areas. As task difficulty increases, learning becomes more specific with respect to both orientation and position, matching the fine spatial retinotopy exhibited by lower areas. Consequently, we enjoy the benefits of learning generalization when possible, and of fine grain but specific training when necessary. The dynamics of learning show a corresponding feature. Improvement begins with easy cases (when the subject is allowed long processing times) and only subsequently proceeds to harder cases. This learning cascade implies that easy conditions guide the learning of hard ones. Taken together, the specificity and dynamics suggest that learning proceeds as a countercurrent along the cortical hierarchy. Improvement begins at higher generalizing levels, which, in turn, direct harder-condition learning to the subdomain of their lower-level inputs. As predicted by this reverse hierarchy model, learning can be effective using only difficult trials, but on condition that learning onset has previously been enabled. A single prolonged presentation suffices to initiate learning. We call this single-encounter enabling effect 'eureka'.