Adaptation to Spiral Motion in Crowding Condition

Introspective inference counteracts perceptual distortion

Introspective agents can recognize the extent to which their internal perceptual experiences deviate from the actual states of the external world. This ability, also known as insight, is critically required for reality testing and is impaired in psychosis, yet little is known about its cognitive underpinnings. We develop a Bayesian modeling framework and a psychophysics paradigm to quantitatively characterize this type of insight while people experience a motion after-effect illusion. People can incorporate knowledge about the illusion into their decisions when judging the actual direction of a motion stimulus, compensating for the illusion (and often overcompensating). Furthermore, confidence, reaction-time, and pupil-dilation data all show signatures consistent with inferential adjustments in the Bayesian insight model. Our results suggest that people can question the veracity of what they see by making insightful inferences that incorporate introspective knowledge about internal distortions.

The Bouma law accounts for crowding in 50 observers

Crowding is the failure to recognize an object due to surrounding clutter. Our visual crowding survey measured 13 crowding distances (or "critical spacings") twice in each of 50 observers. The survey includes three eccentricities (0, 5, and 10 deg), four cardinal meridians, two orientations (radial and tangential), and two fonts (Sloan and Pelli). The survey also tested foveal acuity, twice. Remarkably, fitting a two-parameter model-the well-known Bouma law, where crowding distance grows linearly with eccentricity-explains 82% of the variance for all 13 × 50 measured log crowding distances, cross-validated. An enhanced Bouma law, with factors for meridian, crowding orientation, target kind, and observer, explains 94% of the variance, again cross-validated. These additional factors reveal several asymmetries, consistent with previous reports, which can be expressed as crowding-distance ratios: 0.62 horizontal:vertical, 0.79 lower:upper, 0.78 right:left, 0.55 tangential:radial, and 0.78 Sloan-font:Pelli-font. Across our observers, peripheral crowding is independent of foveal crowding and acuity. Evaluation of the Bouma factor, b (the slope of the Bouma law), as a biomarker of visual health would be easier if there were a way to compare results across crowding studies that use different methods. We define a standardized Bouma factor b' that corrects for differences from Bouma's 25 choice alternatives, 75% threshold criterion, and linearly symmetric flanker placement. For radial crowding on the right meridian, the standardized Bouma factor b' is 0.24 for this study, 0.35 for Bouma (1970), and 0.30 for the geometric mean across five representative modern studies, including this one, showing good agreement across labs, including Bouma's. Simulations, confirmed by data, show that peeking can skew estimates of crowding (e.g., greatly decreasing the mean or doubling the SD of log b). Using gaze tracking to prevent peeking, individual differences are robust, as evidenced by the much larger 0.08 SD of log b across observers than the mere 0.03 test-retest SD of log b measured in half an hour. The ease of measurement of crowding enhances its promise as a biomarker for dyslexia and visual health.

Emergence of crowding: The role of contrast and orientation salience

Crowding causes difficulties in judging attributes of an object surrounded by other objects. We investigated crowding for stimuli that isolated either S-cone or luminance mechanisms or combined them. By targeting different retinogeniculate mechanisms with contrast-matched stimuli, we aim to determine the earliest site at which crowding emerges. Discrimination was measured in an orientation judgment task where Gabor targets were presented parafoveally among flankers. In the first experiment, we assessed flanked and unflanked orientation discrimination thresholds for pure S-cone and achromatic stimuli and their combinations. In the second experiment, to capture individual differences, we measured unflanked detection and orientation sensitivity, along with performance under flanker interference for stimuli containing luminance only or combined with S-cone contrast. We confirmed that orientation sensitivity was lower for unflanked S-cone stimuli. When flanked, the pattern of results for S-cone stimuli was the same as for achromatic stimuli with comparable (i.e. low) contrast levels. We also found that flanker interference exhibited a genuine signature of crowding only when orientation discrimination threshold was reliably surpassed. Crowding, therefore, emerges at a stage that operates on signals representing task-relevant featural (here, orientation) information. Because luminance and S-cone mechanisms have very different spatial tuning properties, it is most parsimonious to conclude that crowding takes place at a neural processing stage after they have been combined.

Invisible Adaptation: The Effect of Awareness on the Strength of the Motion Aftereffect

The ability to process information despite the lack of perceptual awareness is one of the most fascinating aspects of the visual system. Such unconscious processing is often investigated using adaptation, where any presence of the former can be traced by its footprint on aftereffects following the latter. We have investigated the mechanisms of the motion aftereffect (MAE) using random dot displays of varying motion coherence as well as crowding to modulate both the physical as well as the perceptual strength of the adaptation stimulus. Perceptual strength was quantitatively measured as the performance in a forced-choice direction-discrimination task. A motion-nulling technique was used to quantitatively measure the strength of the MAE. We show that the strength of the dynamic MAE is independently influenced by both the physical stimulus strength as well as the subjective perceptual strength, with the effect of the former being more prominent than that of the latter. We further show that the MAE still persists under conditions of subthreshold perception. Our results suggest that perceptual awareness can influence the strength of visual processing, but the latter is not fully dependent on the former and can still take place at its partial or even total absence.

Asymmetries in flanker-target interference at different levels of number processing

Visual stimuli presented in peripheries can be barely recognized when they are surrounded by flankers (crowding). The target-flanker interference can be asymmetrical, and this asymmetry depends on a stimulus type. In particular, recognition of a letter or a number is more disturbed by the presence of a leftward flanker, reflecting the direction of reading. So far, such reading-related asymmetry has been observed with visual recognition tasks. In the following studies, we used numbers as stimuli to examine whether the leftward asymmetry in crowding extends to other levels of information processing, i.e. whether it is present when more abstract, semantic features are extracted. We presented participants with numerical triplets in the left or right visual field, and asked them to classify the middle number according to its magnitude (Experiment 1), physical characteristics (Experiment 2) or parity (Experiment 3). We observed that the leftward flanker interfered stronger with the target than the rightward flanker, but only when magnitude and physical characteristics were classified. Our findings suggest that the leftward asymmetry in crowding extends up to the semantic level of number processing, but only selectively, i.e. when a certain sort of information (magnitude) is extracted.

Multi-level Crowding and the Paradox of Object Recognition in Clutter

In everyday life, we are constantly surrounded by complex and cluttered scenes. In such cluttered environments, visual perception is primarily limited by crowding, the deleterious influence of nearby objects on object recognition. For the past several decades, visual crowding was assumed to occur at a single stage, only between low-level features or object parts, thus dismantling, destroying, or filtering object information. A large and converging body of evidence has demonstrated that this assumption is false: crowding occurs at multiple stages of visual analysis, and information passes through crowding at each of these stages. This converging empirical evidence points to a seeming paradox: crowding happens at multiple levels, which would seem to impair object recognition, and yet visual information at each of those levels is maintained intact and influences subsequent higher-level visual processing. Thus, while crowding impairs the access we have to visual information at many levels, it does not impair the representation of that information. The resolution of this paradox reveals how the visual system strikes a balance between the limits of object selection and the desire to represent multiple levels of visual information throughout cluttered scenes. Understanding crowding is therefore key to resolving the relationship between the richness of object and scene representations and the limits of conscious object recognition.

Crowding and the Furrow Illusion

A spot moves vertically across a large grating of oblique parallel lines. When viewed peripherally, the motion path looks oblique, close to the orientation of the background grating. Even when the grating's orientation is concealed by crowding, it can still deflect the spot's perceived motion path.

Action Recognition in a Crowded Environment

So far, action recognition has been mainly examined with small point-light human stimuli presented alone within a narrow central area of the observer's visual field. Yet, we need to recognize the actions of life-size humans viewed alone or surrounded by bystanders, whether they are seen in central or peripheral vision. Here, we examined the mechanisms in central vision and far periphery (40° eccentricity) involved in the recognition of the actions of a life-size actor (target) and their sensitivity to the presence of a crowd surrounding the target. In Experiment 1, we used an action adaptation paradigm to probe whether static or idly moving crowds might interfere with the recognition of a target's action (hug or clap). We found that this type of crowds whose movements were dissimilar to the target action hardly affected action recognition in central and peripheral vision. In Experiment 2, we examined whether crowd actions that were more similar to the target actions affected action recognition. Indeed, the presence of that crowd diminished adaptation aftereffects in central vision as wells as in the periphery. We replicated Experiment 2 using a recognition task instead of an adaptation paradigm. With this task, we found evidence of decreased action recognition accuracy, but this was significant in peripheral vision only. Our results suggest that the presence of a crowd carrying out actions similar to that of the target affects its recognition. We outline how these results can be understood in terms of high-level crowding effects that operate on action-sensitive perceptual channels.

Crowding, visual awareness, and their respective neural loci

In peripheral vision, object identification can be impeded when a target object is flanked by other objects. This phenomenon of crowding has been attributed to basic processes associated with image encoding by the visual system, but the neural origin of crowding is not known. Determining whether crowding depends on subjective awareness of the flankers can provide information on the neural origin of crowding. However, recent studies that manipulated flanker awareness have yielded conflicting results. In the current study, we suppressed flanker awareness with two methods: interocular suppression (IOS) and adaptation-induced blindness (AIB). We tested two different types of stimuli: gratings and letters. With IOS, we found that the magnitude of crowding increased as the number of physical flankers increased, even when the observers did not report seeing any of the flankers. In contrast, when flanker awareness was manipulated with AIB, the magnitude of crowding increased with the number of perceived flankers. Our results show that whether crowding is contingent on awareness of the flankers depends on the method used to suppress awareness. In addition, our results imply that the locus of crowding is upstream from the neural locus of IOS and close to or downstream from that of AIB. Neurophysiology and neuroimaging studies jointly implicate mid-to-high level visual processing stages for IOS, while direct evidence regarding the neural locus of AIB is limited. The most consistent interpretation of our empirical findings is to place the neural locus of crowding at an early cortical site, such as V1 or V2.

Action Congruency Influences Crowding When Discriminating Biological Motion Direction

Identification and discrimination of peripheral stimuli are often difficult when a few stimuli adjacent to the target are present (crowding). Our previous study showed that crowding occurs for walking direction discrimination of a biological motion stimulus. In the present study, we attempted to examine whether action congruency between the target and flankers would influence the crowding effect on biological motion stimuli. Each biological motion stimulus comprised one action (e.g., walking, throwing wastepaper, etc.) and was rotated in one of five directions around the vertical axis. In Experiment 1, observers discriminated between the directions of the target stimulus actions, which were surrounded by two flankers in the peripheral visual field. The crowding effect was stronger when the flankers performed the same action as the target and the directions differed. The congruency of action type enhanced the crowding effect in the direction-discrimination task. In Experiment 2, observers discriminated between action types of target stimuli. The crowding effect for the action-discrimination task was not modulated by the congruency of action direction. Thus, identical actions induced a larger crowding effect for action-direction discrimination, but congruent directions did not influence crowding for action-type discrimination. These results suggest that the processes involved in direction discrimination of biological motion are partially distinct from action discrimination processes.

Does Temporal Integration Occur for Unrecognizable Words in Visual Crowding?

Visual crowding—the inability to see an object when it is surrounded by flankers in the periphery—does not block semantic activation: unrecognizable words due to visual crowding still generated robust semantic priming in subsequent lexical decision tasks. Based on the previous finding, the current study further explored whether unrecognizable crowded words can be temporally integrated into a phrase. By showing one word at a time, we presented Chinese four-word idioms with either a congruent or incongruent ending word in order to examine whether the three preceding crowded words can be temporally integrated to form a semantic context so as to affect the processing of the ending word. Results from both behavioral (Experiment 1) and Event-Related Potential (Experiment 2 and 3) measures showed congruency effect in only the non-crowded condition, which does not support the existence of unconscious multi-word integration. Aside from four-word idioms, we also found that two-word (modifier + adjective combination) integration—the simplest kind of temporal semantic integration—did not occur in visual crowding (Experiment 4). Our findings suggest that integration of temporally separated words might require conscious awareness, at least under the timing conditions tested in the current study.

Effects of crowding and attention on high-levels of motion processing and motion adaptation

The motion after-effect (MAE) persists in crowding conditions, i.e., when the adaptation direction cannot be reliably perceived. The MAE originating from complex moving patterns spreads into non-adapted sectors of a multi-sector adapting display (i.e., phantom MAE). In the present study we used global rotating patterns to measure the strength of the conventional and phantom MAEs in crowded and non-crowded conditions, and when attention was directed to the adapting stimulus and when it was diverted away from the adapting stimulus. The results show that: (i) the phantom MAE is weaker than the conventional MAE, for both non-crowded and crowded conditions, and when attention was focused on the adapting stimulus and when it was diverted from it, (ii) conventional and phantom MAEs in the crowded condition are weaker than in the non-crowded condition. Analysis conducted to assess the effect of crowding on high-level of motion adaptation suggests that crowding is likely to affect the awareness of the adapting stimulus rather than degrading its sensory representation, (iii) for high-level of motion processing the attentional manipulation does not affect the strength of either conventional or phantom MAEs, neither in the non-crowded nor in the crowded conditions. These results suggest that high-level MAEs do not depend on attention and that at high-level of motion adaptation the effects of crowding are not modulated by attention.

The hierarchical sparse selection model of visual crowding

Because the environment is cluttered, objects rarely appear in isolation. The visual system must therefore attentionally select behaviorally relevant objects from among many irrelevant ones. A limit on our ability to select individual objects is revealed by the phenomenon of visual crowding: an object seen in the periphery, easily recognized in isolation, can become impossible to identify when surrounded by other, similar objects. The neural basis of crowding is hotly debated: while prevailing theories hold that crowded information is irrecoverable - destroyed due to over-integration in early stage visual processing - recent evidence demonstrates otherwise. Crowding can occur between high-level, configural object representations, and crowded objects can contribute with high precision to judgments about the "gist" of a group of objects, even when they are individually unrecognizable. While existing models can account for the basic diagnostic criteria of crowding (e.g., specific critical spacing, spatial anisotropies, and temporal tuning), no present model explains how crowding can operate simultaneously at multiple levels in the visual processing hierarchy, including at the level of whole objects. Here, we present a new model of visual crowding-the hierarchical sparse selection (HSS) model, which accounts for object-level crowding, as well as a number of puzzling findings in the recent literature. Counter to existing theories, we posit that crowding occurs not due to degraded visual representations in the brain, but due to impoverished sampling of visual representations for the sake of perception. The HSS model unifies findings from a disparate array of visual crowding studies and makes testable predictions about how information in crowded scenes can be accessed.

Global-motion aftereffect does not depend on awareness of the adapting motion direction

It has been shown that humans cannot perceive more than three directions from a multidirectional motion stimulus. However, it remains unknown whether adapting to such imperceptible motion directions could generate motion aftereffects (MAEs). A series of psychophysical experiments were conducted to address this issue. Using a display consisting of randomly oriented Gabors, we replicated previous findings that observers were unable to perceive the global directions embedded in a five-direction motion pattern. However, adapting to this multidirectional pattern induced both static and dynamic MAEs, despite the fact that observers were unaware of any global motion directions during adaptation. Furthermore, by comparing the strengths of the dynamic MAEs induced at different levels of motion processing, we found that spatial integration of local illusory signals per se was sufficient to produce a significant global MAE. These psychophysical results show that the generation of a directional global MAE does not require conscious perception of the global motion during adaptation.

Sustained invisibility through crowding and continuous flash suppression: a comparative review

The study of non-conscious vision benefits from several alternative methods that allow the suppression of an image from awareness. Here, we present and compare two of them that are particularly well-suited for creating sustained periods of invisibility, namely visual crowding and continuous flash suppression (CFS). In visual crowding, a peripheral image surrounded by similar flankers becomes impossible to discriminate. In CFS, an image presented to one eye becomes impossible to detect when rapidly changing patterns are presented to the other eye. After discussing the experimental specificities of each method, we give a comparative overview of the main empirical results derived from them, from the mere analysis of low-level features to the extraction of semantic contents. We conclude by proposing practical guidelines and future directions to obtain more quantitative and systematic measures of non-conscious processes under prolonged stimulation.

Semantic priming from crowded words

Vision in a cluttered scene is extremely inefficient. This damaging effect of clutter, known as crowding, affects many aspects of visual processing (e.g., reading speed). We examined observers' processing of crowded targets in a lexical decision task, using single-character Chinese words that are compact but carry semantic meaning. Despite being unrecognizable and indistinguishable from matched nonwords, crowded prime words still generated robust semantic-priming effects on lexical decisions for test words presented in isolation. Indeed, the semantic-priming effect of crowded primes was similar to that of uncrowded primes. These findings show that the meanings of words survive crowding even when the identities of the words do not, suggesting that crowding does not prevent semantic activation, a process that may have evolved in the context of a cluttered visual environment.

Directional remapping in tactile inter-finger apparent motion: a motion aftereffect study

Tactile motion provides critical information for perception and manipulation of objects in touch. Perceived directions of tactile motion are primarily defined in the environmental coordinate, which means they change drastically with body posture even when the same skin sensors are stimulated. Despite the ecological importance of this perceptual constancy, the sensory processing underlying tactile directional remapping remains poorly understood. The present study psychophysically investigated the mechanisms underlying directional remapping in human tactile motion processing by examining whether finger posture modulates the direction of the tactile motion aftereffect (MAE) induced by inter-finger apparent motions. We introduced conflicts in the adaptation direction between somatotopic and environmental spaces by having participants change their finger posture between adaptation and test phases. In a critical condition, they touched stimulators with crossed index and middle fingers during adaptation but with uncrossed fingers during tests. Since the adaptation effect was incongruent between the somatotopic and environmental spaces, the direction of the MAE reflects the coordinate of tactile motion processing. The results demonstrated that the tactile MAE was induced in accordance with the motion direction determined by the environmental rather than the somatotopic space. In addition, it was found that though the physical adaptation of the test fingers was not changed, the tactile MAE disappeared when the adaptation stimuli were vertically aligned or when subjective motion perception was suppressed during adaptation. We also found that the tactile MAE, measured with our procedure, did not transfer across different hands, which implies that the observed MAEs mainly reflect neural adaptations occurring within sensor-specific, tactile-specific processing. The present findings provide a novel behavioral method to analyze the neural representation for directional remapping of tactile motion within tactile sensory processing in the human brain.

Visual crowding: a fundamental limit on conscious perception and object recognition

Crowding, the inability to recognize objects in clutter, sets a fundamental limit on conscious visual perception and object recognition throughout most of the visual field. Despite how widespread and essential it is to object recognition, reading and visually guided action, a solid operational definition of what crowding is has only recently become clear. The goal of this review is to provide a broad-based synthesis of the most recent findings in this area, to define what crowding is and is not, and to set the stage for future work that will extend our understanding of crowding well beyond low-level vision. Here we define six diagnostic criteria for what counts as crowding, and further describe factors that both escape and break crowding. All of these lead to the conclusion that crowding occurs at multiple stages in the visual hierarchy.

The initial interactions underlying binocular rivalry require visual awareness

Current theories of binocular vision suggest that the neural processes that resolve interocular conflict do not involve a single brain region but occur at multiple stages of visual processing. Here, using an adaptation paradigm, we explore the initial mechanisms involved in selecting a stimulus for perceptual dominance during binocular rivalry. When one or both eyes briefly viewed an adapting grating stimulus prior to the presentation of the adapting grating to one eye and an orthogonal, non-adapted grating to the other eye, participants more often reported perceptual dominance of the non-adapted grating. Crowding reduced awareness of the adapting grating. On trials in which subjects were unaware of the orientation of the adaptor grating, there was no effect of the adaptor on perceived dominance during rivalry; participants were just as likely to report dominance of the adapted or non-adapted grating. This implies that the initial events in binocular rivalry involve later stages of visual processing.

Shared attentional resources for global and local motion processing

One of the most important aspects of visual attention is its flexibility; our attentional "window" can be tuned to different spatial scales, allowing us to perceive large-scale global patterns and local features effortlessly. We investigated whether the perception of global and local motion competes for a common attentional resource. Subjects viewed arrays of individual moving Gabors that group to produce a global motion percept when subjects attended globally. When subjects attended locally, on the other hand, they could identify the direction of individual uncrowded Gabors. Subjects were required to devote their attention toward either scale of motion or divide it between global and local scales. We measured direction discrimination as a function of the validity of a precue, which was varied in opposite directions for global and local motion such that when the precue was valid for global motion, it was invalid for local motion and vice versa. There was a trade-off between global and local motion thresholds, such that increasing the validity of precues at one spatial scale simultaneously reduced thresholds at that spatial scale but increased thresholds at the other spatial scale. In a second experiment, we found a similar pattern of results for static-oriented Gabors: Attending to local orientation information impaired the subjects' ability to perceive globally defined orientation and vice versa. Thresholds were higher for orientation compared to motion, however, suggesting that motion discrimination in the first experiment was not driven by orientation information alone but by motion-specific processing. The results of these experiments demonstrate that a shared attentional resource flexibly moves between different spatial scales and allows for the perception of both local and global image features, whether these features are defined by motion or orientation.

Second-order motion without awareness: passive adaptation to second-order motion produces a motion aftereffect

Although second-order motion may be detected by early and automatic mechanisms, some models suggest that perceiving second-order motion requires higher-order processes, such as feature or attentive tracking. These types of attentionally mediated mechanisms could explain the motion aftereffect (MAE) perceived in dynamic displays after adapting to second-order motion. Here we tested whether there is a second-order MAE in the absence of attention or awareness. If awareness of motion, mediated by high-level or top-down mechanisms, is necessary for the second-order MAE, then there should be no measurable MAE if the ability to detect directionality is impaired during adaptation. To eliminate the subject's ability to detect directionality of the adapting stimulus, a second-order drifting Gabor was embedded in a dense array of additional crowding Gabors. We found that a significant MAE was perceived even after adaptation to second-order motion in crowded displays that prevented awareness. The results demonstrate that second-order motion can be passively coded in the absence of awareness and without top-down attentional control.

Seeing Invisible Motion: A Human fMRI Study

A common view about visual consciousness is that it could arise when and where activity reaches some higher level of processing along the cortical hierarchy. Reports showing that activity in striate cortex can be dissociated from awareness , whereas the latter modulates activity in higher areas , point in this direction. In the specific case of visual motion, a central, "perceptual" role has been assigned to area V5: several human and monkey studies have shown V5 activity to correlate with the motion percept. Here we show that activity in this and other higher cortical areas can be also dissociated from perception and follow the physical stimulus instead. The motion information in a peripheral grating modulated fMRI responses, despite being invisible to human volunteers: under crowding conditions , areas V3A, V5, and parietal cortex still showed increased activity when the grating was moving compared to when it was flickering. We conclude that stimulus-specific activation of higher cortical areas does not necessarily result in awareness of the underlying stimulus.