The Y Cell Visual Pathway Implements a Demodulating Nonlinearity

Added load increases the peak frequency of intermuscular coherence

Cortical motor neuron activity appears to drive lower motor neurons through two distinct frequency bands: the β range (15-30 Hz) during weak muscle contractions and γ range (30-50 Hz) during strong contractions. It is unknown whether the frequency of cortical drive shifts continuously or abruptly between the β and γ frequency bands as contraction strength changes. Intermuscular coherence (IMC) between synergistic arm muscles was used to assess how the frequency of common neuronal drive shifts with increasing contraction strength. Muscle activity was recorded by surface electromyography (EMG) from the biceps and brachioradialis in nine healthy adults performing 30-second isometric holds with added loads. IMC was calculated across the two muscle groups during the isometric contraction. Significant IMC was present in the 20 to 50 Hz range with all loads. Repeated measures ANOVA show the peak frequency of IMC increased significantly when load was added, from a peak of 32.7 Hz with no added load, to 35.3 Hz, 35.7 Hz, and 36.3 Hz with three-, five-, and ten-pound loads respectively. An increase in IMC frequency occurs in response to added load, suggesting that cortical drive functions over a range of frequencies as a function of an isometric contraction against load.

Temporal interference stimulation disrupts spike timing in the primate brain

Electrical stimulation can regulate brain activity, producing clear clinical benefits, but focal and effective neuromodulation often requires surgically implanted electrodes. Recent studies argue that temporal interference (TI) stimulation may provide similar outcomes non-invasively. During TI, scalp electrodes generate multiple electrical fields in the brain, modulating neural activity only at their intersection. Despite considerable enthusiasm for this approach, little empirical evidence demonstrates its effectiveness, especially under conditions suitable for human use. Here, using single-neuron recordings in non-human primates, we establish that TI reliably alters the timing, but not the rate, of spiking activity. However, we show that TI requires strategies-high carrier frequencies, multiple electrodes, and amplitude-modulated waveforms-that also limit its effectiveness. Combined, these factors make TI 80 % weaker than other forms of non-invasive brain stimulation. Although unlikely to cause widespread neuronal entrainment, TI may be ideal for disrupting pathological oscillatory activity, a hallmark of many neurological disorders.

Characterization of extracellular spike waveforms recorded in wallaby primary visual cortex

Extracellular recordings were made from 642 units in the primary visual cortex (V1) of a highly visual marsupial, the Tammar wallaby. The receptive field (RF) characteristics of the cells were objectively estimated using the non-linear input model (NIM), and these were correlated with spike shapes. We found that wallaby cortical units had 68% regular spiking (RS), 12% fast spiking (FS), 4% triphasic spiking (TS), 5% compound spiking (CS) and 11% positive spiking (PS). RS waveforms are most often associated with recordings from pyramidal or spiny stellate cell bodies, suggesting that recordings from these cell types dominate in the wallaby cortex. In wallaby, 70-80% of FS and RS cells had orientation selective RFs and had evenly distributed linear and nonlinear RFs. We found that 47% of wallaby PS units were non-orientation selective and they were dominated by linear RFs. Previous studies suggest that the PS units represent recordings from the axon terminals of non-orientation selective cells originating in the lateral geniculate nucleus (LGN). If this is also true in wallaby, as strongly suggested by their low response latencies and bursty spiking properties, the results suggest that significantly more neurons in wallaby LGN are already orientation selective. In wallaby, less than 10% of recorded spikes had triphasic (TS) or sluggish compound spiking (CS) waveforms. These units had a mixture of orientation selective and non-oriented properties, and their cellular origins remain difficult to classify.

Behavioral signatures of Y-like neuronal responses in human vision

Retinal ganglion cells initiating the magnocellular/Y-cell visual pathways respond nonlinearly to high spatial frequencies (SFs) and temporal frequencies (TFs). This nonlinearity is implicated in the processing of contrast modulation (CM) stimuli in cats and monkeys, but its contribution to human visual perception is not well understood. Here, we evaluate human psychophysical performance for CM stimuli, consisting of a high SF grating carrier whose contrast is modulated by a low SF sinewave envelope. Subjects reported the direction of motion of CM envelopes or luminance modulation (LM) gratings at different eccentricities. The performance on SF (for LMs) or carrier SF (for CMs) was measured for different TFs (LMs) or carrier TFs (CMs). The best performance for LMs was at lower TFs and SFs, decreasing systematically with eccentricity. However, performance with CMs was bandpass with carrier SF, largely independent of carrier TF, and at the highest carrier TF (20 Hz) decreased minimally with eccentricity. Since the nonlinear subunits of Y-cells respond better at higher TFs compared to the linear response components and respond best at higher SFs that are relatively independent of eccentricity, these results suggest that behavioral tasks employing CM stimuli might reveal nonlinear contributions of retinal Y-like cells to human perception.

Lower Baseline Variability Gives Rise to Lower Detection Thresholds in Midbrain than Hindbrain Electrosensory Neurons

Understanding how the brain decodes sensory information to give rise to behaviour remains an important problem in systems neuroscience. Across various sensory modalities (e.g. auditory, visual), the time-varying contrast of natural stimuli has been shown to carry behaviourally relevant information. However, it is unclear how such information is actually decoded by the brain to evoke perception and behaviour. Here we investigated how midbrain electrosensory neurons respond to weak contrasts in the electrosensory system of the weakly electric fish Apteronotus leptorhynchus. We found that these neurons displayed lower detection thresholds than their afferent hindbrain electrosensory neurons. Further analysis revealed that the lower detection thresholds of midbrain neurons were not due to increased sensitivity to the stimulus. Rather, these were due to the fact that midbrain neurons displayed lower variability in their firing activities in the absence of stimulation, which is due to lower firing rates. Our results suggest that midbrain neurons play an active role towards enabling the detection of weak stimulus contrasts, which in turn leads to perception and behavioral responses.

Differentiation of Theta Visual Motion from Fourier Motion Requires LC16 and R18C12 Neurons in Drosophila

Many animals perceive features of higher-order visual motion that are beyond the spatiotemporal correlations of luminance defined in first-order motion. Although the neural mechanisms of first-order motion detection have become understood in recent years, those underlying higher-order motion perception remain unclear. Here, we established a paradigm to assess the detection of theta motion—a type of higher-order motion—in freely walking Drosophila. Behavioral screening using this paradigm identified two clusters of neurons in the central brain, designated as R18C12, which were required for perception of theta motion but not for first-order motion. Furthermore, theta motion-activated R18C12 neurons were structurally and functionally located downstream of visual projection neurons in lobula, lobula columnar cells LC16, which activated R18C12 neurons via interactions of acetylcholine (ACh) and muscarinic acetylcholine receptors (mAChRs). The current study provides new insights into LC neurons and the neuronal mechanisms underlying visual information processing in complex natural scenes.

•

Perception of theta motion requires LC16 and R18C12 neurons

•

R18C12 neurons are activated by theta motion

•

R18C12 neurons form synaptic connections with LC16 neurons

•

LC16 neurons activate R18C12 neurons through ACh acting on mAChR

Unraveling nonlinear electrophysiologic processes in the human visual system with full dimension spectral analysis

Natural sensory signals have nonlinear structures dynamically composed of the carrier frequencies and the variation of the amplitude (i.e., envelope). How the human brain processes the envelope information is still poorly understood, largely due to the conventional analysis failing to quantify it directly. Here, we used a recently developed method, Holo-Hilbert spectral analysis, and steady-state visually evoked potential collected using electroencephalography (EEG) recordings to investigate how the human visual system processes the envelope of amplitude-modulated signals, in this case with a 14 Hz carrier and a 2 Hz envelope. The EEG results demonstrated that in addition to the fundamental stimulus frequencies, 4 Hz amplitude modulation residing in 14 Hz carrier and a broad range of carrier frequencies covering from 8 to 32 Hz modulated by 2 Hz amplitude modulation are also found in the two-dimensional frequency spectrum, which have not yet been recognized before. The envelope of the stimulus is also found to dominantly modulate the response to the incoming signal. The findings thus reveal that the electrophysiological response to amplitude-modulated stimuli is more complex than could be revealed by, for example, Fourier analysis. This highlights the dynamics of neural processes in the visual system.

DC shifts, high frequency oscillations, ripples and fast ripples in relation to the seizure onset zone

Efforts to improve epilepsy surgery outcomes have led to increased interest in the study of electroencephalographic oscillations outside the conventional EEG bands. These include fast activity above the gamma band, known as high frequency oscillations (HFOs), and infraslow activity (ISA) below the delta band, sometimes referred to as direct current (DC) or ictal baseline shifts (IBS). HFOs in particular have been extensively studied as potential biomarkers for epileptogenic tissue in light of evidence showing that resection of brain tissue containing HFOs is associated with good surgical outcomes. Not all HFOs are conclusively pathological, however, as they can be recorded in nonepileptic tissue and induced by cognitive, visual, or motor tasks. Consequently, efforts to distinguish between pathological and physiological HFOs have identified several traits specific to pathological HFOs, such as coupling with interictal spikes, association with delta waves, and stereotypical morphologies. On the opposite end of the EEG spectrum, sub-delta oscillations have been shown to co-localize with the seizure onset zones (SOZ) and appear in a narrower spatial distribution than activity in the conventional EEG frequency bands. In this report, we review studies that implicate HFOs and ISA in ictogenesis and discuss current limitations such as inter-observer variability and poor standardization of recording techniques. Furthermore, we propose that HFOs and ISA should be analyzed in addition to activity in the conventional EEG band during intracranial presurgical EEG monitoring to identify the best possible surgical margin.

Real-time experimental control using network-based parallel processing

Modern neuroscience research often requires the coordination of multiple processes such as stimulus generation, real-time experimental control, as well as behavioral and neural measurements. The technical demands required to simultaneously manage these processes with high temporal fidelity is a barrier that limits the number of labs performing such work. Here we present an open-source, network-based parallel processing framework that lowers this barrier. The Real-Time Experimental Control with Graphical User Interface (REC-GUI) framework offers multiple advantages: (i) a modular design that is agnostic to coding language(s) and operating system(s) to maximize experimental flexibility and minimize researcher effort, (ii) simple interfacing to connect multiple measurement and recording devices, (iii) high temporal fidelity by dividing task demands across CPUs, and (iv) real-time control using a fully customizable and intuitive GUI. We present applications for human, non-human primate, and rodent studies which collectively demonstrate that the REC-GUI framework facilitates technically demanding, behavior-contingent neuroscience research.

Editorial note

This article has been through an editorial process in which the authors decide how to respond to the issues raised during peer review. The Reviewing Editor's assessment is that all the issues have been addressed (see decision letter).

Weakly electric fish distinguish between envelope stimuli arising from different behavioral contexts

Understanding how sensory information is processed by the brain in order to give rise to behavior remains poorly understood in general. Here, we investigated the behavioral responses of the weakly electric fish Apteronotus albifrons to stimuli arising from different contexts, by measuring changes in the electric organ discharge (EOD) frequency. Specifically, we focused on envelopes, which can arise either because of movement (i.e. motion envelopes) or because of interactions between the electric fields of three of more fish (i.e. social envelopes). Overall, we found that the animal's EOD frequency effectively tracked the detailed time course of both motion and social envelopes. In general, behavioral sensitivity (i.e. gain) decreased while phase lag increased with increasing envelope and carrier frequency. However, changes in gain and phase lag as a function of changes in carrier frequency were more prominent for motion than for social envelopes in general. Importantly, we compared behavioral responses to motion and social envelopes with similar characteristics. Although behavioral sensitivities were similar, we observed an increased response lag for social envelopes, primarily for low carrier frequencies. Thus, our results imply that the organism can, based on behavioral responses, distinguish envelope stimuli resulting from movement from those that instead result from social interactions. We discuss the implications of our results for neural coding of envelopes and propose that behavioral responses to motion and social envelopes are mediated by different neural circuits in the brain.

Descending pathways generate perception of and neural responses to weak sensory input

Natural sensory stimuli frequently consist of a fast time-varying waveform whose amplitude or contrast varies more slowly. While changes in contrast carry behaviorally relevant information necessary for sensory perception, their processing by the brain remains poorly understood to this day. Here, we investigated the mechanisms that enable neural responses to and perception of low-contrast stimuli in the electrosensory system of the weakly electric fish Apteronotus leptorhynchus. We found that fish reliably detected such stimuli via robust behavioral responses. Recordings from peripheral electrosensory neurons revealed stimulus-induced changes in firing activity (i.e., phase locking) but not in their overall firing rate. However, central electrosensory neurons receiving input from the periphery responded robustly via both phase locking and increases in firing rate. Pharmacological inactivation of feedback input onto central electrosensory neurons eliminated increases in firing rate but did not affect phase locking for central electrosensory neurons in response to low-contrast stimuli. As feedback inactivation eliminated behavioral responses to these stimuli as well, our results show that it is changes in central electrosensory neuron firing rate that are relevant for behavior, rather than phase locking. Finally, recordings from neurons projecting directly via feedback to central electrosensory neurons revealed that they provide the necessary input to cause increases in firing rate. Our results thus provide the first experimental evidence that feedback generates both neural and behavioral responses to low-contrast stimuli that are commonly found in the natural environment.

Feedback input from more central to more peripheral brain areas is found ubiquitously in the central nervous system of vertebrates. In this study, we used a combination of electrophysiological, behavioral, and pharmacological approaches to reveal a novel function for feedback pathways in generating neural and behavioral responses to weak sensory input in the weakly electric fish. We first determined that weak sensory input gives rise to responses that are phase locked in both peripheral sensory neurons and in the central neurons that are their downstream targets. However, central neurons also responded to weak sensory inputs that were not relayed via a feedforward input from the periphery, because complete inactivation of the feedback pathway abolished increases in firing rate but not the phase locking in response to weak sensory input. Because such inactivation also abolished the behavioral responses, our results show that the increases in firing rate in central neurons, and not the phase locking, are decoded downstream to give rise to perception. Finally, we discovered that the neurons providing feedback input were also activated by weak sensory input, thereby offering further evidence that feedback is necessary to elicit increases in firing rate that are needed for perception.

Nonlinear Y-Like Receptive Fields in the Early Visual Cortex: An Intermediate Stage for Building Cue-Invariant Receptive Fields from Subcortical Y Cells

Many of the neurons in early visual cortex are selective for the orientation of boundaries defined by first-order cues (luminance) as well as second-order cues (contrast, texture). The neural circuit mechanism underlying this selectivity is still unclear, but some studies have proposed that it emerges from spatial nonlinearities of subcortical Y cells. To understand how inputs from the Y-cell pathway might be pooled to generate cue-invariant receptive fields, we recorded visual responses from single neurons in cat Area 18 using linear multielectrode arrays. We measured responses to drifting and contrast-reversing luminance gratings as well as contrast modulation gratings. We found that a large fraction of these neurons have nonoriented responses to gratings, similar to those of subcortical Y cells: they respond at the second harmonic (F2) to high-spatial frequency contrast-reversing gratings and at the first harmonic (F1) to low-spatial frequency drifting gratings ("Y-cell signature"). For a given neuron, spatial frequency tuning for linear (F1) and nonlinear (F2) responses is quite distinct, similar to orientation-selective cue-invariant neurons. Also, these neurons respond to contrast modulation gratings with selectivity for the carrier (texture) spatial frequency and, in some cases, orientation. Their receptive field properties suggest that they could serve as building blocks for orientation-selective cue-invariant neurons. We propose a circuit model that combines ON- and OFF-center cortical Y-like cells in an unbalanced push-pull manner to generate orientation-selective, cue-invariant receptive fields.

SIGNIFICANCE STATEMENT

A significant fraction of neurons in early visual cortex have specialized receptive fields that allow them to selectively respond to the orientation of boundaries that are invariant to the cue (luminance, contrast, texture, motion) that defines them. However, the neural mechanism to construct such versatile receptive fields remains unclear. Using multielectrode recording, we found a large fraction of neurons in early visual cortex with receptive fields not selective for orientation that have spatial nonlinearities like those of subcortical Y cells. These are strong candidates for building cue-invariant orientation-selective neurons; we present a neural circuit model that pools such neurons in an imbalanced "push-pull" manner, to generate orientation-selective cue-invariant receptive fields.

Phase-Dependent Interactions in Visual Cortex to Combinations of First- and Second-Order Stimuli

A fundamental task of the visual system is to extract figure-ground boundaries between objects, which are often defined, not only by differences in luminance, but also by "second-order" contrast or texture differences. Responses of cortical neurons to both first- and second-order patterns have been studied extensively, but only for responses to either type of stimulus in isolation. Here, we examined responses of visual cortex neurons to the spatial relationship between superimposed periodic luminance modulation (LM) and contrast modulation (CM) stimuli, the contrasts of which were adjusted to give equated responses when presented alone. Extracellular single-unit recordings were made in area 18 of the cat, the neurons of which show responses to CM and LM stimuli very similar to those in primate area V2 (Li et al., 2014). Most neurons showed a significant dependence on the relative phase of the combined LM and CM patterns, with a clear overall optimal response when they were approximately phase aligned. The degree of this phase preference, and the contributions of suppressive and/or facilitatory interactions, varied considerably from one neuron to another. Such phase-dependent and phase-invariant responses were evident in both simple- and complex-type cells. These results place important constraints on any future model of the underlying neural circuitry for second-order responses. The diversity in the degree of phase dependence between LM and CM stimuli that we observed could help to disambiguate different kinds of boundaries in natural scenes.

SIGNIFICANCE STATEMENT

Many visual cortex neurons exhibit orientation-selective responses to boundaries defined by differences either in luminance or in texture contrast. Previous studies have examined responses to either type of boundary in isolation, but here we measured systematically responses of cortical neurons to the spatial relationship between superimposed periodic luminance-modulated (LM) and contrast-modulated (CM) stimuli with contrasts adjusted to give equated responses. We demonstrate that neuronal responses to these compound stimuli are highly dependent on the relative phase between the LM and CM components. Diversity in the degree of such phase dependence could help to disambiguate different kinds of boundaries in natural scenes, for example, those arising from surface reflectance changes or from illumination gradients such as shading or shadows.

Envelope statistics of self-motion signals experienced by human subjects during everyday activities: Implications for vestibular processing

There is accumulating evidence that the brain's neural coding strategies are constrained by natural stimulus statistics. Here we investigated the statistics of the time varying envelope (i.e. a second-order stimulus attribute that is related to variance) of rotational and translational self-motion signals experienced by human subjects during everyday activities. We found that envelopes can reach large values across all six motion dimensions (~450 deg/s for rotations and ~4 G for translations). Unlike results obtained in other sensory modalities, the spectral power of envelope signals decreased slowly for low (< 2 Hz) and more sharply for high (>2 Hz) temporal frequencies and thus was not well-fit by a power law. We next compared the spectral properties of envelope signals resulting from active and passive self-motion, as well as those resulting from signals obtained when the subject is absent (i.e. external stimuli). Our data suggest that different mechanisms underlie deviation from scale invariance in rotational and translational self-motion envelopes. Specifically, active self-motion and filtering by the human body cause deviation from scale invariance primarily for translational and rotational envelope signals, respectively. Finally, we used well-established models in order to predict the responses of peripheral vestibular afferents to natural envelope stimuli. We found that irregular afferents responded more strongly to envelopes than their regular counterparts. Our findings have important consequences for understanding the coding strategies used by the vestibular system to process natural second-order self-motion signals.

Neural Processing of Second-Order Motion in the Suprasylvian Cortex of the Cat

Neuronal responses to second-order motion, that is, to spatiotemporal variations of texture or contrast, have been reported in several cortical areas of mammals, including the middle-temporal (MT) area in primates. In this study, we investigated whether second-order responses are present in the cat posteromedial lateral suprasylvian (PMLS) cortex, a possible homolog of the primate area MT. The stimuli used were luminance-based sine-wave gratings (first-order) and contrast-modulated carrier stimuli (second-order), which consisted of a high-spatial-frequency static grating (carrier) whose contrast was modulated by a low-spatial-frequency drifting grating (envelope). Results indicate that most PMLS neurons responded to second-order motion and for the vast majority of cells, first- and second-order preferred directions were conserved. However, responses to second-order stimuli were significantly reduced when compared to those evoked by first-order gratings. Circular variance was increased for second-order stimuli, indicating that PMLS direction selectivity was weaker for this type of stimulus. Finally, carrier orientation selectivity was either absent or very broad and had no influence on the envelope's orientation selectivity. In conclusion, our data show that PMLS neurons exhibit similar first- and second-order response profiles and that, akin primate area MT cells, they perform a form-cue invariant analysis of motion signals.

Encoding and estimation of first- and second-order binocular disparity in natural images

•

First- and second-order responses to natural binocular images are correlated.

•

Second-order mechanisms can improve the accuracy of disparity estimation.

•

Second-order mechanisms can extend the depth range of binocular stereopsis.

The first stage of processing of binocular information in the visual cortex is performed by mechanisms that are bandpass-tuned for spatial frequency and orientation. Psychophysical and physiological evidence have also demonstrated the existence of second-order mechanisms in binocular processing, which can encode disparities that are not directly accessible to first-order mechanisms. We compared the responses of first- and second-order binocular filters to natural images. We found that the responses of the second-order mechanisms are to some extent correlated with the responses of the first-order mechanisms, and that they can contribute to increasing both the accuracy, and depth range, of binocular stereopsis.

Coding stimulus amplitude by correlated neural activity

While correlated activity is observed ubiquitously in the brain, its role in neural coding has remained controversial. Recent experimental results have demonstrated that correlated but not single-neuron activity can encode the detailed time course of the instantaneous amplitude (i.e., envelope) of a stimulus. These have furthermore demonstrated that such coding required and was optimal for a nonzero level of neural variability. However, a theoretical understanding of these results is still lacking. Here we provide a comprehensive theoretical framework explaining these experimental findings. Specifically, we use linear response theory to derive an expression relating the correlation coefficient to the instantaneous stimulus amplitude, which takes into account key single-neuron properties such as firing rate and variability as quantified by the coefficient of variation. The theoretical prediction was in excellent agreement with numerical simulations of various integrate-and-fire type neuron models for various parameter values. Further, we demonstrate a form of stochastic resonance as optimal coding of stimulus variance by correlated activity occurs for a nonzero value of noise intensity. Thus, our results provide a theoretical explanation of the phenomenon by which correlated but not single-neuron activity can code for stimulus amplitude and how key single-neuron properties such as firing rate and variability influence such coding. Correlation coding by correlated but not single-neuron activity is thus predicted to be a ubiquitous feature of sensory processing for neurons responding to weak input.

Coding of envelopes by correlated but not single-neuron activity requires neural variability

Understanding how the brain processes sensory information is often complicated by the fact that neurons exhibit trial-to-trial variability in their responses to stimuli. Indeed, the role of variability in sensory coding is still highly debated. Here, we examined how variability influences neural responses to naturalistic stimuli consisting of a fast time-varying waveform (i.e., carrier or first order) whose amplitude (i.e., envelope or second order) varies more slowly. Recordings were made from fish electrosensory and monkey vestibular sensory neurons. In both systems, we show that correlated but not single-neuron activity can provide detailed information about second-order stimulus features. Using a simple mathematical model, we made the strong prediction that such correlation-based coding of envelopes requires neural variability. Strikingly, the performance of correlated activity at predicting the envelope was similarly optimally tuned to a nonzero level of variability in both systems, thereby confirming this prediction. Finally, we show that second-order sensory information can only be decoded if one takes into account joint statistics when combining neural activities. Our results thus show that correlated but not single-neural activity can transmit information about the envelope, that such transmission requires neural variability, and that this information can be decoded. We suggest that envelope coding by correlated activity is a general feature of sensory processing that will be found across species and systems.

Gravity influences the visual representation of object tilt in parietal cortex

Sensory systems encode the environment in egocentric (e.g., eye, head, or body) reference frames, creating inherently unstable representations that shift and rotate as we move. However, it is widely speculated that the brain transforms these signals into an allocentric, gravity-centered representation of the world that is stable and independent of the observer's spatial pose. Where and how this representation may be achieved is currently unknown. Here we demonstrate that a subpopulation of neurons in the macaque caudal intraparietal area (CIP) visually encodes object tilt in nonegocentric coordinates defined relative to the gravitational vector. Neuronal responses to the tilt of a visually presented planar surface were measured with the monkey in different spatial orientations (upright and rolled left/right ear down) and then compared. This revealed a continuum of representations in which planar tilt was encoded in a gravity-centered reference frame in approximately one-tenth of the comparisons, intermediate reference frames ranging between gravity-centered and egocentric in approximately two-tenths of the comparisons, and in an egocentric reference frame in less than half of the comparisons. Altogether, almost half of the comparisons revealed a shift in the preferred tilt and/or a gain change consistent with encoding object orientation in nonegocentric coordinates. Through neural network modeling, we further show that a purely gravity-centered representation of object tilt can be achieved directly from the population activity of CIP-like units. These results suggest that area CIP may play a key role in creating a stable, allocentric representation of the environment defined relative to an "earth-vertical" direction.

Reliability-dependent contributions of visual orientation cues in parietal cortex

Creating accurate 3D representations of the world from 2D retinal images is a fundamental task for the visual system. However, the reliability of different 3D visual signals depends inherently on viewing geometry, such as how much an object is slanted in depth. Human perceptual studies have correspondingly shown that texture and binocular disparity cues for object orientation are combined according to their slant-dependent reliabilities. Where and how this cue combination occurs in the brain is currently unknown. Here, we search for neural correlates of this property in the macaque caudal intraparietal area (CIP) by measuring slant tuning curves using mixed-cue (texture + disparity) and cue-isolated (texture or disparity) planar stimuli. We find that texture cues contribute more to the mixed-cue responses of CIP neurons that prefer larger slants, consistent with theoretical and psychophysical results showing that the reliability of texture relative to disparity cues increases with slant angle. By analyzing responses to binocularly viewed texture stimuli with conflicting texture and disparity information, some cells that are sensitive to both cues when presented in isolation are found to disregard one of the cues during cue conflict. Additionally, the similarity between texture and mixed-cue responses is found to be greater when this cue conflict is eliminated by presenting the texture stimuli monocularly. The present findings demonstrate reliability-dependent contributions of visual orientation cues at the level of the CIP, thus revealing a neural correlate of this property of human visual perception.

Form-cue invariant second-order neuronal responses to contrast modulation in primate area V2

A fundamental task of the visual system is to extract figure-ground boundaries between images of objects, which in natural scenes are often defined not only by luminance differences but also by "second-order" contrast or texture differences. Responses to contrast modulation (CM) and other second-order stimuli have been extensively studied in human psychophysics, but the neuronal substrates of second-order responses in nonhuman primates remain poorly understood. In this study, we have recorded single neurons in area V2 of macaque monkeys, using both CM patterns as well as conventional luminance modulation (LM) gratings. CM stimuli were constructed from stationary sine wave grating carrier patterns, which were modulated by drifting envelope gratings of a lower spatial frequency. We found approximately one-third of visually responsive V2 neurons responded to CM stimuli with a pronounced selectivity to carrier spatial frequencies, and often orientations, that were clearly outside the neurons' passbands for LM gratings. These neurons were "form-cue invariant" in that their tuning to CM envelope spatial frequency and orientation was very similar to that for LM gratings. Neurons were tuned to carrier spatial frequencies that were typically 2-4 octaves higher than their optimal envelope spatial frequencies, similar to results from human psychophysics. These results are distinct from CM responses arising from surround suppression, but could be understood in terms of a filter-rectify-filter model. Such neurons could provide a functionally useful and explicit representation of segmentation boundaries as well as a plausible neural substrate for human perception of second-order boundaries.

Orientation-cue invariant population responses to contrast-modulated and phase-reversed contour stimuli in macaque V1 and V2

Visual scenes can be readily decomposed into a variety of oriented components, the processing of which is vital for object segregation and recognition. In primate V1 and V2, most neurons have small spatio-temporal receptive fields responding selectively to oriented luminance contours (first order), while only a subgroup of neurons signal non-luminance defined contours (second order). So how is the orientation of second-order contours represented at the population level in macaque V1 and V2? Here we compared the population responses in macaque V1 and V2 to two types of second-order contour stimuli generated either by modulation of contrast or phase reversal with those to first-order contour stimuli. Using intrinsic signal optical imaging, we found that the orientation of second-order contour stimuli was represented invariantly in the orientation columns of both macaque V1 and V2. A physiologically constrained spatio-temporal energy model of V1 and V2 neuronal populations could reproduce all the recorded population responses. These findings suggest that, at the population level, the primate early visual system processes the orientation of second-order contours initially through a linear spatio-temporal filter mechanism. Our results of population responses to different second-order contour stimuli support the idea that the orientation maps in primate V1 and V2 can be described as a spatial-temporal energy map.

The synaptic and circuit mechanisms underlying a change in spatial encoding in the retina

Components of neural circuits are often repurposed so that the same biological hardware can be used for distinct computations. This flexibility in circuit operation is required to account for the changes in sensory computations that accompany changes in input signals. Yet we know little about how such changes in circuit operation are implemented. Here we show that a single retinal ganglion cell performs a different computation in dim light--averaging contrast within its receptive field--than in brighter light, when the cell becomes sensitive to fine spatial detail. This computational change depends on interactions between two parallel circuits that control the ganglion cell's excitatory synaptic inputs. Specifically, steady-state interactions through dendro-axonal gap junctions control rectification of the synapses providing excitatory input to the ganglion cell. These findings provide a clear example of how a simple synaptic mechanism can repurpose a neural circuit to perform diverse computations.

The visual representation of 3D object orientation in parietal cortex

An accurate representation of three-dimensional (3D) object orientation is essential for interacting with the environment. Where and how the brain visually encodes 3D object orientation remains unknown, but prior studies suggest the caudal intraparietal area (CIP) may be involved. Here, we develop rigorous analytical methods for quantifying 3D orientation tuning curves, and use these tools to the study the neural coding of surface orientation. Specifically, we show that single neurons in area CIP of the rhesus macaque jointly encode the slant and tilt of a planar surface, and that across the population, the distribution of preferred slant-tilts is not statistically different from uniform. This suggests that all slant-tilt combinations are equally represented in area CIP. Furthermore, some CIP neurons are found to also represent the third rotational degree of freedom that determines the orientation of the image pattern on the planar surface. Together, the present results suggest that CIP is a critical neural locus for the encoding of all three rotational degrees of freedom specifying an object's 3D spatial orientation.

Depth perception from dynamic occlusion in motion parallax: roles of expansion-compression versus accretion-deletion

Motion parallax, or differential retinal image motion from observer movement, provides important information for depth perception. We previously measured the contribution of shear motion parallax to depth, which is only composed of relative motion information. Here, we examine the roles of relative motion and accretion-deletion information in dynamic occlusion motion parallax. Observers performed two-alternative forced choice depth-ordering tasks in response to low spatial frequency patterns of horizontal random dot motion that were synchronized to the observer's head movements. We examined conditions that isolated or combined expansion-compression and accretion-deletion across a range of simulated relative depths. At small depths, expansion-compression provided reliable depth perception while accretion-deletion had a minor contribution: When the two were in conflict, the perceived depth was dominated by expansion-compression. At larger depths in the cue-conflict experiment, accretion-deletion determined the depth-ordering performance. Accretion-deletion in isolation did not yield any percept of depth even though, in theory, it provided sufficient information for depth ordering. Thus, accretion-deletion can substantially enhance depth perception at larger depths but only in the presence of relative motion. The results indicate that expansion-compression contributes to depth from motion parallax across a broad range of depths whereas accretion-deletion contributes primarily at larger depths.

Lobula-specific visual projection neurons are involved in perception of motion-defined second-order motion in Drosophila

A wide variety of animal species including humans and fruit flies see second-order motion although they lack coherent spatiotemporal correlations in luminance. Recent electrophysiological recordings, together with intensive psychophysical studies, are bringing to light the neural underpinnings of second-order motion perception in mammals. However, where and how the higher-order motion signals are processed in the fly brain is poorly understood. Using the rich genetic tools available in Drosophila and examining optomotor responses in fruit flies to several stimuli, we revealed that two lobula-specific visual projection neurons, specifically connecting the lobula and the central brain, are involved in the perception of motion-defined second-order motion, independent of whether the second-order feature is moving perpendicular or opposite to the local first-order motion. By contrast, blocking these neurons has no effect on first-order and flicker-defined second-order stimuli in terms of response delay. Our results suggest that visual neuropils deep in the optic lobe and the central brain, whose functional roles in motion processing were previously unclear, may be specifically required for motion-defined motion processing.

Functional organization of envelope-responsive neurons in early visual cortex: organization of carrier tuning properties

It is well established that visual cortex neurons having similar selectivity for orientation, direction of motion, ocular dominance, and other properties of first-order (luminance-defined) stimuli are clustered into a columnar organization. However, the cortical architecture of neuronal responses to second-order (contrast/texture-defined) stimuli is poorly understood. A useful second-order stimulus is a contrast envelope, consisting of a finely detailed pattern (carrier) whose contrast varies on a coarse spatial scale (envelope). In this study, we analyzed the cortical organization of carrier tuning properties of neurons, which responded to contrast-modulated stimuli. We examined whether neurons tuned to similar carrier properties are clustered spatially and whether such spatial clusters are arranged in columns. To address these questions, we recorded single-unit activity, multiunit activity, and local field potentials simultaneously from area 18 of anesthetized cats, using single-channel microelectrodes and multielectrode arrays. Our data showed that neurons tuned to similar carrier spatial frequency are distributed in a highly clustered manner; neurons tuned to similar carrier orientation are also significantly clustered. Neurons along linear arrays perpendicular to the brain surface always exhibited similar optimal carrier spatial frequency, indicating a columnar organization. Multi-pronged tetrode recordings indicated that the diameter of these columns is ≥450 μm. Optimal carrier orientation was also significantly clustered but with finer-grain organization and greater scatter. These results indicate a fine anatomical structure of cortical organization of second-order information processing and suggest that there are probably more maps in cat area 18 than previously believed.

Motion-defined contour processing in the early visual cortex

From our daily experience, it is very clear that relative motion cues can contribute to correctly identifying object boundaries and perceiving depth. Motion-defined contours are not only generated by the motion of objects in a scene but also by the movement of an observer's head and body (motion parallax). However, the neural mechanism involved in detecting these contours is still unknown. To explore this mechanism, we extracellularly recorded visual responses of area 18 neurons in anesthetized and paralyzed cats. The goal of this study was to determine if motion-defined contours could be detected by neurons that have been previously shown to detect luminance-, texture-, and contrast-defined contours cue invariantly. Motion-defined contour stimuli were generated by modulating the velocity of high spatial frequency sinusoidal luminance gratings (carrier gratings) by a moving squarewave envelope. The carrier gratings were outside the luminance passband of a neuron, such that presence of the carrier alone within the receptive field did not elicit a response. Most neurons that responded to contrast-defined contours also responded to motion-defined contours. The orientation and direction selectivity of these neurons for motion-defined contours was similar to that of luminance gratings. A given neuron also exhibited similar selectivity for the spatial frequency of the carrier gratings of contrast- and motion-defined contours. These results suggest that different second-order contours are detected in a form-cue invariant manner, through a common neural mechanism in area 18.