Response errors explain the failure of independent-channels models of perception of temporal order

Neural-latency noise places limits on human sensitivity to the timing of events

The brain-time account posits that the physical timing of sensory-evoked neural activity determines the perceived timing of corresponding sensory events. A canonical model formalises this account for tasks such as simultaneity and order judgements: Signals arrive at a decision centre in an order, and at a temporal offset, shaped by neural propagation times. This model assumes that the noise affecting people's temporal judgements is primarily neural-latency noise, i.e. variation in propagation times across trials, but this assumption has received little scrutiny. Here, we recorded EEG alongside simultaneity judgements from 50 participants in response to combinations of visual, auditory and tactile stimuli. Bootstrapping of ERP components was used to estimate neural-latency noise, and simultaneity judgements were modelled to estimate the precision of timing judgements. We obtained the predicted correlation between neural and behavioural measures of latency noise, supporting a fundamental feature of the canonical model of perceived timing.

Body-environment integration: Temporal processing of tactile and auditory inputs along the schizophrenia continuum

According to the dimensional approach to psychosis, there is a continuum from low schizotypy to schizophrenia patients. The temporal aspect of sensory processing seems to be compromised across such continuum, as suggested by different studies separately investigating unisensory or multisensory domains. Most of these studies have so far focused primarily on the temporal processing of visual and auditory stimuli, either in schizotypy or schizophrenia, while leaving the tactile domain and the integration of touch with other senses mostly unexplored. Given the relevance of body-related perceptual abnormalities for psychosis proneness, we aimed at filling this gap in the literature across two studies. We asked participants with increasing levels of schizotypy (study 1) and schizophrenia patients (study 2) to perform three simultaneity judgement tasks: a unimodal tactile task, a unimodal auditory task and a bimodal audio-tactile task. Each task allowed estimating a simultaneity range (SR), as a proxy of the individual tolerance to asynchronies in the tactile, auditory and audio-tactile domains, respectively. Results showed larger SRs as the level of schizotypy increases. Specifically, the linear effect of schizotypy levels on the audio-tactile task was stronger than on the auditory task, which in turn was greater than the effect on the tactile task (study 1). Differently, schizophrenia patients showed larger SRs than controls in all the three tasks (study 2). The current study is the first empirical investigation across the continuum from low schizotypy to schizophrenia of the tolerance to asynchronies in the processing of external (auditory) and body-related (tactile) inputs.

Order effects in two-alternative forced-choice tasks invalidate adaptive threshold estimates

Adaptive psychophysical methods are widely used for the quick estimation of percentage points (thresholds) on psychometric functions for two-alternative forced-choice (2AFC) tasks. The use of adaptive methods is supported by numerous simulation studies documenting their performance, which have shown that thresholds can be reasonably estimated with them when their founding assumptions hold. One of these assumptions is that the psychometric function is invariant, but empirical evidence is mounting that human performance in 2AFC tasks needs to be described by two different psychometric functions, one that holds when the test stimulus is presented first in the 2AFC trial and a different one that holds when the test is presented second. The same holds when presentations are instead simultaneous at two spatial locations rather than sequential. We re-evaluated the performance of adaptive methods in the presence of these order effects via simulation studies and an empirical study with human observers. The simulation study showed that thresholds are severely overestimated by adaptive methods in these conditions, and the empirical study corroborated these findings. These results question the validity of threshold estimates obtained with adaptive methods that incorrectly assume the psychometric function to be invariant with presentation order. Alternative ways in which thresholds can be accurately estimated in the presence of order effects are discussed.

Modulation of Individual Alpha Frequency with tACS shifts Time Perception

Previous studies have linked brain oscillation and timing, with evidence suggesting that alpha oscillations (10 Hz) may serve as a "sample rate" for the visual system. However, direct manipulation of alpha oscillations and time perception has not yet been demonstrated. To test this, we had 18 human subjects perform a time generalization task with visual stimuli. Additionally, we had previously recorded resting-state EEG from each subject and calculated their individual alpha frequency (IAF), estimated as the peak frequency from the mean spectrum over posterior electrodes between 8 and 13 Hz. Participants first learned a standard interval (600 ms) and were then required to judge if a new set of temporal intervals were equal or different compared with that standard. After learning the standard, participants performed this task while receiving occipital transcranial Alternating Current Stimulation (tACS). Crucially, for each subject, tACS was administered at their IAF or at off-peak alpha frequencies (IAF ± 2 Hz). Results demonstrated a linear shift in the psychometric function indicating a modification of perceived duration, such that progressively "faster" alpha stimulation led to longer perceived intervals. These results provide the first evidence that direct manipulations of alpha oscillations can shift perceived time in a manner consistent with a clock speed effect.

The Do's and Don'ts of Psychophysical Methods for Interpretability of Psychometric Functions and Their Descriptors

Many areas of research require measuring psychometric functions or their descriptors (thresholds, slopes, etc.). Data for this purpose are collected with psychophysical methods of various types and justification for the interpretation of results arises from a model of performance grounded in signal detection theory. Decades of research have shown that psychophysical data display features that are incompatible with such framework, questioning the validity of interpretations obtained under it and revealing that psychophysical performance is more complex than this framework entertains. This paper describes the assumptions and formulation of the conventional framework for the two major classes of psychophysical methods (single- and dual-presentation methods) and presents various lines of empirical evidence that the framework is inconsistent with. An alternative framework is then described and shown to account for all the characteristics that the conventional framework regards as anomalies. This alternative process model explicitly separates the sensory, decisional, and response components of performance and represents them via parameters whose estimation characterizes the corresponding processes. Retrospective and prospective evidence of the validity of the alternative framework is also presented. A formal analysis also reveals that some psychophysical methods and response formats are unsuitable for separation of the three components of observed performance. Recommendations are thus given regarding practices that should be avoided and those that should be followed to ensure interpretability of the psychometric function, or descriptors (detection threshold, difference limen, point of subjective equality, etc.) obtained with shortcut methods that do not require estimation of psychometric functions.

Nonparametric tests for equality of psychometric functions

Many empirical studies measure psychometric functions (curves describing how observers' performance varies with stimulus magnitude) because these functions capture the effects of experimental conditions. To assess these effects, parametric curves are often fitted to the data and comparisons are carried out by testing for equality of mean parameter estimates across conditions. This approach is parametric and, thus, vulnerable to violations of the implied assumptions. Furthermore, testing for equality of means of parameters may be misleading: Psychometric functions may vary meaningfully across conditions on an observer-by-observer basis with no effect on the mean values of the estimated parameters. Alternative approaches to assess equality of psychometric functions per se are thus needed. This paper compares three nonparametric tests that are applicable in all situations of interest: The existing generalized Mantel-Haenszel test, a generalization of the Berry-Mielke test that was developed here, and a split variant of the generalized Mantel-Haenszel test also developed here. Their statistical properties (accuracy and power) are studied via simulation and the results show that all tests are indistinguishable as to accuracy but they differ non-uniformly as to power. Empirical use of the tests is illustrated via analyses of published data sets and practical recommendations are given. The computer code in MATLAB and R to conduct these tests is available as Electronic Supplemental Material.

Stuck on a Plateau? A Model-Based Approach to Fundamental Issues in Visual Temporal-Order Judgments

Humans are incapable of judging the temporal order of visual events at brief temporal separations with perfect accuracy. Their performance-which is of much interest in visual cognition and attention research-can be measured with the temporal-order judgment (TOJ) task, which typically produces S-shaped psychometric functions. Occasionally, researchers reported plateaus within these functions, and some theories predict such deviation from the basic S shape. However, the centers of the psychometric functions result from the weakest performance at the most difficult presentations and therefore fluctuate strongly, leaving the existence and exact shapes of plateaus unclear. This study set out to investigate whether plateaus disappear if the data accuracy is enhanced, or if we are "stuck on a plateau", or rather with it. For this purpose, highly accurate data were assessed by model-based analysis. The existence of plateaus is confidently confirmed and two plausible mechanisms derived from very different models are presented. Neither model, however, performs well in the presence of a strong attention manipulation, and model comparison remains unclear on the question of which of the models describes the data best. Nevertheless, the present study includes the highest accuracy in visual TOJ data and the most explicit models of plateaus in TOJ studied so far.

Assessing the Role of the 'Unity Assumption' on Multisensory Integration: A Review

There has been longstanding interest from both experimental psychologists and cognitive neuroscientists in the potential modulatory role of various top-down factors on multisensory integration/perception in humans. One such top-down influence, often referred to in the literature as the 'unity assumption,' is thought to occur in those situations in which an observer considers that various of the unisensory stimuli that they have been presented with belong to one and the same object or event (Welch and Warren, 1980). Here, we review the possible factors that may lead to the emergence of the unity assumption. We then critically evaluate the evidence concerning the consequences of the unity assumption from studies of the spatial and temporal ventriloquism effects, from the McGurk effect, and from the Colavita visual dominance paradigm. The research that has been published to date using these tasks provides support for the claim that the unity assumption influences multisensory perception under at least a subset of experimental conditions. We then consider whether the notion has been superseded in recent years by the introduction of priors in Bayesian causal inference models of human multisensory perception. We suggest that the prior of common cause (that is, the prior concerning whether multisensory signals originate from the same source or not) offers the most useful way to quantify the unity assumption as a continuous cognitive variable.

Multisensory simultaneity judgment and proximity to the body

The integration of information across different sensory modalities is known to be dependent upon the statistical characteristics of the stimuli to be combined. For example, the spatial and temporal proximity of stimuli are important determinants with stimuli that are close in space and time being more likely to be bound. These multisensory interactions occur not only for singular points in space/time, but over “windows” of space and time that likely relate to the ecological statistics of real-world stimuli. Relatedly, human psychophysical work has demonstrated that individuals are highly prone to judge multisensory stimuli as co-occurring over a wide range of time—a so-called simultaneity window (SW). Similarly, there exists a spatial representation of peripersonal space (PPS) surrounding the body in which stimuli related to the body and to external events occurring near the body are highly likely to be jointly processed. In the current study, we sought to examine the interaction between these temporal and spatial dimensions of multisensory representation by measuring the SW for audiovisual stimuli through proximal–distal space (i.e., PPS and extrapersonal space). Results demonstrate that the audiovisual SWs within PPS are larger than outside PPS. In addition, we suggest that this effect is likely due to an automatic and additional computation of these multisensory events in a body-centered reference frame. We discuss the current findings in terms of the spatiotemporal constraints of multisensory interactions and the implication of distinct reference frames on this process.

A Roving Dual-Presentation Simultaneity-Judgment Task to Estimate the Point of Subjective Simultaneity

The most popular tasks with which to investigate the perception of subjective synchrony are the temporal order judgment (TOJ) and the simultaneity judgment (SJ). Here, we discuss a complementary approach—a dual-presentation (2x) SJ task—and focus on appropriate analysis methods for a theoretically desirable “roving” design. Two stimulus pairs are presented on each trial and the observer must select the most synchronous. To demonstrate this approach, in Experiment 1 we tested the 2xSJ task alongside TOJ, SJ, and simple reaction-time (RT) tasks using audiovisual stimuli. We interpret responses from each task using detection-theoretic models, which assume variable arrival times for sensory signals at critical brain structures for timing perception. All tasks provide similar estimates of the point of subjective simultaneity (PSS) on average, and PSS estimates from some tasks were correlated on an individual basis. The 2xSJ task produced lower and more stable estimates of model-based (and thus comparable) sensory/decision noise than the TOJ. In Experiment 2 we obtained similar results using RT, TOJ, ternary, and 2xSJ tasks for all combinations of auditory, visual, and tactile stimuli. In Experiment 3 we investigated attentional prior entry, using both TOJs and 2xSJs. We found that estimates of prior-entry magnitude correlated across these tasks. Overall, our study establishes the practicality of the roving dual-presentation SJ task, but also illustrates the additional complexity of the procedure. We consider ways in which this task might complement more traditional procedures, particularly when it is important to estimate both PSS and sensory/decisional noise.

Visual and Auditory Components in the Perception of Asynchronous Audiovisual Speech

Research on asynchronous audiovisual speech perception manipulates experimental conditions to observe their effects on synchrony judgments. Probabilistic models establish a link between the sensory and decisional processes underlying such judgments and the observed data, via interpretable parameters that allow testing hypotheses and making inferences about how experimental manipulations affect such processes. Two models of this type have recently been proposed, one based on independent channels and the other using a Bayesian approach. Both models are fitted here to a common data set, with a subsequent analysis of the interpretation they provide about how experimental manipulations affected the processes underlying perceived synchrony. The data consist of synchrony judgments as a function of audiovisual offset in a speech stimulus, under four within-subjects manipulations of the quality of the visual component. The Bayesian model could not accommodate asymmetric data, was rejected by goodness-of-fit statistics for 8/16 observers, and was found to be nonidentifiable, which renders uninterpretable parameter estimates. The independent-channels model captured asymmetric data, was rejected for only 1/16 observers, and identified how sensory and decisional processes mediating asynchronous audiovisual speech perception are affected by manipulations that only alter the quality of the visual component of the speech signal.

A model-based comparison of three theories of audiovisual temporal recalibration

Observers change their audio-visual timing judgements after exposure to asynchronous audiovisual signals. The mechanism underlying this temporal recalibration is currently debated. Three broad explanations have been suggested. According to the first, the time it takes for sensory signals to propagate through the brain has changed. The second explanation suggests that decisional criteria used to interpret signal timing have changed, but not time perception itself. A final possibility is that a population of neurones collectively encode relative times, and that exposure to a repeated timing relationship alters the balance of responses in this population. Here, we simplified each of these explanations to its core features in order to produce three corresponding six-parameter models, which generate contrasting patterns of predictions about how simultaneity judgements should vary across four adaptation conditions: No adaptation, synchronous adaptation, and auditory leading/lagging adaptation. We tested model predictions by fitting data from all four conditions simultaneously, in order to assess which model/explanation best described the complete pattern of results. The latency-shift and criterion-change models were better able to explain results for our sample as a whole. The population-code model did, however, account for improved performance following adaptation to a synchronous adapter, and best described the results of a subset of observers who reported least instances of synchrony.

The bisection point across variants of the task

Bisection tasks are used in research on normal space and time perception and to assess the perceptual distortions accompanying neurological disorders. Several variants of the bisection task are used, which often yield inconsistent results, prompting the question of which variant is most dependable and which results are to be trusted. We addressed this question using theoretical and experimental approaches. Theoretical performance in bisection tasks is derived from a general model of psychophysical performance that includes sensory components and decisional processes. The model predicts how performance should differ across variants of the task, even when the sensory component is fixed. To test these predictions, data were collected in a within-subjects study with several variants of a spatial bisection task, including a two-response variant in which observers indicated whether a line was transected to the right or left of the midpoint, a three-response variant (which included the additional option to respond "midpoint"), and a paired-comparison variant of the three-response format. The data supported the model predictions, revealing that estimated bisection points were least dependable with the two-response variant, because this format confounds perceptual and decisional influences. Only the three-response paired-comparison format can separate out these influences. Implications for research in basic and clinical fields are discussed.

Shifts of the psychometric function: distinguishing bias from perceptual effects

Morgan, Dillenburger, Raphael, and Solomon have shown that observers can use different response strategies when unsure of their answer, and, thus, they can voluntarily shift the location of the psychometric function estimated with the method of single stimuli (MSS; sometimes also referred to as the single-interval, two-alternative method). They wondered whether MSS could distinguish response bias from a true perceptual effect that would also shift the location of the psychometric function. We demonstrate theoretically that the inability to distinguish response bias from perceptual effects is an inherent shortcoming of MSS, although a three-response format including also an "undecided" response option may solve the problem under restrictive assumptions whose validity cannot be tested with MSS data. We also show that a proper two-alternative forced-choice (2AFC) task with the three-response format is free of all these problems so that bias and perceptual effects can easily be separated out. The use of a three-response 2AFC format is essential to eliminate a confound (response bias) in studies of perceptual effects and, hence, to eliminate a threat to the internal validity of research in this area.