Aberrant occlusal sensitivity in adults with increased somatosensory amplification: a case-control study

OBJECTIVES

Occlusal sensitivity (OS)-the ability to detect fine objects between opposing teeth-mainly relies on the activity of mechanoreceptors located in the periodontal ligament. We tested whether somatosensory amplification (SSA)-the tendency to perceive normal somatic sensations as being intense, noxious, and disturbing, which plays a critical role in hypervigilance-affects OS.

MATERIALS AND METHODS

We measured OS in 66 adults divided into three groups based on their SSA scores (LowSSA, Intermediate - IntSSA, HighSSA) by asking them to bite on aluminum foils (8 to 72 μm thick) and a sham foil, and report whether they felt each foil. We performed 20 trials for each thickness and sham condition (each participant was tested 120 times), and compared the frequency of correct answers (%correct) among groups after adjusting for participants' trait anxiety, depression, self-reported oral behaviors, and masseter cross-sectional area.

RESULTS

%correct was affected by the interaction Foil Thickness-by-SSA (p = 0.007). When tested with the 8 μm foil, the HighSSA group had a lower %correct than the IntSSA (contrast estimate [95% CI]: -14.2 [-25.8 - -2.6]; p = 0.012) and the LowSSA groups (-19.1 [-31.5 - -6.6]; p = 0.001). Similarly, with the 24 μm foil, the HighSSA group had a lower %correct compared to the IntSSA (-12.4 [-24.8-0.1]; p = 0.048) and the LowSSA groups (-10.8 [-22.5-0.8]; p = 0.073).

CONCLUSION

Individuals with high SSA present with an aberrant occlusal sensitivity.

CLINICAL RELEVANCE

Our findings provide novel insights into the relationship between occlusal perception and psychological factors, which may influence an individual's ability to adapt to dental work.

Can metaphors help us better remember wines? The effect of wine evaluation style on short-term recognition of red wines

People are generally poor at remembering complex food stimuli, such as wine. While writing a description has been shown to improve memory performance, talking about wine is generally a difficult task for novices. However, giving novices a framework in which to evaluate the wine may help with the memory process. Using a short-term recognition task, this experiment compared different forms of wine evaluation on the to-be-remembered wine sample, using either 1) a classic smell and taste evaluation, 2) a multisensory metaphor selection task with visual, auditory, and tactile metaphors, or 3) a control condition with no writing. Results from 153 participants revealed that recognition performance between the three groups was not significantly different. Secondary analysis revealed that recognition accuracy was correlated with wine liking for the control group, suggesting that in the absence of explicitly evaluating the wine, participants relied on wine liking as a cue for memory. Implications for theory development and applications in wine education are discussed.

The influence of tDCS on perceived bouncing/streaming

Processing ambiguous situations is a constant challenge in everyday life and sensory input from different modalities needs to be integrated to form a coherent mental representation on the environment. The bouncing/streaming illusion can be studied to provide insights into the ambiguous perception and processing of multi-modal environments. In short, the likelihood of reporting bouncing rather than streaming impressions increases when a sound coincides with the moment of overlap between two moving disks. Neuroimaging studies revealed that the right posterior parietal cortex is crucial in cross-modal integration and is active during the bouncing/streaming illusion. Consequently, in the present study, we used transcranial direct current stimulation to stimulate this brain area. In the active stimulation conditions, a 9 cm² electrode was positioned over the P4-EEG position and the 35 cm² reference positioned over the left upper arm. The stimulation lasted 15 min. Each participant did the bouncing/streaming task three times: before, during and after anodal or sham stimulation. In a sample of N = 60 healthy, young adults, we found no influence of anodal tDCS. Bayesian analysis showed strong evidence against tDCS effects. There are two possible explanations for the finding that anodal tDCS over perceptual areas did not modulate multimodal integration. First, upregulation of multimodal integration is not possible using tDCS over the PPC as the integration process already functions at maximum capacity. Second, prefrontal decision-making areas may have overruled any modulated input from the PPC as it may not have matched their decision-making criterion and compensated for the modulation.

Exposure to multisensory and visual static or moving stimuli enhances processing of nonoptimal visual rhythms

Research has shown that visual moving and multisensory stimuli can efficiently mediate rhythmic information. It is possible, therefore, that the previously reported auditory dominance in rhythm perception is due to the use of nonoptimal visual stimuli. Yet it remains unknown whether exposure to multisensory or visual-moving rhythms would benefit the processing of rhythms consisting of nonoptimal static visual stimuli. Using a perceptual learning paradigm, we tested whether the visual component of the multisensory training pair can affect processing of metric simple two integer-ratio nonoptimal visual rhythms. Participants were trained with static (AVstat), moving-inanimate (AVinan), or moving-animate (AVan) visual stimuli along with auditory tones and a regular beat. In the pre- and posttraining tasks, participants responded whether two static-visual rhythms differed or not. Results showed improved posttraining performance for all training groups irrespective of the type of visual stimulation. To assess whether this benefit was auditory driven, we introduced visual-only training with a moving or static stimulus and a regular beat (Vinan). Comparisons between Vinan and Vstat showed that, even in the absence of auditory information, training with visual-only moving or static stimuli resulted in an enhanced posttraining performance. Overall, our findings suggest that audiovisual and visual static or moving training can benefit processing of nonoptimal visual rhythms.

Sound-modulations of visual motion perception implicate the cortico-vestibular brain

A widely used example of the intricate (yet poorly understood) intertwining of multisensory signals in the brain is the audiovisual bounce inducing effect (ABE). This effect presents two identical objects moving along the azimuth with uniform motion and towards opposite directions. The perceptual interpretation of the motion is ambiguous and is modulated if a transient (sound) is presented in coincidence with the point of overlap of the two objects' motion trajectories. This phenomenon has long been written-off to simple attentional or decision-making mechanisms, although the neurological underpinnings for the effect are not well understood. Using behavioural metrics concurrently with event-related fMRI, we show that sound-induced modulations of motion perception can be further modulated by changing motion dynamics of the visual targets. The phenomenon engages the posterior parietal cortex and the parieto-insular-vestibular cortical complex, with a close correspondence of activity in these regions with behaviour. These findings suggest that the insular cortex is engaged in deriving a probabilistic perceptual solution through the integration of multisensory data.

Hierarchical drift diffusion modeling uncovers multisensory benefit in numerosity discrimination tasks

Studies of accuracy and reaction time in decision making often observe a speed-accuracy tradeoff, where either accuracy or reaction time is sacrificed for the other. While this effect may mask certain multisensory benefits in performance when accuracy and reaction time are separately measured, drift diffusion models (DDMs) are able to consider both simultaneously. However, drift diffusion models are often limited by large sample size requirements for reliable parameter estimation. One solution to this restriction is the use of hierarchical Bayesian estimation for DDM parameters. Here, we utilize hierarchical drift diffusion models (HDDMs) to reveal a multisensory advantage in auditory-visual numerosity discrimination tasks. By fitting this model with a modestly sized dataset, we also demonstrate that large sample sizes are not necessary for reliable parameter estimation.

Task-dependent audiovisual temporal sensitivity is not affected by stimulus intensity levels

Flexibility and robustness of multisensory temporal recalibration is paramount for maintaining perceptual constancy of the surrounding natural world. Different environments impart various impediments, distances and routes that alter the propagation times of sight and sound cues comprising a multimodal event. One's ability to rapidly calibrate and account for these external variations allows for maintained perception of synchrony which is crucial for coherent and consistent perception. The two common paradigms used to compare precision of temporal processing between experimental and control groups, the simultaneity judgment (SJ) and temporal order judgment (TOJ) tasks, often use supra-threshold stimuli. However, few studies have specifically examined the effects of normalizing stimulus intensities to participant's unisensory detection thresholds. The current project presented multiple combinations of auditory and visual stimulus intensity levels, based on individual detection thresholds, during a TOJ and a SJ task. While no effect of stimulus intensity was found on temporal sensitivity or perceived temporal synchrony, there was a significant difference in point of subjective simultaneity (PSS) measures between tasks. In addition, PSS estimates were audio-leading, rather than visual-leading as previously reported, suggesting that exposure to the particular combinations of stimulus intensity levels used influenced temporal synchrony perception. Overall, these results support the use of supra-threshold stimuli in TOJ and SJ tasks as a way of minimizing the confound from differences in unisensory processing.

Perceptual learning evidence for supramodal representation of stimulus orientation at a conceptual level

When stimulus inputs from different senses are integrated to form a coherent percept, inputs from a more precise sense are typically more dominant than those from a less precise sense. Furthermore, we hypothesized that some basic stimulus features, such as orientation, can be supramodal-represented at a conceptual level that is independent of the original modality precision. This hypothesis was tested with perceptual learning experiments. Specifically, participants practiced coarser tactile orientation discrimination, which initially had little impact on finer visual orientation discrimination (tactile vs. visual orientation thresholds = 3:1). However, if participants also practiced a functionally orthogonal visual contrast discrimination task in a double training design, their visual orientation performance was improved at both tactile-trained and untrained orientations, as much as through direct visual orientation training. The complete tactile-to-visual learning transfer is consistent with a conceptual supramodal representation of orientation unconstrained by original modality precision, likely through certain forms of input standardization. Moreover, this conceptual supramodal representation, when improved through perceptual learning in one sense, can in turn facilitate orientation discrimination in an untrained sense.

Perceptual timing precision with vibrotactile, auditory, and multisensory stimuli

The growing use of vibrotactile signaling devices makes it important to understand the perceptual limits on vibrotactile information processing. To promote that understanding, we carried out a pair of experiments on vibrotactile, auditory, and bimodal (synchronous vibrotactile and auditory) temporal acuity. On each trial, subjects experienced a set of isochronous, standard intervals (400 ms each), followed by one interval of variable duration (400 ± 1-80 ms). Intervals were demarcated by short vibrotactile, auditory, or bimodal pulses. Subjects categorized the timing of the last interval by describing the final pulse as either "early" or "late" relative to its predecessors. In Experiment 1, each trial contained three isochronous standard intervals, followed by an interval of variable length. In Experiment 2, the number of isochronous standard intervals per trial varied, from one to four. Psychometric modeling revealed that vibrotactile stimulation produced poorer temporal discrimination than either auditory or bimodal stimulation. Moreover, auditory signals dominated bimodal sensitivity, and inter-individual differences in temporal discriminability were reduced with bimodal stimulation. Additionally, varying the number of isochronous intervals in a trial failed to improve temporal sensitivity in either modality, suggesting that memory played a key role in judgments of interval duration.

Ageing and multisensory integration: A review of the evidence, and a computational perspective

The processing of multisensory signals is crucial for effective interaction with the environment, but our ability to perform this vital function changes as we age. In the first part of this review, we summarise existing research into the effects of healthy ageing on multisensory integration. We note that age differences vary substantially with the paradigms and stimuli used: older adults often receive at least as much benefit (to both accuracy and response times) as younger controls from congruent multisensory stimuli, but are also consistently more negatively impacted by the presence of intersensory conflict. In the second part, we outline a normative Bayesian framework that provides a principled and computationally informed perspective on the key ingredients involved in multisensory perception, and how these are affected by ageing. Applying this framework to the existing literature, we conclude that changes to sensory reliability, prior expectations (together with attentional control), and decisional strategies all contribute to the age differences observed. However, we find no compelling evidence of any age-related changes to the basic inference mechanisms involved in multisensory perception.

Crossmodal correspondences as common ground for joint action

When performing joint actions, people rely on common ground - shared information that provides the required basis for mutual understanding. Common ground can be based on people's interaction history or on knowledge and expectations people share, e.g., because they belong to the same culture or social class. Here, we suggest that people rely on yet another form of common ground, one that originates in their similarities in multisensory processing. Specifically, we focus on 'crossmodal correspondences' - nonarbitrary associations that people make between stimulus features in different sensory modalities, e.g., between stimuli in the auditory and the visual modality such as high-pitched sounds and small objects. Going beyond previous research that focused on investigating crossmodal correspondences in individuals, we propose that people can use these correspondences for communicating and coordinating with others. Initial support for our proposal comes from a communication game played in a public space (an art gallery) by pairs of visitors. We observed that pairs created nonverbal communication systems by spontaneously relying on 'crossmodal common ground'. Based on these results, we conclude that crossmodal correspondences not only occur within individuals but that they can also be actively used in joint action to facilitate the coordination between individuals.

Audio-visual integration in noise: Influence of auditory and visual stimulus degradation on eye movements and perception of the McGurk effect

Seeing a talker’s face can aid audiovisual (AV) integration when speech is presented in noise. However, few studies have simultaneously manipulated auditory and visual degradation. We aimed to establish how degrading the auditory and visual signal affected AV integration. Where people look on the face in this context is also of interest; Buchan, Paré and Munhall (Brain Research, 1242, 162–171, 2008) found fixations on the mouth increased in the presence of auditory noise whilst Wilson, Alsius, Paré and Munhall (Journal of Speech, Language, and Hearing Research, 59(4), 601–615, 2016) found mouth fixations decreased with decreasing visual resolution. In Condition 1, participants listened to clear speech, and in Condition 2, participants listened to vocoded speech designed to simulate the information provided by a cochlear implant. Speech was presented in three levels of auditory noise and three levels of visual blurring. Adding noise to the auditory signal increased McGurk responses, while blurring the visual signal decreased McGurk responses. Participants fixated the mouth more on trials when the McGurk effect was perceived. Adding auditory noise led to people fixating the mouth more, while visual degradation led to people fixating the mouth less. Combined, the results suggest that modality preference and where people look during AV integration of incongruent syllables varies according to the quality of information available.

Coordinating attention requires coordinated senses

From playing basketball to ordering at a food counter, we frequently and effortlessly coordinate our attention with others towards a common focus: we look at the ball, or point at a piece of cake. This non-verbal coordination of attention plays a fundamental role in our social lives: it ensures that we refer to the same object, develop a shared language, understand each other's mental states, and coordinate our actions. Models of joint attention generally attribute this accomplishment to gaze coordination. But are visual attentional mechanisms sufficient to achieve joint attention, in all cases? Besides cases where visual information is missing, we show how combining it with other senses can be helpful, and even necessary to certain uses of joint attention. We explain the two ways in which non-visual cues contribute to joint attention: either as enhancers, when they complement gaze and pointing gestures in order to coordinate joint attention on visible objects, or as modality pointers, when joint attention needs to be shifted away from the whole object to one of its properties, say weight or texture. This multisensory approach to joint attention has important implications for social robotics, clinical diagnostics, pedagogy and theoretical debates on the construction of a shared world.

Senses of place: architectural design for the multisensory mind

Traditionally, architectural practice has been dominated by the eye/sight. In recent decades, though, architects and designers have increasingly started to consider the other senses, namely sound, touch (including proprioception, kinesthesis, and the vestibular sense), smell, and on rare occasions, even taste in their work. As yet, there has been little recognition of the growing understanding of the multisensory nature of the human mind that has emerged from the field of cognitive neuroscience research. This review therefore provides a summary of the role of the human senses in architectural design practice, both when considered individually and, more importantly, when studied collectively. For it is only by recognizing the fundamentally multisensory nature of perception that one can really hope to explain a number of surprising crossmodal environmental or atmospheric interactions, such as between lighting colour and thermal comfort and between sound and the perceived safety of public space. At the same time, however, the contemporary focus on synaesthetic design needs to be reframed in terms of the crossmodal correspondences and multisensory integration, at least if the most is to be made of multisensory interactions and synergies that have been uncovered in recent years. Looking to the future, the hope is that architectural design practice will increasingly incorporate our growing understanding of the human senses, and how they influence one another. Such a multisensory approach will hopefully lead to the development of buildings and urban spaces that do a better job of promoting our social, cognitive, and emotional development, rather than hindering it, as has too often been the case previously.

Multisensory inclusive design with sensory substitution

Sensory substitution techniques are perceptual and cognitive phenomena used to represent one sensory form with an alternative. Current applications of sensory substitution techniques are typically focused on the development of assistive technologies whereby visually impaired users can acquire visual information via auditory and tactile cross-modal feedback. But despite their evident success in scientific research and furthering theory development in cognition, sensory substitution techniques have not yet gained widespread adoption within sensory-impaired populations. Here we argue that shifting the focus from assistive to mainstream applications may resolve some of the current issues regarding the use of sensory substitution devices to improve outcomes for those with disabilities. This article provides a tutorial guide on how to use research into multisensory processing and sensory substitution techniques from the cognitive sciences to design new inclusive cross-modal displays. A greater focus on developing inclusive mainstream applications could lead to innovative technologies that could be enjoyed by every person.

Perception of causality and synchrony dissociate in the audiovisual bounce-inducing effect (ABE)

A sound can cause 2 visual streaming objects appear to bounce (the audiovisual bounce-inducing effect, ABE). Here we examined whether the stream/bounce percept affects perception of audiovisual synchrony. Participants saw 2 disks that either clearly streamed, clearly bounced, or were ambiguous, and heard a sound around the point of contact (POC). They reported, on each trial, whether they perceived the disks to 'stream' or 'bounce', and whether the sound was 'synchronous' or 'asynchronous' with the POC. Results showed that the optimal time of the sound to induce a bounce was before the POC (-59 msec), whereas audiovisual synchrony was maximal when the sound came after the POC (+16 msec). The range of temporal asynchronies perceived as 'synchronous', the temporal binding window (TBW), was wider when disks were perceived as bouncing than streaming, with no difference between ambiguous and non-ambiguous visual displays. These results demonstrate 1) that causality differs from synchrony, 2) that causality widens the TBW, and 3) that the ABE is perceptually real.

Cross-modal psychological refractory period in vision, audition, and haptics

People's parallel-processing ability is limited, as demonstrated by the psychological refractory period (PRP) effect: The reaction time to the second stimulus (RT2) increases as the stimulus onset asynchrony (SOA) between two stimuli decreases. Most theoretical models of PRP are independent of modalities. Previous research on PRP mainly focused on vision and audition as input modalities; tactile stimuli have not been fully explored. Research using other paradigms and involving tactile stimuli, however, found that dual-task performance depended on input modalities. This study explored PRP with all the combinations of input modalities. Thirty participants judged the magnitude (small or large) of two stimuli presented in different modalities with an SOA of 75-1,200 ms. PRP effect was observed, i.e., RT2 increased with a decreasing SOA, in all the modalities. Only in the auditory-tactile condition did the accuracy of Task 2 decrease with a decreasing SOA. In the auditory-tactile and tactile-visual conditions, RT to the first stimulus also increased with a decreasing SOA. Current models could only explain part of the results, and modality characteristics help to explain the overall data pattern better. Limitations and directions for future studies regarding reaction time, task difficulty, and response modalities are discussed.

Older adults sacrifice response speed to preserve multisensory integration performance

Aging has been shown to impact multisensory perception, but the underlying computational mechanisms are unclear. For effective interactions with the environment, observers should integrate signals that share a common source, weighted by their reliabilities, and segregate those from separate sources. Observers are thought to accumulate evidence about the world's causal structure over time until a decisional threshold is reached. Combining psychophysics and Bayesian modeling, we investigated how aging affects audiovisual perception of spatial signals. Older and younger adults were comparable in their final localization and common-source judgment responses under both speeded and unspeeded conditions, but were disproportionately slower for audiovisually incongruent trials. Bayesian modeling showed that aging did not affect the ability to arbitrate between integration and segregation under either unspeeded or speeded conditions. However, modeling the within-trial dynamics of evidence accumulation under speeded conditions revealed that older observers accumulate noisier auditory representations for longer, set higher decisional thresholds, and have impaired motor speed. Older observers preserve audiovisual localization performance, despite noisier sensory representations, by sacrificing response speed.

Interference of irrelevant information in multisensory selection depends on attentional set

In the multisensory world in which we live, certain objects and events are of more relevance than others. In the laboratory, this broadly equates to the distinction between targets and distractors. In selection situations like the flanker task, the evidence suggests that the processing of multisensory distractors is influenced by attention. Here, multisensory distractor processing was investigated by modulating attentional set in three experiments in a flanker interference task, in which the targets were unisensory while the distractors were multisensory. Attentional set was modulated by making the target modality either predictable or unpredictable (Experiments 1 vs. 2, respectively). In Experiment 3, this manipulation was implemented on a within-experiment basis. Furthermore, the third experiment compared audiovisual distractors (used in all experiments) with distractors with one feature in a neutral modality (i.e., touch), that never appeared as the target modality in the flanker task. The results demonstrate that there was no interference from the response-compatible crossmodal distractor feature when the target modality was predictable (i.e., blocked). However, when the modality was varied on a trial-by-trial basis, this crossmodal feature significantly influenced information processing. By contrast, a multisensory distractor with a neutral crossmodal feature never influenced behavior. This finding suggests that the processing of multisensory distractors depends on attentional set. When the target modality varies randomly, participants include features from both modalities in their attentional set and the irrelevant crossmodal feature, now part of the set, influences information processing. In contrast, interference from the crossmodal distractor feature does not occur when it is not part of the attentional set.

Perception it is: Processing level in multisensory selection

When repeatedly exposed to simultaneously presented stimuli, associations between these stimuli are nearly always established, both within as well as between sensory modalities. Such associations guide our subsequent actions and may also play a role in multisensory selection. Thus, crossmodal associations (i.e., associations between stimuli from different modalities) learned in a multisensory interference task might affect subsequent information processing. The aim of this study was to investigate the processing level of multisensory stimuli in multisensory selection by means of crossmodal aftereffects. Either feature or response associations were induced in a multisensory flanker task while the amount of interference in a subsequent crossmodal flanker task was measured. The results of Experiment 1 revealed the existence of crossmodal interference after multisensory selection. Experiments 2 and 3 then went on to demonstrate the dependence of this effect on the perceptual associations between features themselves, rather than on the associations between feature and response. Establishing response associations did not lead to a subsequent crossmodal interference effect (Experiment 2), while stimulus feature associations without response associations (obtained by changing the response effectors) did (Experiment 3). Taken together, this pattern of results suggests that associations in multisensory selection, and the interference of (crossmodal) distractors, predominantly work at the perceptual, rather than at the response, level.

Integration of audiovisual spatial signals is not consistent with maximum likelihood estimation

Multisensory perception is regarded as one of the most prominent examples where human behaviour conforms to the computational principles of maximum likelihood estimation (MLE). In particular, observers are thought to integrate auditory and visual spatial cues weighted in proportion to their relative sensory reliabilities into the most reliable and unbiased percept consistent with MLE. Yet, evidence to date has been inconsistent. The current pre-registered, large-scale (N = 36) replication study investigated the extent to which human behaviour for audiovisual localization is in line with maximum likelihood estimation. The acquired psychophysics data show that while observers were able to reduce their multisensory variance relative to the unisensory variances in accordance with MLE, they weighed the visual signals significantly stronger than predicted by MLE. Simulations show that this dissociation can be explained by a greater sensitivity of standard estimation procedures to detect deviations from MLE predictions for sensory weights than for audiovisual variances. Our results therefore suggest that observers did not integrate audiovisual spatial signals weighted exactly in proportion to their relative reliabilities for localization. These small deviations from the predictions of maximum likelihood estimation may be explained by observers' uncertainty about the world's causal structure as accounted for by Bayesian causal inference.

Metrical congruency and kinematic familiarity facilitate temporal binding between musical and dance rhythms

Although music and dance are often experienced simultaneously, it is unclear what modulates their perceptual integration. This study investigated how two factors related to music-dance correspondences influenced audiovisual binding of their rhythms: the metrical match between the music and dance, and the kinematic familiarity of the dance movement. Participants watched a point-light figure dancing synchronously to a triple-meter rhythm that they heard in parallel, whereby the dance communicated a triple (congruent) or a duple (incongruent) visual meter. The movement was either the participant's own or that of another participant. Participants attended to both streams while detecting a temporal perturbation in the auditory beat. The results showed lower sensitivity to the auditory deviant when the visual dance was metrically congruent to the auditory rhythm and when the movement was the participant's own. This indicated stronger audiovisual binding and a more coherent bimodal rhythm in these conditions, thus making a slight auditory deviant less noticeable. Moreover, binding in the metrically incongruent condition involving self-generated visual stimuli was correlated with self-recognition of the movement, suggesting that action simulation mediates the perceived coherence between one's own movement and a mismatching auditory rhythm. Overall, the mechanisms of rhythm perception and action simulation could inform the perceived compatibility between music and dance, thus modulating the temporal integration of these audiovisual stimuli.

The role of multisensory development in early language learning

In typical development, communicative skills such as language emerge from infants' ability to combine multisensory information into cohesive percepts. For example, the act of associating the visual or tactile experience of an object with its spoken name is commonly used as a measure of early word learning, and social attention and speech perception frequently involve integrating both visual and auditory attributes. Early perspectives once regarded perceptual integration as one of infants' primary challenges, whereas recent work suggests that caregivers' social responses contain structured patterns that may facilitate infants' perception of multisensory social cues. In the current review, we discuss the regularities within caregiver feedback that may allow infants to more easily discriminate and learn from social signals. We focus on the statistical regularities that emerge in the moment-by-moment behaviors observed in studies of naturalistic caregiver-infant play. We propose that the spatial form and contingencies of caregivers' responses to infants' looks and prelinguistic vocalizations facilitate communicative and cognitive development. We also explore how individual differences in infants' sensory and motor abilities may reciprocally influence caregivers' response patterns, in turn regulating and constraining the types of social learning opportunities that infants experience across early development. We end by discussing implications for neurodevelopmental conditions affecting both multisensory integration and communication (i.e., autism) and suggest avenues for further research and intervention.

Auditory capture of visual apparent motion, both laterally and looming

Traditional tests of multisensory stimuli typically support that vision dominates spatial judgments and audition dominates temporal ones. Here, we examine if unambiguous auditory spatial cues can capture ambiguous visual ones in judgments of direction of apparent motion. The visual motion judgments include both lateral movement and movement in depth, each when coupled with auditory stimuli moving at one of four rates. Experiment 1 tested lateral visual movement judgments (leftward vs rightward) coupled with auditory stimuli that moved laterally. Experiment 2 tested depth visual movement judgments (approaching vs receding) coupled with auditory stimuli that got louder or quieter. Results of Experiment 1 revealed and replicated an overall leftward motion bias, but with additional acoustic capture to experience visual movement away from the side on which sound initially occurred, and no effect of auditory motion speed. Results of Experiment 2 revealed and replicated an approaching motion bias, but with no effect of initial sound intensity, and an additional systematic capture effect of auditory motion speed. Faster changes in acoustic intensity produced larger visual motion capture consistent with the direction of acoustic intensity change. Findings of both experiments generalized over conditions of listening device (head phones vs speakers) and test-setting (Laboratory vs Web-based data-collection). The leftward and approaching motion bias results replicate previous research. Our principal new findings, the auditory motion capture effects, confirm the multisensory nature of dynamic spatial perception and support that extent of inter-sensory capture is a function of the relative reliability of spatial information acquired by each sensory modality.

Long-term music training modulates the recalibration of audiovisual simultaneity

To overcome differences in physical transmission time and neural processing, the brain adaptively recalibrates the point of simultaneity between auditory and visual signals by adapting to audiovisual asynchronies. Here, we examine whether the prolonged recalibration process of passively sensed visual and auditory signals is affected by naturally occurring multisensory training known to enhance audiovisual perceptual accuracy. Hence, we asked a group of drummers, of non-drummer musicians and of non-musicians to judge the audiovisual simultaneity of musical and non-musical audiovisual events, before and after adaptation with two fixed audiovisual asynchronies. We found that the recalibration for the musicians and drummers was in the opposite direction (sound leading vision) to that of non-musicians (vision leading sound), and change together with both increased music training and increased perceptual accuracy (i.e. ability to detect asynchrony). Our findings demonstrate that long-term musical training reshapes the way humans adaptively recalibrate simultaneity between auditory and visual signals.

Multimodal mental imagery

When I am looking at my coffee machine that makes funny noises, this is an instance of multisensory perception – I perceive this event by means of both vision and audition. But very often we only receive sensory stimulation from a multisensory event by means of one sense modality, for example, when I hear the noisy coffee machine in the next room, that is, without seeing it. The aim of this paper is to bring together empirical findings about multimodal perception and empirical findings about (visual, auditory, tactile) mental imagery and argue that on occasions like this, we have multimodal mental imagery: perceptual processing in one sense modality (here: vision) that is triggered by sensory stimulation in another sense modality (here: audition). Multimodal mental imagery is not a rare and obscure phenomenon. The vast majority of what we perceive are multisensory events: events that can be perceived in more than one sense modality – like the noisy coffee machine. And most of the time we are only acquainted with these multisensory events via a subset of the sense modalities involved – all the other aspects of these multisensory events are represented by means of multisensory mental imagery. This means that multisensory mental imagery is a crucial element of almost all instances of everyday perception.

Multifactorial assessment of measurement errors affecting intraoral quantitative sensory testing reliability

BACKGROUND AND PURPOSE (AIMS)

Measurement error of intraoral quantitative sensory testing (QST) has been assessed using traditional methods for reliability, such as intraclass correlation coefficients (ICCs). Most studies reporting QST reliability focused on assessing one source of measurement error at a time, e.g., inter- or intra-examiner (test-retest) reliabilities and employed two examiners to test inter-examiner reliability. The present study used a complex design with multiple examiners with the aim of assessing the reliability of intraoral QST taking account of multiple sources of error simultaneously.

METHODS

Four examiners of varied experience assessed 12 healthy participants in two visits separated by 48h. Seven QST procedures to determine sensory thresholds were used: cold detection (CDT), warmth detection (WDT), cold pain (CPT), heat pain (HPT), mechanical detection (MDT), mechanical pain (MPT) and pressure pain (PPT). Mixed linear models were used to estimate variance components for reliability assessment; dependability coefficients were used to simulate alternative test scenarios.

RESULTS

Most intraoral QST variability arose from differences between participants (8.8-30.5%), differences between visits within participant (4.6-52.8%), and error (13.3-28.3%). For QST procedures other than CDT and MDT, increasing the number of visits with a single examiner performing the procedures would lead to improved dependability (dependability coefficient ranges: single visit, four examiners=0.12-0.54; four visits, single examiner=0.27-0.68). A wide range of reliabilities for QST procedures, as measured by ICCs, was noted for inter- (0.39-0.80) and intra-examiner (0.10-0.62) variation.

CONCLUSION

Reliability of sensory testing can be better assessed by measuring multiple sources of error simultaneously instead of focusing on one source at a time. In experimental settings, large numbers of participants are needed to obtain accurate estimates of treatment effects based on QST measurements. This is different from clinical use, where variation between persons (the person main effect) is not a concern because clinical measurements are done on a single person.

IMPLICATIONS

Future studies assessing sensory testing reliability in both clinical and experimental settings would benefit from routinely measuring multiple sources of error. The methods and results of this study can be used by clinical researchers to improve assessment of measurement error related to intraoral sensory testing. This should lead to improved resource allocation when designing studies that use intraoral quantitative sensory testing in clinical and experimental settings.

The organization and reorganization of audiovisual speech perception in the first year of life

The period between six and 12 months is a sensitive period for language learning during which infants undergo auditory perceptual attunement, and recent results indicate that this sensitive period may exist across sensory modalities. We tested infants at three stages of perceptual attunement (six, nine, and 11 months) to determine 1) whether they were sensitive to the congruence between heard and seen speech stimuli in an unfamiliar language, and 2) whether familiarization with congruent audiovisual speech could boost subsequent non-native auditory discrimination. Infants at six- and nine-, but not 11-months, detected audiovisual congruence of non-native syllables. Familiarization to incongruent, but not congruent, audiovisual speech changed auditory discrimination at test for six-month-olds but not nine- or 11-month-olds. These results advance the proposal that speech perception is audiovisual from early in ontogeny, and that the sensitive period for audiovisual speech perception may last somewhat longer than that for auditory perception alone.

"Smooth operator": Music modulates the perceived creaminess, sweetness, and bitterness of chocolate

There has been a recent growth of interest in determining whether sound (specifically music and soundscapes) can enhance not only the basic taste attributes associated with food and beverage items (such as sweetness, bitterness, sourness, etc.), but also other important components of the tasting experience, such as, for instance, crunchiness, creaminess, and/or carbonation. In the present study, participants evaluated the perceived creaminess of chocolate. Two contrasting soundtracks were produced with such texture-correspondences in mind, and validated by means of a pre-test. The participants tasted the same chocolate twice (without knowing that the chocolates were identical), each time listening to one of the soundtracks. The 'creamy' soundtrack enhanced the perceived creaminess and sweetness of the chocolates, as compared to the ratings given while listening to the 'rough' soundtrack. Moreover, while the participants preferred the creamy soundtrack, this difference did not appear to affect their overall enjoyment of the chocolates. Interestingly, and in contrast with previous similar studies, these results demonstrate that in certain cases, sounds can have a perceptual effect on gustatory food attributes without necessarily altering the hedonic experience.

What you see isn't always what you get: Auditory word signals trump consciously perceived words in lexical access

Human speech perception often includes both an auditory and visual component. A conflict in these signals can result in the McGurk illusion, in which the listener perceives a fusion of the two streams, implying that information from both has been integrated. We report two experiments investigating whether auditory-visual integration of speech occurs before or after lexical access, and whether the visual signal influences lexical access at all. Subjects were presented with McGurk or Congruent primes and performed a lexical decision task on related or unrelated targets. Although subjects perceived the McGurk illusion, McGurk and Congruent primes with matching real-word auditory signals equivalently primed targets that were semantically related to the auditory signal, but not targets related to the McGurk percept. We conclude that the time course of auditory-visual integration is dependent on the lexicality of the auditory and visual input signals, and that listeners can lexically access one word and yet consciously perceive another.

Multisensory integration in hemianopia and unilateral spatial neglect: Evidence from the sound induced flash illusion

Recent neuropsychological evidence suggests that acquired brain lesions can, in some instances, abolish the ability to integrate inputs from different sensory modalities, disrupting multisensory perception. We explored the ability to perceive multisensory events, in particular the integrity of audio-visual processing in the temporal domain, in brain-damaged patients with visual field defects (VFD), or with unilateral spatial neglect (USN), by assessing their sensitivity to the 'Sound-Induced Flash Illusion' (SIFI). The study yielded two key findings. Firstly, the 'fission' illusion (namely, seeing multiple flashes when a single flash is paired with multiple sounds) is reduced in both left- and right-brain-damaged patients with VFD, but not in right-brain-damaged patients with left USN. The disruption of the fission illusion is proportional to the extent of the occipital damage. Secondly, a reliable 'fusion' illusion (namely, seeing less flashes when a single sound is paired with multiple flashes) is evoked in USN patients, but neither in VFD patients nor in healthy participants. A control experiment showed that the fusion, but not the fission, illusion is lost in older participants (>50 year-old), as compared with younger healthy participants (<30 year-old). This evidence indicates that the fission and fusion illusions are dissociable multisensory phenomena, altered differently by impairments of visual perception (i.e. VFD) and spatial attention (i.e. USN). The occipital cortex represents a key cortical site for binding auditory and visual stimuli in the SIFI, while damage to right-hemisphere areas mediating spatial attention and awareness does not prevent the integration of audio-visual inputs in the temporal domain.

Combined acoustical and visual performance of noise barriers in mitigating the environmental impact of motorways

This study investigated the overall performance of noise barriers in mitigating environmental impact of motorways, taking into consideration their effects on reducing noise and visual intrusions of moving traffic, but also potentially inducing visual impact themselves. A laboratory experiment was carried out, using computer-visualised video scenes and motorway traffic noise recordings to present experimental scenarios covering two traffic levels, two distances of receiver to road, two types of background landscape, and five barrier conditions including motorway only, motorway with tree belt, motorways with 3 m timber barrier, 5m timber barrier, and 5m transparent barrier. Responses from 30 participants of university students were gathered and perceived barrier performance analysed. The results show that noise barriers were always beneficial in mitigating environmental impact of motorways, or made no significant changes in environmental quality when the impact of motorways was low. Overall, barriers only offered similar mitigation effect as compared to tree belt, but showed some potential to be more advantageous when traffic level went high. 5m timber barrier tended to perform better than the 3m one at the distance of 300 m but not at 100 m possibly due to its negative visual effect when getting closer. The transparent barrier did not perform much differently from the timber barriers but tended to be the least effective in most scenarios. Some low positive correlations were found between aesthetic preference for barriers and environmental impact reduction by the barriers.

Perception of Multisensory Gender Coherence in 6- and 9-month-old Infants

One of the most salient social categories conveyed by human faces and voices is gender. We investigated the developmental emergence of the ability to perceive the coherence of auditory and visual attributes of gender in 6- and 9-month-old infants. Infants viewed two side-by-side video clips of a man and a woman singing a nursery rhyme and heard a synchronous male or female soundtrack. Results showed that 6-month-old infants did not match the audible and visible attributes of gender, and 9-month-old infants matched only female faces and voices. These findings indicate that the ability to perceive the multisensory coherence of gender emerges relatively late in infancy and that it reflects the greater experience that most infants have with female faces and voices.

Perception of the multisensory coherence of fluent audiovisual speech in infancy: its emergence and the role of experience

To investigate the developmental emergence of the perception of the multisensory coherence of native and non-native audiovisual fluent speech, we tested 4-, 8- to 10-, and 12- to 14-month-old English-learning infants. Infants first viewed two identical female faces articulating two different monologues in silence and then in the presence of an audible monologue that matched the visible articulations of one of the faces. Neither the 4-month-old nor 8- to 10-month-old infants exhibited audiovisual matching in that they did not look longer at the matching monologue. In contrast, the 12- to 14-month-old infants exhibited matching and, consistent with the emergence of perceptual expertise for the native language, perceived the multisensory coherence of native-language monologues earlier in the test trials than that of non-native language monologues. Moreover, the matching of native audible and visible speech streams observed in the 12- to 14-month-olds did not depend on audiovisual synchrony, whereas the matching of non-native audible and visible speech streams did depend on synchrony. Overall, the current findings indicate that the perception of the multisensory coherence of fluent audiovisual speech emerges late in infancy, that audiovisual synchrony cues are more important in the perception of the multisensory coherence of non-native speech than that of native audiovisual speech, and that the emergence of this skill most likely is affected by perceptual narrowing.

Audiovisual temporal fusion in 6-month-old infants

The aim of this study was to investigate neural dynamics of audiovisual temporal fusion processes in 6-month-old infants using event-related brain potentials (ERPs). In a habituation-test paradigm, infants did not show any behavioral signs of discrimination of an audiovisual asynchrony of 200 ms, indicating perceptual fusion. In a subsequent EEG experiment, audiovisual synchronous stimuli and stimuli with a visual delay of 200 ms were presented in random order. In contrast to the behavioral data, brain activity differed significantly between the two conditions. Critically, N1 and P2 latency delays were not observed between synchronous and fused items, contrary to previously observed N1 and P2 latency delays between synchrony and perceived asynchrony. Hence, temporal interaction processes in the infant brain between the two sensory modalities varied as a function of perceptual fusion versus asynchrony perception. The visual recognition components Pb and Nc were modulated prior to sound onset, emphasizing the importance of anticipatory visual events for the prediction of auditory signals. Results suggest mechanisms by which young infants predictively adjust their ongoing neural activity to the temporal synchrony relations to be expected between vision and audition.

Object concepts in the chemical senses

This paper examines the applicability of the object concept to the chemical senses, by evaluating them against a set of criteria for object-hood. Taste and chemesthesis do not generate objects. Their parts, perceptible from birth, never combine. Orthonasal olfaction (sniffing) presents a strong case for generating objects. Odorants have many parts yet they are perceived as wholes, this process is based on learning, and there is figure-ground segregation. While flavors are multimodal representations bound together by learning, there is no functional need for flavor objects in the mouth. Rather, food identification occurs prior to ingestion using the eye and nose, with the latter retrieving multimodal flavor objects via sniffing (e.g., sweet smelling caramel). While there are differences in object perception between vision, audition, and orthonasal olfaction, the commonalities suggest that the brain has adopted the same basic solution when faced with extracting meaning from complex stimulus arrays.