1
|
Chang DHF, Thinnes D, Au PY, Maziero D, Stenger VA, Sinnett S, Vibell J. Sound-modulations of visual motion perception implicate the cortico-vestibular brain. Neuroimage 2022; 257:119285. [PMID: 35537600 DOI: 10.1016/j.neuroimage.2022.119285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2022] [Revised: 04/20/2022] [Accepted: 05/05/2022] [Indexed: 11/19/2022] Open
Abstract
A widely used example of the intricate (yet poorly understood) intertwining of multisensory signals in the brain is the audiovisual bounce inducing effect (ABE). This effect presents two identical objects moving along the azimuth with uniform motion and towards opposite directions. The perceptual interpretation of the motion is ambiguous and is modulated if a transient (sound) is presented in coincidence with the point of overlap of the two objects' motion trajectories. This phenomenon has long been written-off to simple attentional or decision-making mechanisms, although the neurological underpinnings for the effect are not well understood. Using behavioural metrics concurrently with event-related fMRI, we show that sound-induced modulations of motion perception can be further modulated by changing motion dynamics of the visual targets. The phenomenon engages the posterior parietal cortex and the parieto-insular-vestibular cortical complex, with a close correspondence of activity in these regions with behaviour. These findings suggest that the insular cortex is engaged in deriving a probabilistic perceptual solution through the integration of multisensory data.
Collapse
Affiliation(s)
- Dorita H F Chang
- Department of Psychology and The State Key Laboratory of Brain and Cognitive Sciences, The University of Hong Kong, Hong Kong.
| | - David Thinnes
- Department of Psychology, University of Hawai'i at Mānoa, Hawaii, USA; Faculty of Medicine, Systems Neuroscience & Neurotechnology Unit, Saarland University & HTW Saar, Germany
| | - Pak Yam Au
- Department of Psychology and The State Key Laboratory of Brain and Cognitive Sciences, The University of Hong Kong, Hong Kong
| | - Danilo Maziero
- Department of Medicine, MR Research Program, John A. Burns School of Medicine, University of Hawai'i, HI, USA
| | - Victor Andrew Stenger
- Department of Medicine, MR Research Program, John A. Burns School of Medicine, University of Hawai'i, HI, USA
| | - Scott Sinnett
- Department of Psychology, University of Hawai'i at Mānoa, Hawaii, USA
| | - Jonas Vibell
- Department of Psychology, University of Hawai'i at Mānoa, Hawaii, USA.
| |
Collapse
|
2
|
Flotho P, Bhamborae MJ, Grün T, Trenado C, Thinnes D, Limbach D, Strauss DJ. Multimodal data acquisition at SARS-CoV-2 drive through screening centers: Setup description and experiences in Saarland, Germany. J Biophotonics 2021; 14:e202000512. [PMID: 33987984 DOI: 10.1002/jbio.202000512] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/28/2020] [Revised: 04/11/2021] [Accepted: 05/04/2021] [Indexed: 06/12/2023]
Abstract
SARS-CoV-2 drive through screening centers (DTSC) have been implemented worldwide as a fast and secure way of mass screening. We use DTSCs as a platform for the acquisition of multimodal datasets that are needed for the development of remote screening methods. Our acquisition setup consists of an array of thermal, infrared and RGB cameras as well as microphones and we apply methods from computer vision and computer audition for the contactless estimation of physiological parameters. We have recorded a multimodal dataset of DTSC participants in Germany for the development of remote screening methods and symptom identification. Acquisition in the early stages of a pandemic and in regions with high infection rates can facilitate and speed up the identification of infection specific symptoms and large-scale data acquisition at DTSC is possible without disturbing the flow of operation.
Collapse
Affiliation(s)
- Philipp Flotho
- Systems Neuroscience and Neurotechnology Unit, Neurocenter, Faculty of Medicine, Saarland University and School of Engineering, htw saar, Saarbrücken, Germany
| | - Mayur J Bhamborae
- Systems Neuroscience and Neurotechnology Unit, Neurocenter, Faculty of Medicine, Saarland University and School of Engineering, htw saar, Saarbrücken, Germany
| | - Tobias Grün
- Systems Neuroscience and Neurotechnology Unit, Neurocenter, Faculty of Medicine, Saarland University and School of Engineering, htw saar, Saarbrücken, Germany
| | - Carlos Trenado
- Systems Neuroscience and Neurotechnology Unit, Neurocenter, Faculty of Medicine, Saarland University and School of Engineering, htw saar, Saarbrücken, Germany
| | - David Thinnes
- Systems Neuroscience and Neurotechnology Unit, Neurocenter, Faculty of Medicine, Saarland University and School of Engineering, htw saar, Saarbrücken, Germany
| | - Dominik Limbach
- Systems Neuroscience and Neurotechnology Unit, Neurocenter, Faculty of Medicine, Saarland University and School of Engineering, htw saar, Saarbrücken, Germany
| | - Daniel J Strauss
- Systems Neuroscience and Neurotechnology Unit, Neurocenter, Faculty of Medicine, Saarland University and School of Engineering, htw saar, Saarbrücken, Germany
| |
Collapse
|
3
|
Chang DHF, Thinnes D, Au PY, Maziero D, Stenger VA, Sinnett S, Vibell J. Bounce or stream? Motion dynamics modulate the audiovisual bounce inducing effect. J Vis 2020. [DOI: 10.1167/jov.20.11.953] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022] Open
Affiliation(s)
| | - David Thinnes
- University of Hawaiʻi at Mānoa
- Systems Neuroscience and Neurotechnology Unit, Homburg/Saarland
| | | | | | | | | | | |
Collapse
|