1
|
Zhao T, Hu A, Su R, Lyu C, Wang L, Yan N. Phonetic versus spatial processes during motor-oriented imitations of visuo-labial and visuo-lingual speech: A functional near-infrared spectroscopy study. Eur J Neurosci 2021; 55:154-174. [PMID: 34854143 DOI: 10.1111/ejn.15550] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2021] [Revised: 10/21/2021] [Accepted: 11/23/2021] [Indexed: 12/28/2022]
Abstract
While a large amount of research has studied the facilitation of visual speech on auditory speech recognition, few have investigated the processing of visual speech gestures in motor-oriented tasks that focus on the spatial and motor features of the articulator actions instead of the phonetic features of auditory and visual speech. The current study examined the engagement of spatial and phonetic processing of visual speech in a motor-oriented speech imitation task. Functional near-infrared spectroscopy (fNIRS) was used to measure the haemodynamic activities related to spatial processing and audiovisual integration in the superior parietal lobe (SPL) and the posterior superior/middle temporal gyrus (pSTG/pMTG) respectively. In addition, visuo-labial and visuo-lingual speech were compared with examine the influence of visual familiarity and audiovisual association on the processes in question. fNIRS revealed significant activations in the SPL but found no supra-additive audiovisual activations in the pSTG/pMTG, suggesting that the processing of audiovisual speech stimuli was primarily focused on spatial processes related to action comprehension and preparation, whereas phonetic processes related to audiovisual integration was minimal. Comparisons between visuo-labial and visuo-lingual speech imitations revealed no significant difference in the activation of the SPL or the pSTG/pMTG, suggesting that a higher degree of visual familiarity and audiovisual association did not significantly influence how visuo-labial speech was processed compared with visuo-lingual speech. The current study offered insights on the pattern of visual-speech processing under a motor-oriented task objective and provided further evidence for the modulation of multimodal speech integration by voluntary selective attention and task objective.
Collapse
Affiliation(s)
- Tinghao Zhao
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.,Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Anming Hu
- Department of Rehabilitation Medicine, Beijing Tiantan Hospital, Capital Medical University, Beijing, China
| | - Rongfeng Su
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.,Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Chengchen Lyu
- Institute of Software, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, China
| | - Lan Wang
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.,Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| | - Nan Yan
- CAS Key Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.,Guangdong-Hong Kong-Macao Joint Laboratory of Human-Machine Intelligence-Synergy Systems, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China
| |
Collapse
|
2
|
Frankford SA, Nieto-Castañón A, Tourville JA, Guenther FH. Reliability of single-subject neural activation patterns in speech production tasks. BRAIN AND LANGUAGE 2021; 212:104881. [PMID: 33278802 PMCID: PMC7781091 DOI: 10.1016/j.bandl.2020.104881] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/09/2020] [Revised: 09/25/2020] [Accepted: 11/06/2020] [Indexed: 06/12/2023]
Abstract
Speech neuroimaging research targeting individual speakers could help elucidate differences that may be crucial to understanding speech disorders. However, this research necessitates reliable brain activation across multiple speech production sessions. In the present study, we evaluated the reliability of speech-related brain activity measured by functional magnetic resonance imaging data from twenty neuro-typical subjects who participated in two experiments involving reading aloud simple speech stimuli. Using traditional methods like the Dice and intraclass correlation coefficients, we found that most individuals displayed moderate to high reliability. We also found that a novel machine-learning subject classifier could identify these individuals by their speech activation patterns with 97% accuracy from among a dataset of seventy-five subjects. These results suggest that single-subject speech research would yield valid results and that investigations into the reliability of speech activation in people with speech disorders are warranted.
Collapse
Affiliation(s)
- Saul A Frankford
- Department of Speech, Language, & Hearing Sciences, Boston University, Boston, MA 02215, USA.
| | - Alfonso Nieto-Castañón
- Department of Speech, Language, & Hearing Sciences, Boston University, Boston, MA 02215, USA
| | - Jason A Tourville
- Department of Speech, Language, & Hearing Sciences, Boston University, Boston, MA 02215, USA.
| | - Frank H Guenther
- Department of Speech, Language, & Hearing Sciences, Boston University, Boston, MA 02215, USA; Department of Biomedical Engineering, Boston University, Boston, MA 02215, USA; Department of Radiology, Massachusetts General Hospital, Boston, MA 02114, USA.
| |
Collapse
|
3
|
Grabski K, Sato M. Adaptive phonemic coding in the listening and speaking brain. Neuropsychologia 2020; 136:107267. [DOI: 10.1016/j.neuropsychologia.2019.107267] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2019] [Revised: 10/23/2019] [Accepted: 11/15/2019] [Indexed: 10/25/2022]
|
4
|
Longcamp M, Hupé JM, Ruiz M, Vayssière N, Sato M. Shared premotor activity in spoken and written communication. BRAIN AND LANGUAGE 2019; 199:104694. [PMID: 31586790 DOI: 10.1016/j.bandl.2019.104694] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/07/2018] [Revised: 09/12/2019] [Accepted: 09/15/2019] [Indexed: 06/10/2023]
Abstract
The aim of the present study was to uncover a possible common neural organizing principle in spoken and written communication, through the coupling of perceptual and motor representations. In order to identify possible shared neural substrates for processing the basic units of spoken and written language, a sparse sampling fMRI acquisition protocol was performed on the same subjects in two experimental sessions with similar sets of letters being read and written and of phonemes being heard and orally produced. We found evidence of common premotor regions activated in spoken and written language, both in perception and in production. The location of those brain regions was confined to the left lateral and medial frontal cortices, at locations corresponding to the premotor cortex, inferior frontal cortex and supplementary motor area. Interestingly, the speaking and writing tasks also appeared to be controlled by largely overlapping networks, possibly indicating some domain general cognitive processing. Finally, the spatial distribution of individual activation peaks further showed more dorsal and more left-lateralized premotor activations in written than in spoken language.
Collapse
Affiliation(s)
| | - Jean-Michel Hupé
- CNRS, Université de Toulouse Paul Sabatier, CerCo, Toulouse, France
| | - Mathieu Ruiz
- CNRS, Université de Toulouse Paul Sabatier, CerCo, Toulouse, France
| | - Nathalie Vayssière
- CNRS, Université de Toulouse Paul Sabatier, CerCo, Toulouse, France; Toulouse Mind and Brain Institute, France
| | - Marc Sato
- CNRS, Aix-Marseille Univ, LPL, Aix-en-Provence, France
| |
Collapse
|
5
|
Carey D, Krishnan S, Callaghan MF, Sereno MI, Dick F. Functional and Quantitative MRI Mapping of Somatomotor Representations of Human Supralaryngeal Vocal Tract. Cereb Cortex 2018; 27:265-278. [PMID: 28069761 PMCID: PMC5808730 DOI: 10.1093/cercor/bhw393] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2016] [Indexed: 12/15/2022] Open
Abstract
Speech articulation requires precise control of and coordination between the effectors of the vocal tract (e.g., lips, tongue, soft palate, and larynx). However, it is unclear how the cortex represents movements of and contact between these effectors during speech, or how these cortical responses relate to inter-regional anatomical borders. Here, we used phase-encoded fMRI to map somatomotor representations of speech articulations. Phonetically trained participants produced speech phones, progressing from front (bilabial) to back (glottal) place of articulation. Maps of cortical myelin proxies (R1 = 1/T1) further allowed us to situate functional maps with respect to anatomical borders of motor and somatosensory regions. Across participants, we found a consistent topological map of place of articulation, spanning the central sulcus and primary motor and somatosensory areas, that moved from lateral to inferior as place of articulation progressed from front to back. Phones produced at velar and glottal places of articulation activated the inferior aspect of the central sulcus, but with considerable across-subject variability. R1 maps for a subset of participants revealed that articulator maps extended posteriorly into secondary somatosensory regions. These results show consistent topological organization of cortical representations of the vocal apparatus in the context of speech behavior.
Collapse
Affiliation(s)
- Daniel Carey
- Department of Psychology, Royal Holloway, University of London, London, TW20 0EX, UK.,The Irish Longitudinal Study on Ageing, Department of Medical Gerontology, Trinity College Dublin, Dublin 2, Ireland.,Department of Psychological Sciences, Birkbeck College, University of London, Malet St, London, WC1E 7HX, UK
| | - Saloni Krishnan
- Department of Psychological Sciences, Birkbeck College, University of London, Malet St, London, WC1E 7HX, UK.,Department of Experimental Psychology, Tinbergen Building, 9 South Parks Road, Oxford, OX1 3UD, UK
| | - Martina F Callaghan
- Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, 12 Queen Square, London, WC1N 3BG, UK
| | - Martin I Sereno
- Department of Psychological Sciences, Birkbeck College, University of London, Malet St, London, WC1E 7HX, UK.,Birkbeck/UCL Centre for Neuroimaging, 26 Bedford Way, London, WC1H 0AP, UK.,Department of Experimental Psychology, UCL Division of Psychology and Language Sciences, 26 Bedford Way, London, WC1H 0AP, UK.,Department of Psychology, College of Sciences, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-4611, USA
| | - Frederic Dick
- Department of Psychological Sciences, Birkbeck College, University of London, Malet St, London, WC1E 7HX, UK.,Birkbeck/UCL Centre for Neuroimaging, 26 Bedford Way, London, WC1H 0AP, UK
| |
Collapse
|
6
|
Custead R, Oh H, Wang Y, Barlow S. Brain encoding of saltatory velocity through a pulsed pneumotactile array in the lower face. Brain Res 2017; 1677:58-73. [PMID: 28958864 DOI: 10.1016/j.brainres.2017.09.025] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/30/2017] [Revised: 08/31/2017] [Accepted: 09/20/2017] [Indexed: 12/25/2022]
Abstract
Processing dynamic tactile inputs is a primary function of the somatosensory system. Spatial velocity encoding mechanisms by the nervous system are important for skilled movement production and may play a role in recovery of sensorimotor function following neurological insult. Little is known about tactile velocity encoding in mechanosensory trigeminal networks required for speech, suck, mastication, and facial gesture. High resolution functional magnetic resonance imaging (fMRI) was used to investigate the neural substrates of velocity encoding in the human orofacial somatosensory system during unilateral saltatory pneumotactile stimulation of perioral and buccal hairy skin in 20 neurotypical adults. A custom multichannel, scalable pneumotactile array consisting of 7 TAC-Cells was used to present 5 stimulus conditions: 5cm/s, 25cm/s, 65cm/s, ALL-ON synchronous activation, and ALL-OFF. The spatiotemporal organization of whole-brain blood oxygen level-dependent (BOLD) response was analyzed with general linear modeling (GLM) and fitted response estimates of percent signal change to compare activations associated with each velocity, and the main effect of velocity alone. Sequential saltatory inputs to the right lower face produced localized BOLD responses in 6 key regions of interest (ROI) including; contralateral precentral and postcentral gyri, and ipsilateral precentral, superior temporal (STG), supramarginal gyri (SMG), and cerebellum. The spatiotemporal organization of the evoked BOLD response was highly dependent on velocity, with the greatest amplitude of BOLD signal change recorded during the 5cm/s presentation in the contralateral hemisphere. Temporal analysis of BOLD response by velocity indicated rapid adaptation via a scalability of networks processing changing pneumotactile velocity cues.
Collapse
Affiliation(s)
- Rebecca Custead
- Special Education and Communication Disorders, University of Nebraska, Lincoln, NE, USA; Center for Brain, Biology and Behavior, University of Nebraska, Lincoln, NE, USA.
| | - Hyuntaek Oh
- Biological Systems Engineering, University of Nebraska, Lincoln, NE, USA; Center for Brain, Biology and Behavior, University of Nebraska, Lincoln, NE, USA.
| | - Yingying Wang
- Special Education and Communication Disorders, University of Nebraska, Lincoln, NE, USA; Biological Systems Engineering, University of Nebraska, Lincoln, NE, USA; Center for Brain, Biology and Behavior, University of Nebraska, Lincoln, NE, USA.
| | - Steven Barlow
- Special Education and Communication Disorders, University of Nebraska, Lincoln, NE, USA; Biological Systems Engineering, University of Nebraska, Lincoln, NE, USA; Center for Brain, Biology and Behavior, University of Nebraska, Lincoln, NE, USA.
| |
Collapse
|