Construction of An Unbiased Spatio-Temporal Atlas of the Tongue During Speech

Quantifying Velopharyngeal Motion Variation in Speech Sound Production Using an Audio-Informed Dynamic MRI Atlas

New developments in dynamic magnetic resonance imaging (MRI) facilitate high-quality data acquisition of human velopharyngeal deformations in real-time speech. With recently established speech motion atlases, group analysis is made possible via spatially and temporally aligned datasets in the atlas space from a desired population of interest. In practice, when analyzing motion characteristics from various subjects performing a designated speech task, it is observed that different subjects' velopharyngeal deformation patterns could vary during the pronunciation of the same utterance, regardless of the spatial and temporal alignment of their MRI. Since such variation can be subtle, identification and extraction of unique patterns out of these high-dimensional datasets is a challenging task. In this work, we present a method that computes and visualizes subtle deformation variation patterns as principal components of a subject group's dynamic motion fields in the atlas space. Coupled with the real-time speech audio recordings during image acquisition, the key time frames that contain maximum speech variations are identified by the principal components of temporally aligned audio waveforms, which in turn inform the temporal location of the maximum spatial deformation variation. Henceforth, the motion fields between the key frames and the reference frame for each subject are computed and warped into the common atlas space, enabling a direct extraction of motion variation patterns via quantitative analysis. The method was evaluated on a dataset of twelve healthy subjects. Subtle velopharyngeal motion differences were visualized quantitatively to reveal pronunciation-specific patterns among different subjects.

3D Dynamic Spatiotemporal Atlas of the Vocal Tract during Consonant–Vowel Production from 2D Real Time MRI

In this work, we address the problem of creating a 3D dynamic atlas of the vocal tract that captures the dynamics of the articulators in all three dimensions in order to create a global speaker model independent of speaker-specific characteristics. The core steps of the proposed method are the temporal alignment of the real-time MR images acquired in several sagittal planes and their combination with adaptive kernel regression. As a preprocessing step, a reference space was created to be used in order to remove anatomical information of the speakers and keep only the variability in speech production for the construction of the atlas. The adaptive kernel regression makes the choice of atlas time points independently of the time points of the frames that are used as an input for the construction. The evaluation of this atlas construction method was made by mapping two new speakers to the atlas and by checking how similar the resulting mapped images are. The use of the atlas helps in reducing subject variability. The results show that the use of the proposed atlas can capture the dynamic behavior of the articulators and is able to generalize the speech production process by creating a universal-speaker reference space.

4D magnetic resonance imaging atlas construction using temporally aligned audio waveforms in speech

Magnetic resonance (MR) imaging is becoming an established tool in capturing articulatory and physiological motion of the structures and muscles throughout the vocal tract and enabling visual and quantitative assessment of real-time speech activities. Although motion capture speed has been regularly improved by the continual developments in high-speed MR technology, quantitative analysis of multi-subject group data remains challenging due to variations in speaking rate and imaging time among different subjects. In this paper, a workflow of post-processing methods that matches different MR image datasets within a study group is proposed. Each subject's recorded audio waveform during speech is used to extract temporal domain information and generate temporal alignment mappings from their matching pattern. The corresponding image data are resampled by deformable registration and interpolation of the deformation fields, achieving inter-subject temporal alignment between image sequences. A four-dimensional dynamic MR speech atlas is constructed using aligned volumes from four human subjects. Similarity tests between subject and target domains using the squared error, cross correlation, and mutual information measures all show an overall score increase after spatiotemporal alignment. The amount of image variability in atlas construction is reduced, indicating a quality increase in the multi-subject data for groupwise quantitative analysis.

Reliability of Sonoelastography Measurement of Tongue Muscles and Its Application on Obstructive Sleep Apnea

Few studies have explored the feasibility of shear-wave ultrasound elastography (SWUE) for evaluating the upper airways of patients with obstructive sleep apnea (OSA). This study aimed to establish a reliable SWUE protocol for evaluating tongue muscle elasticity and its feasibility and utility in differentiating patients with OSA. Inter-rater and intra-rater reliability of SWUE measurements were tested using the intraclass correlation coefficients. Submental ultrasound was used to measure tongue thickness and stiffness. Association between the ultrasound measurements and presence of OSA was analyzed using multivariate logistic regression. One-way analysis of variance was used to examine if the values of the ultrasound parameters varied among patients with different severities of OSA. Overall, 37 healthy subjects and 32 patients with OSA were recruited. The intraclass correlation coefficients of intra‐ and inter-rater reliability for SWUE for tongue stiffness ranged from 0.84 to 0.90. After adjusting for age, sex, neck circumference, and body mass index, the risk for OSA was positively associated with tongue thickness [odds ratio 1.16 (95% confidence interval 1.01–1.32)] and negatively associated with coronal imaging of tongue muscle stiffness [odds ratio 0.72 (95% confidence interval 0.54–0.95)]. There were no significant differences in tongue stiffness among OSA patients with varying disease severity. SWUE provided a reliable evaluation of tongue muscle stiffness, which appeared to be softer in patients with OSA. Future longitudinal studies are necessary to investigate the relationship between tongue softening and OSA, as well as response to treatment.

Analysis of fiber strain in the human tongue during speech

This study investigates mechanical cooperation among tongue muscles. Five volunteers were imaged using tagged magnetic resonance imaging to quantify spatiotemporal kinematics while speaking. Waveforms of strain in the line of action of fibers (SLAF) were estimated by projecting strain tensors onto a model of fiber directionality. SLAF waveforms were temporally aligned to determine consistency across subjects and correlation across muscles. The cohort exhibited consistent patterns of SLAF, and muscular extension-contraction was correlated. Volume-preserving tongue movement in speech generation can be achieved through multiple paths, but the study reveals similarities in motion patterns and muscular action-despite anatomical (and other) dissimilarities.

Quantifying Tensor Field Similarity With Global Distributions and Optimal Transport

Strain tensor fields quantify tissue deformation and are important for functional analysis of moving organs such as the heart and the tongue. Strain data can be readily obtained using medical imaging. However, quantification of similarity between different data sets is difficult. Strain patterns vary in space and time, and are inherently multidimensional. Also, the same type of mechanical deformation can be applied to different shapes; hence, automatic quantification of similarity should be unaffected by the geometry of the objects being deformed. This work introduces the application of global distributions used to classify shapes and vector fields in the pattern recognition literature, in the context of tensorial strain data. In particular, the distribution of mechanical properties of a field are approximated using a 3D histogram, and the Wasserstein distance from optimal transport theory is used to measure the similarity between histograms. To measure the method's consistency in matching deformations across different objects, the proposed approach was evaluated by sorting strain fields according to their similarity. Performance was compared to sorting via maximum shear distribution (a 1D histogram) and tensor residual magnitude (in perfectly registered objects). The technique was also applied to correlate muscle activation to muscular contraction observed via tagged MRI. The results show that the proposed approach accurately matches deformation regardless of the shape of the object being deformed. Sorting accuracy surpassed 1D shear distribution and was on par with residual magnitude, but without the need for registration between objects.

Inverse Biomechanical Modeling of the Tongue via Machine Learning and Synthetic Training Data

The tongue's deformation during speech can be measured using tagged magnetic resonance imaging, but there is no current method to directly measure the pattern of muscles that activate to produce a given motion. In this paper, the activation pattern of the tongue's muscles is estimated by solving an inverse problem using a random forest. Examples describing different activation patterns and the resulting deformations are generated using a finite-element model of the tongue. These examples form training data for a random forest comprising 30 decision trees to estimate contractions in 262 contractile elements. The method was evaluated on data from tagged magnetic resonance data from actual speech and on simulated data mimicking flaps that might have resulted from glossectomy surgery. The estimation accuracy was modest (5.6% error), but it surpassed a semi-manual approach (8.1% error). The results suggest that a machine learning approach to contraction pattern estimation in the tongue is feasible, even in the presence of flaps.

Speech Map: A Statistical Multimodal Atlas of 4D Tongue Motion During Speech from Tagged and Cine MR Images

Quantitative measurement of functional and anatomical traits of 4D tongue motion in the course of speech or other lingual behaviors remains a major challenge in scientific research and clinical applications. Here, we introduce a statistical multimodal atlas of 4D tongue motion using healthy subjects, which enables a combined quantitative characterization of tongue motion in a reference anatomical configuration. This atlas framework, termed Speech Map, combines cine- and tagged-MRI in order to provide both the anatomic reference and motion information during speech. Our approach involves a series of steps including (1) construction of a common reference anatomical configuration from cine-MRI, (2) motion estimation from tagged-MRI, (3) transformation of the motion estimations to the reference anatomical configuration, and (4) computation of motion quantities such as Lagrangian strain. Using this framework, the anatomic configuration of the tongue appears motionless, while the motion fields and associated strain measurements change over the time course of speech. In addition, to form a succinct representation of the high-dimensional and complex motion fields, principal component analysis is carried out to characterize the central tendencies and variations of motion fields of our speech tasks. Our proposed method provides a platform to quantitatively and objectively explain the differences and variability of tongue motion by illuminating internal motion and strain that have so far been intractable. The findings are used to understand how tongue function for speech is limited by abnormal internal motion and strain in glossectomy patients.

A Spatio-Temporal Atlas and Statistical Model of the Tongue During Speech from Cine-MRI

Statistical modeling of tongue motion during speech using cine magnetic resonance imaging (MRI) provides key information about the relationship between structure and motion of the tongue. In order to study the variability of tongue shape and motion in populations, a consistent integration and characterization of inter-subject variability is needed. In this paper, a method to construct a spatio-temporal atlas comprising a mean motion model and statistical modes of variation during speech is presented. The model is based on the cine-MRI from twenty two normal speakers and consists of several steps involving both spatial and temporal alignment problems independently. First, all images are registered into a common reference space, which is taken to be a neutral resting position of the tongue. Second, the tongue shapes of each individual relative to this reference space are produced. Third, a time warping approach (several are evaluated) is used to align the time frames of each subject to a common time series of initial mean images. Finally, the spatio-temporal atlas is created by time-warping each subject, generating new mean images at each time, and producing shape statistics around these mean images using principal component analysis at each reference time frame. Experimental results consist of comparison of various parameters and methods in creation of the atlas and a demonstration of the final modes of variations at various key time frames in a sample phrase.