Accelerated three-dimensional upper airway MRI using compressed sensing

In speech-production research, three-dimensional (3D) MRI of the upper airway has provided insights into vocal tract shaping and data for its modeling. Small movements of articulators can lead to large changes in the produced sound, therefore improving the resolution of these data sets, within the constraints of a sustained speech sound (6-12 s), is an important area for investigation. The purpose of the study is to provide a first application of compressed sensing (CS) to high-resolution 3D upper airway MRI using spatial finite difference as the sparsifying transform, and to experimentally determine the benefit of applying constraints on image phase. Estimates of image phase are incorporated into the CS reconstruction to improve the sparsity of the finite difference of the solution. In a retrospective subsampling experiment with no sound production, 5x and 4x were the highest acceleration factors that produced acceptable image quality when using a phase constraint and when not using a phase constraint, respectively. The prospective use of a 5x undersampled acquisition and phase-constrained CS reconstruction enabled 3D vocal tract MRI during sustained sound production of English consonants /s/, /integral/, /l/, and /r/ with 1.5 x 1.5 x 2.0 mm(3) spatial resolution and 7 s of scan time.

Closure duration analysis of incomplete stop consonants due to stop-stop interaction

An incomplete stop consonant is characterized either by an indistinguishable closure or a missing burst. If an incomplete stop happens due to a stop following another stop [stop-stop interaction (SSI)], its acoustics typically resemble that of a complete stop-one closure followed by a single burst. As a consequence, stop detectors would fail to distinguish an SSI from a complete stop. Analysis of the TIMIT corpus shows 35.04% incomplete stops (14.97% SSI). It is shown that by using automatically estimated (and hand-labeled) closure duration, complete stops can be distinguished from incomplete stops due to SSI with 69.66% (79.14%) accuracy.

Effect of bandwidth extension to telephone speech recognition in cochlear implant users

The present study investigated a bandwidth extension method to enhance telephone speech understanding for cochlear implant (CI) users. The acoustic information above telephone speech transmission range (i.e., 3400 Hz) was estimated based on trained models describing the relation between narrow-band and wide-band speech. The effect of the bandwidth extension method was evaluated with IEEE sentence recognition tests in seven CI users. Results showed a relatively modest but significant improvement in the speech recognition with the proposed method. The effect of bandwidth extension method was also observed to be highly dependent on individual CI users.

Exploiting Acoustic and Syntactic Features for Automatic Prosody Labeling in a Maximum Entropy Framework

In this paper, we describe a maximum entropy-based automatic prosody labeling framework that exploits both language and speech information. We apply the proposed framework to both prominence and phrase structure detection within the Tones and Break Indices (ToBI) annotation scheme. Our framework utilizes novel syntactic features in the form of supertags and a quantized acoustic-prosodic feature representation that is similar to linear parameterizations of the prosodic contour. The proposed model is trained discriminatively and is robust in the selection of appropriate features for the task of prosody detection. The proposed maximum entropy acoustic-syntactic model achieves pitch accent and boundary tone detection accuracies of 86.0% and 93.1% on the Boston University Radio News corpus, and, 79.8% and 90.3% on the Boston Directions corpus. The phrase structure detection through prosodic break index labeling provides accuracies of 84% and 87% on the two corpora, respectively. The reported results are significantly better than previously reported results and demonstrate the strength of maximum entropy model in jointly modeling simple lexical, syntactic, and acoustic features for automatic prosody labeling.

Effect of spectral normalization on different talker speech recognition by cochlear implant users

In cochlear implants (CIs), different talkers often produce different levels of speech understanding because of the spectrally distorted speech patterns provided by the implant device. A spectral normalization approach was used to transform the spectral characteristics of one talker to those of another talker. In Experiment 1, speech recognition with two talkers was measured in CI users, with and without spectral normalization. Results showed that the spectral normalization algorithm had small but significant effect on performance. In Experiment 2, the effects of spectral normalization were measured in CI users and normal-hearing (NH) subjects; a pitch-stretching technique was used to simulate six talkers with different fundamental frequencies and vocal tract configurations. NH baseline performance was nearly perfect with these pitch-shift transformations. For CI subjects, while there was considerable intersubject variability in performance with the different pitch-shift transformations, spectral normalization significantly improved the intelligibility of these simulated talkers. The results from Experiments 1 and 2 demonstrate that spectral normalization toward more-intelligible talkers significantly improved CI users' speech understanding with less-intelligible talkers. The results suggest that spectral normalization using optimal reference patterns for individual CI patients may compensate for some of the acoustic variability across talkers.

Pathological voice assessment

While there are number of guidelines and methods used in practice, there is no standard universally agreed upon system for assessment of pathological voices. Pathological voices are primarily labeled based on the perceptual judgments of specialists, a process that may result in different label(s) being assigned to a given voice sample. This paper focuses on the recognition of five specific pathologies. The main goal is to compare two different classification methods. The first method considers single label classification by assigning a new label (single label) to the ensembles to which they most likely belong. The second method employs all labels originally assigned to the voice samples. Our results show that the pathological voice assessment performance in the second method is improved with respect to the first method.

Automatic Prosodic Event Detection Using Acoustic, Lexical, and Syntactic Evidence

With the advent of prosody annotation standards such as tones and break indices (ToBI), speech technologists and linguists alike have been interested in automatically detecting prosodic events in speech. This is because the prosodic tier provides an additional layer of information over the short-term segment-level features and lexical representation of an utterance. As the prosody of an utterance is closely tied to its syntactic and semantic content in addition to its lexical content, knowledge of the prosodic events within and across utterances can assist spoken language applications such as automatic speech recognition and translation. On the other hand, corpora annotated with prosodic events are useful for building natural-sounding speech synthesizers. In this paper, we build an automatic detector and classifier for prosodic events in American English, based on their acoustic, lexical, and syntactic correlates. Following previous work in this area, we focus on accent (prominence, or "stress") and prosodic phrase boundary detection at the syllable level. Our experiments achieved a performance rate of 86.75% agreement on the accent detection task, and 91.61% agreement on the phrase boundary detection task on the Boston University Radio News Corpus.

Robust Speech Rate Estimation for Spontaneous Speech

In this paper, we propose a direct method for speech rate estimation from acoustic features without requiring any automatic speech transcription. We compare various spectral and temporal signal analysis and smoothing strategies to better characterize the underlying syllable structure to derive speech rate. The proposed algorithm extends the methods of spectral subband correlation by including temporal correlation and the use of prominent spectral subbands for improving the signal correlation essential for syllable detection. Furthermore, to address some of the practical robustness issues in previously proposed methods, we introduce some novel components into the algorithm such as the use of pitch confidence for filtering spurious syllable envelope peaks, magnifying window for tackling neighboring syllable smearing, and relative peak measure thresholds for pseudo peak rejection. We also describe an automated approach for learning algorithm parameters from data, and find the optimal settings through Monte Carlo simulations and parameter sensitivity analysis. Final experimental evaluations are conducted based on a portion of the Switchboard corpus for which manual phonetic segmentation information, and published results for direct comparison are available. The results show a correlation coefficient of 0.745 with respect to the ground truth based on manual segmentation. This result is about a 17% improvement compared to the current best single estimator and a 11% improvement over the multiestimator evaluated on the same Switchboard database.

Toward articulatory-acoustic models for liquid approximants based on MRI and EPG data. Part I. The laterals

Magnetic resonance images of the vocal tract during the sustained phonation of /l/ (both dark and light allophones) by four native American English talkers are employed for measuring lengths, area functions, and cavity volumes and for the analysis of 3-D vocal tract and tongue shapes. Electropalatography contact profiles are used for studying inter- and intra-talker variabilities and as a source of converging evidence for the magnetic resonance imaging study. The general 3-D tongue body shapes for both allophones of /l/ are characterized by a linguo-alveolar contact together with inward lateral compression and convex cross sections of the posterior tongue body region. The lateral compression along the midsagittal plane enables the creation of flow channels along the sides of the tongue. The bilateral flow channels exhibit somewhat different areas, a characteristic which is talker-dependent. Dark /l/s show smaller pharyngeal areas than the light varieties due to tongue-root retraction and/or posterior tongue body raising. The acoustic implications of the observed geometries are discussed.

A nonlinear dynamical systems analysis of fricative consonants

Acoustic waveforms of the strident fricatives /s/, /z/, /integral of/, and /3/ spoken by two native American English speakers are analyzed using modern chaotic analysis techniques. Fricative data are extracted from both intervocalic and sustained utterances. For comparison, acoustic waveforms of the vowels /a/, /i/, and /u/ are also analyzed. For 44% of the unvoiced fricative tokens in VCV contexts and 59% of the sustained voiced fricatives, indications of low-dimensional dynamics could be found with the given limitations of stationarity. The low-dimensional chaotic behavior is exhibited by a correlation dimension (D2) ranging between 3 and 7.2, and by positive maximum Lyapunov exponents (LEs). For the remaining fricatives, results suggest that the dimensional complexity therein is greater than the maximum D2 value that could be reliably estimated from the available data (about 7.8 for the intervocalic cases and 9 for the sustained cases). Intervocalic voiced fricatives are excluded from the analysis due to stationarity requirements. Analysis of vowels, on the other hand, indicates nonchaotic behavior demonstrated by folded limit cycles and nonpositive maximum LEs; this is consistent with results of previous studies. Findings are interpreted in terms of posited articulatory and aerodynamic parameters of turbulence in the production of fricative consonants.