Modeling global and focal hyperarticulation during human-computer error resolution

Using Different Error Handling Strategies to Facilitate Older Users' Interaction With Chatbots in Learning Information and Communication Technologies

To support older users' accessibility and learning of the prevalent information and communication technologies (ICTs), libraries, as informal learning institutes, are committed to information literacy education activities with friendly interfaces. Chatbots using Voice User Interfaces (VUIs) with natural and intuitive interactions have received growing research and practical attention; however, older users report regular frustrations and problems in using them. To serve as a basis for the subsequent design and development of an automated dialog mechanism in senior-friendly chatbots, a between-subject user experiment was conducted with 30 older adults divided into three groups. The preliminary findings on their interactions with the voice chatbots designed with different error handling strategies were reported. Participants' behavioral patterns, performances, and the tactics they employed in interacting with the three types of chatbots were analyzed. The results of the study showed that the use of multiple error handling strategies is beneficial for older users to achieve effectiveness and satisfaction in human-robot interactions, and facilitate their attitude toward information technology. This study contributes empirical evidence in the genuine and pragmatic field of gerontechnology and expands upon voice chatbots research by exploring conversation errors in human-robot interactions that could be of further application in designing educational and living gerontechnology.

Increased speech contrast induced by sensorimotor adaptation to a nonuniform auditory perturbation

When auditory feedback is perturbed in a consistent way, speakers learn to adjust their speech to compensate, a process known as sensorimotor adaptation. Although this paradigm has been highly informative for our understanding of the role of sensory feedback in speech motor control, its ability to induce behaviorally relevant changes in speech that affect communication effectiveness remains unclear. Because reduced vowel contrast contributes to intelligibility deficits in many neurogenic speech disorders, we examine human speakers' ability to adapt to a nonuniform perturbation field that was designed to affect vowel distinctiveness, applying a shift that depended on the vowel being produced. Twenty-five participants were exposed to this "vowel centralization" feedback perturbation in which the first two formant frequencies were shifted toward the center of each participant's vowel space, making vowels less distinct from one another. Speakers adapted to this nonuniform shift, learning to produce corner vowels with increased vowel space area and vowel contrast to partially overcome the perceived centralization. The increase in vowel contrast occurred without a concomitant increase in duration and persisted after the feedback shift was removed, including after a 10-min silent period. These findings establish the validity of a sensorimotor adaptation paradigm to increase vowel contrast, showing that complex, nonuniform alterations to sensory feedback can successfully drive changes relevant to intelligible communication.NEW & NOTEWORTHY To date, the speech motor learning evoked in sensorimotor adaptation studies has had little ecological consequences for communication. By inducing complex, nonuniform acoustic errors, we show that adaptation can be leveraged to cause an increase in speech sound contrast, a change that has the capacity to improve intelligibility. This study is relevant for models of sensorimotor integration across motor domains, showing that complex alterations to sensory feedback can successfully drive changes relevant to ecological behavior.

Variation in the speech signal as a window into the cognitive architecture of language production

The pronunciation of words is highly variable. This variation provides crucial information about the cognitive architecture of the language production system. This review summarizes key empirical findings about variation phenomena, integrating corpus, acoustic, articulatory, and chronometric data from phonetic and psycholinguistic studies. It examines how these data constrain our current understanding of word production processes and highlights major challenges and open issues that should be addressed in future research.

Dynamically adapted context-specific hyper-articulation: Feedback from interlocutors affects speakers' subsequent pronunciations

We ask whether speakers can adapt their productions when feedback from their interlocutors suggests that previous productions were perceptually confusable. To address this question, we use a novel web-based task-oriented paradigm for speech recording, in which participants produce instructions towards a (simulated) partner with naturalistic response times. We manipulate (1) whether a target word with a voiceless plosive (e.g., pill) occurs in the presence of a voiced competitor (bill) or an unrelated word (food) and (2) whether or not the simulated partner occasionally misunderstands the target word. Speakers hyper-articulated the target word when a voiced competitor was present. Moreover, the size of the hyper-articulation effect was nearly doubled when partners occasionally misunderstood the instruction. A novel type of distributional analysis further suggests that hyper-articulation did not change the target of production, but rather reduced the probability of perceptually ambiguous or confusable productions. These results were obtained in the absence of explicit clarification requests, and persisted across words and over trials. Our findings suggest that speakers adapt their pronunciations based on the perceived communicative success of their previous productions in the current environment. We discuss why speakers make adaptive changes to their speech and what mechanisms might underlie speakers' ability to do so.

Dynamic hyperarticulation of coda voicing contrasts

This study investigates the capacity for targeted hyperarticulation of contextually-relevant contrasts. Participants communicated target words with final /s/ or /z/ when a voicing minimal-pair (e.g., target dose, minimal-pair doze) either was or was not available as an alternative in the context. The results indicate that talkers enhance the durational cues associated with the word-final voicing contrast based on whether the context requires it, and that this can involve both elongation as well as shortening, depending on what enhances the contextually-relevant contrast. This suggests that talkers are capable of targeted, context-sensitive temporal enhancements.

Tongue-palate contact pressure, oral air pressure, and acoustics of clear speech

PURPOSE

The authors compared articulatory contact pressure (ACP), oral air pressure (Po), and speech acoustics for conversational versus clear speech. They also assessed the relationship of these measures to listener perception.

METHOD

Twelve adults with normal speech produced monosyllables in a phrase using conversational and clear speech. Target phonemes were /t, d, s, z, l, n/. ACP was measured at a point of articulatory contact; Po was sensed by a catheter open in the posterior oral cavity.

RESULTS

ACP was increased to a greater extent in clear speech for /t, d, z/. Po was increased to a greater extent for /t, d/. Acoustic changes also occurred in terms of segment durations, speaking rate, and CV dB ratio. Regression analysis indicated that segment duration was the strongest predictor of listener ratings of speech clarity, followed by an index of articulatory effort and speaking rate.

CONCLUSION

Articulatory effort, as indexed by ACP, Po, and CV dB ratio, was increased to varying degrees depending on the consonant. Greatest changes occurred for /t, d/. Durational measures at both the segment and the phrase level were also important for predicting listener ratings of speech clarity.

Recovery of speech following total glossectomy: an acoustic and perceptual appraisal

This study involved an acoustic and perceptual analysis of the speech produced by a 31-year-old female following total glossectomy. Speech samples were collected on three occasions within the first 3 months following glossectomy. Vowel articulation was examined acoustically as a function of vowel space and the Euclidean distance separating corner vowels. Perceptual analyses involved presentation of the participant's CV productions to 30 healthy adult listeners who made forced-choice identifications of consonant type. Acoustic analysis revealed improvements in vowel space area and an increase in the Euclidean distances. The perceptual results revealed a statistically significant deterioration in consonants over the 3-month period with anterior sounds being perceived more correctly than medial and posterior sounds. The current study highlights the variable nature of speech following glossectomy, with greater improvements in vowel articulation compared to consonant articulation during the earliest stages of surgical recovery.

A new method for eliciting three speaking styles in the laboratory

In this study, a method was developed to elicit three different speaking styles, reduced, citation, and hyperarticulated, using controlled sentence materials in a laboratory setting. In the first set of experiments, the reduced style was elicited by having twelve talkers read a sentence while carrying out a distractor task that involved recalling from short-term memory an individually-calibrated number of digits. The citation style corresponded to read speech in the laboratory. The hyperarticulated style was elicited by prompting talkers (twice) to reread the sentences more carefully. The results of perceptual tests with naïve listeners and an acoustic analysis showed that six of the twelve talkers produced a reduced style of speech for the test sentences in the distractor task relative to the same sentences in the citation style condition. In addition, all talkers consistently produced sentences in the citation and hyperarticulated styles. In the second set of experiments, the reduced style was elicited by increasing the number of digits in the distractor task by one (a heavier cognitive load). The procedures for eliciting citation and hyperarticulated sentences remained unchanged. Ten talkers were recorded in the second experiment. The results showed that six out of ten talkers differentiated all three styles as predicted (70% of all sentences recorded). In addition, all talkers consistently produced sentences in the citation and hyperarticulated styles. Overall, the results demonstrate that it is possible to elicit controlled sentence stimulus materials varying in speaking style in a laboratory setting, although the method requires further refinement to elicit these styles more consistently from individual participants.

Linguistic adaptations during spoken and multimodal error resolution

Fragile error handling in recognition-based systems is a major problem that degrades their performance, frustrates users, and limits commercial potential. The aim of the present research was to analyze the types and magnitude of linguistic adaptation that occur during spoken and multimodal human-computer error resolution. A semiautomatic simulation method with a novel error-generation capability was used to collect samples of users' spoken and pen-based input immediately before and after recognition errors, and at different spiral depths in terms of the number of repetitions needed to resolve an error. When correcting persistent recognition errors, results revealed that users adapt their speech and language in three qualitatively different ways. First, they increase linguistic contrast through alternation of input modes and lexical content over repeated correction attempts. Second, when correcting with verbatim speech, they increase hyperarticulation by lengthening speech segments and pauses, and increasing the use of final falling contours. Third, when they hyperarticulate, users simultaneously suppress linguistic variability in their speech signal's amplitude and fundamental frequency. These findings are discussed from the perspective of enhancement of linguistic intelligibility. Implications are also discussed for corroboration and generalization of the Computer-elicited Hyperarticulate Adaptation Model (CHAM), and for improved error handling capabilities in next-generation spoken language and multimodal systems.