On Dynamic Pitch Benefit for Speech Recognition in Speech Masker

Exploring the Effects of Online Physician Voice Pitch Range and Filled Pauses on Patient Satisfaction in Mobile Health Communication

The convenience of mobile devices has driven the widespread use of voice technology in mobile health communication, significantly improving the timeliness of online service. However, the issue of listening to therapeutic content, which requires great cognitive effort and may exceed the patient's information processing capacity (i.e., information overload), is of concern. Based on information processing theory, this study reports how online physicians' voice characteristics (pitch range and filled pauses) affect patient satisfaction. We obtained 10,585 mobile voice consultation records of 1,416 doctors from China's largest mHealth platform and analyzed them using audio mining and empirical methods. Results showed that pitch range (β = 0.0539, p < .01) and filled pauses (β = 0.0365, p < .01) in doctors' voice positively influenced online patient satisfaction. However, the effect of filled pauses becomes weaker for patients with higher health literacy and higher disease risk. This suggests that there is heterogeneity in the way different patients process audio information. This study provides important insights for guiding online physician behaviors, enhancing patient satisfaction, and improving mobile health platform management.

Recognition of Speech With Dynamic Pitch Manipulation in Noise: Effects of Manipulation Methods

PURPOSE

Dynamic pitch, which is defined as the variation in fundamental frequency in speech, is one of the acoustic cues that affect speech recognition in noise. Built on the evidence that a symmetrical manipulation of dynamic pitch led to poorer speech recognition, the present study examined the effect of an asymmetrical manipulation method on speech recognition in noise by younger and older adults.

METHOD

Speech recognition accuracy in noise was measured from younger adults with normal hearing in Experiment 1, and speech reception threshold (in dB SNR) from older adults with normal hearing to mild-moderate hearing loss in Experiment 2. The dynamic pitch contours of the speech stimuli were manipulated using both symmetrical and asymmetrical methods.

RESULTS

Younger adults recognized speech better in noise with asymmetrical than symmetrical manipulation, and with weakened than strengthened dynamic pitch. A substantial amount of variability was observed in a group of older listeners. This variability was predominately predicted by the listeners' age but not hearing thresholds or their ability to perceive dynamic pitch in fluctuating noise.

CONCLUSIONS

The asymmetrical manipulation of dynamic pitch had a less negative effect than the symmetrical manipulation. This effect also interacted with pitch-change direction. These findings suggest the influence of perceptual naturalness on speech recognition with signal modification. Directions for future research are also discussed.

Towards a simultaneously speaking bilingual robot: Primary study on the effect of gender and pitch of the robot's voice

With fast and reliable international transportation, more people with different language backgrounds can interact now. As a result, the need for communicative agents fluent in several languages to assist those people is highlighted. The high cost of hiring human attendants fluent in several languages makes using social robots a more affordable alternative in international gatherings. A social robot capable of presenting a piece of information in more than one language at the same time to its audience is the goal of this line of study. However, the negative effect of background noise on speech comprehension in humans is well-established. Hence, presenting a piece of information in two different languages at the same time by the robot creates an adverse listening condition for both individuals listening to the speech of such a bilingual robot. In this study, we investigated whether manipulating the pitch and gender of the robot's voice could affect human subjects' memory of the presented information in the presence of background noise. The results indicate that the pitch and gender of the speaking voice do indeed affect our memory of the presented information. when a male voice was used, a higher pitch resulted in significantly better memory performance than a lower pitch. Contrarily, when a female voice was used, a lower pitch resulted in significantly better memory in participants than a higher pitch. Both male and female subjects performed significantly better with a female voice in a noisy background. In nutshell, the result of this study suggests using a female voice for robots in noisy conditions, as in the case of simultaneously speaking robots, can significantly improve the retrieval of presented information in human subjects.

Speech intelligibility prediction based on modulation frequency-selective processing

Speech intelligibility models can provide insights regarding the auditory processes involved in human speech perception and communication. One successful approach to modelling speech intelligibility has been based on the analysis of the amplitude modulations present in speech as well as competing interferers. This review covers speech intelligibility models that include a modulation-frequency selective processing stage i.e., a modulation filterbank, as part of their front end. The speech-based envelope power spectrum model [sEPSM, Jørgensen and Dau (2011). J. Acoust. Soc. Am. 130(3), 1475-1487], several variants of the sEPSM including modifications with respect to temporal resolution, spectro-temporal processing and binaural processing, as well as the speech-based computational auditory signal processing and perception model [sCASP; Relaño-Iborra et al. J. Acoust. Soc. Am. 146(5), 3306-3317], which is based on an established auditory signal detection and masking model, are discussed. The key processing stages of these models for the prediction of speech intelligibility across a variety of acoustic conditions are addressed in relation to competing modeling approaches. The strengths and weaknesses of the modulation-based analysis are outlined and perspectives presented, particularly in connection with the challenge of predicting the consequences of individual hearing loss on speech intelligibility.

Older Listeners' Perception of Speech With Strengthened and Weakened Dynamic Pitch Cues in Background Noise

Purpose Dynamic pitch, which is defined as the variation in fundamental frequency, is an acoustic cue that aids speech perception in noise. This study examined the effects of strengthened and weakened dynamic pitch cues on older listeners' speech perception in noise, as well as how these effects were modulated by individual factors including spectral perception ability. Method The experiment measured speech reception thresholds in noise in both younger listeners with normal hearing and older listeners whose hearing status ranged from near-normal hearing to mild-to-moderate sensorineural hearing loss. The pitch contours of the target speech were manipulated to create four levels of dynamic pitch strength: weakened, original, mildly strengthened, and strengthened. Listeners' spectral perception ability was measured using tests of spectral ripple and frequency modulation discrimination. Results Both younger and older listeners performed worse with manipulated dynamic pitch cues than with original dynamic pitch. The effects of dynamic pitch on older listeners' speech recognition were associated with their age but not with their perception of spectral information. Those older listeners who were relatively younger were more negatively affected by dynamic pitch manipulations. Conclusions The findings suggest the current pitch manipulation strategy is detrimental for older listeners to perceive speech in noise, as compared to original dynamic pitch. While the influence of age on the effects of dynamic pitch is likely due to age-related declines in pitch perception, the spectral measures used in this study were not strong predictors for dynamic pitch effects. Taken together, these results indicate next steps in this line of work should be focused on how to manipulate acoustic cues in speech in order to improve speech perception in noise for older listeners.

The ability to glimpse dynamic pitch in noise by younger and older listeners

While dynamic pitch is helpful for speech perception in temporally-modulated noise, the ability to benefit from this cue varies substantially among older listeners. To examine the perceptual factors that contribute to this variability, this study aimed to characterize individuals' ability to perceive dynamic pitch in temporally-modulated noise using dynamic pitch segments extracted from real speech and embedded in temporally modulated noise. Data from younger and older listeners showed stronger pitch contours were more easily perceived than weaker pitch contours. The metric significantly predicted speech-in-noise ability in older listeners. Potential implications of this work are discussed.

Test Administration Methods and Cognitive Test Scores in Older Adults with Hearing Loss

In light of the high prevalence of hearing loss and cognitive impairment in the aging population, it is important to know how cognitive tests should be administered for older adults with hearing loss. The purpose of the present study is to examine this question with a cognitive screening test and a working memory test. Specifically, we asked the following questions in 2 experiments. First, does a controlled amplification method affect cognitive test scores? Second, does test modality (visual vs. auditory) impact cognitive test scores? Three test administration conditions were created for both Montreal Cognitive Assessment (MoCA) and working memory test (a word recognition and recall test): auditory amplified, auditory unamplified, and visual. The auditory administration was implemented through a computer program to control for presentation level while the visual condition was implemented through timed computer slides. Data were collected from older individuals with mild-to-moderate sensorineural hearing loss. We did not find any effect of amplification or test modality on the total score of the cognitive screening test (i.e., MoCA). Amplification improved working memory performance as measured by word recall performance, but test modality (auditory vs. visual) did not. These results are consistent with literature in demonstrating a downstream effect of audibility on working memory performance. From a clinical perspective, these findings are informative for developing clinical administration protocols of these tests for older individuals with hearing loss.