Perceiving vocal age and gender: an adaptation approach

Vocal emotion adaptation aftereffects within and across speaker genders: Roles of timbre and fundamental frequency

While the human perceptual system constantly adapts to the environment, some of the underlying mechanisms are still poorly understood. For instance, although previous research demonstrated perceptual aftereffects in emotional voice adaptation, the contribution of different vocal cues to these effects is unclear. In two experiments, we used parameter-specific morphing of adaptor voices to investigate the relative roles of fundamental frequency (F0) and timbre in vocal emotion adaptation, using angry and fearful utterances. Participants adapted to voices containing emotion-specific information in either F0 or timbre, with all other parameters kept constant at an intermediate 50% morph level. Full emotional voices and ambiguous voices were used as reference conditions. All adaptor stimuli were either of the same (Experiment 1) or opposite speaker gender (Experiment 2) of subsequently presented target voices. In Experiment 1, we found consistent aftereffects in all adaptation conditions. Crucially, aftereffects following timbre adaptation were much larger than following F0 adaptation and were only marginally smaller than those following full adaptation. In Experiment 2, adaptation aftereffects appeared massively and proportionally reduced, with differences between morph types being no longer significant. These results suggest that timbre plays a larger role than F0 in vocal emotion adaptation, and that vocal emotion adaptation is compromised by eliminating gender-correspondence between adaptor and target stimuli. Our findings also add to mounting evidence suggesting a major role of timbre in auditory adaptation.

Parameter-Specific Morphing Reveals Contributions of Timbre and Fundamental Frequency Cues to the Perception of Voice Gender and Age in Cochlear Implant Users

Purpose Using naturalistic synthesized speech, we determined the relative importance of acoustic cues in voice gender and age perception in cochlear implant (CI) users. Method We investigated 28 CI users' abilities to utilize fundamental frequency (F0) and timbre in perceiving voice gender (Experiment 1) and vocal age (Experiment 2). Parameter-specific voice morphing was used to selectively control acoustic cues (F0; time; timbre, i.e., formant frequencies, spectral-level information, and aperiodicity, as defined in TANDEM-STRAIGHT) in voice stimuli. Individual differences in CI users' performance were quantified via deviations from the mean performance of 19 normal-hearing (NH) listeners. Results CI users' gender perception seemed exclusively based on F0, whereas NH listeners efficiently used timbre. For age perception, timbre was more informative than F0 for both groups, with minor contributions of temporal cues. While a few CI users performed comparable to NH listeners overall, others were at chance. Separate analyses confirmed that even high-performing CI users classified gender almost exclusively based on F0. While high performers could discriminate age in male and female voices, low performers were close to chance overall but used F0 as a misleading cue to age (classifying female voices as young and male voices as old). Satisfaction with CI generally correlated with performance in age perception. Conclusions We confirmed that CI users' gender classification is mainly based on F0. However, high performers could make reasonable usage of timbre cues in age perception. Overall, parameter-specific morphing can serve to objectively assess individual profiles of CI users' abilities to perceive nonverbal social-communicative vocal signals.

Towards the automatic detection of social biomarkers in autism spectrum disorder: introducing the simulated interaction task (SIT)

Social interaction deficits are evident in many psychiatric conditions and specifically in autism spectrum disorder (ASD), but hard to assess objectively. We present a digital tool to automatically quantify biomarkers of social interaction deficits: the simulated interaction task (SIT), which entails a standardized 7-min simulated dialog via video and the automated analysis of facial expressions, gaze behavior, and voice characteristics. In a study with 37 adults with ASD without intellectual disability and 43 healthy controls, we show the potential of the tool as a diagnostic instrument and for better description of ASD-associated social phenotypes. Using machine-learning tools, we detected individuals with ASD with an accuracy of 73%, sensitivity of 67%, and specificity of 79%, based on their facial expressions and vocal characteristics alone. Especially reduced social smiling and facial mimicry as well as a higher voice fundamental frequency and harmony-to-noise-ratio were characteristic for individuals with ASD. The time-effective and cost-effective computer-based analysis outperformed a majority vote and performed equal to clinical expert ratings.

The Jena Speaker Set (JESS)-A database of voice stimuli from unfamiliar young and old adult speakers

Here we describe the Jena Speaker Set (JESS), a free database for unfamiliar adult voice stimuli, comprising voices from 61 young (18-25 years) and 59 old (60-81 years) female and male speakers uttering various sentences, syllables, read text, semi-spontaneous speech, and vowels. Listeners rated two voice samples (short sentences) per speaker for attractiveness, likeability, two measures of distinctiveness ("deviation"-based [DEV] and "voice in the crowd"-based [VITC]), regional accent, and age. Interrater reliability was high, with Cronbach's α between .82 and .99. Young voices were generally rated as more attractive than old voices, but particularly so when male listeners judged female voices. Moreover, young female voices were rated as more likeable than both young male and old female voices. Young voices were judged to be less distinctive than old voices according to the DEV measure, with no differences in the VITC measure. In age ratings, listeners almost perfectly discriminated young from old voices; additionally, young female voices were perceived as being younger than young male voices. Correlations between the rating dimensions above demonstrated (among other things) that DEV-based distinctiveness was strongly negatively correlated with rated attractiveness and likeability. By contrast, VITC-based distinctiveness was uncorrelated with rated attractiveness and likeability in young voices, although a moderate negative correlation was observed for old voices. Overall, the present results demonstrate systematic effects of vocal age and gender on impressions based on the voice and inform as to the selection of suitable voice stimuli for further research into voice perception, learning, and memory.

Investigating the common set of acoustic parameters in sexual orientation groups: A voice averaging approach

While the perception of sexual orientation in voices often relies on stereotypes, it is unclear whether speech stereotypes and accurate perceptions of sexual orientation are each based on acoustic cues common to speakers of a given group. We ask if the stereotypical belief, that members of the same sexual orientation group share similar acoustic patterns, is accurate to some degree. To address this issue, we are the first to use a novel voice morphing technique to create voice averages from voices that represent extremes of a given sexual orientation group either in terms of actual or perceived sexual orientation. Importantly, averaging preserves only those acoustic cues shared by the original speakers. 144 German listeners judged the sexual orientation of twelve natural-sounding sentence stimuli, each representing an average of five original utterances. Half of the averages were based on targets' self-ratings of sexual orientation: On a 7-point Kinsey-like scale, we selected targets who were most typical for a certain sexual orientation group according to their self-identifications. The other half were based on extreme ratings by others (i.e., on speech-related sexual-orientation stereotypes). Listeners judged sexual orientation from the voice averages with above-chance accuracy suggesting 1) that the perception of actual and stereotypical sexual orientation, respectively, are based on acoustic cues shared by speakers of the same group, and 2) that the stereotypical belief that members of the same sexual orientation group share similar acoustic patterns is accurate to some degree. Mean fundamental frequency and other common acoustic parameters showed systematic variation depending on speaker gender and sexual orientation. Effects of sexual orientation were more pronounced for stereotypical voice averages than for those based on speakers' self-ratings, suggesting that sexual-orientation stereotypes exaggerate even those differences present in the most salient groups of speakers. Implications of our findings for stereotyping and discrimination are discussed.

Electrophysiological correlates of voice memory for young and old speakers in young and old listeners

Faces of one's own-age group are easier to recognize than other-age faces. Using behavioral measures and EEG, we studied whether an own-age bias (OAB) also exists in voice memory. Young (19 - 26 years) and old (60-75 years) participants studied young (18-25 years) and old (60-77 years) unfamiliar voices from short sentences. Subsequently, they classified studied and novel voices as "old" (i.e. studied) or "new", from the same sentences. Recognition performance was higher in young compared to old participants, and for old compared to young voices, with no OAB. At the same time, we found evidence for higher distinctiveness of old compared to young voices, both in terms of acoustic measures and subjective ratings (independent of rater age). Analyses of event-related brain potentials (ERPs) indicated more negative-going deflections (400-1000ms) for old compared to young voices in young participants. In old participants, we observed a reversed OLD/NEW memory effect, with overall more positive amplitudes for novel compared to studied old (but not young) voices (400-1000ms). Time-frequency analyses revealed less beta power (16-26Hz) for young compared to old voices at left anterior sites, and also reduced beta power for correctly recognized studied (compared to novel) voices at left posterior sites (300-900ms). These findings could suggest an engagement of cortical areas during stimulus-specific recollection from about 300ms, in a task that emphasized the analysis of individual acoustic features.

Neural effects of environmental advertising: An fMRI analysis of voice age and temporal framing

Ecological information offered to society through advertising enhances awareness of environmental issues, encourages development of sustainable attitudes and intentions, and can even alter behavior. This paper, by means of functional Magnetic Resonance Imaging (fMRI) and self-reports, explores the underlying mechanisms of processing ecological messages. The study specifically examines brain and behavioral responses to persuasive ecological messages that differ in temporal framing and in the age of the voice pronouncing them. The findings reveal that attitudes are more positive toward future-framed messages presented by young voices. The whole-brain analysis reveals that future-framed (FF) ecological messages trigger activation in brain areas related to imagery, prospective memories and episodic events, thus reflecting the involvement of past behaviors in future ecological actions. Past-framed messages (PF), in turn, elicit brain activations within the episodic system. Young voices (YV), in addition to triggering stronger activation in areas involved with the processing of high-timbre, high-pitched and high-intensity voices, are perceived as more emotional and motivational than old voices (OV) as activations in anterior cingulate cortex and amygdala. Messages expressed by older voices, in turn, exhibit stronger activation in areas formerly linked to low-pitched voices and voice gender perception. Interestingly, a link is identified between neural and self-report responses indicating that certain brain activations in response to future-framed messages and young voices predicted higher attitudes toward future-framed and young voice advertisements, respectively. The results of this study provide invaluable insight into the unconscious origin of attitudes toward environmental messages and indicate which voice and temporal frame of a message generate the greatest subconscious value.

The Perception of Operational Sex Ratios by Voice

Adult sex ratios in a local environment are linked to a wide variety of reproductive behaviors in humans and other animals. When sex ratios are biased, the more numerous sex faces increased competition for mates and is more likely to yield to the sociosexual preferences of the less numerous sex. Despite widespread evidence of the relationship between sex ratios and behavior, we know little about whether or how sex ratios are encoded and perceived. In two experiments men and women showed perceived sex ratios that correlated with actual sex ratios after 1500 ms exposures to groups of simultaneous voices. However, men perceived more female voices than women did, and women perceived more male voices than men did. Women showed better accuracy than men, but only when sex ratios departed markedly from 50%. Increasing the number of simultaneous voices reduced accuracy, but only at extreme sex ratios. Talker age also significantly affected perceived sex ratios, suggesting that perceived operational sex ratios are adaptively linked to the reproductive viability of the local population. The results suggest that listeners automatically encode vocal sex ratio information and that perceived sex ratios are influenced by characteristics of the local population and characteristics of the listener.

To hear or not to hear: Voice processing under visual load

Adaptation to female voices causes subsequent voices to be perceived as more male, and vice versa. This contrastive aftereffect disappears under spatial inattention to adaptors, suggesting that voices are not encoded automatically. According to Lavie, Hirst, de Fockert, and Viding (2004), the processing of task-irrelevant stimuli during selective attention depends on perceptual resources and working memory. Possibly due to their social significance, faces may be an exceptional domain: That is, task-irrelevant faces can escape perceptual load effects. Here we tested voice processing, to study whether voice gender aftereffects (VGAEs) depend on low or high perceptual (Exp. 1) or working memory (Exp. 2) load in a relevant visual task. Participants adapted to irrelevant voices while either searching digit displays for a target (Exp. 1) or recognizing studied digits (Exp. 2). We found that the VGAE was unaffected by perceptual load, indicating that task-irrelevant voices, like faces, can also escape perceptual-load effects. Intriguingly, the VGAE was increased under high memory load. Therefore, visual working memory load, but not general perceptual load, determines the processing of task-irrelevant voices.

Cell-phone vs microphone recordings: Judging emotion in the voice

Emotional states can be conveyed by vocal cues such as pitch and intensity. Despite the ubiquity of cellular telephones, there is limited information on how vocal emotional states are perceived during cell-phone transmissions. Emotional utterances (neutral, happy, angry) were elicited from two female talkers and simultaneously recorded via microphone and cell-phone. Ten-step continua (neutral to happy, neutral to angry) were generated using the straight algorithm. Analyses compared reaction time (RT) and emotion judgment as a function of recording type (microphone vs cell-phone). Logistic regression revealed no judgment differences between recording types, though there were interactions with emotion type. Multi-level model analyses indicated that RT data were best fit by a quadratic model, with slower RT at the middle of each continuum, suggesting greater ambiguity, and slower RT for cell-phone stimuli across blocks. While preliminary, results suggest that critical acoustic cues to emotion are largely retained in cell-phone transmissions, though with effects of recording source on RT, and support the methodological utility of collecting speech samples by phone.

Change of speech fundamental frequency explains the satisfaction with voice in response to testosterone therapy in female-to-male gender dysphoric individuals

Previous studies demonstrated that there is a significant change in speaking fundamental frequency after testosterone therapy in female-to-male gender dysphoric individuals. It is yet an open question how the satisfaction with voice alteration can be predicted because until now it is not clear whether a testosterone therapy is sufficiently effective. The aim of the current study was not only to measure satisfaction with voice, but additionally to detect factors that predict or explain satisfaction with voice after testosterone therapy. Therefore, nine female-to-male gender dysphoric individuals were examined during the first year of testosterone treatment at different points of time. The patients underwent several voice analyses within 1 year and had to fill out several questionnaires concerning their voice, depressive symptoms, quality of life and voice handicap index. Multiple regression analyses were performed to find the factors that explained satisfaction with altered voice after 1 year. The difference of voice frequency in semitones before the treatment and after 1 year is the only significant predictor for satisfaction after 1 year (B = 0.442; SE = 0.049) and more important than the absolute fundamental frequency.

Adding years to your life (or at least looking like it): a simple normalization underlies adaptation to facial age

Adaptation has been widely used to probe how experience shapes the visual encoding of faces, but the pattern of perceptual changes produced by adaptation and the neural mechanisms these imply remain poorly characterized. We explored how adaptation alters the perceived age of faces, a fundamental facial attribute which can uniquely and reliably be scaled by observers. This allowed us to measure how adaptation to one age level affected the full continuum of perceived ages. Participants guessed the ages of faces ranging from 18-89, before or after adapting to a different set of faces composed of younger, older, or middle-aged adults. Adapting to young or old faces induced opposite linear shifts in perceived age that were independent of the model's age. Specifically, after adapting to younger or older faces, faces of all ages appeared 2 to 3 years older or younger, respectively. In contrast, middle-aged adaptors induced no aftereffects. This pattern suggests that adaptation leads to a simple and uniform renormalization of age perception, and is consistent with a norm-based neural code for the mechanisms mediating the perception of facial age.