Automatic speech and singing classification in ambulatory recordings for normal and disordered voices

Closed-loop network of skin-interfaced wireless devices for quantifying vocal fatigue and providing user feedback

Vocal fatigue is a measurable form of performance fatigue resulting from overuse of the voice and is characterized by negative vocal adaptation. Vocal dose refers to cumulative exposure of the vocal fold tissue to vibration. Professionals with high vocal demands, such as singers and teachers, are especially prone to vocal fatigue. Failure to adjust habits can lead to compensatory lapses in vocal technique and an increased risk of vocal fold injury. Quantifying and recording vocal dose to inform individuals about potential overuse is an important step toward mitigating vocal fatigue. Previous work establishes vocal dosimetry methods, that is, processes to quantify vocal fold vibration dose but with bulky, wired devices that are not amenable to continuous use during natural daily activities; these previously reported systems also provide limited mechanisms for real-time user feedback. This study introduces a soft, wireless, skin-conformal technology that gently mounts on the upper chest to capture vibratory responses associated with vocalization in a manner that is immune to ambient noises. Pairing with a separate, wirelessly linked device supports haptic feedback to the user based on quantitative thresholds in vocal usage. A machine learning-based approach enables precise vocal dosimetry from the recorded data, to support personalized, real-time quantitation and feedback. These systems have strong potential to guide healthy behaviors in vocal use.

Amount and Characteristics of Speaking and Singing Voice Use in Vocally Healthy Female College Student Singers During a Typical Week

OBJECTIVES

Singers, college students, and females are groups known to be at an elevated risk of developing functional/hyperfunctional voice disorders; therefore, female college students majoring in vocal performance may be at an even higher risk. To mitigate this risk, it would be helpful to know the "safe limits" for voice use that would help maintain vocal health in this vulnerable group, but there is a paucity of high-quality objective information upon which to base such limits. This study employed weeklong ambulatory voice monitoring in a large group of vocally healthy female college student singers to begin providing the types of objective data that could be used to help develop improved vocal health guidelines.

METHODS

Participants included 64 vocally healthy females currently enrolled in a vocal performance or similar program at a college or university. An ambulatory voice monitor recorded neck-surface acceleration throughout a typical week. A singing classifier was applied to the data to separate singing from speech. Weeklong vocal dose measures and distributional characteristics for standard voice measures were computed separately for singing and speech, and for both types of phonation combined.

RESULTS

Participants spent 6.2% of the total monitoring time speaking and 2.1% singing (with total phonation time being 8.4%). Singing had a higher f_o mode, more pitch variability, higher average sound pressure level (SPL), negatively skewed SPL distributions, lower average CPP, and higher H₁-H₂ values than speaking.

CONCLUSIONS

These results provide a basis for beginning to establish vocal health guidelines for female students enrolled in college-level vocal performance programs and for future studies of the types of voice disorders that are common in this group. Results also demonstrate the potential value that ambulatory voice monitoring may have in helping to objectively identify vocal behaviors that could contribute to voice problems in this population.

Automated Relative Fundamental Frequency Algorithms for Use With Neck-Surface Accelerometer Signals

OBJECTIVE

Relative fundamental frequency (RFF) has been suggested as a potential acoustic measure of vocal effort. However, current clinical standards for RFF measures require time-consuming manual markings. Previous semi-automated algorithms have been developed to calculate RFF from microphone signals. The current study aimed to develop fully automated algorithms to calculate RFF from neck-surface accelerometer signals for ecological momentary assessment and ambulatory monitoring of voice.

METHODS

Training a set of 2646 /vowel-fricative-vowel/ utterances from 317 unique speakers, with and without voice disorders, was used to develop automated algorithms to calculate RFF values from neck-surface accelerometer signals. The algorithms first rejected utterances with poor vowel-to-noise ratios, then identified fricative locations, then used signal features to determine voicing boundary cycles, and finally calculated corresponding RFF values. These automated RFF values were compared to the clinical gold-standard of manual RFF calculated from simultaneously collected microphone signals in a novel test set of 639 utterances from 77 unique speakers.

RESULTS

Automated accelerometer-based RFF values resulted in an average mean bias error (MBE) across all cycles of 0.027 ST, with an MBE of 0.152 ST and -0.252 ST in the offset and onset cycles closest to the fricative, respectively.

CONCLUSION

All MBE values were smaller than the expected changes in RFF values following successful voice therapy, suggesting that the current algorithms could be used for ecological momentary assessment and ambulatory monitoring via neck-surface accelerometer signals.

Relationships Among Personality, Daily Speaking Voice Use, and Phonotrauma in Adult Female Singers

PURPOSE

This study sought to determine whether personality traits related to extraversion and impulsivity are more strongly associated with singers with nodules compared to vocally healthy singers and to understand the relationship between personality and the types of daily speaking voice use.

METHOD

Weeklong ambulatory voice recordings and personality inventories were obtained for 47 female singers with nodules and 47 vocally healthy female singers. Paired t tests investigated trait differences between groups. Relationships between traits and weeklong speaking voice measures (vocal dose, sound pressure level [SPL], neck surface acceleration magnitude [NSAM], fundamental frequency, cepstral peak prominence [CPP], and the ratio of the first two harmonic magnitudes [H 1 -H 2]) were examined using pairwise Pearson r coefficients. Multiple regressions were performed to estimate voice parameters that correlated with two or more traits.

RESULTS

Singers with nodules scored higher on the Social Potency scale (reflecting a tendency toward social dominance) and lower on the Control scale (reflecting impulsivity) compared to the vocally healthy singers. In vocally healthy singers, vocal dose measures were positively correlated with a combination of Wellbeing (i.e., happiness) and Social Potency, mean SPL was positively correlated with Wellbeing, SPL variability was positively correlated with Social Potency and negatively with Harm Avoidance, and CPP mean was positively correlated with Wellbeing. Singers with nodules had a negative correlation between NSAM skewness and Social Potency. Both groups had negative correlations between H 1 -H 2 mean and Social Potency and Social Closeness.

CONCLUSIONS

Singers with nodules are more socially dominant and impulsive than vocally healthy singers. Personality traits are related to daily speaking voice use, particularly in vocally healthy singers. Individuals with higher levels of traits related to happiness and social dominance and lower Harm Avoidance tended to speak more, with higher laryngeal forces, with more SPL variability, and with more pressed glottal closure, which could increase risk of phonotrauma.

Towards a Singing Voice Multi-Sensor Analysis Tool: System Design, and Assessment Based on Vocal Breathiness

Singing voice is a human quality that requires the precise coordination of numerous kinetic functions and results in a perceptually variable auditory outcome. The use of multi-sensor systems can facilitate the study of correlations between the vocal mechanism kinetic functions and the voice output. This is directly relevant to vocal education, rehabilitation, and prevention of vocal health issues in educators; professionals; and students of singing, music, and acting. In this work, we present the initial design of a modular multi-sensor system for singing voice analysis, and describe its first assessment experiment on the ‘vocal breathiness’ qualitative characteristic. A system case study with two professional singers was conducted, utilizing signals from four sensors. Participants sung a protocol of vocal trials in various degrees of intended vocal breathiness. Their (i) vocal output, (ii) phonatory function, and (iii) respiratory behavior-per-condition were recorded through a condenser microphone (CM), an Electroglottograph (EGG), and thoracic and abdominal respiratory effort transducers (RET), respectively. Participants’ individual respiratory management strategies were studied through qualitative analysis of RET data. Microphone audio samples breathiness degree was rated perceptually, and correlation analysis was performed between sample ratings and parameters extracted from CM and EGG data. Smoothed Cepstral Peak Prominence (CPPS) and vocal folds’ Open Quotient (OQ), as computed with the Howard method (HOQ), demonstrated the higher correlation coefficients, when analyzed individually. DECOM method-computed OQ (DOQ) was also examined. Interestingly, the correlation coefficient of pitch difference between estimates from CM and EGG signals appeared to be (based on the Pearson correlation coefficient) statistically insignificant (a result that warrants investigation in larger populations). The study of multi-variate models revealed even higher correlation coefficients. Models studied were the Acoustic Breathiness Index (ABI) and the proposed multiple regression model CDH (CPPS, DOQ, and HOQ), which was attempted in order to combine analysis results from microphone and EGG signals. The model combination of ABI and the proposed CDH appeared to yield the highest correlation with perceptual breathiness ratings. Study results suggest potential for the use of a completed system version in vocal pedagogy and research, as the case study indicated system practicality, a number of pertinent correlations, and introduced topics with further research possibilities.

Changes in the Daily Phonotrauma Index Following the Use of Voice Therapy as the Sole Treatment for Phonotraumatic Vocal Hyperfunction in Females

Purpose The aim of this study was to use the Daily Phonotrauma Index (DPI) to quantify group-based changes in the daily voice use of patients with phonotraumatic vocal hyperfunction (PVH) after receiving voice therapy as the sole treatment. This is part of an ongoing effort to validate an updated theoretical framework for PVH. Method A custom-designed ambulatory voice monitor was used to collect 1 week of pre- and posttreatment data from 52 female patients with PVH. Normative weeklong data were also obtained from 52 matched controls. Each week was represented by the DPI, which is a combination of neck-surface acceleration magnitude skewness and the standard deviation of the difference between the first and second harmonic magnitudes. Results Compared to pretreatment, the DPI statistically decreased towards normal in the patient group after treatment (Cohen's d = -0.25). The posttreatment patient group's DPI was still significantly higher than the control group (d = 0.68). Conclusions The DPI showed the pattern of improved ambulatory voice use in a group of patients with PVH following voice therapy that was predicted by the updated theoretical framework. Per the prediction, voice therapy was associated with a decreased potential for phonotrauma in daily voice use, but the posttreatment patient group data were still significantly different from the normative control group data. This posttreatment difference is interpreted as reflecting the impact on voice use of the persistence of phonotrauma-induced structural changes to the vocal folds. Further validation of the DPI is needed to better understand its potential clinical use.

Differences Between Female Singers With Phonotrauma and Vocally Healthy Matched Controls in Singing and Speaking Voice Use During 1 Week of Ambulatory Monitoring

Purpose Previous ambulatory voice monitoring studies have included many singers and have combined speech and singing in the analyses. This study applied a singing classifier to the ambulatory recordings of singers with phonotrauma and healthy controls to determine if analyzing speech and singing separately would reveal voice use differences that could provide new insights into the etiology and pathophysiology of phonotrauma in this at-risk population. Method Forty-two female singers with phonotrauma (vocal fold nodules or polyps) and 42 healthy matched controls were monitored using an ambulatory voice monitor. Weeklong statistics (average, standard deviation, skewness, kurtosis) for sound pressure level (SPL), fundamental frequency, cepstral peak prominence, the magnitude ratio of the first two harmonics (H₁-H₂ ), and three vocal dose measures were computed from the neck surface acceleration signal and separated into singing and speech using a singing classifier. Results Mixed analysis of variance models found expected differences between singing and speech in each voice parameter, except SPL kurtosis. SPL skewness, SPL kurtosis, and all H₁-H₂ distributional parameters differentiated patients and controls when singing and speech were combined. Interaction effects were found in H₁-H₂ kurtosis and all vocal dose measures. Patients had significantly higher vocal doses in speech compared to controls. Conclusions Consistent with prior work, the pathophysiology of phonotrauma in singers is characterized by more abrupt/complete glottal closure (decreased mean and variation for H₁-H₂ ) and increased laryngeal forces (negatively skewed SPL distribution) during phonation. Application of a singing classifier to weeklong data revealed that singers with phonotrauma spent more time speaking on a weekly basis, but not more time singing, compared to controls. Results are used as a basis for hypothesizing about the role of speaking voice in the etiology of phonotraumatic vocal hyperfunction in singers.

Changes in a Daily Phonotrauma Index After Laryngeal Surgery and Voice Therapy: Implications for the Role of Daily Voice Use in the Etiology and Pathophysiology of Phonotraumatic Vocal Hyperfunction

Purpose This study attempts to gain insights into the role of daily voice use in the etiology and pathophysiology of phonotraumatic vocal hyperfunction (PVH) by applying a logistic regression-based daily phonotrauma index (DPI) to predict group-based improvements in patients with PVH after laryngeal surgery and/or postsurgical voice therapy. Method A custom-designed ambulatory voice monitor was used to collect 1 week of pre- and postsurgery data from 27 female patients with PVH; 13 of these patients were also monitored after postsurgical voice therapy. Normative weeklong data were obtained from 27 matched controls. Each week was represented by the DPI, standard deviation of the difference between the first and second harmonic amplitudes (H1-H2). Results Compared to pretreatment, the DPI significantly decreased in the patient group after surgery (Cohen's d effect size = -0.86) and voice therapy (d = -1.06). The patient group DPI only normalized after voice therapy. Conclusions The DPI produced the expected pattern of improved ambulatory voice use across laryngeal surgery and postsurgical voice therapy in a group of patients with PVH. The results were interpreted as providing new objective information about the role of daily voice use in the etiology and pathophysiology of PVH. The DPI is viewed as an estimate of potential vocal fold trauma that relies on combining the long-term distributional characteristics of two parameters representing the magnitude of phonatory forces (neck-surface acceleration magnitude) and vocal fold closure dynamics (H1-H2). Further validation of the DPI is needed to better understand its potential clinical use.

Ambulatory monitoring of Lombard-related vocal characteristics in vocally healthy female speakers

Speakers typically modify their voice in the presence of increased background noise levels, exhibiting the classic Lombard effect. Lombard-related characteristics during everyday activities were recorded from 17 vocally healthy women who wore an acoustic noise dosimeter and ambulatory voice monitor. The linear relationship between vocal sound pressure level and environmental noise level exhibited an average slope of 0.54 dB/dB and value of 72.8 dB SPL at 50 dBA when correlation coefficients were greater than 0.4. These results, coupled with analyses of spectral and cepstral vocal function measures, provide normative ambulatory Lombard characteristics for comparison with patients with voice-use related disorders.

Differences in Weeklong Ambulatory Vocal Behavior Between Female Patients With Phonotraumatic Lesions and Matched Controls

Purpose Previous work using ambulatory voice recordings has shown no differences in average vocal behavior between patients with phonotraumatic vocal hyperfunction and matched controls. This study used larger groups to replicate these results and expanded the analysis to include distributional characteristics of ambulatory voice use and measures indicative of glottal closure. Method Subjects included 180 adult women: 90 diagnosed with vocal fold nodules or polyps and 90 age-, sex-, and occupation-matched controls with no history of voice disorders. Weeklong summary statistics (average, variability, skewness, kurtosis) of voice use were computed from neck-surface acceleration recorded using an ambulatory voice monitor. Voice measures included estimates of sound pressure level (SPL), fundamental frequency (f _o), cepstral peak prominence, and the difference between the first and second harmonic magnitudes (H1-H2). Results Statistical comparisons resulted in medium-large differences (Cohen's d ≥ 0.5) between groups for SPL skewness, f _o variability, and H1-H2 variability. Two logistic regressions (theory-based and stepwise) found SPL skewness and H1-H2 variability to classify patients and controls based on their weekly voice data, with an area under the receiver operating characteristic curve of 0.85 and 0.82 on training and test sets, respectively. Conclusion Compared to controls, the weekly voice use of patients with phonotraumatic vocal hyperfunction reflected higher SPL tendencies (negatively skewed SPL) with more abrupt glottal closure (reduced H1-H2 variability, especially toward higher values). Further work could examine posttreatment data (e.g., after surgery and/or therapy) to determine the extent to which these differences are associated with the etiology and pathophysiology of phonotraumatic vocal fold lesions.