Evaluating Camera Mouse as a computer access system for augmentative and alternative communication in cerebral palsy: A case study

Camera Mouse is a freely available software program that visually tracks the movement of facial features to allow individuals with motor impairments to control a computer mouse. The goal of this case study was to provide an evaluation of Camera Mouse as a computer access method as part of a multiple modality communication system for an individual with cerebral palsy. The participant was asked to reproduce sentences and respond to ethical dilemmas for language sampling. Tasks were completed using natural speech and an AAC solution consisting of Camera Mouse paired with an orthographic selection interface and speech synthesis. The participant completed a questionnaire for satisfaction with the introduced assistive technology. Camera Mouse resulted in higher intelligibility than natural speech, while natural speech had a higher rate. She used more complex language with her natural speech. The participant rated Camera Mouse as at least 3/5 on all measures, including 5/5 on weight and safety. The results of this case study suggest Camera Mouse is a promising computer access system for communication supported by the participant's satisfaction rating, expressive language, and synthesized speech production capabilities.

Microstructural Properties of the Cerebellar Peduncles in Children with Developmental Language Disorder

Children with developmental language disorder (DLD) struggle to learn their native language for no apparent reason. While research on the neurobiological underpinnings of the disorder has focused on the role of cortico-striatal systems, little is known about the role of the cerebellum in DLD. Cortico-cerebellar circuits might be involved in the disorder as they contribute to complex sensorimotor skill learning, including the acquisition of spoken language. Here, we used diffusion-weighted imaging data from 77 typically developing and 54 children with DLD and performed probabilistic tractography to identify the cerebellum's white matter tracts: the inferior, middle, and superior cerebellar peduncles. Children with DLD showed lower fractional anisotropy (FA) in the inferior cerebellar peduncles (ICP), fiber tracts that carry motor and sensory input via the inferior olive to the cerebellum. Lower FA in DLD was driven by lower axial diffusivity. Probing this further with more sophisticated modeling of diffusion data, we found higher orientation dispersion but no difference in neurite density in the ICP of DLD. Reduced FA is therefore unlikely to be reflecting microstructural differences in myelination in this tract, rather the organization of axons in these pathways is disrupted. ICP microstructure was not associated with language or motor coordination performance in our sample. We also found no differences in the middle and superior peduncles, the main pathways connecting the cerebellum with the cortex. To conclude, it is not cortico-cerebellar but atypical olivocerebellar white matter connections that characterize DLD and suggest the involvement of the olivocerebellar system in speech acquisition and development.

Quantitative MRI reveals differences in striatal myelin in children with DLD

Developmental language disorder (DLD) is a common neurodevelopmental disorder characterised by receptive or expressive language difficulties or both. While theoretical frameworks and empirical studies support the idea that there may be neural correlates of DLD in frontostriatal loops, findings are inconsistent across studies. Here, we use a novel semiquantitative imaging protocol - multi-parameter mapping (MPM) - to investigate microstructural neural differences in children with DLD. The MPM protocol allows us to reproducibly map specific indices of tissue microstructure. In 56 typically developing children and 33 children with DLD, we derived maps of (1) longitudinal relaxation rate R1 (1/T1), (2) transverse relaxation rate R2* (1/T2*), and (3) Magnetization Transfer saturation (MTsat). R1 and MTsat predominantly index myelin, while R2* is sensitive to iron content. Children with DLD showed reductions in MTsat values in the caudate nucleus bilaterally, as well as in the left ventral sensorimotor cortex and Heschl's gyrus. They also had globally lower R1 values. No group differences were noted in R2* maps. Differences in MTsat and R1 were coincident in the caudate nucleus bilaterally. These findings support our hypothesis of corticostriatal abnormalities in DLD and indicate abnormal levels of myelin in the dorsal striatum in children with DLD.

Ability-Based Methods for Personalized Keyboard Generation

This study introduces an ability-based method for personalized keyboard generation, wherein an individual’s own movement and human–computer interaction data are used to automatically compute a personalized virtual keyboard layout. Our approach integrates a multidirectional point-select task to characterize cursor control over time, distance, and direction. The characterization is automatically employed to develop a computationally efficient keyboard layout that prioritizes each user’s movement abilities through capturing directional constraints and preferences. We evaluated our approach in a study involving 16 participants using inertial sensing and facial electromyography as an access method, resulting in significantly increased communication rates using the personalized keyboard (52.0 bits/min) when compared to a generically optimized keyboard (47.9 bits/min). Our results demonstrate the ability to effectively characterize an individual’s movement abilities to design a personalized keyboard for improved communication. This work underscores the importance of integrating a user’s motor abilities when designing virtual interfaces.

LaDIVA: A neurocomputational model providing laryngeal motor control for speech acquisition and production

Many voice disorders are the result of intricate neural and/or biomechanical impairments that are poorly understood. The limited knowledge of their etiological and pathophysiological mechanisms hampers effective clinical management. Behavioral studies have been used concurrently with computational models to better understand typical and pathological laryngeal motor control. Thus far, however, a unified computational framework that quantitatively integrates physiologically relevant models of phonation with the neural control of speech has not been developed. Here, we introduce LaDIVA, a novel neurocomputational model with physiologically based laryngeal motor control. We combined the DIVA model (an established neural network model of speech motor control) with the extended body-cover model (a physics-based vocal fold model). The resulting integrated model, LaDIVA, was validated by comparing its model simulations with behavioral responses to perturbations of auditory vocal fundamental frequency (fo) feedback in adults with typical speech. LaDIVA demonstrated capability to simulate different modes of laryngeal motor control, ranging from short-term (i.e., reflexive) and long-term (i.e., adaptive) auditory feedback paradigms, to generating prosodic contours in speech. Simulations showed that LaDIVA’s laryngeal motor control displays properties of motor equivalence, i.e., LaDIVA could robustly generate compensatory responses to reflexive vocal fo perturbations with varying initial laryngeal muscle activation levels leading to the same output. The model can also generate prosodic contours for studying laryngeal motor control in running speech. LaDIVA can expand the understanding of the physiology of human phonation to enable, for the first time, the investigation of causal effects of neural motor control in the fine structure of the vocal signal.

Elevated iron concentration in putamen and cortical speech motor network in developmental stuttering

Theoretical accounts of developmental stuttering implicate dysfunctional cortico-striatal-thalamo-cortical motor loops through the putamen. However, the analysis of conventional MRI brain scans in individuals who stutter has failed to yield strong support for this theory in terms of reliable differences in the structure or function of the basal ganglia. Here, we performed quantitative mapping of brain tissue, which can be used to measure iron content alongside markers sensitive to myelin and thereby offers particular sensitivity to the measurement of iron-rich structures such as the basal ganglia. Analysis of these quantitative maps in 41 men and women who stutter and 32 individuals who are typically fluent revealed significant group differences in maps of R2*, indicative of higher iron content in individuals who stutter in the left putamen and in left hemisphere cortical regions important for speech motor control. Higher iron levels in brain tissue in individuals who stutter could reflect elevated dopamine levels or lysosomal dysfunction, both of which are implicated in stuttering. This study represents the first use of these quantitative measures in developmental stuttering and provides new evidence of microstructural differences in the basal ganglia and connected frontal cortical regions.

Listener Age and Gender Diversity: Effects on Voice-based Perception of Gender

OBJECTIVE

An important clinical outcome of voice masculinization treatments in transmasculine speakers is voice-based perception of gender. Rigorous assessments of voice treatment that utilize ratings of perception of gender typically do not control for demographic characteristics of the listeners. The objective of the present study was to determine the effect of listeners' age and gender diversity on voice-based judgments of speaker gender.

METHODS

Speech stimuli were produced by a single transmasculine individual over approximately one year of hormone replacement therapy, during which he experienced significant changes in his voice. Three groups of listeners rated speech stimuli on a visual analog scale with anchors ranging from "definitely male" to "guessing male" to "guessing female" to "definitely female." Listener groups were N = 10 cisgender young adults, N = 10 cisgender older adults, and N = 10 gender diverse individuals.

RESULTS

All groups rated the speaker as consistently female through week 14 of hormone replacement therapy and consistently male after week 28. Mean responses of the three groups of listeners were highly correlated (Pearson's correlations all r > 0.97).

CONCLUSION

Given reasonable group sizes, average ratings of gender perception of a transmasculine speaker are not highly influenced by varying listener age and gender minority status.

Oral configurations during vowel nasalization in English

Speech nasalization is achieved primarily through the opening and closing of the velopharyngeal port. However, the resultant acoustic features can also be influenced by tongue configuration. Although vowel nasalization is not contrastive in English, two previous studies have found possible differences in the oral articulation of nasal and oral vowel productions, albeit with inconsistent results. In an attempt to further understand the conflicting findings, we evaluated the oral kinematics of nasalized and non-nasalized vowels in a cohort of both male and female American English speakers via electromagnetic articulography. Tongue body and lip positions were captured during vowels produced in nasal and oral contexts (e.g., /mɑm/, /bɑb/). Large contrasts were seen in all participants between tongue position of /æ/ in oral and nasal contexts, in which tongue positions were higher and more forward during /mæm/ than /bæb/. Lip aperture was smaller in a nasal context for /æ/. Lip protrusion was not different between vowels in oral and nasal contexts. Smaller contrasts in tongue and lip position were seen for vowels /ɑ, i, u/; this is consistent with biomechanical accounts of vowel production that suggest that /i, u/ are particularly constrained, whereas /æ/ has fewer biomechanical constraints, allowing for more flexibility for articulatory differences in different contexts. Thus we conclude that speakers of American English do indeed use different oral configurations for vowels that are in nasal and oral contexts, despite vowel nasalization being non-contrastive. This effect was consistent across speakers for only one vowel, perhaps accounting for previously-conflicting results.

Functional organisation for verb generation in children with developmental language disorder

Developmental language disorder (DLD) is characterised by difficulties in learning one's native language for no apparent reason. These language difficulties occur in 7% of children and are known to limit future academic and social achievement. Our understanding of the brain abnormalities associated with DLD is limited. Here, we used a simple four-minute verb generation task (children saw a picture of an object and were instructed to say an action that goes with that object) to test children between the ages of 10-15 years (DLD N = 50, typically developing N = 67). We also tested 26 children with poor language ability who did not meet our criteria for DLD. Contrary to our registered predictions, we found that children with DLD did not have (i) reduced activity in language relevant regions such as the left inferior frontal cortex; (ii) dysfunctional striatal activity during overt production; or (iii) a reduction in left-lateralised activity in frontal cortex. Indeed, performance of this simple language task evoked activity in children with DLD in the same regions and to a similar level as in typically developing children. Consistent with previous reports, we found sub-threshold group differences in the left inferior frontal gyrus and caudate nuclei, but only when analysis was limited to a subsample of the DLD group (N = 14) who had the poorest performance on the task. Additionally, we used a two-factor model to capture variation in all children studied (N = 143) on a range of neuropsychological tests and found that these language and verbal memory factors correlated with activity in different brain regions. Our findings indicate a lack of support for some neurological models of atypical language learning, such as the procedural deficit hypothesis or the atypical lateralization hypothesis, at least when using simple language tasks that children can perform. These results also emphasise the importance of controlling for and monitoring task performance.

Integrated Head-Tilt and Electromyographic Cursor Control

This study evaluated the performance of two alternate computer access methods that could be used for two-dimensional cursor control. The first method, ACC/sEMG, integrates head acceleration and facial surface electromyography. The second method, Camera Mouse, is a free-to-use, computer vision-based access method. Twenty-four healthy adults performed a target acquisition task using each computer access method across two lighting conditions and three computer orientations. Performance in the task was evaluated using metrics of target selection accuracy, movement time, and path efficiency. Using ACC/sEMG resulted in better mean path efficiency and target selection accuracy, whereas using Camera Mouse resulted in faster target selection. Moreover, performance in the task when using Camera Mouse depended on lighting conditions in the room. The findings of this study show that the ACC/sEMG system is an effective computer access method across different lighting conditions and computer orientations. However, there is a tradeoff between speed and accuracy: ACC/sEMG system provided higher target selection accuracy compared to Camera Mouse, while the latter provided faster target selection. Future development should focus on evaluating performance of each method in populations with limited motor abilities.

The Effects of Modulating Fundamental Frequency and Speech Rate on the Intelligibility, Communication Efficiency, and Perceived Naturalness of Synthetic Speech

Purpose This study investigated how modulating fundamental frequency (f0) and speech rate differentially impact the naturalness, intelligibility, and communication efficiency of synthetic speech. Method Sixteen sentences of varying prosodic content were developed via a speech synthesizer. The f0 contour and speech rate of these sentences were altered to produce 4 stimulus sets: (a) normal rate with a fixed f0 level, (b) slow rate with a fixed f0 level, (c) normal rate with prosodically natural f0 variation, and (d) normal rate with prosodically unnatural f0 variation. Sixteen listeners provided orthographic transcriptions and judgments of naturalness for these stimuli. Results Sentences with f0 variation were rated as more natural than those with a fixed f0 level. Conversely, sentences with a fixed f0 level demonstrated higher intelligibility than those with f0 variation. Speech rate did not affect the intelligibility of stimuli with a fixed f0 level. Communication efficiency was highest for sentences produced at a normal rate and a fixed f0 level. Conclusions Sentence-level f0 variation increased naturalness ratings of synthesized speech, whether the variation was prosodically natural or not. However, these f0 variations reduced intelligibility. There is evidence of a trade-off in naturalness and intelligibility of synthesized speech, which may impact future speech synthesis designs. Supplemental Material https://doi.org/10.23641/asha.8847833.

Optimized and Predictive Phonemic Interfaces for Augmentative and Alternative Communication

Purpose We empirically assessed the results of computational optimization and prediction in communication interfaces that were designed to allow individuals with severe motor speech disorders to select phonemes and generate speech output. Method Interface layouts were either random or optimized, in which phoneme targets that were likely to be selected together were located in proximity. Target sizes were either static or predictive, such that likely targets were dynamically enlarged following each selection. Communication interfaces were evaluated by 36 users without motor impairments using an alternate access method. Each user was assigned to 1 of 4 interfaces varying in layout and whether prediction was implemented (random/static, random/predictive, optimized/static, optimized/predictive) and participated in 12 sessions over a 3-week period. Six participants with severe motor impairments used both the optimized/static and optimized/predictive interfaces in 1-2 sessions. Results In individuals without motor impairments, prediction provided significantly faster communication rates during training (Sessions 1-9), as users were learning the interface target locations and the novel access method. After training, optimization acted to significantly increase communication rates. The optimization likely became relevant only after training when participants knew the target locations and moved directly to the targets. Participants with motor impairments could use the interfaces with alternate access methods and generally rated the interface with prediction as preferred. Conclusions Optimization and prediction led to increases in communication rates in users without motor impairments. Predictive interfaces were preferred by users with motor impairments. Future research is needed to translate these results into clinical practice. Supplemental Material https://doi.org/10.23641/asha.8636948.

Longitudinal Case Study of Transgender Voice Changes Under Testosterone Hormone Therapy

The purpose of this study was to comprehensively evaluate voice and speech changes in one healthy 30-year-old transgender male undergoing testosterone therapy for transition. Testing occurred at three timepoints before cross-sex hormone therapy and every 2 weeks thereafter for 1 year. Data collected included measures of acoustics, aerodynamics, and laryngeal structure and function via flexible laryngoscopy. Analysis included acoustic correlates of pitch, loudness, voice quality, and vocal tract length, as well as perceptual measures of voice quality and gender. Speaking fundamental frequency (f_o) lowered from 183 Hz to 134 Hz. Phonatory frequency range (ie, minimum and maximum singing range) shifted from a range of D#3-E6 to a range of A2-A5. Perceptual measures of voice quality indicated no negative changes. Naïve listeners reliably rated the participant's speech samples as male after 37 weeks on testosterone. Few studies document in detail the variety of voice changes that occur during cross-sex hormone therapy, focusing instead on f_o alone. This study adds to the literature a comprehensive case study of speech and voice changes experienced by one transmasculine participant undergoing testosterone therapy.

Prediction of Optimal Facial Electromyographic Sensor Configurations for Human-Machine Interface Control

Surface electromyography (sEMG) is a promising computer access method for individuals with motor impairments. However, optimal sensor placement is a tedious task requiring trial-and-error by an expert, particularly when recording from facial musculature likely to be spared in individuals with neurological impairments. We sought to reduce the sEMG sensor configuration complexity by using quantitative signal features extracted from a short calibration task to predict human-machine interface (HMI) performance. A cursor control system allowed individuals to activate specific sEMG-targeted muscles to control an onscreen cursor and navigate a target selection task. The task was repeated for a range of sensor configurations to elicit a range of signal qualities. Signal features were extracted from the calibration of each configuration and examined via a principle component factor analysis in order to predict the HMI performance during subsequent tasks. Feature components most influenced by the energy and the complexity of the EMG signal and muscle activity between the sensors were significantly predictive of the HMI performance. However, configuration order had a greater effect on performance than the configurations, suggesting that non-experts can place sEMG sensors in the vicinity of usable muscle sites for computer access and healthy individuals will learn to efficiently control the HMI system.

The Impact of Glottal Configuration on Speech Breathing

OBJECTIVE

The purpose of this study was to examine whether changes in respiratory patterns occurred in response to volitional changes in glottal configuration.

METHODS

Twelve vocally healthy participants read a passage while wearing the Inductotrace respiratory inductive plethysmograph, which measures the excursions of the rib cage and abdomen. Participants read the passage 5 times in a typical speaking voice (baseline phase), 10 times in an experimental voice, which was similar to a breathy vocal quality (experimental phase), and 5 times again in a typical speaking voice (return phase). Kinematic estimates of lung volume (LV) initiation, LV termination, and LV excursion were collected for each speech breath.

RESULTS

Participants spoke with larger LV excursions during the experimental phase, characterized by increased LV initiation and decreased LV termination compared with the baseline phase.

CONCLUSION

In response to volitional changes in glottal configuration, healthy individuals spoke with increased LV excursion. They both responded to changes (decreasing LV termination) and planned for more efficient future utterances (increasing LV initiation) during the experimental phase. This study demonstrated that respiratory patterns change in response to changes in glottal configuration; future work will examine these patterns in individuals with voice disorders.

Video Game Rehabilitation of Velopharyngeal Dysfunction: A Case Series

PURPOSE

Video games provide a promising platform for rehabilitation of speech disorders. Although video games have been used to train speech perception in foreign language learners and have been proposed for aural rehabilitation, their use in speech therapy has been limited thus far. We present feasibility results from at-home use in a case series of children with velopharyngeal dysfunction (VPD) using an interactive video game that provided real-time biofeedback to facilitate appropriate nasalization.

METHOD

Five participants were recruited across a range of ages, VPD severities, and VPD etiologies. Participants completed multiple weeks of individual game play with a video game that provides feedback on nasalization measured via nasal accelerometry. Nasalization was assessed before and after training by using nasometry, aerodynamic measures, and expert perceptual judgments.

RESULTS

Four participants used the game at home or school, with the remaining participant unwilling to have the nasal accelerometer secured to his nasal skin, perhaps due to his young age. The remaining participants showed a tendency toward decreased nasalization after training, particularly for the words explicitly trained in the video game.

CONCLUSION

Results suggest that video game-based systems may provide a useful rehabilitation platform for providing real-time feedback of speech nasalization in VPD.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.5116828.

Kinematic Analysis of Speech Sound Sequencing Errors Induced by Delayed Auditory Feedback

PURPOSE

Delayed auditory feedback (DAF) causes speakers to become disfluent and make phonological errors. Methods for assessing the kinematics of speech errors are lacking, with most DAF studies relying on auditory perceptual analyses, which may be problematic, as errors judged to be categorical may actually represent blends of sounds or articulatory errors.

METHOD

Eight typical speakers produced nonsense syllable sequences under normal and DAF (200 ms). Lip and tongue kinematics were captured with electromagnetic articulography. Time-locked acoustic recordings were transcribed, and the kinematics of utterances with and without perceived errors were analyzed with existing and novel quantitative methods.

RESULTS

New multivariate measures showed that for 5 participants, kinematic variability for productions perceived to be error free was significantly increased under delay; these results were validated by using the spatiotemporal index measure. Analysis of error trials revealed both typical productions of a nontarget syllable and productions with articulatory kinematics that incorporated aspects of both the target and the perceived utterance.

CONCLUSIONS

This study is among the first to characterize articulatory changes under DAF and provides evidence for different classes of speech errors, which may not be perceptually salient. New methods were developed that may aid visualization and analysis of large kinematic data sets.

SUPPLEMENTAL MATERIAL

https://doi.org/10.23641/asha.5103067.

Surface electromyographic control of a novel phonemic interface for speech synthesis

Many individuals with minimal movement capabilities use AAC to communicate. These individuals require both an interface with which to construct a message (e.g., a grid of letters) and an input modality with which to select targets. This study evaluated the interaction of two such systems: (a) an input modality using surface electromyography (sEMG) of spared facial musculature, and (b) an onscreen interface from which users select phonemic targets. These systems were evaluated in two experiments: (a) participants without motor impairments used the systems during a series of eight training sessions, and (b) one individual who uses AAC used the systems for two sessions. Both the phonemic interface and the electromyographic cursor show promise for future AAC applications.

Discrete Versus Continuous Mapping of Facial Electromyography for Human-Machine Interface Control: Performance and Training Effects

Individuals with high spinal cord injuries are unable to operate a keyboard and mouse with their hands. In this experiment, we compared two systems using surface electromyography (sEMG) recorded from facial muscles to control an onscreen keyboard to type five-letter words. Both systems used five sEMG sensors to capture muscle activity during five distinct facial gestures that were mapped to five cursor commands: move left, move right, move up, move down, and "click". One system used a discrete movement and feedback algorithm in which the user produced one quick facial gesture, causing a corresponding discrete movement to an adjacent letter. The other system was continuously updated and allowed the user to control the cursor's velocity by relative activation between different sEMG channels. Participants were trained on one system for four sessions on consecutive days, followed by one crossover session on the untrained system. Information transfer rates (ITRs) were high for both systems compared to other potential input modalities, both initially and with training (Session 1: 62.1 bits/min, Session 4: 105.1 bits/min). Users of the continuous system showed significantly higher ITRs than the discrete users. Future development will focus on improvements to both systems, which may offer differential advantages for users with various motor impairments.