A novel speech analysis algorithm to detect cognitive impairment in a Spanish population

Objective

Early detection of cognitive impairment in the elderly is crucial for diagnosis and appropriate care. Brief, cost-effective cognitive screening instruments are needed to help identify individuals who require further evaluation. This study presents preliminary data on a new screening technology using automated voice recording analysis software in a Spanish population.

Method

Data were collected from 174 Spanish-speaking individuals clinically diagnosed as cognitively normal (CN, n = 87) or impaired (mild cognitive impairment [MCI], n = 63; all-cause dementia, n = 24). Participants were recorded performing four common language tasks (Animal fluency, alternating fluency [sports and fruits], phonemic "F" fluency, and Cookie Theft Description). Recordings were processed via text-transcription and digital-signal processing techniques to capture neuropsychological variables and audio characteristics. A training sample of 122 subjects with similar demographics across groups was used to develop an algorithm to detect cognitive impairment. Speech and task features were used to develop five independent machine learning (ML) models to compute scores between 0 and 1, and a final algorithm was constructed using repeated cross-validation. A socio-demographically balanced subset of 52 participants was used to test the algorithm. Analysis of covariance (ANCOVA), covarying for demographic characteristics, was used to predict logistically-transformed algorithm scores.

Results

Mean logit algorithm scores were significantly different across groups in the testing sample (p < 0.01). Comparisons of CN with impaired (MCI + dementia) and MCI groups using the final algorithm resulted in an AUC of 0.93/0.90, with overall accuracy of 88.4%/87.5%, sensitivity of 87.5/83.3, and specificity of 89.2/89.2, respectively.

Conclusion

Findings provide initial support for the utility of this automated speech analysis algorithm as a screening tool for cognitive impairment in Spanish speakers. Additional study is needed to validate this technology in larger and more diverse clinical populations.

H - 52 Impact of Spanish Dialect on Artificial Intelligence in Assessing Expressive Language

OBJECTIVE

AcceXible is a new Artificial Intelligence (AI) tool that identifies speech biomarkers for cognitive impairment that was developed in Spain. Good sensitivity (85.7%) and specificity (92.8%) in distinguishing patients with mild cognitive impairment from individuals with normal functioning have been established with populations in Spain, and preliminary data are similar in the United States and the United Kingdom. In this study, we assessed transcription accuracy for Spanish speakers in the US originating from Latin American countries (LAC) and the effect it may have on the performance of Accexible's AI model.

METHOD

AcceXible involves participants completing several verbal tasks via computer interface, including Animal Naming (AN). Transcription accuracy for 16 Spanish-speaking patients from LAC in Texas was assessed by comparing data obtained from hand scoring of AcceXible AN to data from different customized expressive language transcription tools used in AcceXible's platform. The transcription tool was updated based on inconsistencies found.

RESULTS

AcceXible's customized transcription tool was initially found to miss 34% of total items on AN due to differences in vocabulary and pronunciation between LAC and Spain. The transcription tool was modified to recognize the LAC vocabulary and pronunciations, which resulted in 25.84% increased accuracy for an overall transcription accuracy of 91%.

CONCLUSIONS

AcceXible is a useful tool for efficiently assessing expressive language, and the current findings highlight the need to consider not just language, but also nationality when adapting tests across cultures.

Reliable Digit Span Scores Among Latin American Monolingual Spanish Speakers

OBJECTIVE

The Reliable Digit Span (RDS) is an embedded measure used to assess performance validity. The purpose of this study was to determine whether the RDS can be used with Spanish speakers from Latin America.

METHOD

The Digit Span subtest of the Wechsler Adult Intelligence Scale-IV was administered to 66 Spanish-speaking patients, 66 English-speaking patients, and 30 Spanish-speaking normal controls. The Spanish-speaking patients had a mean age of 65.82 (standard deviation [SD] = 12.44) and mean education of 6.62 years (SD = 4.91), whereas the controls had a mean age of 48.27 (SD = 10.97) and mean education of 8.33 (SD = 3.04). The English-speaking patients had a mean age of 56.44 (SD = 13.53) and mean education of 12.14 (SD = 2.37).

RESULTS

The average RDS among the Spanish-speaking patients was 5.36 (SD = 1.27) and the average RDS among the controls was 5.90 (SD = 1.06). The mean RDS of the English-speaking patients was 7.83 (SD = 1.72). The modal RDS among Spanish-speaking patients was 5. The modal RDS among Spanish-speaking controls was also 5, though the distribution was bimodal in nature with nearly as many 7 s as 5 s. The modal RDS among the English-speaking patients was 7. Among patients with ≥9 years of education and controlling for age, there remained a significant difference between both language patient groups on the RDS (F(1, 79) = 30.85, p < .0001).

CONCLUSIONS

These data suggest that the RDS may not be a valuable measure in assessing performance validity among monolingual Spanish speakers from Latin America, particularly among those with low levels of education.

Regulatory issues and ethical considerations in cross-cultural neuropsychology with a special emphasis on Limited English Proficiency

Regulations addressing provision of healthcare services to individuals from diverse cultures and with limited English proficiency (LEP) are reviewed and balanced against ethical considerations with respect to a provider's competency in conducting such services. Federal regulations require providers at institutions receiving Federal financial assistance to provide services to individuals without allowing factors such as LEP to act as barriers to those services. However, regulations addressing these factors are not absolute, and must be applied reasonably, with respect to: the number of individuals who speak a particular language in an area, the criticalness of the services, and the costs to and resources of the provider's institution. Thresholds based on numbers of individuals speaking a language requiring provision of language assistance services are reviewed. It is also noted that a provider's competency in conducting a particular service to patients from diverse backgrounds may be factored as a resource, while accessing appropriate tools and knowledge to provide services to a particular group may be factored as a cost. A review of relevant issues pertaining to competence in conducting cross-cultural evaluations with an emphasis on LEP is presented, followed by case samples in which potential conflicts between regulations and provider ethical obligations are resolved.

Brief report of the reliability of a new method for scoring organizational approach on the Mesulam Cancellation Test

The Mesulam Cancellation Test is a low cost, quick measure used to screen for hemispatial inattention. The task is also useful for observing a patient's organizational approach. We developed a method to quantify the organizational strategy used by patients in completing this visual scanning task. In this study, we evaluated the reliability of the new method. Participants were 40 patients seen in an outpatient neuropsychology clinic. The 34 men, 5 women, and 1 transitioning individual had a mean age of 49 (SD = 13.94), and mean education of 13.59 years (SD = 2.15). Two raters blinded to patient information provided independent ratings of organization using our new scoring method. Scores ranged from 1 to 5 based on increasing organization. Cohen's weighted Kappa and Spearman's rank order correlation indicated good interrater reliability (Κ(weighted) = .84; rs = .89). The average absolute difference between the raters was .25 (SD = .54). We also found preliminary evidence for the validity of the organizational measure. This study supports the reliability of this new embedded measure of organization. A next step will be to gather normative data, to further establish its validity, and to assess the reliability of the scoring system with other cancellation tests.

Screening for more than level of cognitive functioning: the BNI screen for higher cerebral functions

OBJECTIVE

This paper describes the BNI Screen for Higher Cerebral Functions (BNIS) and reviews studies that comment on its reliability, validity, and clinical and research utility. The ability of the BNIS to assess non-cognitive higher brain functions is also described.

METHODS

We reviewed the original administration manual, studies published in the BNI Quarterly of the Barrow Neurological Institute, and peer-reviewed studies on the BNI Screen identified by an academic database, PubMed and Google Scholar. Thirty-two studies were reviewed that describe normative data, psychometric properties, sensitivity and specificity estimates, the relationship of demographic factors to test performance, and its research utility.

RESULTS

The BNIS is a time efficient screening test often taking no longer than 12-18 minutes. In addition to cognitive functioning, it aids in assessing conation, awareness of memory impairment, and affects expression and perception. Sensitivity estimates ranged from 80% to 92.3%. Specificity estimates ranged from 38.9% to 90%. Its construct, concurrent, and predictive validity have been supported by a series of international studies using different language translations of the test.

CONCLUSION

The BNIS is a useful screening test for identifying patients with underlying brain disorders that uniquely measures domains of functioning not sampled by other existing screening tests.

Commentary on the use of 96119 in billing for neuropsychological services

OBJECTIVE

This commentary seeks to outline the problematic issues with the use of 96119 in billing for neuropsychological evaluations. Issues covered include billing for the provider's time when using technicians as well as the use of trainees when billing Medicare.

METHOD

Resources from the Centers for Medicare and Medicaid Services (CMS) were reviewed along with a search of available documentation from regional carriers and the general literature.

CONCLUSIONS

There has been much confusion in the professional community regarding billing for neuropsychological evaluation services when utilizing technicians and/or trainees. Much of this confusion has stemmed from discrepancies in interpretation of testing codes between the American Medical Association and CMS. Ultimately, clinicians must use codes according to the requirements of the payor, whether Medicare, Medicaid, or a private third-party payor. Failure to do so can be construed as fraud and result in unwanted penalties.

Neuropsychological test performance of children with ADHD relative to test norms and parent behavioral ratings

Neuropsychological test performance relative to published norms and parent ratings was assessed archivally among 78 children with attention deficit hyperactivity disorder (ADHD). Despite average to low average IQs and academic achievement, the children performed poorly relative to test norms on most measures sensitive to fronto-executive functioning (span of attention, sustained attention, response inhibition, and working memory). They also performed poorly on most memory tests requiring free recall/retrieval, a skill dependent in part on intact frontal/subcortical functioning. In contrast, performance fell within the average to low average range on all measures of retention, although some mean scores fell significantly below normative means. In addition, higher levels of inattention or hyperactivity as assessed from parent reports were associated with worse performance on neuropsychological tests. That is, correlations calculated between neuropsychological test scores and parent ratings on the Adaptive Behavior Scale (ABS) were both significant at the P<.05 level and in the expected direction in 33% of the analyses for ratings of attention, in 43% of the analyses for ratings of hyperactivity, and in 5% of the analyses for ratings of persistence.

Visuoconstructional drawing ability in the differential diagnosis of neurologic compromise versus depression

Neuropsychologists are often called upon to assist in making differential diagnoses between neurologic compromise and depression. However, the literature has been inconsistent about the impact of depression on neurocognitive functioning. This makes interpretation of neuropsychological test data among depressed individuals difficult. In a retrospective cross-sectional study of 200 patients, the relationship between depression (Beck Depression Inventory or Geriatric Depression Scale) and visuoconstructional drawing ability/praxis (Rey-Osterrieth Copy [RO]) was evaluated. Age and education both correlated significantly with RO performance (p <.001 and.005, respectively). There were no significant relationships found between copy performance and depression, even when controlling for the significant effects of age and education. A prospective longitudinal study of 20 psychiatric inpatients with depression tested on admission and again at discharge matched with 20 nondepressed normal controls yielded similar results in that the two groups did not differ in their initial RO performance. Moreover, an improvement in depression as measured by a decrease in Beck Depression Inventory scores did not coincide with any improvement in RO scores. By comparison, a matched positive control group of 20 neurologically compromised patients was significantly worse than both the normal control and the depressed groups on the RO (p <.05). These data suggest that poor performance on the RO should not be interpreted simply as a consequence of depression. Age and education, though, should be considered when interpreting the RO.

Differential diagnosis of the major progressive dementias and depression in middle and late adulthood: a summary of the literature of the early 1990s

There is a preponderance of research on the neuropsychology of the various dementias. There are also direct comparisons between two or more dementias available in the literature. This paper sought to summarize the most recent literature, primarily from 1990 through mid-1996, including recent reviews of the literature from previous decades. The purpose was to provide, in one location, a summary of neuropsychological (i.e., cognitive, motor, and psychiatric) characteristics of major noninfectious, progressive dementias and depression of middle and late adulthood. It is hoped that this review, particularly a summary table provided, will serve as a guide in the differential diagnosis of the dementias by clinicians. In addition to Alzheimer's disease, vascular dementias, Parkinson's disease, Lewy body dementia, Huntington's disease, and frontal lobe dementia, the impact of depression on cognitive functioning is covered given the frequency with which neuropsychologists are asked to differentiate depression from primary dementia.

Differentiating patients with higher cerebral dysfunction from patients with psychiatric or acute medical illness using the BNI Screen for Higher Cerebral Functions

The BNI Screen for Higher Cerebral Functions (BNIS) was administered to 41 patients with known cerebral dysfunction. 22 psychiatric patients (some of whom were psychotic) without documented brain lesions, and 22 medical inpatients without neurological or psychiatric diagnoses. Patients with cerebral dysfunction scored significantly lower than the medical and psychiatric patients (p < 0.05). Utilizing the recommended cutoff score of 47, 40 of the 41 brain-dysfunctional patients were correctly classified as impaired, but only seven of the medical and five of the psychiatric patients were correctly classified. Using age-based T-scores, 36 of the 41 brain-dysfunctional patients (87.8%) were correctly classified. Specificity improved slightly, but these numbers were still low (55%), primarily because psychotic patients performed like neurological patients (100%). This study provides further empirical validation of this screening instrument in identifying patients with brain disorders.

Handedness and language among the mentally retarded: implications for the model of pathological left-handedness and gender differences in hemispheric specialization

The relationship between handedness and language was examined among 238 mentally retarded subjects representing the four DSM III classification ranges of mental retardation. Language ability was found to be significantly related to handedness, with an increased prevalence of left-handedness among those individuals with language deficits. In addition, both expressive and receptive language ability interacted with gender, with the main effects of each language variable being stronger for females than males. Although no main effects were found for severity of retardation, a significant interaction was found between severity and gender. Results support the model of pathological left-handedness and are discussed within the context of this theory, as well as with regard to gender differences in cerebral lateralization of language and cognitive functioning.

The clinical significance of congenital agenesis of the corpus callosum

Three cases of congenital agenesis of the corpus callosum (ACC) are presented along with typical clinical features observed on computerized tomography and magnetic resonance imaging. Most congenital ACC cases do not display hemi-syndromes or callosal syndromes, but do exhibit some deficit in terms of hemispheric integration. The most common neuropsychological impairments are in the areas of motor and perceptual-motor functioning. Clinical guidelines in the assessment of ACC patients are given.

Scales for measuring fear of AIDS and homophobia

Two scales, one for measuring attitudes toward the fear of Acquired Immune Deficiency Syndrome (AIDS) and one for measuring attitude toward homosexuality, were constructed using Thurstone's method of equal-appearing intervals. After construction, the scales were given to 528 students at the University of Texas at Austin in the fall of 1985 to determine their respective reliabilities. Factor analyses were also done to determine what factors underlie the attitudes measured by the two scales and to determine if fear of AIDS and homophobia are in fact simply two facets of the same attitude: fear of homosexuals. The results indicated high reliabilities of both scales and a relatively low correlation between the two scales suggesting that the scales do measure different attitudes.

No relationship between handedness and sexual preference

Self-reports of handedness and sexual preference were compared in a group of 89 University students out of 2500 pretested. No relationship was found between the two variables. These results are discussed in relation to Geschwind and Galaburda's 1985 speculation about handedness and homosexuality.

Study of microscopic porphyrin fluorescence

A technique of porphyrin (hematoporphyrin derivative [Hpd]) fluorescence microscopy was evaluated by studying Hpd uptake, retention, and loss in established human cancer cell lines. Hpd uptake appeared to be qualitatively identical in the three cell lines used, but the rate of loss was slowest in the renal-carcinoma-derived line, suggesting a cellular characteristic. The technique was readily applied to demonstrate porphyrin fluorescence in exfoliated cells in urine of bladder cancer patients.