The test-retest reliability of the Mini-Mental State Examination in chronic schizophrenic patients

Developing a machine learning-based short form of the positive and negative syndrome scale

BACKGROUND AND HYPOTHESIS

The Positive and Negative Syndrome Scale (PANSS) consists of 30 items and takes up to 50 minutes to administer and score. Therefore, this study aimed to develop and validate a machine learning-based short form of the PANSS (PANSS-MLSF) that reproduces the PANSS scores. Moreover, the PANSS-MLSF estimated the removed-item scores.

STUDY DESIGN

The PANSS-MLSF was developed using an artificial neural network, and the removed-item scores were estimated using the eXtreme Gradient Boosting classifier algorithm. The reliability of the PANSS-MLSF was examined using Cronbach's alpha. The concurrent validity was examined by the association (Pearson's r) between the PANSS-MLSF and the PANSS. The convergent validity was examined by the association (Pearson's r) between the PANSS-MLSF and the Clinical Global Impression-Severity, Mini-Mental State Examination, and Lawton Instrumental Activities of Daily Living Scale. The agreement of the estimated removed-item scores with their original scores was examined using Cohen's kappa.

STUDY RESULTS

Our analysis included data from 573 patients with moderate severity. The two versions of the PANSS-MLSF comprised 15 items and 9 items were proposed. The PANSS-MLSF scores were similar to the PANSS scores (mean squared error=2.6-24.4 points). The reliability, concurrent validity, and convergent validity of the PANSS-MLSF were good. Moderate to good agreement between the estimated removed-item scores and the original item scores was found in 60% of the removed items.

CONCLUSION

The PANSS-MLSF offers a viable way to reduce PANSS administration time, maintain score comparability, uphold reliability and validity, and even estimate scores for the removed items.

Validity of the Test of Visual Perceptual Skills-Fourth Edition in People With Schizophrenia

IMPORTANCE

The Test of Visual Perceptual Skills-Fourth Edition (TVPS-4), which utilizes seven subscales, comprehensively assesses visual perception.

OBJECTIVE

To examine the ecological, convergent, and discriminative validities of the TVPS-4 in the assessment of people with schizophrenia.

DESIGN

Prospective and observational study.

SETTING

Community rehabilitation facilities.

PARTICIPANTS

Eighty people with schizophrenia.

OUTCOMES AND MEASURES

To examine ecological validity, we estimated correlations (Pearson's r) between the TVPS-4 scores (i.e., overall scale and subscales) and the Activities of Daily Living Rating Scale-III (ADLRS-III). To examine convergent validity, we computed the correlations (r) between the TVPS-4 scores and two cognitive measures. An analysis of variance was applied to compare the TVPS-4 scores in groups with different disease severities to examine discriminative validity.

RESULTS

Moderate to strong correlations were found between TVPS-4 scores and ADLRS-III scores (rs = .33-.61) and between TVPS-4 scores and two cognitive measures (rs = .44-.70). TVPS-4 scores could discriminate among the three groups with different severities of psychiatric illnesses (p < .05), except for two subscales (Visual Memory and Spatial Relationships).

CONCLUSIONS AND RELEVANCE

The TVPS-4 has sufficient ecological validity, good convergent validity, and acceptable discriminative validity in assessing people with schizophrenia. The Spatial Relationships subscale, which displays a strong correlation with the ADLRS-III, can reveal the status of daily living in people with schizophrenia. Two subscales showing inadequate discriminative validity should be used cautiously to discriminate people with different severities of schizophrenia. What This Article Adds: The TVPS-4 showed three sound validities (ecological, convergent, and discriminative) for assessing visual perception in people with schizophrenia.

Psychometric Properties of Two Tasks in the Allen Cognitive Level Screen-Sixth Edition for Community-Dwelling People Living With Schizophrenia

IMPORTANCE

Empirical evidence is needed on the psychometric properties of the Allen Cognitive Level Screen-Sixth Edition (ACLS-6), an instrument that assesses cognitive functions and is commonly used for people living with schizophrenia.

OBJECTIVE

To examine the convergent validity, discriminative validity, and test-retest reliability of two tasks, stitching and copying, in the ACLS-6 for community-dwelling people living with schizophrenia.

DESIGN

Prospective observational study.

SETTING

Psychiatric center.

PARTICIPANTS

110 people living with schizophrenia.

OUTCOMES AND MEASURES

To examine convergent validity, we calculated correlations (Pearson's r) between the two tasks and between these two tasks and three cognitive measures. We checked for floor and ceiling effects and conducted independent t tests to evaluate discriminative validity. We calculated intraclass correlation coefficients (ICCs) to investigate test-retest reliability.

RESULTS

We found a strong correlation (r = .88) between the two tasks and moderate correlations (rs = .32-.52) between the two tasks and the three cognitive measures. No floor or ceiling effects were observed for the two tasks, and t tests showed significant differences between two participant groups with marginal and mild clinical symptoms (p < .001). The ICC values for the two tasks were .71-.74.

CONCLUSION AND RELEVANCE

The stitching and copying tasks of the ACLS-6 have good convergent validity, discriminative validity, and test-retest reliability for community-dwelling people living with schizophrenia. The copying task showed a strong correlation with the stitching task and a similar score range, so practitioners can consider using the copying task as a substitute for the stitching task. What This Article Adds: The stitching and copying tasks of the ACLS-6 have sound psychometric properties for measuring cognitive functions in community-dwelling people living with schizophrenia.

Test-retest reliability of the Wisconsin Card Sorting Test in people with schizophrenia

PURPOSE

The aim of this study was to examine the test-retest reliability of the Wisconsin Card Sorting Test in people with schizophrenia. In this study, minimal detectable change (MDC) was calculated and systematic measurement errors were evaluated.

METHOD

Sixty-three people with schizophrenia underwent the WCST twice with a two-week interval. Test-retest reliability was evaluated using intraclass correlation coefficient. Systematic measurement error was examined using paired t-test and effect size (Cohen's d).

RESULTS

The values of intraclass correlation coefficient were >0.70, except for two indices ("nonperseverative errors" and "failure to maintain set" with intraclass correlation coefficient of 0.56 and 0.30, respectively). Seven indices showed nonsignificant differences between the two assessments (t(62)= -0.84 to 1.38, p > 0.05) and negligible effect sizes (d = 0.03-0.13). The values of MDC with 95% certainty were 32.3, 42.0, 31.2, 36.9, 40.1, 3.3, and 3.8 for the "total number correct," "perseverative responses," "perseverative errors," "nonperseverative errors," "conceptual level responses," "number of categories completed," and "failure to maintain set" indices, respectively.

CONCLUSIONS

The WCST has acceptable test-retest reliability. Two indices ("nonperseverative errors" and "failure to maintain set") revealed lower levels of consistency in scores over repeated assessments. Clinicians and researchers should be cautious when using these two indices to interpret of the re-assessment results in people with schizophrenia.IMPLICATIONS FOR REHABILITATIONThe Wisconsin Card Sorting Test showed acceptable test-retest reliability in people with schizophrenia.Six indices of the Wisconsin Card Sorting Test revealed substantial random measurement errors, which should be used cautiously to interpret executive functions over repeated assessments.

A comparison between the original and Tablet-based Symbol Digit Modalities Test in patients with schizophrenia: Test-retest agreement, random measurement error, practice effect, and ecological validity

We aimed to compare the test-retest agreement, random measurement error, practice effect, and ecological validity of the original and Tablet-based Symbol Digit Modalities Test (T-SDMT) over five serial assessments, and to examine the concurrent validity of the T-SDMT in patients with schizophrenia. Sixty patients with chronic schizophrenia completed five serial assessments (one week apart) of the SDMT and T-SDMT and one assessment of the Activities of Daily Living Rating Scale III at the first time point. Both measures showed high test-retest agreement, similar levels of random measurement error over five serial assessments. Moreover, the practice effects of the two measures did not reach a plateau phase after five serial assessments in young and middle-aged participants. Nevertheless, only the practice effect of the T-SDMT became trivial after the first assessment. Like the SDMT, the T-SDMT had good ecological validity. The T-SDMT also had good concurrent validity with the SDMT. In addition, only the T-SDMT had discriminative validity to discriminate processing speed in young and middle-aged participants. Compared to the SDMT, the T-SDMT had overall slightly better psychometric properties, so it can be an alternative measure to the SDMT for assessing processing speed in patients with schizophrenia.

Development of a Performance-Based Measure of Executive Functions in Patients with Schizophrenia

A performance-based measure for assessing executive functions (EF) is useful to understand patients’ real life performance of EF. This study aimed to develop a performance-based measure of executive functions (PEF) based on the Lezak model and to examine psychometric properties (i.e., unidimensionality and reliability) of the PEF using Rasch analysis in patients with schizophrenia. We developed the PEF in three phases: (1) designing the preliminary version of PEF; (2) consultation with experts, cognitive interviews with patients, and pilot tests on patients to revise the preliminary PEF; (3) establishment of the final version of the PEF and examination of unidimensionality and Rasch reliability. Two hundred patients were assessed using the revised PEF. After deleting items which did not satisfy the Rasch model’s expectations, the final version of the PEF contained 1 practice item and 13 test items for assessing the four domains of EF (i.e., volition, planning, purposive action, and effective performance). For unidimensional and multidimensional Rasch analyses, the 4 domains showed good reliability (i.e., 0.77–0.85 and 0.87–0.90, respectively). Our results showed that the PEF had satisfactory unidimensionality and Rasch reliability. Therefore, clinicians and researchers could use the PEF to assess the four domains of EF in patients with schizophrenia.

Correlation between age and MMSE in schizophrenia

BACKGROUND

The Mini-Mental State Examination (MMSE) is widely used in schizophrenia, although normative data are lacking in this population. This review and meta-regression analysis studies the effect of aging on MMSE scores in schizophrenic patients.

METHODS

We entered the search terms schizophrenia and MMSE in PubMed and PsychInfo. Bibliographies of pertinent articles were also examined. We included every study presenting the MMSE scores in schizophrenic patients along with a corresponding mean age. We conducted our analyses using simple linear regression weighted for the inverse of within-trial variance of the age variable, thus conferring more importance to studies with narrower age groups.

RESULTS

We identified 56 articles (n = 5,588) published between 1990 and 2012. The MMSE scores of schizophrenic patients decline by approximately 1 point for every four years (y = 34.939-0.247x, 95% Confidence Interval (CI) [-0.304, -0.189], R 2 = 0,545), which is five times the rate in the general population. Institutionalized patients account for a large proportion of this decline (y = 37.603-0.308x, 95% CI [-0.349, -0.267], R 2 = 0.622) whereas community-dwelling patients are relatively stable throughout aging (y = 27.591-0.026x, 95% CI [-0.074, 0.023], R 2 = 0.037).

CONCLUSIONS

Subgroup analyses show different trajectories between institutionalized and outpatients with schizophrenia. The deterioration observed in institutionalized patients may have to do with greater illness severity, heavier medication load, vascular risk factors, and lack of stimulation in institutional settings. Studies documenting the role of these variables would be useful. Cognitive screening tools that assess executive functions would be interesting to study in schizophrenics, as they may reveal more subtle age-related cognitive changes not measured by the MMSE.

Clinical usefulness of the screen for cognitive impairment in psychiatry (SCIP-S) scale in patients with type I bipolar disorder

BACKGROUND

The relevance of persistent cognitive deficits to the pathogenesis and prognosis of bipolar disorders (BD) is understudied, and its translation into clinical practice has been limited by the absence of brief methods assessing cognitive status in Psychiatry. This investigation assessed the psychometric properties of the Spanish version of the Screen for Cognitive Impairment in Psychiatry (SCIP-S) for the detection of cognitive impairment in BD.

METHODS

After short training, psychiatrists at 40 outpatient clinics administered the SCIP three times over two weeks to a total of 76 consecutive type I BD admissions. Experienced psychologists also administered a comprehensive battery of standard neuropsychological instruments to clinical sample and 45 healthy control subjects.

RESULTS

Feasibility was supported by a brief administration time (approximately 15 minutes) and minimal scoring errors. The reliability of the SCIP was confirmed by good equivalence of forms, acceptable stability (ICC range 0.59 to 0.87) and adequate internal consistency (Chronbach's alpha of 0.74). Construct validity was granted by extraction of a single factor (accounting 52% of the variance), acceptable correlations with conventional neuropsychological instruments, and a clear differentiation between bipolar I and normal samples. Efficiency was also provided by the adequate sensitivity and specificity.

LIMITATIONS

The sample size is not very large. The SCIP and the neurocognitive battery do not cover all potentially relevant cognitive domains. Also, sensitivity to change remains unexplored.

CONCLUSION

With minimal training, physicians obtained a reliable and valid estimate of cognitive impairment in approximately 15 minutes from an application of the SCIP to type I BD patients.

Neighborhood environment and self-reported health status: a multilevel analysis

OBJECTIVES

This study examined whether neighborhood socioeconomic environment helps to explain the proportion of community members with self-reported poor health status.

METHODS

A random sample of 9240 persons aged 25 to 74 years were interviewed during 1988 and 1989. The socioeconomic environment of each respondent's neighborhood was measured with the Care Need Index (CNI) and the Townsend score. The data were analyzed with a multilevel model adjusted for the independent variables. The second-level variables were the 2 neighborhood scores.

RESULTS

There was a clear gradient for poor health and education within every CNI interval so that with an increasing CNI (indicating more deprivation), the prevalence of poor health increased in all 3 education groups (P = .001). In the full model, decreasing educational level, obesity, length and frequency of smoking, physical inactivity, and increasing CNI were associated with poor health. Persons living in the most deprived neighborhoods had a prevalence ratio of 1.69 (95% confidence interval = 1.44, 1.98) for poor health compared with those living in the most affluent areas.

CONCLUSIONS

Both neighborhood socioeconomic environment and individual educational status are associated with self-reported poor health.

A factor analysis of the Mini-Mental State Examination in schizophrenic disorders

The Mini-Mental State Examination (MMSE) is frequently used in schizophrenia studies. Therefore, it is surprising that no factor analysis of MMSE has been carried out in schizophrenic patients. The MMSE ratings of 80 long-term in-patients with DSM-III-R schizophrenia or schizoaffective disorder were introduced into a principal-component analysis with varimax rotation which generated three MMSE factors or subscales (frontal, memory and spatial). The limitations of this pilot study are the small sample size and the inclusion of only hospitalized patients.

Mini-Mental State Examination (MMSE) performance of partially remitted community-dwelling patients with schizophrenia

Patients with schizophrenia perform worse than healthy controls on many neuro-psychological tests. However, previous studies of neuro-cognitive function have mostly been carried out on acutely ill or institutionalized patients. The objective of this study was to generate norms for performance of partially remitted community-dwelling patients with schizophrenia on the Mini-Mental State Examination (MMSE). Partially remitted outpatients attending a depot antipsychotic clinic or a clozapine clinic (n = 272) were tested using the MMSE. Demographic and clinical characteristics associated with MMSE performance, as well as the performance of specific items, were examined. MMSE score was significantly associated with educational status and race. Patients in our sample performed approximately 2-3 points below the population norms at all ages, but the mean score for the group was not in the impaired range. There was no apparent widening of this gap with advancing age. Patients who did poorly most frequently had difficulty with memory, attention and construction tasks. The MMSE is easy to administer to outpatients with schizophrenia and most patients score in the un-impaired range. The MMSE may be used to identify a subgroup of patients who score in the impaired range, for further investigations.

The MOS 36-Item Short Form Health Survey: reliability, validity, and preliminary findings in schizophrenic outpatients

OBJECTIVES

The authors test the reliability and validity of the Medical Outcomes Study Short Form 36-Item Health Survey (SF-36) as a written, self-administered survey in outpatients with chronic schizophrenia.

METHODS

Thirty-six schizophrenic outpatients completed a written and oral form of the SF-36. A psychiatrist rated the patients using the Brief Psychiatric Rating Scale to determine severity of psychopathology. Cognitive functioning and academic achievement were also assessed. Internal consistency, test-retest reliability, concurrent and discriminative validity of the oral and written versions were determined.

RESULTS

The SF-36 in both forms was shown to have good internal consistency, stability, and concurrent validity. The mental health SF-36 subscales had poor discriminant validity, compared with the physical functioning scale that demonstrated good discriminant validity.

CONCLUSIONS

The validity of using the written form of the SF-36 on a sample of patients with chronic mental illness was demonstrated. The SF-36 appears to be an appropriate outcome measure for changes in physical and role functioning in consumers of outpatient mental health programs.

Olfactory identification in elderly schizophrenia and Alzheimer's disease

In the present study we assessed olfactory identification ability using the University of Pennsylvania Smell Identification Test (UPSIT) in 16 elderly patients with schizophrenia (ES), 20 patients with a diagnosis of probable Alzheimer's disease (AD), and 20 healthy elderly controls (EC). Both patient groups exhibited marked deficits in UPSIT performance relative to controls. ES and AD patients with similar levels of general cognitive impairment did not differ on the UPSIT, suggesting that the two disorders may share a common dysfunction in olfactory brain regions. While there have been recent reports of greater olfactory impairment in males, neither patient group exhibited significant gender differences on the UPSIT.

A study of bedside screening procedures for cognitive deficits in chronic psychiatric inpatients

The Mini-Mental State Examination (MMSE) and the Trail Making Test, part B (TMT-B) were compared for screening cognitive deficits in a sample of 104 psychiatric inpatients at a long-term hospital. The TMT-B classified 84% of the patients as impaired. The classic MMSE cutoff score (< or = 23) using serial sevens and spelling classified 42% and 24% of the patients respectively, as impaired. The new MMSE cutoff classified 71% as impaired. Education, gender, and medication may influence impairment when using the MMSE classic cutoff scores. In TMT-B only, impairment was influenced by chronicity. The TMT-B was able to distinguish patients impaired in the MMSE. The TMT-B, therefore, may be more useful than the MMSE to screen for cognitive deficits in chronic psychiatric inpatients.