When less is more: reducing redundancy in mental health and psychosocial instruments using Item Response Theory

Reliability and Mental Health Correlates of a Single-Item Measure of Self-Rated Mental Health (SRMH) in the Chinese Context

The use of single-item measures of self-rated mental health (SRMH) has been increasingly valued in epidemiologic research. However, little is known about the reliability and mental health correlates of SRMH in Chinese populations. This study examined the reliability and mental health correlates of SRMH in three Chinese samples. We analyzed data collected from two convenience samples of Chinese adults from Hong Kong and/or Taiwan (Sample 1: N = 205; Sample 2: N = 377), and a random sample of Taiwan psychiatric inpatients (Sample 3: N = 100). Our results showed that the single-item measure of SRMH had moderate to good test-retest reliability (intraclass correlation [ICC] = 0.75) in Sample 1 and acceptable reliability between the self-report and interviewer-administered versions (ICC = 0.58) in Sample 3. It had a high positive correlation with self-esteem and a moderately high negative correlation with depression. It also had a consistently negative correlation with borderline personality disorder symptoms and post-traumatic stress disorder symptoms. The SRMH score was also associated with psychiatric service usage. These findings contribute to the body of knowledge regarding the use of a single-item measure of SRMH to assess overall self-perceived mental health in Chinese communities.

Item response theory-based psychometric analysis of the Short Warwick-Edinburgh Mental Well-Being Scale (SWEMWBS) among adolescents in the UK

BACKGROUND

Middle and late adolescence is the period in a person's life that is most vulnerable to mental health problems. To enable an evidence base that can support policies to prevent such problems, it is crucial to have good quality, reliable, and accurate measurement tools for mental well-being. One of them is the Short Warwick-Edinburgh Mental Well-Being Scale (SWEMWBS). This study aimed to test the psychometric properties of the SWEMWBS on a large sample of adolescents aged 16 to 19 from the United Kingdom (UK) (N = 8,090). Data were from four waves of the longitudinal panel study Understanding Society.

METHODS

The analysis was conducted using Item Response Theory (IRT), which is the most appropriate method for testing psychometric properties. The Graded Response Model (GRM) was applied to the data. The reliability and criterion validity of the SWEMWBS were also examined.

RESULTS

The presented results confirm the very good psychometric properties of the SWEMWBS amongst adolescents aged 16 to 19 years. The assumptions for the use (unidimensionality, local non-independence, monotonicity) of IRT were met. The results of GRM showed very high discriminant power for all items. The five-category response scale performed optimally; however, differences were found between points on the response scale both between and within items. In general, the scale as a whole showed very good functioning, but particularly in the negative values of mental well-being.

CONCLUSIONS

The SWEMWBS was confirmed as a concise, reliable, and valid instrument for measuring mental well-being among older UK adolescents.

A Novel Approach for Developing Efficient and Convenient Short Assessments to Approximate a Long Assessment

This paper describes a novel Long to Short approach that uses machine learning to develop efficient and convenient short assessments to approximate a long assessment. This approach is applicable to any assessments used to assess people's behaviors, opinions, attitudes, mental and physical states, traits, aptitudes, abilities, and mastery of a subject matter. We demonstrated the Long to Short approach on the Depression Anxiety Stress Scale (DASS-42) for assessing anxiety levels in adults. We first obtained data for the original assessment from a large sample of participants. We then derived the total scores from participants' responses to all items of the long assessment as the ground truths. Next, we used feature selection techniques to select participants' responses to a subset of items of the long assessment to predict the ground truths accurately. We then trained machine learning models that uses the minimal number of items needed to achieve the prediction accuracy similar to that when the responses to all items of the whole long assessment are used. We generated all possible combinations of minimal number of items to create multiple short assessments of similar predictive accuracies for use if the short assessment is to be done repeatedly. Finally, we implemented the short anxiety assessments in a web application for convenient use with any future participant of the assessment.

Reliability and validity of a perinatal depression screening instrument in rural Mali

Background

In order to reduce the burden of perinatal depression in low- and middle-income countries, health systems must be able to identify and treat women suffering from depression. The objective of our study was to develop a locally valid and reliable screening instrument for use in identifying pregnant women and mothers of young children with a local depression syndrome, dusukasi, in rural Mali.

Methods

We administered a locally adapted screening instrument containing items from the Edinburgh Postpartum Depression Scale (EPDS) and Hopkins Symptom Checklist (HSCL-25) to 180 pregnant women and mothers of children under age 2 in Sélingué, Mali to assess the instrument's psychometric properties and validity. Item Response Theory was used to develop an abbreviated version of the measure and the validity and psychometric properties of this shortened version were compared with the full-length scale.

Results

The full 28-item scale exhibited a single factor structure with good internal consistency (Cronbach's alpha = 0.92). Women who self-identified as suffering from dusukasi (n = 87) in a known groups analysis to assess construct validity had significantly higher depression and anxiety symptom scores (p < 0.0001) and functional impairment scores (p < 0.0001) compared to women not reporting dusukasi (n = 93). The shortened 16-item scale performed as well as the full scale in identifying women with dusukasi.

Conclusions

Construct validity of our adapted screening instrument was supported for identifying dusukasi in rural Malian women. Our methodology can be applied in other settings to develop similarly valid screening instruments for perinatal depression.

The measurement of war-related trauma amongst internally displaced men and women in South Sudan: Psychometric analysis of the Harvard Trauma Questionnaire

BACKGROUND

Studies from armed conflict settings, including South Sudan, have revealed the deleterious mental health impact of exposure to war atrocities. However, there is little consensus on what is meant by war trauma, how it should be measured, and how levels of trauma vary across men and women.

METHODS

We used psychometric analyses to measure war trauma among 1178 internally displaced adults (mean age = 39 years, 50% women) in the Malakal region of South Sudan. We used cross-sectional survey data and applied classical test theory, factor analysis, item response theory, and differential item functioning with the war events subscale (17 items) of the Harvard Trauma Questionnaire (HTQ).

RESULTS

We found good validity and internal consistency reliability for the HTQ. We found evidence for unidimensionality using factor analyses, and item response theory models showed that some war events (like witnessing the killing of family or friends) were more sensitive to the underlying 'war-related trauma' trait than others (like abduction). Differential item functioning analyses revealed that the measure performed differently for men and women, indicating the need for sex-stratified analysis in the measurement of trauma.

LIMITATIONS

The use of self-report may lead to recall and response bias, and the study sample may not be representative of the broader population in South Sudan.

CONCLUSION

This study emphasizes the need for cultural adaptation and psychometric evaluation of commonly used measurement instruments, especially in humanitarian settings where survey data are used to set priorities for mental health and psychosocial support services.

Adaptation of the Texas Christian University Organisational Readiness for Change Short Form (TCU-ORC-SF) for use in primary health facilities in South Africa

OBJECTIVES

The Texas Christian University Organisational Readiness for Change Scale (TCU-ORC) assesses factors influencing adoption of evidence-based practices. It has not been validated in low-income and middle-income countries (LMIC). This study assessed its psychometric properties in a South African setting with the aim of adapting it into a shorter measure.

METHODS

This study was conducted in 24 South African primary healthcare clinics in the Western Cape Province. The TCU-ORC and two other measures, the Organisational Readiness to Change Assessment (ORCA) and the Checklist for Assessing Readiness for Implementation (CARI) were administered. The questionnaire was readministered after 2 weeks to obtain data on test-retest reliability. Three hundred and ninety-five surveys were completed: 281 participants completed the first survey, and 118 recompleted the assessments.

RESULTS

We used exploratory factor analysis (EFA) to identify latent dimensions represented in the data. Cronbach's alpha for each subscale was assessed and we examined the extent to which the subscales and total scale scores for the first and retest surveys correlated. Convergent validity was assessed by the correlation coefficient between the TCU-ORC, ORCA and CARI total scale scores. EFA resulted in a three-factor solution. The three subscales proposed are Clinic Organisational Climate (8 items), Motivational Readiness for Change (13 items) and Individual Change Efficacy (5 items) (26 items total). Cronbach's alpha for each subscale was >0.80. The overall shortened scale had a test-retest correlation of r=0.80, p<0.01, acceptable convergent validity with the ORCA scale (r=0.56, p<0.05), moderate convergence with the CARI (r=39, p<0.05) and strong correlation with the original scale (r=0.79, p<0.05).

CONCLUSIONS

This study presents the first psychometric data on the TCU-ORC from an LMIC. The proposed shortened tool may be more feasible for use in LMICs.

TRIAL REGISTRATION NUMBER

Results stage. Project MIND trial. Pan-African Clinical Trials Registry. PACTR201610001825405.

Brief Spanish Zung Self-Rating Anxiety Scale: Dimensionality, Internal Consistency, Nomological Validity, and Differential Item Functioning Among Chronic Obstructive Pulmonary Disease Patients in Colombia

BACKGROUND AND PURPOSES

The brief Spanish Zung Self-Rating Anxiety Scale is a tenitem tool available; however, there is little information about its psychometric performance. The study aimed to test the dimensionality, internal consistency, nomological validity, and gender differential item functioning of the scale in patients with chronic obstructive pulmonary disease (COPD) in Colombia.

METHODS

A psychometric study was designed. It participated in 408 COPD patients aged between 40 and 102 years.

RESULTS

The confirmatory factor analysis showed both one- and two-dimension structures with acceptable goodness-of-fit indicators. The brief Spanish Zung Self-Rating Anxiety Scale presented high internal consistency, acceptable nomological validity, and non-gender differential functioning.

CONCLUSIONS

The brief Spanish Zung Self-Rating Anxiety Scale presents some good indicators of validity and reliability, and it can be used in the measurement of anxiety in COPD patients.

Brief Screening Tool for Stepped-Care Management of Mental and Substance Use Disorders

OBJECTIVE

Stepped mental health care requires a rapid method for nonspecialists to detect illness. This study aimed to develop and validate a brief instrument, the Mental Wellness Tool (mwTool), for identification and classification.

METHODS

Cross-sectional development and validation samples included adults at six health facilities in Mozambique. Mini International Neuropsychiatric Interview diagnoses were the criterion standard. Candidate items were from nine mental disorder and functioning assessments. Regression modeling and expert consultation determined best items for identifying any mental disorder and classifying positives into disorder categories (severe mental disorder, common mental disorder, substance use disorder, and suicide risk). For validation, sensitivity and specificity were calculated for any mental disorder (index and proxy respondents) and disorder categories (index).

RESULTS

From the development sample (911 participants, mean±SD age=32.0±11 years, 63% female), 13 items were selected-three with 0.83 sensitivity (95% confidence interval [CI]=0.79-0.86) for any mental disorder and 10 additional items classifying participants with a specificity that ranged from 0.72 (severe mental disorder) to 0.90 (suicide risk). For validation (453 participants, age 31±11 years, 65% female), sensitivity for any mental disorder was 0.94 (95% CI=0.89-0.97) with index responses and 0.73 (95% CI=0.58-0.85) with family proxy responses. Specificity for categories ranged from 0.47 (severe mental disorder) to 0.93 (suicide risk). Removing one item increased severe mental disorder specificity to 0.63 (95% CI=0.58-0.68).

CONCLUSIONS

The mwTool performed well for identification of any mental disorder with index and proxy responses to three items and for classification into treatment categories with index responses to nine additional items.