Shortening patient-reported outcome measures through optimal test assembly: application to the Social Appearance Anxiety Scale in the Scleroderma Patient-centered Intervention Network Cohort

External validation of a shortened screening tool using individual participant data meta-analysis: A case study of the Patient Health Questionnaire-Dep-4

Shortened versions of self-reported questionnaires may be used to reduce respondent burden. When shortened screening tools are used, it is desirable to maintain equivalent diagnostic accuracy to full-length forms. This manuscript presents a case study that illustrates how external data and individual participant data meta-analysis can be used to assess the equivalence in diagnostic accuracy between a shortened and full-length form. This case study compares the Patient Health Questionnaire-9 (PHQ-9) and a 4-item shortened version (PHQ-Dep-4) that was previously developed using optimal test assembly methods. Using a large database of 75 primary studies (34,698 participants, 3,392 major depression cases), we evaluated whether the PHQ-Dep-4 cutoff of ≥ 4 maintained equivalent diagnostic accuracy to a PHQ-9 cutoff of ≥ 10. Using this external validation dataset, a PHQ-Dep-4 cutoff of ≥ 4 maximized the sum of sensitivity and specificity, with a sensitivity of 0.88 (95% CI 0.81, 0.93), 0.68 (95% CI 0.56, 0.78), and 0.80 (95% CI 0.73, 0.85) for the semi-structured, fully structured, and MINI reference standard categories, respectively, and a specificity of 0.79 (95% CI 0.74, 0.83), 0.85 (95% CI 0.78, 0.90), and 0.83 (95% CI 0.80, 0.86) for the semi-structured, fully structured, and MINI reference standard categories, respectively. While equivalence with a PHQ-9 cutoff of ≥ 10 was not established, we found the sensitivity of the PHQ-Dep-4 to be non-inferior to that of the PHQ-9, and the specificity of the PHQ-Dep-4 to be marginally smaller than the PHQ-9.

Development and Evaluation of Short-Form Measures of the HIV/AIDS Knowledge Assessment Tool Among Sexual and Gender Minorities in Brazil: Cross-sectional Study

Background

In theoretical models of health behavior, knowledge about disease transmission and self-protective behaviors are conceptualized as important drivers of behavior change. Several studies conducted in Brazil point to an unfortunate convergence of sexual and gender minority (SGM) populations with low levels of HIV knowledge and younger age, lower education, engagement in higher-risk sexual behavior, and never having tested for HIV. Measures to assess level of HIV knowledge have been previously published, including the 12-item HIV/AIDS Knowledge Assessment (HIV-KA) tool. However, measure length can be a barrier to assessment.

Objective

We started from the 12-item HIV-KA tool and developed candidate short forms using statistical procedures, evaluated their psychometric properties, and tested the equivalency of their associations with other measures of HIV knowledge compared to the 12-item version.

Methods

A convenience sample of SGM was recruited during September 2020 to complete an online survey through advertisements on two social networking apps (Grindr and Hornet). The survey instrument included items on sociodemographic information, prior HIV testing and HIV test results, preexposure prophylaxis (PrEP) and antiretroviral treatment use, sexual behavior, and 3 HIV knowledge measures: the HIV-KA, World Health Organization Knowledge About HIV Transmission Prevention Indicator, and the Brief HIV Knowledge Questionnaire. We used exploratory factor analysis and confirmatory factor analysis (CFA) to assess the factor structure of the of the HIV-KA. We used optimal test assembly (OTA) methods to develop candidate short forms of the HIV-KA and evaluated them based on prespecified reliability, concurrent validity, and statistically equivalent convergent validity criteria.

Results

Among 2552 SGM individuals from Brazil, mean age was 35.1 years, 98.2% (2507/2552) cisgender men and 1.8% (45/2552) transgender/nonbinary, 56.5% (1441/2552) White, and 31.0% (792/2552) self-reported HIV positive. CFA indicated a 1-factor structure for the 12-item HIV-KA. Concurrent validity correlations were high for all short forms with 6 items, but only versions with 9 items were as reliable as the full-length form and demonstrated equivalency for convergent validity correlations. Suggesting post hoc convergent validity, HIV knowledge scores using the 9- and 10-item short forms were higher for participants who perceived the Undetectable Equals Untransmittable (U=U) slogan as completely accurate versus not accurate. Suggesting post hoc concurrent validity, participants of younger age, of Black, Pardo or indigenous race, and reporting lower education and lower income scored lower on HIV knowledge. Participants who never tested for HIV scored lower than those who tested negative or positive, while those currently using PrEP scored higher than those reporting past or never use.

Conclusions

OTA methods were used to shorten the 12-item HIV-KA to 9-item and 10-item versions while maintaining comparable reliability and validity among a large sample of Brazilian SGM. However, these short forms did not shorten sufficiently to justify deviation from the full measure.

Mask-Wearing Behavior During the COVID-19 Pandemic: A Cross-Cultural Comparison Between the United States and South Korea

This study identified and compared factors that directly and indirectly influenced face mask-wearing in the United States and South Korea during the COVID-19 pandemic by applying the theory of reasoned action. The overall levels of attitudes and future mask-wearing behavioral intention were lower for United States than Korean participants (N _US = 150 and N _{South Korea} = 150). Differences between the groups were noted in the impact of norms and background characteristics on attitudes and behavioral intention. Messages communicating the importance of wearing masks should be tailored to different cultures. Americans should be encouraged to try wearing masks on their own while Koreans should hear about the social benefits of mask-wearing.

Shortening the Edinburgh postnatal depression scale using optimal test assembly methods: Development of the EPDS-Dep-5

AIMS

This study used a large database to develop a reliable and valid shortened form of the Edinburgh Postnatal Depression Scale (EPDS), a self-report questionnaire used for depression screening in pregnancy and postpartum, based on objective criteria.

METHODS

Item responses from the 10-item EPDS were obtained from 5157 participants (765 major depression cases) from 22 primary screening accuracy studies that compared the EPDS to the Structured Clinical Interview for DSM (SCID). Unidimensionality of the EPDS latent construct was verified using confirmatory factor analysis, and an item response theory model was fit. Optimal test assembly (OTA) methods identified a maximally informative shortened form for each possible scale length between 1 and 9 items. The final shortened form was selected based on pre-specified validity and reliability criteria and non-inferiority of screening accuracy of the EPDS as compared to the SCID.

RESULTS

A 5-item short form of the EPDS (EPDS-Dep-5) was selected. The EPDS-Dep-5 had a Cronbach's alpha of 0.82. Sensitivity and specificity of the EPDS-Dep-5 for a cutoff of 4 or greater were 0.83 (95% CI, 0.73, 0.89) and 0.86 (95% CI, 0.80, 0.90) and were statistically non-inferior to the EPDS. The correlation of total scores with the full EPDS was high (r = 0.91).

CONCLUSION

The EPDS-Dep-5 is a valid short form with minimal loss of information when compared to the full-length EPDS. The EPDS-Dep-5 was developed with OTA methods using objective, pre-specified criteria, but the approach is data-driven and exploratory. Thus, there is a need to replicate results of this study in different populations.

A Short Measure of Acceptability of Intimate Partner Violence Against Women: Development and Validation of the A-IPVAW-8 Scale

Attitudes of acceptability of intimate partner violence against women (IPVAW) are considered one of the main risk factors of this type of violence. The aim of this study is to develop and validate a short version of the acceptability of IPVAW scale, the A-IPVAW-8, for large scale studies where space and time are limited. A panel of experts were asked to assess item content validity. Two samples were recruited to assemble an 8-item short version of the scale using automated test assembly, and to reassess the psychometric properties of the A-IPVAW-8 in an independent sample. Results showed that the A-IPVAW-8 had adequate internal consistency (α = .72-.76, ω = .73-.81), a stable one-factor latent structure (comparative fit index [CFI] = 0.94, Tucker-Lewis index = 0.92, root mean square error of approximation = 0.077), validity evidences based on its relationships to other variables in both samples, and was also invariant across gender (ΔCFI < |0.02|). This study provides a short, easy-to-use tool to evaluate attitudes of acceptability of IPVAW for large scale studies.

A Short Form of the Chinese Version of the Weinstein Noise Sensitivity Scale through Optimal Test Assembly

This study developed a short form of the traditional Chinese version of the Weinstein Noise Sensitivity Scale (WNSS) through optimal test assembly (OTA). A total of 1069 Chinese adults (64.8% female) completed the territory-wide cross-sectional study. We first removed Items 12 and 5 which had negative factor loading and gender-related differential item functioning (DIF), respectively. The optimal length was then determined as the minimal one that reasonably resembled the reliability and validity of the scale without DIF items. OTA identified an 8-item WNSS (WNSS-8) which retained 67.2% of the test information of the original 21-item scale and had a Cronbach's alpha of 0.83. It also showed significant correlations of 0.272 and -0.115 with the neuroticism and extraversion scales of Chinese NEO-Five Factor Inventory, respectively. Adequate model fit of the WNSS-8 was demonstrated by the confirmatory factor analysis. The Chinese WNSS-8 can be used to assess noise sensitivity without compromising reliability and validity.

Assessing differential item functioning for the Social Appearance Anxiety Scale: a Scleroderma atient-centred Intervention Network (SPIN) Cohort Study

OBJECTIVES

The Social Appearance Anxiety Scale (SAAS) is a 16-item questionnaire developed to evaluate fear of appearance-based evaluation by others. The primary objective of this research was to investigate the existence of differential item functioning (DIF) for the 16 SAAS items, comparing patients who completed the SAAS in English and French, either to confirm that scores are comparable or provide guidance on calculating comparable scores. A secondary research objective was to investigate the existence of DIF based on sex and disease status. A tertiary research objective was to assess DIF related to language, sex, and disease status on the recently developed SAAS-5.

DESIGN

This was a cross-sectional analysis using baseline data from patients enrolled in the Scleroderma Patient-centred Intervention Network (SPIN).

SETTING

SPIN patients included in the present study were enrolled at 43 centres in Canada, USA, UK, France and Australia, with questionnaires completed in April 2014 to July 2019.

PARTICIPANTS

1640 SPIN patients completed the SAAS in French (n=600) or English (n=1040).

PRIMARY AND SECONDARY MEASURES

The SAAS was collected along with demographic and disease characteristics.

RESULTS

Six items were identified with statistically significant language-based DIF, four with sex-based DIF and one with disease type-based DIF. However, factor scores before and after accounting for DIF were similar (Pearson correlation >0.99), and individual score differences were small. This was true for both the full and shortened versions of the SAAS.

CONCLUSION

SAAS and SAAS-5 scores are comparable across language, sex, and disease-type, despite small differences in how patients respond to some items.

Nonrestorative sleep scale: a reliable and valid short form of the traditional Chinese version

Purpose

Previous research has suggested the essential unidimensionality of the 12-item traditional Chinese version of the Nonrestorative Sleep Scale (NRSS). This study aimed to develop a short form of the traditional Chinese version of the NRSS without compromising its reliability and validity.

Methods

Data were collected from 2 cross-sectional studies with identical target groups of adults residing in Hong Kong. An iterative Wald test was used to assess differential item functioning by gender. Based on the generalized partial credit model, we first obtained a shortened version such that further shortening would result in substantial sacrifice of test information and standard error of measurement. Another shortened version was obtained by the optimal test assembly (OTA). The two shortened versions were compared for test information, Cronbach’s alpha, and convergent validity.

Results

Data from a total of 404 Chinese adults (60.0% female) who had completed the Chinese NRSS were gathered. All items were invariant by gender. A 6-item version was obtained beyond which the test performance substantially deteriorated, and a 9-item version was obtained by OTA. The 9-item version performed better than the 6-item version in test information and convergent validity. It had discrimination and difficulty indices ranging from 0.44 to 2.23 and − 7.58 to 2.13, respectively, and retained 92% of the test information of the original 12-item version.

Conclusion

The 9-item Chinese NRSS is a reliable and valid tool to measure nonrestorative sleep for epidemiological studies.

Electronic supplementary material

The online version of this article (10.1007/s11136-020-02523-4) contains supplementary material, which is available to authorized users.