A review of Evidence‐Based Evaluation of Measures for Assessing Social and Emotional Well‐Being in Indigenous Australians

Against the use of the Strengths and Difficulties Questionnaire for Aboriginal and Torres Strait Islander children aged 2-15 years

OBJECTIVE

The Strengths and Difficulties Questionnaire is a widely used screening tool for emotional and behavioural problems in children. Recent quantitative analyses have raised concerns regarding its structural validity in Aboriginal and Torres Strait Islander communities. This paper aims to extend upon existing findings by analysing the factor structure of both the parent- and teacher-reported Strengths and Difficulties Questionnaire in this population across a broader age range than in previous studies.

METHODS

Participants were the caregivers and teachers of 1624 Aboriginal and Torres Strait Islander children (820 male, 804 female) aged 2-15 years from Waves 2-11 of the Longitudinal Study of Indigenous Children. The majority of children were Aboriginal living in major cities and inner regional areas. Internal consistency was estimated with McDonald's Omega. Exploratory structural equation modelling was conducted to investigate the factor structure of the parent-reported and teacher-reported versions of the Strengths and Difficulties Questionnaire.

RESULTS

Responses from teachers demonstrated higher internal consistency than responses from parents, which was unacceptably low across most age groups. The purported five-factor structure of the Strengths and Difficulties Questionnaire failed to be replicated across both parent- and teacher-reported questionnaires. The results of bifactor and hierarchical exploratory structural equation models also failed to approximate the higher-order summary scales. These results indicate that the Strengths and Difficulties Questionnaire subscales and summary scores do not provide a valid index of emotional and behavioural problems in Aboriginal and Torres Strait Islander children.

CONCLUSION

The Strengths and Difficulties Questionnaire should not be used with Aboriginal and Torres Strait Islander children.

Health-related quality of life measures used with Indigenous children/youth in the Pacific Rim: a scoping review

OBJECTIVE

To identify and describe (1) which health-related quality of life (HRQoL) measures have been used with Indigenous children/youth (aged 8-17 years) within the Pacific Rim; and (2) studies that refer to Indigenous health concepts in the use of child/youth HRQoL measures.

DESIGN

A scoping review.

DATA SOURCES

Ovid (Medline), PubMed, Scopus, Web of Science and CINAHL were searched up until 25 June 2020.

ELIGIBILITY CRITERIA

Eligible papers were identified by two independent reviewers. Eligible papers were written in English, published between January 1990 and June 2020 and included an HRQoL measure used in research with Indigenous child/youth populations (aged between 8 and 17 years) in the Pacific Rim region.

DATA EXTRACTION AND SYNTHESIS

Data extracted included study characteristics (year, country, Indigenous population, Indigenous sample size, age group), HRQoL measure characteristics (generic or condition-specific measure, child or adult measure, who completed the measure(s), dimensions, items and response scale of measure) and consideration of Indigenous concepts (created for Indigenous population, modified for Indigenous population, validated for Indigenous population, reliability in Indigenous populations, Indigenous involvement, reference to Indigenous theories/models/frameworks).

RESULTS

After removing duplicates, 1393 paper titles and abstracts were screened, and 543 had full-text review for eligibility. Of these, 40 full-text papers were eligible, reporting on 32 unique studies. Twenty-nine HRQoL measures were used across eight countries. Thirty-three papers did not acknowledge Indigenous concepts of health, and only two measures were specifically created for use with Indigenous populations.

CONCLUSIONS

There is a paucity of research investigating HRQoL measures used with Indigenous children/youth and a lack of involvement of Indigenous peoples in the development and use of HRQoL measures. We strongly recommend explicit consideration of Indigenous concepts when developing, validating, assessing and using HRQoL measures with Indigenous populations.

Exploring Indigenous Community Conceptions of Parent Wellbeing: A Qualitative Analysis

Using non-Indigenous perspectives of parental social and emotional wellbeing in the design and application of parent support programs can undermine program effectiveness as it may not account for Indigenous family structures and community values. With a clearer understanding of Indigenous parent wellbeing and its determinants, parenting interventions can be more appropriately designed and tailored to provide support for Indigenous families. This study utilised a community-based participatory action research approach involving collaboration between the research team, participants, and community advisory groups to explore Indigenous parents' and carers' conceptions of wellbeing. Participants' cultural perspectives on parent wellbeing were collected through semi-structured focus groups and in-depth interviews (N = 20). Thematic analysis was undertaken using theory-driven and interpretative phenomenological analysis. Eleven themes emerged as risk and protective factors across three domains: child domain (i.e., school attendance and education, respect, routine, development), parent domain (i.e., role modelling, self-regulation of body, self-regulation of mind and emotions, parenting strategies), and context domain (i.e., connections to family and kinship, community, access to services). It is noteworthy that parents reported three super-ordinate intersecting themes across all domains: connection to culture, Country, and spirituality. In addition, Indigenous parents' and carers' conception of their own wellbeing is closely linked to their children's wellbeing, their lived community context, and expected personal indicators. In recognising and working with this holistic view of Indigenous parent wellbeing, parent support programs can be optimally designed and implemented in Indigenous communities.

Counting what counts: a systematic scoping review of instruments used in primary healthcare services to measure the wellbeing of Indigenous children and youth

BACKGROUND

Primary healthcare services have principal responsibility for providing child and youth wellbeing and mental health services, but have lacked appropriate measurement instruments to assess the wellbeing of Indigenous children and youth or to evaluate the effectiveness of programs and services designed to meet their needs. This review assesses the availability and characteristics of measurement instruments that have been applied in primary healthcare services in Canada, Australia, New Zealand and the United States (CANZUS countries) to assess the wellbeing of Indigenous children and youth.

METHODS

Fifteen databases and 12 websites were searched in December 2017 and again in October 2021. Pre-defined search terms pertained to Indigenous children and youth, CANZUS country names, and wellbeing or mental health measures. PRISMA guidelines were followed, with eligibility criteria guiding screening of titles and abstracts, and selected full-text papers. Results are presented based on the characteristics of documented measurement instruments assessed according to five desirability criteria: development for Indigenous youth populations, adherence to relational strength-based constructs, administration by child and or youth self-report, reliability and validity, and usefulness for identifying wellbeing or risk levels.

RESULTS

Twenty-one publications were found that described the development and or use by primary healthcare services of 14 measurement instruments, employed across 30 applications. Four of the 14 measurement instruments were developed specifically for Indigenous youth populations, four focused solely on strength-based wellbeing concepts but none included all Indigenous wellbeing domains.

CONCLUSION

There is a diversity of measurement instruments available, but few fit our desirability criteria. Although it is possible that we missed relevant papers and reports, this review clearly supports the need for further research to develop, refine or adapt instruments cross-culturally to measure the wellbeing of Indigenous children and youth.

Measuring emotional well-being through subjective report: a scoping review of reviews

OBJECTIVE

This scoping review of reviews aims to describe the current landscape of measures of emotional well-being (EWB).

METHODS

Following established practices for scoping reviews, we searched the PsycInfo, ERIC, Scopus and PubMed databases in June and July 2021 for reviews of measures of EWB that described their review methods and psychometric properties of included measures. From each eligible article, two coders independently extracted the authors' (1) definition of EWB, (2) purpose for the review, (3) methods (eg, search terms, inclusion and exclusion criteria), (4) identified measures (including any noted adaptations) and (5) the scope of psychometric information presented. Descriptive and content analyses were used to examine data.

RESULTS

Forty-nine reviews were included in this scoping review. Reviews included anywhere between 1 and 34 measures of EWB and 135 unique EWB measures were captured across all reviews. We found that there was no consistent definition of EWB, identified measures varied widely and reviews were published in a range of disciplines. Psychometric evidence varied as did authors' purposes for conducting the reviews.

CONCLUSIONS

Overall, these reviews suggest that literature on EWB measurement is disjointed and diffuse. Conceptual integration and harmonisation of measures is needed to advance knowledge of EWB and its measurement.

TRIAL REGISTRATION NUMBERS

10.17605/OSF.IO/BQDS7 and 10.17605/OSF.IO/WV8PF.

Community Engagement and Psychometric Methods in Aboriginal and Torres Strait Islander Patient-Reported Outcome Measures and Surveys-A Scoping Review and Critical Analysis

(1) Background: In healthcare settings, patient-reported outcome measures (PROMs) and surveys are accepted, patient-centered measures that provide qualitative information on dimensions of health and wellbeing. The level of psychometric assessment and engagement with end users for their design can vary significantly. This scoping review describes the psychometric and community engagement processes for PROMs and surveys developed for Aboriginal and Torres Strait Islander communities. (2) Methods: The PRISMA ScR guidelines for scoping reviews were followed, aimed at those PROMs and surveys that underwent psychometric assessment. The Aboriginal and Torres Strait Islander Quality Appraisal Tool and a narrative synthesis approach were used. (3) Results: Of 1080 articles, 14 were eligible for review. Most articles focused on a validity assessment of PROMs and surveys, with reliability being less common. Face validity with Aboriginal and Torres Strait Islander communities was reported in most studies, with construct validity through exploratory factor analyses. Methodological design risks were identified in the majority of studies, notably the absence of explicit Indigenous knowledges. Variability existed in the development of PROMs and surveys for Aboriginal and Torres Strait Islander communities. (4) Conclusions: Improvement in inclusion of Indigenous knowledges and research approaches is needed to ensure relevance and appropriate PROM structures. We provide suggestions for research teams to assist in future design.

Understanding Aboriginal Models of Selfhood: The National Empowerment Project's Cultural, Social, and Emotional Wellbeing Program in Western Australia

Culturally safe and responsive interventions that acknowledge Aboriginal models of selfhood are needed. Such interventions empower Aboriginal peoples and communities by increasing self-determination over individual and community social and emotional wellbeing (SEWB). In response to this need, the National Empowerment Project developed the Cultural, Social, and Emotional Wellbeing Program (CSEWB). The CSEWB aims to strengthen SEWB and cultural identity and subsequently reduce psychological distress in Aboriginal peoples. An Aboriginal Participatory Action Research approach ensured community ownership and engagement. Seven research questions and a culturally modified adaption of the Most Significant Change technique informed a thematic analysis of the evaluation content. Aboriginal adults (n = 49; 53% ≥50 years, 66% female, 34% male) from three Western Australian urban communities participated in the program evaluation workshops. Participants reported the benefits of enhanced SEWB and reduced psychological distress. This research reaffirms the need for culturally safe programs that acknowledge social determinants of health and are guided by the SEWB framework. Long-term commitment from the government is needed to support such programs.

Assessment tools measuring health-related empowerment in psychosocially vulnerable populations: a systematic review

BACKGROUND

Many programs are undertaken to facilitate the empowerment of vulnerable populations across the world. However, an overview of appropriate empowerment measurements to evaluate such initiatives remains incomplete to date. This systematic review aims to describe and summarise psychometric properties, feasibility and clinical utility of the available tools for measuring empowerment in psychosocially vulnerable populations.

METHODS

A systematic literature review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines was completed. A descriptive approach was used for data analysis. Papers were eligible if they explored the development, validation, cross-cultural translation or the utility of an empowerment measurement tool in the context of psychosocially vulnerable populations.

RESULTS

Twenty-six included articles described twenty-six separate studies in which 16 empowerment measurement tools were developed, validated/translated, or used. There was heterogeneity in empowerment constructs, samples targeted, and psychometric properties measured. The measurement of reliability of the included instruments was satisfactory in most cases. However, the validity, responsiveness, interpretability, feasibility and clinical utility of the identified measurement tools were often not adequately described or measured.

CONCLUSION

This systematic review provides a useful snapshot of the strengths as well as limitations of existing health related empowerment measurement tools used with psychosocially vulnerable populations in terms of their measurement properties, and constructs captured. It highlights significant gaps in empowerment tool measurement, development and evaluation processes. In particular, the results suggest that in addition to systematic assessments of psychometric properties, the inclusion of feasibility and clinical utility as outcome measures are important to assess relevance to clinical practice.

Development and validation of a parent-proxy health-related quality of life survey for Australian First Nations children

OBJECTIVE

Within Australia, Aboriginal and Torres Strait Islander (First Nations) populations perceive health and well-being differently to non-Indigenous Australians. Existing health-related quality of life (HR-QoL) measurement tools do not account for these differences. The objective of this study was to develop and validate a culturally specific parent-proxy HR-QoL measurement tool for First Nations children.

DESIGN

Scale development was informed by parents/carers of children with a chronic illness and an expert panel. The preliminary 39-item survey was reviewed (n=12) and tested (n=163) with parents/carers of First Nations children aged 0-12 years at baseline with comparative scales: the Kessler Psychological Distress Scale, generic HR-QoL (Paediatric QoL Inventory 4.0, PedsQL4.0) and Spence Children's Anxiety Scale, and repeated (n=46) 4 weeks later. Exploratory Factor Analysis was used for scale reduction. Reliability and validity were assessed by internal consistency, test-retest, and correlations with comparison scales.

RESULTS

Items within our First Nations-Child Quality of Life (FirstNations-CQoL) were internally consistent with Cronbach's alpha coefficients of ≥0.7 (quality of life, 0.808; patient experience, 0.880; patient support, 0.768) and overall test-retest reliability was good (r=0.75; 95% CI 0.593 to 0.856). Convergent validity was observed with the PedsQL4.0 with Pearson's coefficients of r=0.681 (ages 2-4 years); r=0.651 (ages 5-12 years) and with the Kessler Psychological Distress scale (r=-0.513). Divergent validity against the Spence Anxiety Scale was not demonstrated.

CONCLUSIONS

The FirstNations-CQoL scale was accepted by the participants, reliable and demonstrated convergent validity with comparison measures. This tool requires further evaluation to determine responsiveness, its minimal important difference and clinical utility.

Measuring Social and Emotional Wellbeing in Aboriginal Youth Using Strong Souls: A Rasch Measurement Approach

Currently, there are few robustly evaluated social and emotional wellbeing (SEWB) measures available for use with Aboriginal youth in research, policy, and practice. As such, this study used a Rasch measurement approach to examine the psychometric properties of Strong Souls, a 25-item self-reported SEWB instrument, created for use with Aboriginal youth in the Northern Territory. Our sample (N = 154) included youth (15–25 years old) living on Whadjuk (metropolitan Western Australia; N = 91) and Kamilaroi countries (rural New South Wales; N = 63). Using Rasch modelling techniques, evidence for multidimensionality in the scale was observed, resulting in subsequent analyses conducted separately on two subscales: Psychological Distress and Resilience. The Resilience subscale did not meet the Rasch model assumptions, with poor person and item separation and reliability indexes suggesting the scale was not reliably differentiating between participants’ Resilience scores. The Psychological Distress subscale had mixed separation and reliability index results, with good construct validity implied but poorer ability to target the distress of participants. Our findings provide novel evidence demonstrating the functioning of Strong Souls in a contemporary sample of Aboriginal youth, suggesting further modifications of the instrument are required before it can be used with confidence as a reliable measure in this population group.

Social and emotional wellbeing among Indigenous Australian correctional officers

The workplace challenges of correctional officers have been well documented. Yet little is known about how such stressors impact the wellbeing of correctional officers from minority cultural-subgroups. This study explored Indigenous Australian correctional officers' social and emotional wellbeing (SEWB) when managing workplace adversity. Specifically, the study sought to identify culturally-specific stressors, protective factors and associated response tendencies to gain a more robust understanding of the factors that impact Indigenous officers' wellbeing in the workplace. In total, 15 Indigenous Australian correctional officers from the Northern Territory participated in the study through focus groups or semi-structured interviews. Thematic analysis of participant data revealed that Indigenous correctional officers' SEWB is influenced by four broad categories: cultural connection, morale, social functioning and somatic health. Results are discussed in terms of common themes, future research and implications for assisting the SEWB of Indigenous officers.

Instruments for Measuring the Resilience of Indigenous Adolescents: An Exploratory Review

Introduction: Resilience is enabled by internal, individual assets as well as the resources available in a person's environment to support healthy development. For Indigenous people, these resources and assets can include those which enhance cultural resilience. Measurement instruments which capture these core resilience constructs are needed, yet there is a lack of evidence about which instruments are most appropriate and valid for use with Indigenous adolescents. The current study reviews instruments which have been used to measure the resilience of Indigenous adolescents in Canada, Australia, New Zealand, and the United States (the CANZUS nations). The aim is to provide guidance for the future use of instruments to measure resilience among Indigenous adolescents and provide recommendations for research to strengthen evidence in this area. Method: Instruments were identified through a systematic search of resilience intervention and indicator studies targeting Indigenous youth from CANZUS nations. The studies were analyzed for information on the constructs of resilience measured in the instruments, their use with the targeted groups, and their psychometric properties. A second search was conducted to fill in any gaps in information. Instruments were included if they measured at least one construct of resilience reflecting individual assets, environmental resources, and/or cultural resilience. Results: A total of 20 instruments were identified that measured constructs of resilience and had been administered to Indigenous adolescents in the CANZUS nations. Instruments which measured both individual assets and environmental resources (n = 7), or only environmental resources (n = 6) were most common. Several instruments (n = 5) also measured constructs of cultural resilience, and two instruments included items addressing all three constructs of individual assets, environmental resources, and cultural resilience. The majority of the reviewed studies tested the reliability (75%) and content or face validity (80%) of instruments with the target population. Conclusion: There are several validated instruments available to appropriately measure constructs of resilience with Indigenous adolescents from CANZUS nations. Further work is needed on developing a consistent framework of resilience constructs to guide research efforts. Future instrument development and testing ought to focus on measures which include elements of all three core constructs critical to Indigenous adolescent resilience.

Social and emotional wellbeing assessment instruments for use with Indigenous Australians: A critical review

RATIONALE

There is growing recognition that in addition to universally recognised domains and indicators of wellbeing (such as population health and life expectancy), additional frameworks are required to fully explain and measure Indigenous wellbeing. In particular, Indigenous Australian wellbeing is largely determined by colonisation, historical trauma, grief, loss, and ongoing social marginalisation. Dominant mainstream indicators of wellbeing based on the biomedical model may therefore be inadequate and not entirely relevant in the Indigenous context. It is possible that "standard" wellbeing instruments fail to adequately assess indicators of health and wellbeing within societies that have a more holistic view of health.

OBJECTIVE

The aim of this critical review was to identify, document, and evaluate the use of social and emotional wellbeing measures within the Australian Indigenous community.

METHOD

The instruments were systematically described regarding their intrinsic properties (e.g., generic v. disease-specific, domains assessed, extent of cross-cultural adaptation and psychometric characteristics) and their purpose of utilisation in studies (e.g., study setting, intervention, clinical purpose or survey). We included 33 studies, in which 22 distinct instruments were used.

RESULTS

Three major categories of social and emotional wellbeing instruments were identified: unmodified standard instruments (10), cross-culturally adapted standard instruments (6), and Indigenous developed measures (6). Recommendations are made for researchers and practitioners who assess social and emotional wellbeing in Indigenous Australians, which may also be applicable to other minority groups where a more holistic framework of wellbeing is applied.

CONCLUSION

It is advised that standard instruments only be used if they have been subject to a formal cross-cultural adaptation process, and Indigenous developed measures continue to be developed, refined, and validated within a diverse range of research and clinical settings.

Interplay wellbeing framework: a collaborative methodology 'bringing together stories and numbers' to quantify Aboriginal cultural values in remote Australia

BACKGROUND

Wellbeing has been difficult to understand, measure and strengthen for Aboriginal people in remote Australia. Part of the challenge has been genuinely involving community members and incorporating their values and priorities into assessment and policy. Taking a 'shared space' collaborative approach between remote Aboriginal communities, governments and scientists, we merged Aboriginal knowledge with western science - by bringing together stories and numbers. This research aims to statistically validate the holistic Interplay Wellbeing Framework and Survey that bring together Aboriginal-identified priorities of culture, empowerment and community with government priorities including education, employment and health.

METHOD

Quantitative survey data were collected from a cohort of 842 Aboriginal people aged 15-34 years, recruited from four different Aboriginal communities in remote Australia. Aboriginal community researchers designed and administered the survey.

RESULTS

Structural equation modeling showed good fit statistics (χ/df = 2.69, CFI = 0.95 and RMSEA = 0.045) confirming the holistic nature of the Interplay Wellbeing Framework. The strongest direct impacts on wellbeing were 'social and emotional wellbeing' (r = 0.23; p < 0.001), 'English literacy and numeracy' (r = 0.15; p < 0.001), 'Aboriginal literacy' (r = 0.14; p < 0.001), 'substances' (lack thereof; r = 0.13; p = 0.003), 'work' (r = 0.12; p = 0.02) and 'community' (r = 0.08; p = 0.05). Correlation analyses suggested cultural factors have indirect impacts on wellbeing, such as through Aboriginal literacy. All cultural variables correlated highly with each other, and with empowerment and community. Empowerment also correlated highly with all education and work variables. 'Substances' (lack thereof) was linked with positive outcomes across culture, education and work. Specific interrelationships will be explored in detail separately.

CONCLUSION

The Interplay Wellbeing Framework and Survey were statistically validated as a collaborative approach to assessing wellbeing that is inclusive of other cultural worldviews, values and practices. New community-derived social and cultural indicators were established, contributing valuable insight to psychometric assessment across cultures. These analyses confirm that culture, empowerment and community play key roles in the interplay with education, employment and health, as part of a holistic and quantifiable system of wellbeing. This research supports the holistic concept of wellbeing confirming that everything is interrelated and needs to be considered at the 'whole of system' level in policy approaches.

Mental health problems in a regional population of Australian adolescents: association with socio-demographic characteristics

BACKGROUND

Population level data regarding the general mental health status, and the socio-demographic factors associated with the mental health status of adolescents in Australia aged 12-16 years is limited. This study assessed prevalence of mental health problems in a regional population of Australian students in Grades 7-10, and investigated associations between mental health problems and socio-demographic factors.

METHODS

A web-based survey was conducted in 21 secondary schools located in disadvantaged local government areas in one regional local health district of NSW Australia. Mental health problems were measured using the youth self-report Strengths and Difficulties Questionnaire (SDQ) total SDQ score and three subscale scores (internalising problems, externalising problems and prosocial behaviour). Associations between each SDQ outcome and student socio-demographic characteristics (age, gender, Aboriginal and/or Torres Strait Islander Status, remoteness of residential location and socio-economic disadvantage) were investigated.

RESULTS

Data are reported for 6793 students aged 12-16 years. Nineteen percent of participants scored in the 'very high' range for the total SDQ, 18.0 % for internalising problems, 11.3 % for externalising problems and 8.9 % for prosocial behaviour problems. Gender and Aboriginal status were associated with all four SDQ outcomes, while age was associated with two, excluding externalising problems and prosocial behaviour. Aboriginal adolescents scored higher for mental health problems than non-Aboriginal adolescents for all four SDQ outcomes. Females scored higher than males for total SDQ and internalising problems, with mean difference greatest at age 15. Males scored higher for externalising problems and lower for prosocial behaviour than females.

CONCLUSIONS

The finding that mental health problems significantly varied by age, gender and Aboriginality may suggest a need for tailored interventions for groups of adolescents with highest levels of mental health problems. Trial Registration ANZCTR ACTRN12611000606987. Registered 14/06/2011.