Measuring Community-Engaged Research Contexts, Processes, and Outcomes: A Mapping Review

Future Directions for Community-Engaged Research in Clinical Psychological Science with Youth

Despite advances in clinical science, the burden of mental health problems among youth is not improving. To tackle this burden, clinical science with youth needs methods that include youth and family perspectives on context and public health. In this paper, we illustrate how community-engaged research (CEnR) methods center these perspectives. Although CEnR methods are well-established in other disciplines (e.g. social work, community psychology), they are underutilized in clinical science with youth. This is due in part to misperceptions of CEnR as resource-intensive, overly contextualized, incompatible with experimentally controlled modes of inquiry, or irrelevant to understanding youth mental health. By contrast, CEnR methods can provide real-world impact, contextualized clinical solutions, and sustainable outcomes. A key advantage of CEnR strategies is their flexibility-they fall across a continuum that centers community engagement as a core principle, and thus can be infused in a variety of research efforts, even those that center experimental control (e.g. randomized controlled trials). This paper provides a brief overview of this continuum of strategies and its application to youth-focused clinical science. We then discuss future directions of CEnR in clinical science with youth, as well as structural changes needed to advance this work. The goals of this paper are to help demystify CEnR and encourage clinical scientists to consider adopting methods that better consider context and intentionally engage the communities that our work seeks to serve.

Using Participatory Implementation Science to Advance Health Equity

Participatory approaches to implementation science (IS) offer an inclusive, collaborative, and iterative perspective on implementing and sustaining evidence-based interventions (EBIs) to advance health equity. This review provides guidance on the principles and practice of participatory IS, which enables academic researchers, community members, implementers, and other actors to collaboratively integrate practice-, community-, and research-based evidence into public health and health care services. With a foundational focus on supporting academics in coproducing knowledge and action, participatory IS seeks to improve health, reduce inequity, and create transformational change. The three main sections of this review provide (a) a rationale for participatory approaches to research in implementation science, (b) a framework for integrating participatory approaches in research utilizing IS theory and methods, and (c) critical considerations for optimizing the practice and impact of participatory IS. Ultimately, participatory approaches can move IS activities beyond efforts to make EBIs work within harmful systems toward transformative solutions that reshape these systems to center equity.

Validation of the Measurement Approaches to Partnership Success (MAPS) Questionnaire

Conceptualizing and testing factors that contribute to the success of community-academic partnerships are critical to understanding their contributions to the health and well-being of communities. Most measures to date focus on factors that contribute to the development of new partnerships, and only a few have been adequately tested and validated. Methods. The Measurement Approaches to Partnership Success (MAPS) study followed a community-based participatory research (CBPR) approach and a multiphase process that included the construction and pilot testing of a questionnaire, and a national survey to validate the psychometric properties of the questionnaire in long-standing CBPR partnerships (existing ≥ six years). All members within partnerships were recruited to complete the survey (55 partnerships with 563 partners). We used confirmatory factor analysis (CFA), Cronbach's alpha statistics, and a pairwise correlations approach to assess discriminant and convergent validity, and assessed internal consistency, and test-retest reliability. Results. All MAPS Questionnaire dimensions demonstrated strong validity and reliability and demonstrated agreement over time. Conclusion. The MAPS Questionnaire includes seven dimensions and 81 items related to the MAPS conceptual model and provides a scientific, in-depth measurement tool that allows long-standing CBPR partnerships to evaluate their work toward achieving health equity.

Diversity, equity, and inclusivity in observational ambulatory assessment: Recommendations from two decades of Electronically Activated Recorder (EAR) research

Ambient audio sampling methods such as the Electronically Activated Recorder (EAR) have become increasingly prominent in clinical and social sciences research. These methods record snippets of naturalistically assessed audio from participants' daily lives, enabling novel observational research about the daily social interactions, identities, environments, behaviors, and speech of populations of interest. In practice, these scientific opportunities are equaled by methodological challenges: researchers' own cultural backgrounds and identities can easily and unknowingly permeate the collection, coding, analysis, and interpretation of social data from daily life. Ambient audio sampling poses unique and significant challenges to cultural humility, diversity, equity, and inclusivity (DEI) in scientific research that require systematized attention. Motivated by this observation, an international consortium of 21 researchers who have used ambient audio sampling methodologies created a workgroup with the aim of improving upon existing published guidelines. We pooled formally and informally documented challenges pertaining to DEI in ambient audio sampling from our collective experience on 40+ studies (most of which used the EAR app) in clinical and healthy populations ranging from children to older adults. This article presents our resultant recommendations and argues for the incorporation of community-engaged research methods in observational ambulatory assessment designs looking forward. We provide concrete recommendations across each stage typical of an ambient audio sampling study (recruiting and enrolling participants, developing coding systems, training coders, handling multi-linguistic participants, data analysis and interpretation, and dissemination of results) as well as guiding questions that can be used to adapt these recommendations to project-specific constraints and needs.

Top Tips for Designing and Managing a Public Engagement Laboratory

Public engagement with science is a core facet of the broader science ecosystem, in particular the science research and science education sectors. In this article we demarcate the benefits of dedicated laboratories along with practitioner advice pertaining to the design and running of a public engagement learning environment. A practicing public engagement laboratory and one that is currently being developed are used as illustrative cases to provide real-world insights to public engagement practitioners.

Adaptations to a statewide walking program: Use of iterative feedback cycles between research and delivery systems improves fit for over 10 years

FitEx is an 8-week, group-based physical activity and fruit and vegetable consumption program co-created and implemented with the Cooperative Extension System. Effectiveness and delivery personnel perceptions of the program are promising; however, ongoing adaptations are required to continuously meet shifting needs of both researchers and delivery systems. We applied the APDER iterative cycles of implementation over 15 years to understand dynamic and ongoing adaptations as well as implications for FitEx sustainability. Each year, an IRPP between delivery (FitEx deliverers) and research (FitEx developers) systems shared feedback on program core elements and strategies for adaptation through regular team meetings, emails, and evaluations. While the core elements (delivering to groups, goal setting, feedback, and self-monitoring) of FitEx remained consistent, changes were made to address logistical factors, emergent research questions, and technological advancements. For example, program deliverers suggested decreasing training time and making program content available on demand rather than through traditional in-person training. Using APDER with a long-standing IRPP allowed the delivery system to provide feedback to program developers to co-create ongoing adaptations and data-driven decisions. Future work in response to shifting needs includes Fitbit integration and technological updates to the usability of the FitEx platform. Our aim is to report the 15+ years of applying the Assess, Plan, Do, Evaluate, Report (APDER) process with an integrated research-practice partnership (IRPP) for co-creation of ongoing adaptations of FitEx and to share methods for capturing relevant data for decision-making to integrate health promotion programs in community settings.

Assessing Nurse Competency in the Veterans Health Administration Registered Nurse Transition-to-Practice Residency Program: Item Pool Content Validation

BACKGROUND

New graduate registered nurse (RN) competencies are complex and difficult to measure. Psychometrically sound tools are needed to evaluate competencies for nurses participating in nurse residencies.

PURPOSE

Project aims were to develop an item pool for a tool to measure new graduate RN competencies for the Veterans Health Administration RN Transition-to-Practice Residency Program; validate item pool content; and use consensus methods to improve item pool content validity.

METHODS

A sequential, mixed-methods design was used. Item pool creation, content validation, and revisions included a collaborative process with the evaluation team, operational stakeholders, and subject matter experts (SMEs).

RESULTS

Inclusion of SMEs in item development enhanced item pool content validity to measure nurse competency. Stakeholder feedback ensured programmatic logistical and evaluation concerns were met.

CONCLUSIONS

Engaging SMEs in conceptualization, item development, and aligning existing standards enhanced item pool content validity to measure nurse competencies for new graduate RNs.

Synthesizing Best Practices to Promote Health Equity for Older Adults Through Community-Engaged Research

Achieving health equity requires creating evidence that reflects the nuance and diversity of experiences among populations disproportionately impacted by age- and race-related disparities. Community-engaged research (CEnR) is one way to pursue equity in research on health and aging to ensure the relevance and translational potential of findings. The current review synthesizes best practices regarding CEnR that promote health equity among older adults, including an overview of CEnR, benefits, and fundamental principles, and three research exemplars from the authors' CEnR. Finally, we discuss these best practices and considerations for advancing CEnR to reduce health disparities experienced by historically underserved older adults and their families. [Research in Gerontological Nursing, 17(1), 9-16.].

Effectiveness of Community-Engaged Research Strategies in Increasing COVID-19 Vaccinations in Targeted Communities: The Community Engagement Alliance (CEAL) Regional Teams

Objectives. To explore a key outcome of interest for the Community Engagement Alliance (CEAL) Regional Teams by examining COVID-19 vaccinations over time in US counties where CEAL teams operated and comparing them to demographically similar counties in the same state. Methods. Our evaluation used a nonequivalent control group design. Each county where a CEAL team operated was matched to a unique non-CEAL county in the same state. Components of the Centers for Disease Control and Prevention's Social Vulnerability Index were used as the matching criteria. COVID-19 vaccinations (county-level percentage of residents aged 18 years or older who are fully vaccinated) for CEAL and matched counties were analyzed over time. Results. The mean percentage of vaccinated adults was significantly higher in CEAL counties than in matched non-CEAL counties. Sensitivity analyses confirmed conclusions did not change depending on the CEAL cohort or closeness of matches. Conclusions. Our findings support CEAL team efforts to increase COVID-19 vaccinations in target communities and employ community-engaged research more broadly within public health contexts. (Am J Public Health. 2024;114(S1):S124-S127. https://doi.org/10.2105/AJPH.2023.307517).

A scoping review of the globally available tools for assessing health research partnership outcomes and impacts

BACKGROUND

Health research partnership approaches have grown in popularity over the past decade, but the systematic evaluation of their outcomes and impacts has not kept equal pace. Identifying partnership assessment tools and key partnership characteristics is needed to advance partnerships, partnership measurement, and the assessment of their outcomes and impacts through systematic study.

OBJECTIVE

To locate and identify globally available tools for assessing the outcomes and impacts of health research partnerships.

METHODS

We searched four electronic databases (Ovid MEDLINE, Embase, CINAHL + , PsychINFO) with an a priori strategy from inception to June 2021, without limits. We screened studies independently and in duplicate, keeping only those involving a health research partnership and the development, use and/or assessment of tools to evaluate partnership outcomes and impacts. Reviewer disagreements were resolved by consensus. Study, tool and partnership characteristics, and emerging research questions, gaps and key recommendations were synthesized using descriptive statistics and thematic analysis.

RESULTS

We screened 36 027 de-duplicated citations, reviewed 2784 papers in full text, and kept 166 studies and three companion reports. Most studies originated in North America and were published in English after 2015. Most of the 205 tools we identified were questionnaires and surveys targeting researchers, patients and public/community members. While tools were comprehensive and usable, most were designed for single use and lacked validity or reliability evidence. Challenges associated with the interchange and definition of terms (i.e., outcomes, impacts, tool type) were common and may obscure partnership measurement and comparison. Very few of the tools identified in this study overlapped with tools identified by other, similar reviews. Partnership tool development, refinement and evaluation, including tool measurement and optimization, are key areas for future tools-related research.

CONCLUSION

This large scoping review identified numerous, single-use tools that require further development and testing to improve their psychometric and scientific qualities. The review also confirmed that the health partnership research domain and its measurement tools are still nascent and actively evolving. Dedicated efforts and resources are required to better understand health research partnerships, partnership optimization and partnership measurement and evaluation using valid, reliable and practical tools that meet partners' needs.

Accelerating knowledge translation to improve cardiovascular outcomes and health services: opportunities for bridging science and clinical practice

Knowledge translation (KT) is the exchange between knowledge producers and users to understand, synthesize, share, and apply evidence to accelerate the benefits of research to improve health and health systems. Knowledge translation practice (activities/strategies to move evidence into practice) and KT science (study of the methodology and approaches to promote the uptake of research) benefit from the use of conceptual thinking, the meaningful inclusion of patients, and the application of intersectionality. In spite of multiple barriers, there are opportunities to develop strong partnerships and evidence to drive an impactful research agenda and increase the uptake of cardiovascular research.

Exploring Online Crowdfunding for Cancer-Related Costs Among LGBTQ+ (Lesbian, Gay, Bisexual, Transgender, Queer, Plus) Cancer Survivors: Integration of Community-Engaged and Technology-Based Methodologies

BACKGROUND

Cancer survivors frequently experience cancer-related financial burdens. The extent to which Lesbian, Gay, Bisexual, Transgender, Queer, Plus (LGBTQ+) populations experience cancer-related cost-coping behaviors such as crowdfunding is largely unknown, owing to a lack of sexual orientation and gender identity data collection and social stigma. Web-scraping has previously been used to evaluate inequities in online crowdfunding, but these methods alone do not adequately engage populations facing inequities.

OBJECTIVE

We describe the methodological process of integrating technology-based and community-engaged methods to explore the financial burden of cancer among LGBTQ+ individuals via online crowdfunding.

METHODS

To center the LGBTQ+ community, we followed community engagement guidelines by forming a study advisory board (SAB) of LGBTQ+ cancer survivors, caregivers, and professionals who were involved in every step of the research. SAB member engagement was tracked through quarterly SAB meeting attendance and an engagement survey. We then used web-scraping methods to extract a data set of online crowdfunding campaigns. The study team followed an integrated technology-based and community-engaged process to develop and refine term dictionaries for analyses. Term dictionaries were developed and refined in order to identify crowdfunding campaigns that were cancer- and LGBTQ+-related.

RESULTS

Advisory board engagement was high according to metrics of meeting attendance, meeting participation, and anonymous board feedback. In collaboration with the SAB, the term dictionaries were iteratively edited and refined. The LGBTQ+ term dictionary was developed by the study team, while the cancer term dictionary was refined from an existing dictionary. The advisory board and analytic team members manually coded against the term dictionary and performed quality checks until high confidence in correct classification was achieved using pairwise agreement. Through each phase of manual coding and quality checks, the advisory board identified more misclassified campaigns than the analytic team alone. When refining the LGBTQ+ term dictionary, the analytic team identified 11.8% misclassification while the SAB identified 20.7% misclassification. Once each term dictionary was finalized, the LGBTQ+ term dictionary resulted in a 95% pairwise agreement, while the cancer term dictionary resulted in an 89.2% pairwise agreement.

CONCLUSIONS

The classification tools developed by integrating community-engaged and technology-based methods were more accurate because of the equity-based approach of centering LGBTQ+ voices and their lived experiences. This exemplar suggests integrating community-engaged and technology-based methods to study inequities is highly feasible and has applications beyond LGBTQ+ financial burden research.

Team-focused implementation strategies to improve implementation of mental health screening and referral in rural Children's Advocacy Centers: study protocol for a pilot cluster randomized hybrid type 2 trial

BACKGROUND

Children's Advocacy Centers (CACs) use multidisciplinary teams to investigate and respond to maltreatment allegations. CACs play a critical role in connecting children with mental health needs to evidence-based mental health treatment, especially in low-resourced rural areas. Standardized mental health screening and referral protocols can improve CACs' capacity to identify children with mental health needs and encourage treatment engagement. In the team-based context of CACs, teamwork quality is likely to influence implementation processes and outcomes. Implementation strategies that target teams and apply the science of team effectiveness may enhance implementation outcomes in team-based settings.

METHODS

We will use Implementation Mapping to develop team-focused implementation strategies to support the implementation of the Care Process Model for Pediatric Traumatic Stress (CPM-PTS), a standardized screening and referral protocol. Team-focused strategies will integrate activities from effective team development interventions. We will pilot team-focused implementation in a cluster-randomized hybrid type 2 effectiveness-implementation trial. Four rural CACs will implement the CPM-PTS after being randomized to either team-focused implementation (n = 2 CACs) or standard implementation (n = 2 CACs). We will assess the feasibility of team-focused implementation and explore between-group differences in hypothesized team-level mechanisms of change and implementation outcomes (implementation aim). We will use a within-group pre-post design to test the effectiveness of the CPM-PTS in increasing caregivers' understanding of their child's mental health needs and caregivers' intentions to initiate mental health services (effectiveness aim).

CONCLUSIONS

Targeting multidisciplinary teams is an innovative approach to improving implementation outcomes. This study will be one of the first to test team-focused implementation strategies that integrate effective team development interventions. Results will inform efforts to implement evidence-based practices in team-based service settings.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT05679154 . Registered on January 10, 2023.

Coping Strategies as a Moderator for the Association between Intimate Partner Violence and Depression and Anxiety Symptoms among Transgender Women

Evidence suggests that intimate partner violence (IPV) is associated with negative mental health outcomes. There is currently limited research on the impact of IPV on the outcomes of mental health for transgender women. The current study aimed to examine the relationship between intimate partner violence, coping skills, depression, and anxiety in a sample of transgender women. Hierarchical regression analyses were conducted examining the relationship of IPV and depression and anxiety symptoms, where coping skills moderate this relationship. The results suggest that those with experiences of IPV are more likely to have symptoms of depression and anxiety. For individuals with no experiences of IPV and low depression, high levels of emotional processing coping and acceptance coping buffered this relationship. For individuals with more experiences of IPV and more depressive symptoms, coping skills did not show to buffer this relationship. These same coping skills did not show evidence for buffering anxiety symptoms for transgender women with low or high levels of IPV. The results, implications, and limitations of this study and suggestions for further research are discussed.

Community engagement processes in low- and middle-income countries health research settings: a systematic review of the literature

BACKGROUND

Community Engagement is an important ethical imperative in research. Although substantial research emphasizes its real value and strategic importance, much of the available literature focuses primarily on the success of community participation, with little emphasis given to specific community engagement processes, mechanisms and strategies in relation to intended outcomes in research environments. The systematic literature review's objective was to explore the nature of community engagement processes, strategies and approaches in health research settings in low- and middle-income countries.

METHODS

The systematic literature review design was informed by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. We searched for peer-reviewed, English-language literature published between January 2011 and December 2021 through three databases on the internet (PubMed, Web of Science and Google Scholar). The terms "community engagement," "community involvement," "participation," "research settings," and "low- and middle-income countries" were merged in the search.

RESULTS

The majority of publications [8/10] were led by authors from low- and middle-income countries, with many of them, [9/10] failing to continuously include important aspects of study quality. Even though consultation and information sessions were less participatory, articles were most likely to describe community engagement in these types of events. The articles covered a wide range of health issues, but the majority were concerned with infectious diseases such as malaria, human immunodeficiency virus, and tuberculosis, followed by studies on the environment and broader health factors. Articles were largely under-theorized.

CONCLUSIONS

Despite the lack of theoretical underpinnings for various community engagement processes, strategies and approaches, community engagement in research settings was variable. Future studies should go deeper into community engagement theory, acknowledge the power dynamics underpin community engagement, and be more practical about the extent to which communities may participate.

Evaluation of a stakeholder advisory board for an adolescent mental health randomized clinical trial

INTRODUCTION

Community engagement in research is widely accepted as best practice, despite gaps in existing frameworks to evaluate its process, context, and impact on research. The Screening in High Schools to Identify, Evaluate, and Lower Depression (SHIELD) study evaluated the use of a school-based major depressive disorder screening tool in the identification of symptoms and treatment initiation among adolescents, and was developed, implemented, and disseminated in partnership with a Stakeholder Advisory Board (SAB). We summarize outcomes of the evaluation strategy applied through our partnership with the SAB and explore gaps in the available engagement evaluation tools for mixed stakeholder populations including youth.

METHODS

SHIELD study SAB members (n = 13; adolescents, parents, mental health and primary care providers, and professionals from education and mental health organizations) advised on study design, implementation, and dissemination over a three-year period. Both SAB members and study team members (i.e., clinician researchers, project managers) were invited to quantitatively and qualitatively evaluate stakeholder engagement after each project year. At the conclusion of the study, SAB members and study team members were asked to evaluate the application of engagement principles in overall stakeholder engagement across the study period, using portions of the Research Engagement Survey Tool (REST).

RESULTS

SAB members and study team members responded similarly when evaluating engagement process (i.e., valued on team, voice represented); means ranged from 3.9 to 4.8 out of 5 points across all three project years. Reported engagement within study-specific engagement activities (i.e., meetings, study newsletter) varied from year to year, with some discrepancy between SAB member and study team evaluations. Using REST, SAB members reported the alignment of their experience with key engagement principles the same or higher than study team members. Qualitative feedback at the conclusion of the study generally matched quantitative measures; adolescent SAB members, however, reported disengagement from stakeholder activities that was not accurately or effectively captured in evaluation strategies employed across the study period.

CONCLUSIONS

Challenges exist in effectively engaging stakeholders and evaluating their engagement, particularly among heterogenous groups that include youth. Evaluation gaps should be addressed through the development of validated instruments that quantify the process, context, and impact of stakeholder engagement on study outcomes. Consideration should be given to collecting parallel feedback from stakeholders and study team members to fully understand the application and execution of engagement strategy.

Challenges to Engaging Women Veterans in Quality Improvement From Patient Care to Policy: Women's Health Managers' Perspectives

INTRODUCTION

Patients are uniquely positioned to identify issues and to provide innovative solutions to problems impacting their care. Yet, patient engagement in quality improvement (QI) and health care governance remains limited and underexplored. In the Veterans Health Administration, the work of women's health managers (WHMs) includes engaging women veterans, a numerical minority with unique health care needs, in QI. We aimed to understand the extent to which WHMs engage women veterans along a continuum, highlight challenges to engagement, and identify potential strategies to facilitate multilevel patient engagement.

METHODS

Data were generated from a multisite evaluation to improve delivery of comprehensive women's health care in Veterans Health Administration primary care sites. We conducted 39 semistructured interviews with WHMs across 21 sites. Guided by Carman et al.'s patient engagement framework, we analyzed the interviews using rapid-qualitative and content analysis methods.

RESULTS

When effectively engaged, women veterans were important champions and partners in QI activities to improve the structure and delivery of care. However, most WHMs engaged women veterans in mainly informal or passive ways-that is, solicited feedback through comment cards, surveys, focus groups, and townhall meetings-and did not report pursuing more in-depth or long-term forms of engagement. WHMs also identified a variety of facilitators and challenges to engaging women veterans in QI.

CONCLUSIONS

There may be unanticipated benefits to health care policy from engaging patients in QI, especially for patients with unique health care needs who represent a minority within the health care system. However, managers require training and workflow integration of patient engagement tasks to increase their efficiency and allow for meaningful patient engagement.

Tools for assessing health research partnership outcomes and impacts: a systematic review

OBJECTIVE

To identify and assess the globally available valid, reliable and acceptable tools for assessing health research partnership outcomes and impacts.

METHODS

We searched Ovid MEDLINE, Embase, CINAHL Plus and PsycINFO from origin to 2 June 2021, without limits, using an a priori strategy and registered protocol. We screened citations independently and in duplicate, resolving discrepancies by consensus and retaining studies involving health research partnerships, the development, use and/or assessment of tools to evaluate partnership outcomes and impacts, and reporting empirical psychometric evidence. Study, tool, psychometric and pragmatic characteristics were abstracted using a hybrid approach, then synthesized using descriptive statistics and thematic analysis. Study quality was assessed using the quality of survey studies in psychology (Q-SSP) checklist.

RESULTS

From 56 123 total citations, we screened 36 027 citations, assessed 2784 full-text papers, abstracted data from 48 studies and one companion report, and identified 58 tools. Most tools comprised surveys, questionnaires and scales. Studies used cross-sectional or mixed-method/embedded survey designs and employed quantitative and mixed methods. Both studies and tools were conceptually well grounded, focusing mainly on outcomes, then process, and less frequently on impact measurement. Multiple forms of empirical validity and reliability evidence was present for most tools; however, psychometric characteristics were inconsistently assessed and reported. We identified a subset of studies (22) and accompanying tools distinguished by their empirical psychometric, pragmatic and study quality characteristics. While our review demonstrated psychometric and pragmatic improvements over previous reviews, challenges related to health research partnership assessment and the nascency of partnership science persist.

CONCLUSION

This systematic review identified multiple tools demonstrating empirical psychometric evidence, pragmatic strength and moderate study quality. Increased attention to psychometric and pragmatic requirements in tool development, testing and reporting is key to advancing health research partnership assessment and partnership science. PROSPERO CRD42021137932.

Toward Meaningful Cultural Adaptation Across Implementation Stages: Lessons Learned From a Culturally Based HIV Stigma Intervention in Gaborone, Botswana

A culturally based stigma intervention for pregnant women living with HIV in Gaborone, Botswana highlights the importance of conceptualizing and formalizing cultural adaptation across all stages of implementation.

How are health research partnerships assessed? A systematic review of outcomes, impacts, terminology and the use of theories, models and frameworks

Background

Accurate, consistent assessment of outcomes and impacts is challenging in the health research partnerships domain. Increased focus on tool quality, including conceptual, psychometric and pragmatic characteristics, could improve the quantification, measurement and reporting partnership outcomes and impacts. This cascading review was undertaken as part of a coordinated, multicentre effort to identify, synthesize and assess a vast body of health research partnership literature.

Objective

To systematically assess the outcomes and impacts of health research partnerships, relevant terminology and the type/use of theories, models and frameworks (TMF) arising from studies using partnership assessment tools with known conceptual, psychometric and pragmatic characteristics.

Methods

Four electronic databases were searched (MEDLINE, Embase, CINAHL Plus and PsycINFO) from inception to 2 June 2021. We retained studies containing partnership evaluation tools with (1) conceptual foundations (reference to TMF), (2) empirical, quantitative psychometric evidence (evidence of validity and reliability, at minimum) and (3) one or more pragmatic characteristics. Outcomes, impacts, terminology, definitions and TMF type/use were abstracted verbatim from eligible studies using a hybrid (independent abstraction–validation) approach and synthesized using summary statistics (quantitative), inductive thematic analysis and deductive categories (qualitative). Methodological quality was assessed using the Quality Assessment Tool for Studies with Diverse Designs (QATSDD).

Results

Application of inclusion criteria yielded 37 eligible studies. Study quality scores were high (mean 80%, standard deviation 0.11%) but revealed needed improvements (i.e. methodological, reporting, user involvement in research design). Only 14 (38%) studies reported 48 partnership outcomes and 55 impacts; most were positive effects (43, 90% and 47, 89%, respectively). Most outcomes were positive personal, functional, structural and contextual effects; most impacts were personal, functional and contextual in nature. Most terms described outcomes (39, 89%), and 30 of 44 outcomes/impacts terms were unique, but few were explicitly defined (9, 20%). Terms were complex and mixed on one or more dimensions (e.g. type, temporality, stage, perspective). Most studies made explicit use of study-related TMF (34, 92%). There were 138 unique TMF sources, and these informed tool construct type/choice and hypothesis testing in almost all cases (36, 97%).

Conclusion

This study synthesized partnership outcomes and impacts, deconstructed term complexities and evolved our understanding of TMF use in tool development, testing and refinement studies. Renewed attention to basic concepts is necessary to advance partnership measurement and research innovation in the field.

Systematic review protocol registration: PROSPERO protocol registration: CRD42021137932 https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=137932.

Building capacity in quantitative research and data storytelling to enhance knowledge translation: a training curriculum for peer researchers

BACKGROUND

Many community-based HIV research studies incorporate principles of greater involvement and meaningful engagement of people living with HIV (GIPA/MEPA) by training people with HIV as peer researchers. Unfortunately, there are still some aspects of research (e.g., quantitative data analysis and interpretation) where many projects fall short in realizing GIPA/MEPA principles. To address these gaps, we developed an eight-week training course that aimed to build the capacity of peer researchers around the understanding and interpretation of quantitative data and incorporating lived experience to increase the impact of the knowledge transfer and exchange phase of a study.

METHODS

Peer researchers (n = 8) participated from British Columbia, Alberta, and Ontario and lessons learned from the training were implemented throughout the dissemination of research findings from the People Living with HIV Stigma Index study. This paper presents the curriculum and main training components, course evaluation results, and challenges and lessons learned. The manuscript was created in collaboration with and includes the perspectives of both the peer researchers involved in the training, as well the course facilitators.

RESULTS

Throughout the course, peer researchers' self-assessed knowledge and understanding of quantitative research and data storytelling improved and, through interactive activities and practice, they gained the confidence to deliver a full research presentation. This improved their understanding of research findings, which was beneficial for discussing results with community partners and study participants. The peer researchers also agreed that learning about integrating lived experience with quantitative data has helped them to make research findings more relatable and convey key messages in a more meaningful way.

CONCLUSIONS

Our training curriculum provides a template for research teams to build capacity in areas of research where peer researchers and community members are less often engaged. In doing so, we continue to uphold the principles of GIPA/MEPA and enhance the translation of research knowledge in communities most greatly affected.

Promoting community stakeholder engagement in research on treatment for pregnant women with opioid use disorder

Aim: Community stakeholder engagement in research (CSER) can improve research relevance and efficiency as well as prevent harmful practices, particularly for vulnerable populations. Despite potential benefits, researchers lack familiarity with CSER methods. Methods: We describe CSER strategies used across the research continuum, including proposal development, study planning and the first years of a comparative effectiveness study of care for pregnant women with opioid use disorder. Results: We highlight successful strategies, grounded in principles of engagement, to establish and maintain stakeholder relationships, foster bidirectional communication and trust and support active participation of women with opioid use disorder in the research process. Conclusion: CSER methods support research with a disenfranchised population. Future work will evaluate the impact of CSER strategies on study outcomes and dissemination.

Feasibility of a best-worst scaling exercise to set priorities for autism research

Background

The preferences of autism stakeholders regarding the top priorities for future autism research are largely unknown.

Objective

This study had two objectives: First, to examine what autism stakeholders think new research investments should be and the attributes of investment that they consider important, and second, to explore the feasibility, acceptability and outcomes of two prioritization exercises among autism stakeholders regarding their priorities for future research in autism.

Design

This was  a prospective stakeholder‐engaged iterative study consisting of best–worst scaling (BWS) and direct prioritization exercise.

Setting and Participants

A national snowball sample of 219 stakeholders was included: adults with autism, caregivers, service providers and researchers.

Main Outcome Measures

The main outcomes measures were attributes that participants value in future research investments, and priority research investments for future research.

Results

Two hundred and nineteen participants completed the exercises, of whom 11% were adults with autism, 58% were parents/family members, 37% were service providers and 21% were researchers. Among stakeholders, the BWS exercises were easier to understand than the direct prioritization, less frequently skipped and yielded more consistent results. The proportion of children with autism affected by the research was the most important attribute for all types of stakeholders. The top three priorities among future research investments were (1) evidence on which child, family and intervention characteristics lead to the best/worst outcomes; (2) evidence on how changes in one area of a child's life are related to changes in other areas; and (3) evidence on dietary interventions. Priorities were similar for all stakeholder types.

Conclusions

The values and priorities examined here provide a road map for investigators and funders to pursue autism research that matters to stakeholders.

Patient or Public Contribution

Stakeholders completed a BWS and direct prioritization exercise to inform us about their priorities for future autism research.

Veteran Engagement in Health Services Research: a Conceptual Model

With 20 million living veterans and millions more immediate family members, and approximately 9 million veterans enrolled in the nationally networked VA healthcare system, representing the interests and needs of veterans in this complex community is a substantial endeavor. Based on the importance of engaging Veterans in research, the VA Health Services Research and Development (HSR&D) Service convened a Working Group of VA researchers and Veterans to conduct a review of patient engagement models and develop recommendations for an approach to engage Veterans in health research that would incorporate their unique lived experiences and interests, and their perspectives on research priorities. The Working Group considered the specific context for Veteran engagement in research that includes other VA stakeholders from the operational and clinical leadership of the VA Health Administration (VHA). The resulting model identifies the range of potential stakeholders and three domains of relevant constructs-processes expected to facilitate Veteran engagement in research with other stakeholders, individual stakeholder and external factors, and outcomes. The expectation is that Veteran engagement will benefit research to policy and practice translation, including increasing the transparency of research and producing knowledge that is readily accepted and implemented in healthcare.

Evaluating the Impacts of Patient Engagement on Health Services Research Teams: Lessons from the Veteran Consulting Network

BACKGROUND

Despite increasing commitment to patient engagement in research, evaluation of the impact of these efforts on research processes, products, and teams is limited.

OBJECTIVE

To explore the impacts of engaging patients as consultants to research studies by examining the experiences, impacts, and lessons learned from a program facilitating patient engagement at a Veterans Health Administration research center.

DESIGN

We developed a logic model to articulate the activities being implemented to support patient engagement and their anticipated outcomes. Then, we conducted qualitative, semi-structured interviews with participants in the local Veteran Consulting Network to qualitatively explore these outcomes.

PARTICIPANTS

Twelve researchers and eleven Veteran patients with experience working on at least one grant or funded study.

APPROACH

Interview transcripts were inductively coded using a consensus-based approach. Findings were synthesized using framework analysis and mapped back onto our logic model of expected patient engagement impacts.

KEY RESULTS

Patient engagement improved the perceived quality and relevance of research studies as patient consultants challenged researchers' assumptions about patient populations and clinical contexts and gave feedback that helped improve the feasibility of proposed grants, readability of study materials, comprehensiveness of study assessments, and cultural sensitivity and relevance of interventions. Patient engagement also had personal benefits to researchers and patients. Researchers reported improved communication skills and higher job satisfaction. Patients reported a sense of purpose and satisfaction from their work with greater awareness of and appreciation for research.

CONCLUSIONS

Engaging patients in research can have multiple benefits to the people and work involved. Our evaluation process can serve as a template for other organizations to plan for and assess the impact of their own patient engagement programs. Creating logic models and updating them based on feedback from program users make engagement goals explicit, help verify expected mechanisms to achieve impact, and facilitate organizational learning.

Moving Food Assistance into the Digital Age: A Scoping Review

One of the most basic needs globally, food assistance refers to the multitude of programs, both governmental and non-governmental, to improve food access and consumption by food-insecure individuals and families. Despite the importance of digital and mobile Health (mHealth) strategies in food insecurity contexts, little is known about their specific use in food assistance programs. Therefore, the purpose of this study was to address that gap by conducting a scoping review of the literature. Keywords were defined within the concepts of food assistance and digital technology. The search included relevant peer-reviewed and grey literature from 2011 to 2021. Excluded articles related to agriculture and non-digital strategies. PRISMA guidelines were followed to perform a partnered, two-round scoping literature review. The final synthesis included 39 studies of which most (84.6%) were from the last five years and United States-based (93.2%). The top three types of articles or studies included text and opinion, qualitative research, and website, application, or model development (17.9%). The top three types of digital tools were websites (56.4%), smartphone applications (20.5%), and chatbots (5.1%). Nineteen digital features were identified as desirable. Most tools included just one or two features. The most popular feature to include was online shopping (n = 14), followed by inventory management, and client tracking. Digital tools for individual food assistance represent an opportunity for equitable and stable access to programs that can enhance or replace in-person services. While this review identified 39 tools, all are in early development and/or implementation stages. Review findings highlight an overall lack of these tools, an absence of user-centered design in their development, and a critical need for research on their effectiveness globally. Further analysis and testing of current digital tool usage and interventions examining the health and food security impacts of such tools should be explored in future studies, including in the context of pandemics, where digital tools allow for help from a distance.

Partnering to Increase Colorectal Cancer Screening: Perspectives of Community Advisory Board Members

The Patient-Centered Outcomes Research Institute (PCORI) defines engagement in research as the meaningful involvement of patients, caregivers, clinicians, insurers, and others throughout the entire research process-from planning to conducting the study to disseminating study results. The purposes of this paper are to (a) describe methods used to engage community members across the various phases of a PCORI-funded comparative effectiveness trial to increase colorectal cancer screening; and (b) report results of qualitative and quantitative evaluations of community advisory board members' experiences on this project. Decisions to join and stay engaged with the study included feeling valued and appreciated, being compensated, the opportunity to contribute to research based on their skills and expertise, and being committed to colon cancer prevention efforts. Challenges identified by advisory board members included the significant time commitment, transportation, and meeting location. Lessons learned and guidance for researchers committed to patient and community engagement are described.

Utilizing Participatory Research to Engage Underserved Populations to Improve Health-Related Outcomes in Delaware

Cooperative Extension is a community outreach program. Despite its large reach, there is a need for the evaluation of changes in health-related outcomes for individuals engaged with Cooperative Extension. A team-based challenge was developed using community-engaged participatory research integrated with Cooperative Extension to encourage healthy eating and physical activity behaviors through Cooperative Extension programming. Thus, the primary purpose of this secondary analysis was to (1) evaluate changes in anthropometric outcomes and (2) evaluate changes in health behavior outcomes. Associations of anthropometric changes and health behavior changes with engagement in the three-month team-based challenge were explored. Anthropometrics were measured using standard procedures, and intake of fruits and vegetables and physical activity were self-reported. Of the 145 participants in the community-engaged participatory research portion of the study, 52.4% (n = 76) had complete anthropometrics before and after the team-based challenge and were included in this study. At 3 months, there was a significant reduction in body mass index (−0.3 kg/m², p = 0.024) and no significant change in waist circumference (p = 0.781). Fruit and vegetable intake significantly increased (+0.44 servings/day, p = 0.018). Physical activity did not significantly change based on (1) the number of days 30 or more minutes of physical activity was conducted (p = 0.765) and (2) Godin Leisure-Time Exercise Questionnaire scores (p = 0.612). Changes in anthropometrics and health behaviors were not associated with engagement in the team-based challenge. Using community-engaged participatory research with community outreach programs, such as Cooperative Extension, can improve health-related outcomes in underserved populations. However, despite a participatory approach, changes in anthropometrics and health behaviors were not associated with engagement in the developed team-based challenge.

Scales of Practices and Outcomes for Community-Engaged Research

Despite the growth of research on community-engaged research (CEnR), recent reviews suggest there has been limited development of validated scales to measure key contexts, mechanisms, and outcomes, impairing testing and refinement of theoretical models. The purpose of this study is to present the psychometric properties of scales from the Engage for Equity (E2) project, stemming from a long-term research partnership examining community-engaged research projects. This study used a three-stage, cross-sectional format: (a) a sampling frame of 413 CEnR projects was identified; (b) 210 principal investigators completed a project-level survey and nominated partners for another survey; (c) 457 investigators and partners completed a survey about project contexts, processes, interventions, and outcomes. Factorial validity was established through confirmatory factor analysis supporting seven scales: contextual capacity, commitment to collective empowerment, relationships, community engagement in research actions, synergy, partner and partnership transformation, and projected outcomes. Convergent validity was established through examining covariances among the scales. This study largely yielded results consistent with a previous psychometric study of related measures, while demonstrating improved ceiling effects of the items and refined conceptualization of core theoretical constructs.

Patient engagement in fertility research: bench research, ethics, and social justice

BACKGROUND

Patient and Public Involvement (PPI) in research is increasingly being utilized to better connect patients and researchers. The Patient Engagement Studio (PES) supports PPI in research by working directly with researchers throughout various stages of their projects. Recently, two researchers presented to the PES for assistance with their project, Embryo+™. The purpose of Embryo+™ is to decrease miscarriage rates using RNA sequencing technology that screens for the most viable embryos. To date, no examples of PPI directly in the planning or implementation of bench research concerning in vitro fertilization and embryo transfer have been identified.

MAIN BODY

Embryo+™ researchers met in-person with the PES two times (fall 2019; each meeting had 9 PES members in attendance) for initial feedback and protocol development. After these meetings, PES leadership and Embryo+™ researchers decided that the unique nature of the project merited a PPI evaluation. Subsequent evaluation of engagement efforts occurred by reviewing the PES reports for the Embryo+™ researchers, conducting two recorded web-based discussion meetings with the PES (summer 2020; meeting 1 n = 7; meeting 2 n = 6), and a brief survey (n = 13). The discussion meetings provided an opportunity for the PES members to define engagement themes through consensus via verbal agreement to the studio director's periodic summaries during the discussions. Combining survey results and PES themes allowed for a broad discussion for meaningful engagement. The Embryo+™ researchers established trust with the patients by changing some of their language in response to patient suggestions, allowing for unintended ethical conversations, and implementing the patient developed protocols. Overall, the patient experts thought this project was very meaningful and valuable, quantified by a mean loyalty score 89.43 (s.d. 10.29).

CONCLUSION

Bench science researchers may need additional PPI training prior to engaging with patient groups. PPI in this project was successful in large part due to this training, where the director emphasized the importance of gaining trust with the patients. The researchers applied what they learned and several examples of how to develop trust with patients are discussed. If trust is established, PPI in an ethically charged, basic science research study can be both valuable and successful.

Using community engagement to implement evidence-based practices for opioid use disorder: A data-driven paradigm & systems science approach

Community-driven responses are essential to ensure the adoption, reach and sustainability of evidence-based practices (EBPs) to prevent new cases of opioid use disorder (OUD) and reduce fatal and non-fatal overdoses. Most organizational approaches for selecting and implementing EBPs remain top-down and individually oriented without community engagement (CE). Moreover, few CE approaches have leveraged systems science to integrate community resources, values and priorities. This paper provides a novel CE paradigm that utilizes a data-driven and systems science approach; describes the composition, functions, and roles of researchers in CE; discusses unique ethical considerations that are particularly salient to CE research; and provides a description of how systems science and data-driven approaches to CE may be employed to select a range of EBPs that collectively address community needs. Finally, we conclude with scientific recommendations for the use of CE in research. Greater investment in CE research is needed to ensure contextual, equitable, and sustainable access to EBPs, such as medications for OUD (MOUD) in communities heavily impacted by the opioid epidemic. A data-driven approach to CE research guided by systems science has the potential to ensure adequate saturation and sustainability of EBPs that could significantly reduce opioid overdose and health inequities across the US.