Forging a link between mentoring and collaboration: a new training model for implementation science

Training in eight low-and middle-income countries: lessons learned from a pilot study using the WHO-TDR dissemination and implementation massive open online course

Introduction

Non-communicable diseases (NCDs) are a leading cause of morbidity and mortality in low-and middle- income countries (LMICs). Despite this, a lack of funding, training and mentorship for NCD investigators in LMICs exists. In an effort to gain knowledge and skills to address these gaps, participants from the Global Research on Implementation and Translation Science (GRIT), a consortium of studies in eight LMICs and their networks, attended the dissemination and implementation (D&I) massive open online course (MOOC) developed by the Special Programme for Research and Training in Tropical Diseases at the World Health Organization to strengthen D&I capacity building. Here, we report on the pilot of this MOOC, which was implemented during the SARS COVID-19 pandemic from April- November 2020.

Methods

Participants completed pre-and post-training questionnaires to assess self-reported D&I competencies, general research skills, and research mentor access and quality. D&I competencies were measured by use of a scale developed for a US-based training program, with change in competency scores assessed by paired t test. We used univariate statistics to analyze the data for all other outcomes.

Results

Of the 247 participants enrolled, 32 (13%) completed all course requirements, 21 (9%) completed the pre-and post-surveys and are included in the analysis. D&I competency scores suggest improvement for those who had complete pre- and post-assessments. Trainee's average score on the full competency scale improved 1.45 points (0-5 scale) from pre- to post-test; all four subscales also showed evidence of improvements. There were small but not significant increases in competencies for grant writing, proposal/ manuscript writing and presentations from pre- to post-test assessment. 40% of trainees reported access to a research mentor and 12% reported access to a D&I specific mentor. Participants reported barriers (e.g., unstable internet access and challenges due to COVID-19) and facilitators (e.g., topical interests, collaboration with colleagues) to completing the MOOC.

Conclusions

Although COVID-19 affected program usage and completion, the MOOC was feasible. We also had signals of effectiveness, meaning among LMIC participants completing the course, there was improvement in self-report D&I competency scores. Recommendations for future D&I trainings in LMICs include (1) adding more topic specific modules (i.e., NCD research, general research skills) for scalability; (2) fostering more collaboration with participants across LMICs; and (3) establishing partnerships with D&I mentors for course participants.

Researchers' views on and practices of knowledge translation: an international survey of transfusion medicine researchers

BACKGROUND

Health research is often driven by the desire to improve the care and health of the community; however, the translation of research evidence into policy and practice is not guaranteed. Knowledge translation (KT) activities, such as dissemination and end-user engagement by researchers, are important to achieving this goal. This study examined researchers' views on and practices of KT in the field of transfusion medicine.

METHODS

An anonymous, cross-sectional survey was distributed to transfusion medicine researchers in May 2022 by emailing corresponding authors of papers in four major blood journals, emailing grant recipients, posting on social media, and through international blood operator networks. Comparative analyses were conducted for career stage, work setting, research type, and KT training.

RESULTS

The final sample included 117 researchers from 33 countries. Most participants reported that research translation was important (86%) and felt it was their responsibility (69%). Fewer than half felt they had the skills to translate their research (45%) or knew which strategies to employ (45%). When examining how research findings are shared, most reported using diffusion activities (86%), including publishing in peer-reviewed journals (74%), or presenting at academic conferences (72%). Fewer used dissemination methods (60%), such as developing educational materials (29%) or writing plain language summaries (30%). Greater use of tailored dissemination strategies was seen among researchers with KT training, whilst traditional diffusion strategies were used more by those working in an academic setting. Most participants had engaged end-users in their research (72%), primarily to consult on a research component (47%) or to involve them in the research process (45%). End-user engagement was greater among researchers with established careers, working in both academic and applied settings, and with KT training.

CONCLUSIONS

Whilst participating researchers acknowledged the importance of KT, they typically focused on traditional diffusion strategies. This is despite well-established knowledge of the limited impact of these strategies in achieving KT. Those with KT training were more likely to use tailored dissemination strategies and engage end-users in their research. This demonstrates the value of sharing knowledge from the KT field with health researchers to facilitate KT.

A Nurse-Led Implementation Science Specialist Program

ABSTRACT

A well-known challenge in health care is integrating evidence into practice. Implementation science (IS) is a growing field that promotes the sustainable application of evidence-based practice (EBP) to clinical care. Health care organizations have an opportunity to support sustainable change by creating robust IS infrastructures that engage nurses in the clinical environment. Integrating IS into a nursing shared governance model is an ideal vehicle to empower direct care nurses to sustain EBP. Importantly, an IS infrastructure may also promote nurse retention and increase interdisciplinary collaboration. This article, the first in a series on applying IS, describes how a multisite health care organization developed a systemwide nurse-led IS Specialist program within a shared governance model.

Creating an Interdisciplinary Collaborative Network of Scholars in Child Maltreatment Prevention: A Network Analysis of the Doris Duke Fellowships for the Promotion of Child Well-Being

Background

Child maltreatment is a complex problem requiring interdisciplinary collaborative research to generate innovative solutions. The Doris Duke Fellowships for the Promotion of Child Well-Being were designed to identify and nurture emerging scholars committed to child maltreatment prevention and create a supportive interdisciplinary learning network.

Objective

This paper examines connectivity within the collaborative network created by the fellowships program using longitudinal social network data.

Participants and Setting

Participants were 120 individuals selected as Doris Duke Fellows during their doctoral training at universities in the United States.

Methods

Fellows completed annual, voluntary web-based surveys to assess their interactions with other fellows during the past year. Social network analysis methods were used to assess the strength and quality of the learning network over a four-year period.

Results

Across four years of data, there were increases in the number of connections, proportion of cross-cohort connections, and proportion of interdisciplinary connections. Network analyses showed a highly connected network consisting of primarily medium- and high-quality connections between fellows from different disciplines. The number of scientific journal articles authored by two or more fellows grew substantially over time.

Conclusions

Findings indicate the collaborative network created by the fellowships program is growing and strengthening over time. The new Child Well-Being Research Network extends the fellowships network to a broader group of scholars and professionals to advance diversity, equity, inclusion, and justice in the field of child well-being research.

Learning to Prioritize Our Collaborative Opportunities: Overcoming the Bright Shiny Object Syndrome

ABSTRACT

There are multiple opportunities to participate in team science, leading to long-term benefits (eg, research impact, novelty, productivity). Scholars are not well-trained in how to choose among these opportunities, often learning via trial and error. The ability to navigate collaborations is framed by several principles and considerations: (1) locus of control (what control we have over our own behavior) and how it affects academic job satisfaction; (2) the scarcity mindset that may manifest as a result of the fear of missing future opportunities; and (3) power dynamics and inequities (eg, among women and racial/ethnic minority individuals). To provide a more systematic approach to weighing academic opportunities, the authors offer 30 questions across six overlapping domains. The domains include: the big picture (eg, Is the opportunity a building block for your career?), context (eg, How much do you have on your plate?), person (eg, Who is asking?), team (eg, Is the team productive?), role (eg, Will you lead or assist?), and outcomes (eg, Might the opportunity lead to publications and/or grants?). We offer advice for decision-making. For example, when presented with an opportunity involving a significant time commitment, it is useful to allow at least 24 hours before deciding. The authors offer advice and sample language for communicating your decision. Although every situation is different, there are several fundamental issues and questions to consider when one is presented with a new opportunity-these questions are suggested for mentors and mentees.

Advancing translational science education

In this communication, the authors offer considerations for how the scientific community can capitalize on decades of translational science advances and experiential knowledge to develop new education opportunities for a diverse and highly skilled translational science workforce. Continued advancement of the field of translational science will require new education approaches that distill key concepts in translational science from past and ongoing research initiatives and teach this foundational knowledge to current and future translational scientists. These key concepts include generalizable scientific and operational principles to guide translational science, as well as evidence-informed practices. Inspired by this approach, the National Center for Advancing Translational Sciences (NCATS) has developed an initial set of guiding principles for translational science generated via case studies of multiple highly successful translational science initiatives, and is now teaching them via new education activities that aim to reach a broad scientific audience interested in translational science. Our goal with this review is to prompt continued conversation with the translational science community regarding capitalizing on our collective translational science knowledge to advance core content for translational science education and disseminating this content to a broad range of scientific audiences.

Collaboration networks of the implementation science centers for cancer control: a social network analysis

BACKGROUND

Multi-center research initiatives offer opportunities to develop and strengthen connections among researchers. These initiatives often have goals of increased scientific collaboration which can be examined using social network analysis.

METHODS

The National Cancer Institute (NCI)-funded Implementation Science Centers in Cancer Control (ISC3) initiative conducted an online social network survey in its first year of funding (2020) to (1) establish baseline network measures including the extent of cross-center collaboration and (2) assess factors associated with a network member's access to the network such as one's implementation science (IS) expertise. Members of the seven funded centers and NCI program staff identified collaborations in planning/conducting research, capacity building, product development, scientific dissemination, and practice/policy dissemination.

RESULTS

Of the 192 invitees, 182 network members completed the survey (95%). The most prevalent roles were faculty (60%) and research staff (24%). Almost one-quarter (23%) of members reported advanced expertise in IS, 42% intermediate, and 35% beginner. Most members were female (69%) and white (79%). One-third (33%) of collaboration ties were among members from different centers. Across all collaboration activities, the network had a density of 14%, suggesting moderate cohesion. Degree centralization (0.33) and betweenness centralization (0.07) measures suggest a fairly dispersed network (no single or few central member(s) holding all connections). The most prevalent and densely connected collaboration was in planning/conducting research (1470 ties; 8% density). Practice/policy dissemination had the fewest collaboration, lowest density (284 ties' 3% density), and the largest number of non-connected members (n=43). Access to the ISC3 network varied significantly depending on members' level of IS expertise, role within the network, and racial/ethnic background. Across all collaboration activities, most connected members included those with advanced IS expertise, faculty and NCI staff, and Hispanic or Latino and white members.

CONCLUSIONS

Results establish a baseline for assessing the growth of cross-center collaborations, highlighting specific areas in need of particular growth in network collaborations such as increasing engagement of racial and ethnic minorities and trainees or those with less expertise in IS.

Training the next generation of aging and cognitive health researchers

Dementia is a growing public health concern, and African Americans and Latinos are disproportionately affected compared to White Americans. Improving cognitive health outcomes and reducing disparities requires a diverse, interdisciplinary workforce. The US Centers for Disease Control and Prevention's Healthy Brain Research Network (HBRN) Scholars Program trained racially/ethnically and gender-diverse scholars through mentored, collaborative research. Entry, exit, and alumni surveys and a Scholar Spotlight Series queried motivation, goals, acquired skills, accomplishments, program impact, and scholar perspectives. Scholars (n = 41) were majority female (n = 31, 75.6%), graduate students (n = 23, 56.1%), and racially/ethnically diverse (n = 20, 48.7%). Scholars primarily represented Medicine (n = 19, 46.3%) and Public Health (n = 12, 29.3%). Exiting scholars (n = 25) secured faculty/professional positions (n = 9, 36.0%), awards/funding (n = 12, 48.0%), and publications (n = 8, 32.0%). Alumni (n = 10) secured cognitive health-related positions/fellowships (n = 7, 70.0%). The HBRN Scholars Program is an adaptable model for other thematic networks to prepare scholars in collaborative skills critical for effective research and practice.

Operationalizing Implementation Science in Nutrition: The Implementation Science Initiative in Kenya and Uganda

BACKGROUND

Implementation science (IS) has the potential to improve the implementation and impact of policies, programs, and interventions. Most of the training, guidance, and experience has focused on implementation research, which is only 1 part of the broader field of IS. In 2018, the Society for Implementation Science in Nutrition borrowed concepts from IS in health to develop a broader and more integrated conceptual framework, adapted to the particular case of nutrition and with language and concepts more familiar to the nutrition community: it is called the IS in Nutrition (ISN) framework.

OBJECTIVE

The purpose of this research was to generate knowledge concerning challenges and strategies in operationalizing the ISN framework in low- and middle-income country (LMIC) settings.

METHODS

The ISN framework was operationalized in partnership with country teams in Kenya and Uganda over a 3-y period as part of the Implementation Science Initiative. An action research methodology (developmental evaluation) was used to provide timely feedback to the country teams, facilitate adaptations and adjustments, and generate the data presented in this article concerning challenges and strategies.

RESULTS

Operationalization of the ISN framework proceeded by first articulating a set of guiding principles as touchstones for the country teams and further articulating 6 components of an IS system to facilitate development of work streams. Challenges and strategies in implementing these 6 components were then documented. The knowledge gained through this experience led to the development of an IS system operational model to assist the application of IS in other LMIC settings.

CONCLUSIONS

Future investments in IS should prioritize a system- and capacity-building approach in order to realize its full potential and become institutionalized at country level. The operational model can guide others to improve the implementation of IS within a broad range of programs.

Learning needs assessment for multi-stakeholder implementation science training in LMIC settings: findings and recommendations

Background

Despite significant progress in the field of implementation science (IS), current training programs are inadequate to meet the global need, especially in low-and middle-income countries (LMICs). Even when training opportunities exist, there is a “knowledge-practice gap,” where implementation research findings are not useful to practitioners in a field designed to bridge that gap. This is a critical challenge in LMICs where complex public health issues must be addressed. This paper describes results from a formal assessment of learning needs, priority topics, and delivery methods for LMIC stakeholders.

Methods

We first reviewed a sample of articles published recently in Implementation Science to identify IS stakeholders and assigned labels and definitions for groups with similar roles. We then employed a multi-step sampling approach and a random sampling strategy to recruit participants (n = 39) for a semi-structured interview that lasted 30–60 min. Stakeholders with inputs critical to developing training curricula were prioritized and selected for interviews. We created memos from audio-recorded interviews and used a deductively created codebook to conduct thematic analysis. We calculated kappa coefficients for each memo and used validation techniques to establish rigor including incorporating feedback from reviewers and member checking.

Results

Participants included program managers, researchers, and physicians working in over 20 countries, primarily LMICs. The majority had over 10 years of implementation experience but fewer than 5 years of IS experience. Three main themes emerged from the data, pertaining to past experience with IS, future IS training needs, and contextual issues. Most respondents (even with formal training) described their IS knowledge as basic or minimal. Preferences for future training were heterogeneous, but findings suggest that curricula must encompass a broader set of competencies than just IS, include mentorship/apprenticeship, and center the LMIC context.

Conclusion

While this work is the first systematic assessment of IS learning needs among LMIC stakeholders, findings reflect existing research in that current training opportunities may not meet the demand, trainings are too narrowly focused to meet the heterogeneous needs of stakeholders, and there is a need for a broader set of competencies that moves beyond only IS. Our research also demonstrates the timely and unique needs of developing appropriately scoped, accessible training and mentorship support within LMIC settings. Therefore, we propose the novel approach of intelligent swarming as a solution to help build IS capacity in LMICs through the lens of sustainability and equity.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-021-00238-2.

Evaluating an Implementation Science Training Program: Impact on Professional Research and Practice

Real-world application and implementation of evidence-based practice continue to be a challenge across multiple sectors, including behavioral health settings. Providing the opportunity for future researchers and practitioners to gain capacity and knowledge through structured experiential learning in implementation science is critical to closing the research to practice gap. The Institute for Translational Research Education in Adolescent Drug Abuse (ITRE) is a graduate certificate program that offers specific coursework, a large-scale service-learning project based in the community, and mentorship related to implementation science research and practice. The purpose of this evaluation was to examine, from the perspective of ITRE scholars, the perceived impact on the development of professional research and practice skills once graduated from the ITRE program. Fifty-eight semi-structured interviews across five cohorts were selected randomly for in-depth thematic analysis (n = 58). Results suggest that the ITRE provides a unique approach grounded in implementation science for building robust and transferable skills for future researchers and practitioners working in a variety of behavioral healthcare settings.

Teaching for implementation: A framework for building implementation research and practice capacity within the translational science workforce

Implementation science offers a compelling value proposition to translational science. As such, many translational science stakeholders are seeking to recruit, teach, and train an implementation science workforce. The type of workforce that will make implementation happen consists of both implementation researchers and practitioners, yet little guidance exists on how to train such a workforce. We-members of the Advancing Dissemination and Implementation Sciences in CTSAs Working Group-present the Teaching For Implementation Framework to address this gap. We describe the differences between implementation researchers and practitioners and demonstrate what and how to teach them individually and in co-learning opportunities. We briefly comment on educational infrastructures and resources that will be helpful in furthering this type of approach.

Competences for implementation science: what trainees need to learn and where they learn it

Education in implementation science, which involves the training of health professionals in how to implement evidence-based findings into health practice systematically, has become a highly relevant topic in health sciences education. The present study advances education in implementation science by compiling a competence profile for implementation practice and research and by exploring implementation experts' sources of expertise. The competence profile is theoretically based on educational psychology, which implies the definition of improvable and teachable competences. In an online-survey, an international, multidisciplinary sample of 82 implementation experts named competences that they considered most helpful for conducting implementation practice and implementation research. For these competences, they also indicated whether they had acquired them in their professional education, additional training, or by self-study and on-the-job experience. Data were analyzed using a mixed-methods approach that combined qualitative content analyses with descriptive statistics. The participants deemed collaboration knowledge and skills most helpful for implementation practice. For implementation research, they named research methodology knowledge and skills as the most important ones. The participants had acquired most of the competences that they found helpful for implementation practice in self-study or by on-the-job experience. However, participants had learned most of their competences for implementation research in their professional education. The present results inform education and training activities in implementation science and serve as a starting point for a fluid set of interdisciplinary implementation science competences that will be updated continuously. Implications for curriculum development and the design of educational activities are discussed.

From Research Training to Scientific Advancement-Contributions from the Implementation Research Institute: An Introduction to the Special Issue

The special series is designed to provide examples of funded implementation research conducted by alumni of the first four cohorts of the Implementation Research Institute (IRI). The introduction links the six substantive papers to the conceptual and methodological challenges laid out in a 2009 publication in this journal which led to the IRI training program in the emerging science of implementation with a special focus on behavior health settings. The 7th paper in the series illustrates an innovative evaluative approach to design and measurement of IRI fellow publications and grants informed by the training program such as bibliometrics. The introduction also notes some elements identified in the 2009 foundational paper not represented in these papers such as costs as well as important developments and foci in the decade since 2009 such as de-implementation, sustainability, dynamic adaptation processes, and hybrid designs that need to be an integral part of training programs in implementation research.

Building the Next Generation of Researchers: Mentored Training in Dissemination and Implementation Science

PROBLEM

Dissemination and implementation (D&I) science provides the tools needed to close the gap between known intervention strategies and their effective application. The authors report on the Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) program-a D&I training program for postdoctoral or early-career cancer prevention and control scholars.

APPROACH

MT-DIRC was a 2-year training institute in which fellows attended 2 annual Summer Institutes and other conferences and received didactic, group, and individual instruction; individualized mentoring; and other supports (e.g., pilot funding). A quasi-experimental design compared changes in 3 areas: mentoring, skills, and network composition. To evaluate mentoring and D&I skills, data from fellows on their mentors' mentoring competencies, their perspectives on the importance of and satisfaction with mentoring priority areas, and their self-rated skills in D&I competency domains were collected. Network composition data were collected from faculty and fellows for 3 core social network domains: contact, mentoring, and collaboration. Paired t tests (mentoring), linear mixed models (skills), and descriptive analyses (network composition) were performed.

OUTCOMES

Mentors were rated as highly competent across all mentoring competencies, and each mentoring priority area showed reductions in gaps between satisfaction and importance between the 6 and 18 months post-first Summer Institute. Fellows' self-rated skills in D&I competencies improved significantly in all domains over time (range: 42.5%-52.9% increase from baseline to 18 months post-first Summer Institute). Mentorship and collaboration networks grew over time, with the highest number of collaboration network ties for scholarly manuscripts (n = 199) in 2018 and for research projects (n = 160) in 2019.

NEXT STEPS

Building on study findings and existing literature, mentored training of scholars is an important approach for building D&I skills and networks, and thus to better applying the vast amount of available intervention evidence to benefit cancer control.

Building capacity in dissemination and implementation science: a systematic review of the academic literature on teaching and training initiatives

BACKGROUND

The field of dissemination and implementation (D&I) science has grown significantly over recent years. Alongside this, an increased demand for training in D&I from researchers and implementers has been seen. Research describing and evaluating D&I training opportunities, referred to here as 'capacity building initiatives' (CBIs), can help provide an understanding of different methods of training as well as training successes and challenges. However, to gain a more detailed understanding of the evidence-base and how D&I CBIs are being reported in publications, a field-wide examination of the academic literature is required.

METHODS

Systematic review to identify the type and range of D&I CBIs discussed and/or appraised in the academic literature. EMBASE, Medline and PsycINFO were searched between January 2006 and November 2019. Articles were included if they reported on a D&I CBI that was developed by the authors (of each of the included articles) or the author's host institution. Two reviewers independently screened the articles and extracted data using a standardised form.

RESULTS

Thirty-one articles (from a total of 4181) were included. From these, 41 distinct D&I CBIs were identified which focussed on different contexts and professions, from 8 countries across the world. CBIs ranged from short courses to training institutes to being part of academic programmes. Nearly half were delivered face-face with the remainder delivered remotely or using a blended format. CBIs often stipulated specific eligibility criteria, strict application processes and/or were oversubscribed. Variabilities in the way in which the D&I CBIs were reported and/or evaluated were evident.

CONCLUSIONS

Increasing the number of training opportunities, as well as broadening their reach (to a wider range of learners), would help address the recognised deficit in D&I training. Standardisation in the reporting of D&I CBIs would enable the D&I community to better understand the findings across different contexts and scientific professions so that training gaps can be identified and overcome. More detailed examination of publications on D&I CBIs as well as the wider literature on capacity building would be of significant merit to the field.

Developing implementation research capacity: longitudinal evaluation of the King's College London Implementation Science Masterclass, 2014-2019

BACKGROUND

Despite an increasing number of training opportunities in implementation science becoming available, the demand for training amongst researchers and practitioners is unmet. To address this training shortfall, we developed the King's College London 'Implementation Science Masterclass' (ISM), an innovative 2-day programme (and currently the largest of its kind in Europe), developed and delivered by an international faculty of implementation experts.

METHODS

This paper describes the ISM and provides delegates' quantitative and qualitative evaluations (gathered through a survey at the end of the ISM) and faculty reflections over the period it has been running (2014-2019).

RESULTS

Across the 6-year evaluation, a total of 501 delegates have attended the ISM, with numbers increasing yearly from 40 (in 2014) to 147 (in 2019). Delegates represent a diversity of backgrounds and 29 countries from across the world. The overall response rate for the delegate survey was 64.5% (323/501). Annually, the ISM has been rated 'highly' in terms of delegates' overall impression (92%), clear and relevant learning objectives (90% and 94%, respectively), the course duration (85%), pace (86%) and academic level 87%), and the support provided on the day (92%). Seventy-one percent of delegates reported the ISM would have an impact on how they approached their future work. Qualitative feedback revealed key strengths include the opportunities to meet with an international and diverse pool of experts and individuals working in the field, the interactive nature of the workshops and training sessions, and the breadth of topics and contexts covered.

CONCLUSIONS

Yearly, the UK ISM has grown, both in size and in its international reach. Rated consistently favourably by delegates, the ISM helps to tackle current training demands from all those interested in learning and building their skills in implementation science. Evaluation of the ISM will continue to be an annual iterative process, reflective of changes in the evidence base and delegates changing needs as the field evolves.

The "secret sauce" for a mentored training program: qualitative perspectives of trainees in implementation research for cancer control

BACKGROUND

Mentored training approaches help build capacity for research through mentoring networks and skill building activities. Capacity for dissemination and implementation (D&I) research in cancer is needed and mentored training programs have been developed. Evaluation of mentored training programs through quantitative approaches often provides us with information on "what" improved for participants. Qualitative approaches provide a deeper understanding of "how" programs work best.

METHODS

Qualitative interviews were conducted with 21 fellows of the National Cancer Institute-funded Mentored Training for Dissemination and Implementation in Cancer to gain understanding of their experiences with mentoring received during the program. Fellows were selected from all 55 trained participants based upon their gain in D&I research skills (highest and lowest) and number of collaborative connections in the program network (highest and lowest) reported in previous quantitative surveys. Phone interviews were recorded with permission, transcribed verbatim, and de-identified for analysis. Codes were developed a priori to reflect interview guide concepts followed by further development and iterative coding of three common themes that emerged: 1) program and mentoring structure, 2) importance of mentor attributes, and 3) enhanced capacity: credentials, confidence, credibility and connections.

RESULTS

Interviews provided valuable information about program components that worked best and impacts attributed to participation in the program. Fellows reported that regular monthly check-in calls with mentors helped to keep their research moving forward and that group mentoring structures aided in their learning of basic D&I research concepts and their application. Accessible, responsive, and knowledgeable mentors were commonly mentioned by fellows as a key to their success in the program. Fellows mentioned various forms of impact that they attributed to their participation in the program including gaining credibility in the field, a network of peers and experts, and career developments (e.g., collaborative publications and grant funding).

CONCLUSIONS

These findings suggest that mentored training works best when mentoring is structured and coupled with applied learning and when respected and dedicated mentors are on board. Increased scientific collaborations and credibility within a recognized network are important trainee experiences that should be considered when designing, implementing, and sustaining mentored training programs.

The TDR MOOC training in implementation research: evaluation of feasibility and lessons learned in Rwanda

Background

Hypertension (HTN) affects nearly 1 billion people globally and is a major cause of morbidity and mortality. In low- and middle-income countries (LMICs), HTN represents an unmet health care gap that can be addressed by strengthening national health care systems. The National Heart, Lung, and Blood Institute recently funded the T4 Translation Research Capacity Building Initiative in Low Income Countries (TREIN) program to build capacity in dissemination and implementation (D&I) research in HTN in LMICs. The Special Programme for Research and Training in Tropical Diseases (TDR) at the World Health Organization (WHO) recently developed a massive open online course (MOOC) to train in D&I. Herein, we report on the use of the TDR WHO MOOC in D&I for the TREIN program in Rwanda, assessing feasibility of the MOOC and D&I competencies after MOOC training.

Methods

Participants in one-group MOOC training completed pre- and post-training questionnaires to assess dissemination and implementation (D&I) competency outcomes and feasibility. D&I competencies were measured by use of a scale developed for a US-based training program, with the change in competency scores assessed by paired t test. Feasibility was measured by completion of homework and final project assignment and analyzed using descriptive statistics.

Results

Of the 92 trainees enrolled, 35 (38%) completed all MOOC components. D&I competency scores showed strong evidence of improvements from pre- to post-test. The full-scale average score improved by an average of 1.09 points, representing an effect size of 1.25 (CI 0.48-2.00); all four subscales also showed strong evidence of improvements. Trainees reported challenges to MOOC course completion that included technological issues (i.e., limited internet access) and competing demands (i.e., work, family).

Conclusions

In the context of LMIC training, the MOOC course was feasible and course completion showed improvement in D&I competency scores. While the program was designed with a focus on training for tropical diseases, there is potential for scalability to a wider audience of health care researchers, workers, administrators, and policymakers in LMIC interested in D&I research in non-communicable diseases.

Mentored training and its association with dissemination and implementation research output: a quasi-experimental evaluation

Background

There is a continued need to evaluate training programs in dissemination and implementation (D&I) research. Scientific products yielded from trainees are an important and objective measure to understand the capacity growth within the D&I field. This study evaluates our mentored training program in terms of scientific productivity among applicants.

Methods

Post-doctoral and early-career cancer researchers were recruited and applied to the R25 Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) between 2014 and 2017. Using application details and publicly available bibliometric and funding data, we compared selected fellows with unsuccessful applicants (nonfellows). We extracted Scopus citations and US federal grant funding records for all applicants (N = 102). Funding and publication abstracts were de-identified and coded for D&I focus and aggregated to the applicant level for analysis. Logistic regression models were explored separately for the odds of (1) a D&I publication and (2) US federal grant funding post year of application among fellows (N = 55) and nonfellows (N = 47). Additional models were constructed to include independent variables that attenuated the program’s association by 5% or more. Only US-based applicants (N = 87) were included in the grant funding analysis.

Results

Fellows and nonfellows were similar across several demographic characteristics. Fellows were more than 3 times more likely than nonfellows to have grant funding after MT-DIRC application year (OR 3.2; 95% CI 1.1–11.0) while controlling for time since application year; the association estimate was 3.1 (95% CI 0.98–11.0) after adjusting for both cancer research area and previous grant funding. For publications, fellows were almost 4 times more likely to publish D&I-focused work adjusting for time (OR 3.8; 95% CI 1.7–9.0). This association lessened after adjusting for previous D&I publication and years since undergraduate degree (OR 2.9; 95% CI 1.2–7.5).

Conclusions

We document the association of a mentored training approach with built-in networks of peers to yield productive D&I researchers. Future evaluation efforts could be expanded to include other forms of longer-term productivity such as policy or practice change as additional objective measures. D&I research trainings in the USA and internationally should consider common evaluation measures.

The University of California San Francisco (UCSF) Training Program in Implementation Science: Program Experiences and Outcomes

Purpose: We evaluated outcomes of trainees who have completed the Certificate program in Implementation Science at the University of California San Francisco.

Methods: All students who completed the in-person Certificate Program between 2008 and 2015 (n = 71), or the online Certificate Program between 2016 and 2017 (n = 13), were eligible for our study. We assessed the potential impact of the Certificate Program on the professional development of trainees, through participant surveys on their self-reported level of comfort with pre-defined competencies, and on academic productivity.

Results: Of eligible trainees, 54 in-person (77%) and 13 online (100%) Certificate Program participants completed surveys. In-person trainees reported a total of 147 implementation science-related publications in peer-reviewed journals (median 3 publications/trainee, IQR 1–15). Thirty-four trainees (63%) reported being a Principal Investigator (PI) of 64 funded implementation science-related grants (median 2 grants/trainee, IQR 1–4). Fifteen percent (15%, n = 8) of participants reported receiving an NIH grant on which they were the PI, including R01 or P01 level funding (n = 4, 7%) and K awards (n = 3, 6%). Both in-person and online trainees reported median high to moderate confidence for all 12 competencies assessed. Confidence waned in skills aligning with later stages of implementation research for all trainees.

Conclusion: The moderate to high confidence in all competencies assessed and reported high level of academic productivity support the benefits of intensive, graduate-level training focused on applied methods to support career development of implementation scientists.

Evaluation of the Implementation Research Institute: Trainees' Publications and Grant Productivity

With growth in the field of dissemination and implementation (D&I) research, there has been growth in capacity building, with many training opportunities. As such, it is important to continue to evaluate D&I research training programs. This paper reports the results of an evaluation of the Implementation Research Institute (IRI), a R25 funded by the National Institute of Mental Health with additional funding by the Department of Veterans Affairs (VA). The fourth cohort also had a supplement from the National Institute on Drug Abuse. Using bibliometrics data, we report on a quasi-experimental retrospective cohort study assessing whether the rates of scholarly productivity in D&I science of IRI fellows (those who applied and were accepted to the training) were greater than those who applied but were not accepted to IRI. Our findings show that Selected Applicants' odds of publishing in implementation science were higher for earlier alumni, starting at 12% 1 year out and increasing to 94% for those who were 4 years out from starting training. Chances for Non-Selected Applicants remained relatively stable, starting at 47% at 1 year and going to 33% at 4 years since their application, a pattern that was stable even after controlling for demographic characteristics. These results support the hypothesis that IRI is increasing the D&I research productivity of those selected to the program, and that our fellows are advancing the field of D&I compared to those investigators not selected to our institute. Our finding also indicates the importance of a 2-year training.

Building the Next Generation of Implementation Science Careers to Advance Health Equity

Rapid advancements in translational research have produced innovative clinical discoveries and evidence-based interventions that are ready for uptake in real-world settings, creating vast opportunities and challenges for implementation science. However, there is an inadequate research workforce to study effective strategies and delivery of implementation to advance the field. Novel career development initiatives will build scholars for the next generation of implementation science to bridge research to practice for diverse populations to advance health equity, specifically with a strategic focus on heart, lung, blood and sleep diseases and conditions. Along with traditional mentoring and curricula, research training includes state-of-the-art approaches using complex methods and multi-disciplinary collaborations between researchers, practice settings, and diverse communities. Implementation science scholars strive not only to decrease the lag time between the discovery of evidence-based interventions and successful implementation but also how to advance health equity and to reduce disparities for underserved populations that suffer disproportionally.

Training scholars in dissemination and implementation research for cancer prevention and control: a mentored approach

BACKGROUND

As the field of D&I (dissemination and implementation) science grows to meet the need for more effective and timely applications of research findings in routine practice, the demand for formalized training programs has increased concurrently. The Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) Program aims to build capacity in the cancer control D&I research workforce, especially among early career researchers. This paper outlines the various components of the program and reports results of systematic evaluations to ascertain its effectiveness.

METHODS

Essential features of the program include selection of early career fellows or more experienced investigators with a focus relevant to cancer control transitioning to a D&I research focus, a 5-day intensive training institute, ongoing peer and senior mentoring, mentored planning and work on a D&I research proposal or project, limited pilot funding, and training and ongoing improvement activities for mentors. The core faculty and staff members of the MT-DIRC program gathered baseline and ongoing evaluation data regarding D&I skill acquisition and mentoring competency through participant surveys and analyzed it by iterative collective reflection.

RESULTS

A majority (79%) of fellows are female, assistant professors (55%); 59% are in allied health disciplines, and 48% focus on cancer prevention research. Forty-three D&I research competencies were assessed; all improved from baseline to 6 and 18 months. These effects were apparent across beginner, intermediate, and advanced initial D&I competency levels and across the competency domains. Mentoring competency was rated very highly by the fellows--higher than rated by the mentors themselves. The importance of different mentoring activities, as rated by the fellows, was generally congruent with their satisfaction with the activities, with the exception of relatively greater satisfaction with the degree of emotional support and relatively lower satisfaction for skill building and opportunity initially.

CONCLUSIONS

These first years of MT-DIRC demonstrated the program's ability to attract, engage, and improve fellows' competencies and skills and implement a multicomponent mentoring program that was well received. This account of the program can serve as a basis for potential replication and evolution of this model in training future D&I science researchers.

Reducing Cardiovascular Disparities Through Community-Engaged Implementation Research: A National Heart, Lung, and Blood Institute Workshop Report

Cardiovascular disparities remain pervasive in the United States. Unequal disease burden is evident among population groups based on sex, race, ethnicity, socioeconomic status, educational attainment, nativity, or geography. Despite the significant declines in cardiovascular disease mortality rates in all demographic groups during the last 50 years, large disparities remain by sex, race, ethnicity, and geography. Recent data from modeling studies, linked micromap plots, and small-area analyses also demonstrate prominent variation in cardiovascular disease mortality rates across states and counties, with an especially high disease burden in the southeastern United States and Appalachia. Despite these continued disparities, few large-scale intervention studies have been conducted in these high-burden populations to examine the feasibility of reducing or eliminating cardiovascular disparities. To address this challenge, on June 22 and 23, 2017, the National Heart, Lung, and Blood Institute convened experts from a broad range of biomedical, behavioral, environmental, implementation, and social science backgrounds to summarize the current state of knowledge of cardiovascular disease disparities and propose intervention strategies aligned with the National Heart, Lung, and Blood Institute mission. This report presents the themes, challenges, opportunities, available resources, and recommended actions discussed at the workshop.

Assessing Research Collaboration through Co-authorship Network Analysis

Interdisciplinary research collaboration is needed to perform transformative science and accelerate innovation. The Science of Team Science strives to investigate, evaluate, and foster team science, including institutional policies that may promote or hinder collaborative interdisciplinary research and the resources and infrastructure needed to promote team science within and across institutions. Social network analysis (SNA) has emerged as a useful method to measure interdisciplinary science through the evaluation of several types of collaboration networks, including co-authorship networks. Likewise, research administrators are responsible for conducting rigorous evaluation of policies and initiatives. Within this paper, we present a case study using SNA to evaluate inter-programmatic collaboration (evidenced by co-authoring scientific papers) from 2007-2014 among scientists who are members of four formal research programs at an NCI-designated Cancer Center, the Markey Cancer Center (MCC) at the University of Kentucky. We evaluate change in network descriptives over time and implement separable temporal exponential-family random graph models (STERGMs) to estimate the effect of author and network variables on the tendency to form a co-authorship tie. We measure the diversity of the articles published over time (Blau's Index) to understand whether the changes in the co-authorship network are reflected in the diversity of articles published by research members. Over the 8-year period, we found increased inter-programmatic collaboration among research members as evidenced by co-authorship of published scientific papers. Over time, MCC Members collaborated more with others outside of their research program and outside their initial dense co-authorship groups, however tie formation continues to be driven by co-authoring with individuals of the same research program and academic department. Papers increased in diversity over time on all measures with the exception of author gender. This inter-programmatic research was fostered by policy changes in cancer center administration encouraging interdisciplinary research through both informal (e.g., annual retreats, seminar series) and formal (e.g., requiring investigators from more than two research programs on applications for pilot funding) means. Within this cancer center, interdisciplinary co-authorship increased over time as policies encouraging this collaboration were implemented. Yet, there is room for improvement in creating more interdisciplinary and diverse ties between research program members.

Bridging the Chasm: Challenges, Opportunities, and Resources for Integrating a Dissemination and Implementation Science Curriculum into Medical Education

BACKGROUND

Physicians are charged with implementing evidence-based medicine, yet few are trained in the science of Dissemination and Implementation (D&I). In view of the potential of evidence-based training in D&I to help close the gap between research and practice, the goal of this review is to examine the importance of D&I training in medical education, describe challenges to implementing such training, and provide strategies and resources for building D&I capacity.

METHODS

We conducted (1) a systematic review to identify US-based D&I training efforts and (2) a critical review of additional literature to inform our evaluation of the challenges and opportunities of integrating D&I training in medical education.

RESULTS

Out of 269 unique articles reviewed, 11 described US-based D&I training. Although vibrant and diverse training opportunities exist, their capacity is limited, and they are not designed to meet physicians' needs. Synthesis of relevant literature using a critical review approach identified challenges inherent to changing medical education, as well as challenges related to D&I science. Finally, selected strategies and resources are available for facilitating incorporation of D&I training into medical education and overcoming existing challenges.

CONCLUSIONS

Integrating D&I training in the medical education curriculum, and particularly in residency and fellowship training, holds promise for bridging the chasm between scientific discoveries and improved patient care and outcomes. However, unique challenges should be addressed, including the need for greater evidence.

A content analysis of dissemination and implementation science resource initiatives: what types of resources do they offer to advance the field?

BACKGROUND

The recent growth in organized efforts to advance dissemination and implementation (D & I) science suggests a rapidly expanding community focused on the adoption and sustainment of evidence-based practices (EBPs). Although promising for the D & I of EBPs, the proliferation of initiatives is difficult for any one individual to navigate and summarize. Such proliferation may also result in redundant efforts or missed opportunities for participation and advancement. A review of existing D & I science resource initiatives and their unique merits would be a significant step for the field. The present study aimed to describe the global landscape of these organized efforts to advance D & I science.

METHODS

We conducted a content analysis between October 2015 and March 2016 to examine resources and characteristics of D & I science resource initiatives using public, web-based information. Included resource initiatives must have engaged in multiple efforts to advance D & I science beyond conferences, offered D & I science resources, and provided content in English. The sampling method included an Internet search using D & I terms and inquiry among internationally representative D & I science experts. Using a coding scheme based on a priori and grounded approaches, two authors consensus coded website information including interactive and non-interactive resources and information regarding accessibility (membership, cost, competitive application, and location).

RESULTS

The vast majority (83%) of resource initiatives offered at least one of seven interactive resources (consultation/technical assistance, mentorship, workshops, workgroups, networking, conferences, and social media) and one of six non-interactive resources (resource library, news and updates from the field, archived talks or slides, links pages, grant writing resources, and funding opportunities). Non-interactive resources were most common, with some appearing frequently across resource initiatives (e.g., news and updates from the field).

CONCLUSION

Findings generated by this study offer insight into what types of D & I science resources exist and what new resources may have the greatest potential to make a unique and needed contribution to the field. Additional interactive resources may benefit the field, particularly mentorship opportunities and resources that can be accessed virtually. Moving forward, it may be useful to consider strategic attention to the core tenets of D & I science put forth by Glasgow and colleagues to most efficiently and effectively advance the field.

Building capacity for dissemination and implementation research: one university's experience

Background

While dissemination and implementation (D&I) science has grown rapidly, there is an ongoing need to understand how to build and sustain capacity in individuals and institutions conducting research. There are three inter-related domains for capacity building: people, settings, and activities. Since 2008, Washington University in St. Louis has dedicated significant attention and resources toward building D&I research capacity. This paper describes our process, challenges, and lessons with the goal of informing others who may have similar aims at their own institution.

Activities

An informal collaborative, the Washington University Network for Dissemination and Implementation Research (WUNDIR), began with a small group and now has 49 regular members. Attendees represent a wide variety of settings and content areas and meet every 6 weeks for half-day sessions. A logic model organizes WUNDIR inputs, activities, and outcomes. A mixed-methods evaluation showed that the network has led to new professional connections and enhanced skills (e.g., grant and publication development). As one of four, ongoing, formal programs, the Dissemination and Implementation Research Core (DIRC) was our first major component of D&I infrastructure. DIRC’s mission is to accelerate the public health impact of clinical and health services research by increasing the engagement of investigators in later stages of translational research. The aims of DIRC are to advance D&I science and to develop and equip researchers with tools for D&I research. As a second formal component, the Washington University Institute for Public Health has provided significant support for D&I research through pilot projects and a small grants program. In a third set of formal programs, two R25 training grants (one in mental health and one in cancer) support post-doctoral scholars for intensive training and mentoring in D&I science. Finally, our team coordinates closely with D&I functions within research centers across the university. We share a series of challenges and potential solutions.

Conclusion

Our experience in developing D&I research at Washington University in St. Louis shows how significant capacity can be built in a relatively short period of time. Many of our ideas and ingredients for success can be replicated, tailored, and improved upon by others.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-017-0634-4) contains supplementary material, which is available to authorized users.