Training the next generation of delivery science researchers: 10-year experience of a post-doctoral research fellowship program within an integrated care system

Evaluation of an Interdisciplinary Educational Program to Foster Learning Health Systems: Education Evaluation

Background

Learning health systems (LHS) have the potential to use health data in real time through rapid and continuous cycles of data interrogation, implementing insights to practice, feedback, and practice change. However, there is a lack of an appropriately skilled interprofessional informatics workforce that can leverage knowledge to design innovative solutions. Therefore, there is a need to develop tailored professional development training in digital health, to foster skilled interprofessional learning communities in the health care workforce in Australia.

Objective

This study aimed to explore participants' experiences and perspectives of participating in an interprofessional education program over 13 weeks. The evaluation also aimed to assess the benefits, barriers, and opportunities for improvements and identify future applications of the course materials.

Methods

We developed a wholly online short course open to interdisciplinary professionals working in digital health in the health care sector. In a flipped classroom model, participants (n=400) undertook 2 hours of preclass learning online and then attended 2.5 hours of live synchronous learning in interactive weekly Zoom workshops for 13 weeks. Throughout the course, they collaborated in small, simulated learning communities (n=5 to 8), engaging in various activities and problem-solving exercises, contributing their unique perspectives and diverse expertise. The course covered a number of topics including background on LHS, establishing learning communities, the design thinking process, data preparation and machine learning analysis, process modeling, clinical decision support, remote patient monitoring, evaluation, implementation, and digital transformation. To evaluate the purpose of the program, we undertook a mixed methods evaluation consisting of pre- and postsurveys rating scales for usefulness, engagement, value, and applicability for various aspects of the course. Participants also completed identical measures of self-efficacy before and after (n=200), with scales mapped to specific skills and tasks that should have been achievable following each of the topics covered. Further, they undertook voluntary weekly surveys to provide feedback on which aspects to continue and recommendations for improvements, via free-text responses.

Results

From the evaluation, it was evident that participants found the teaching model engaging, useful, valuable, and applicable to their work. In the self-efficacy component, we observed a significant increase (P<.001) in perceived confidence for all topics, when comparing pre- and postcourse ratings. Overall, it was evident that the program gave participants a framework to organize their knowledge and a common understanding and shared language to converse with other disciplines, changed the way they perceived their role and the possibilities of data and technologies, and provided a toolkit through the LHS framework that they could apply in their workplaces.

Conclusions

We present a program to educate the health workforce on integrating the LHS model into standard practice. Interprofessional collaborative learning was a major component of the value of the program. This evaluation shed light on the multifaceted challenges and expectations of individuals embarking on a digital health program. Understanding the barriers and facilitators of the audience is crucial for creating an inclusive and supportive learning environment. Addressing these challenges will not only enhance participant engagement but also contribute to the overall success of the program and, by extension, the broader integration of digital health solutions into health care practice and, ultimately, patient outcomes.

Partnership-building considerations for implementation science in learning health systems: a case study of the Implementation Science Collaborative in Alberta, Canada

Introduction

Implementation of health innovations is inherently collaborative, requiring trans-sectoral partnerships between implementation researchers, innovation teams, and implementation practitioners. Implementation science has been shown to improve implementation successes; however, challenges that hinder partnerships to advance implementation science continue to persist. Using a whole-system approach to assess and respond to implementation science partnership barriers may shed light on effective responses.

Methods

We conducted a case study of Alberta's learning health system, using semi-structured group and individual interviews to create a nuanced understanding of the considerations required for implementation research collaborations. We interviewed 53 participants representing 21 offices in the health system, academia, professional associations, and government who regularly plan, evaluate, and/or study health system implementation initiatives in Alberta. Using the Partnership Model for Research Capacity Building, we identified current facilitators and challenges for partnerships for conducting and using implementation science, at different levels of Alberta's health-research ecosystem.

Results

Alberta's healthcare system is well set up to readily embed intervention effectiveness and efficacy research. Infrastructure was also in place to strengthen implementation practice. However, weaknesses around exchanging knowledge and skills, providing feedback and mentoring, and accommodating diversity affected the ability of both individuals and teams to build implementation science capacity. Without this capacity, teams could not participate in embedded implementation research collaborations. We report the response of the Alberta Strategy for Patient-Oriented Research SUPPORT Unit to these barriers to provide practical guidance on various program options to strengthen individual- and organization-level implementation science capacity.

Discussion

This study applied a whole-system approach to assess factors across Alberta's health-research ecosystem, which affect partnerships to advance implementation science. Our findings illustrated that partnership considerations go beyond interpersonal factors and include system-wide considerations. With the results, health organization leaders have (1) a method for assessing organizational capability to readily embed implementation research and (2) a catalog of potential responses to create conditions to readily engage with implementation science in their day-to-day implementation processes.