Health research systems in change: the case of 'Push the Pace' in the National Institute for Health Research

How successful was the use of a community of practice for the implementation of evidence-based practices for heart failure within the United States Department of Veterans Affairs: Insights from a formative evaluation

Background

Communities of Practice (CoPs) are a promising approach to facilitate the implementation of evidence-based practices (EBPs) to improve care for chronic conditions like heart failure (HF). CoPs involve a complex process of acquiring and converting both explicit and tacit knowledge into clinical activities. This study describes the conceptualization, creation, capacity-building and dissemination of a CoP sustained over 9 years, and evaluates its value and impact on EBP.

Methods

In July 2006, a CoP called the Heart Failure Provider Network (HF Network) was established within the United States Department of Veterans Affairs (VA) with the overarching goal of improving the quality of care for HF patients. We assessed (formative) the HF Network in terms of its various activities (inputs) and proximal impacts (mediators) at the individual level, and its distal impacts (outcomes) at the site level including implementation of new/improved EBPs at the systemwide level.

Results

The HF Network membership grew steadily over the 9 years. The CoP has involved a total of 1341 multidisciplinary and multilevel members at all 144 VA Health Care Systems (sites). Most members were practising clinicians (n = 891, 66.4%), followed by administrators (n = 342, 25.5%), researchers (n = 70, 5.2%) and others (n = 38, 2.8%). Participation was assessed to be “active” for 70.6% versus “passive” for 29.4% of members. The distribution of active members (clinicians 64.7%, administrators 21.6%) was similar to the distribution of overall membership.

Conclusions

Survey respondents perceived the HF Network as useful in terms of its varied activities and resources relevant for patient care. Strong evidence shows that these members, particularly those who considered themselves influential in improving quality of care, noted multiple benefits of membership, which included confirmation of their own clinical practices, evidence-based changes to their practice and help in understanding facilitators and barriers in setting up or running HF clinics and other programmes. Such CoPs have strong impacts on the quality of care being delivered for both mandated and non-mandated initiatives.

How to increase value and reduce waste in research: initial experiences of applying Lean thinking and visual management in research leadership

Waste in research has been well documented, but initiatives to reduce it are scarce. Here, we share our initial experiences of implementing Lean thinking and visual management into hospital research units in the Region of Southern Denmark. A Transformation Guiding Team (TGT) anchored in the top management was established with participation from leaders, researchers and patient representatives. The role of the TGT was to implement Lean methods, considering patients as primary end-users of the research results. This is in line with an explicit decision on setting patient values first in clinical settings at participating hospitals. The leaders of the research units were instructed in Lean thinking and Lean methods during a five-module course focusing on increasing value and reducing waste in research production. Initial experiences were that Lean tools could create a patient-centred vision that through visual management could identify waste in work processes. Concerns were lack of evidence for using Lean methods in research leadership and that the model itself could be a time consumer. Some lessons learnt were that adding Lean tools in research leadership should not just provide increased research productivity, but also improve other important key performance indicators such as quality of research and patient-relevant results. We intend to evaluate the value of the initiative by follow-up research and publish the outcome of key behavioural and key performance indicators.

What works for peer review and decision-making in research funding: a realist synthesis

INTRODUCTION

Allocation of research funds relies on peer review to support funding decisions, and these processes can be susceptible to biases and inefficiencies. The aim of this work was to determine which past interventions to peer review and decision-making have worked to improve research funding practices, how they worked, and for whom.

METHODS

Realist synthesis of peer-review publications and grey literature reporting interventions in peer review for research funding.

RESULTS

We analysed 96 publications and 36 website sources. Sixty publications enabled us to extract stakeholder-specific context-mechanism-outcomes configurations (CMOCs) for 50 interventions, which formed the basis of our synthesis. Shorter applications, reviewer and applicant training, virtual funding panels, enhanced decision models, institutional submission quotas, applicant training in peer review and grant-writing reduced interrater variability, increased relevance of funded research, reduced time taken to write and review applications, promoted increased investment into innovation, and lowered cost of panels.

CONCLUSIONS

Reports of 50 interventions in different areas of peer review provide useful guidance on ways of solving common issues with the peer review process. Evidence of the broader impact of these interventions on the research ecosystem is still needed, and future research should aim to identify processes that consistently work to improve peer review across funders and research contexts.

A systems approach for optimizing implementation to impact: meeting report and proceedings of the 2019 In the Trenches: Implementation to Impact International Summit

Background

The In the Trenches series of cutting-edge knowledge sharing events on impact for front-line experts and practitioners provides an engagement platform for diverse stakeholders across government, research funding organizations, industry, and academia to share emerging knowledge and practical experiences. The second event of the series In the Trenches: Implementation to Impact International Summit was held in Banff, Alberta, Canada, on June 7–8, 2019. The overarching vision for the Summit was to create an engagement platform for addressing key challenges and finding practical solutions to move from implementation (i.e. putting findings into effect) to impact (i.e. creating benefits to society and the economy).

Processes and proceedings

The Summit used diverse approaches to facilitate active engagement and knowledge sharing between 80 delegates across sectors and jurisdictions. Summit sessions mostly consisted of short talks and moderated panels grouped into eight thematic sessions. Each presentation included a summary of Key Messages, along with a summary of the Actionable Insights which concluded each session. The presentations and discussions are analysed, synthesized and described in this proceedings paper using a systems approach. This demonstrates how the Summit focused on each of the necessary functions (and associated components) that should be undertaken, and combined, for effective research and innovation: stewardship and governance, securing finance, creating capacity, and producing and using research. The approach also identifies relevant challenges.

Conclusions

There is increased interest globally in the benefits that can accrue from adopting a systems approach to research and innovation. Various organizations in Canada and internationally have made considerable progress on Implementation to Impact, often as a result of well-planned initiatives. The Summit highlights the value of 1) collaboration between researchers and potential users, and 2) the adoption by funders of approaches involving an increasing range of responsibilities and activities. The Summit website (https://inthetrenchessummit.com/) will be periodically updated with new resources and information about future In the Trenches events.

How do organisations implement research impact assessment (RIA) principles and good practice? A narrative review and exploratory study of four international research funding and administrative organisations

Background

Public research funding agencies and research organisations are increasingly accountable for the wider impacts of the research they support. While research impact assessment (RIA) frameworks and tools exist, little is known and shared of how these organisations implement RIA activities in practice.

Methods

We conducted a review of academic literature to search for research organisations’ published experiences of RIAs. We followed this with semi-structured interviews from a convenience sample (n = 7) of representatives of four research organisations deploying strategies to support and assess research impact.

Results

We found only five studies reporting empirical evidence on how research organisations put RIA principles into practice. From our interviews, we observed a disconnect between published RIA frameworks and tools, and the realities of organisational practices, which tended not to be reported.

We observed varying maturity and readiness with respect to organisations’ structural set ups for conducting RIAs, particularly relating to leadership, skills for evaluation and automating RIA data collection. Key processes for RIA included efforts to engage researcher communities to articulate and plan for impact, using a diversity of methods, frameworks and indicators, and supporting a learning approach. We observed outcomes of RIAs as having supported a dialogue to orient research to impact, underpinned shared learning from analyses of research, and provided evidence of the value of research in different domains and to different audiences.

Conclusions

Putting RIA principles and frameworks into practice is still in early stages for research organisations. We recommend that organisations (1) get set up by considering upfront the resources, time and leadership required to embed impact strategies throughout the organisation and wider research ‘ecosystem’, and develop methodical approaches to assessing impact; (2) work together by engaging researcher communities and wider stakeholders as a core part of impact pathway planning and subsequent assessment; and (3) recognise the benefits that RIA can bring about as a means to improve mutual understanding of the research process between different actors with an interest in research.