Moving Comparative Effectiveness Research Into Practice: Implementation Science And The Role Of Academic Medicine

Development of a Rubric to Evaluate Implementation Quality of Simulation-Based Courses: A Consensus Study

INTRODUCTION

Simulation-based education is a recognized way of developing medical competencies, and there is overwhelming scientific evidence to support its efficacy. However, it is still underused, which can often be related to poor implementation process. In addition, best practices for implementation of simulation-based courses based on implementation science are not widely known nor applied. The purpose of this study was to develop a rubric, the Implementation Quality Rubric for Simulation (IQR-SIM), to evaluate the implementation quality of simulation-based courses.

METHODS

A 3-round, modified Delphi process involving international simulation and implementation experts was initiated to gather and converge opinions regarding criteria for evaluating the implementation quality of simulation-based courses. Candidate items for Round 1 were developed based on the Adapted Implementation Model for Simulation. Items were revised and expanded to include descriptive anchors for evaluation in Round 2. Criterion for inclusion was 70% of respondents selecting an importance rating of 4 or 5/5. Round 3 provided refinement and final approval of items and anchors.

RESULTS

Thirty-three experts from 9 countries participated. The initial rubric of 32 items was reduced to 18 items after 3 Delphi rounds, resulting in the IQR-SIM: a 3-point rating scale, with nonscored options "Don't know/can't assess" and "Not applicable," and a comments section.

CONCLUSIONS

The IQR-SIM is an operational tool that can be used to evaluate the implementation quality of simulation-based courses and aid in the implementation process to identify gaps, monitor the process, and promote the achievement of desired implementation and learning outcomes.

Variation in Stakeholder Preferences for Implementing Evidence-Based Practices in Behavioral Health Care

Objective

The authors examined whether stakeholders in behavioral health care differ in their preferences for strategies that support the implementation of evidence-based practices (EBPs).

Methods

Using data collected in March and April 2019 in a survey of stakeholders in Philadelphia Medicaid’s behavioral health care system, the authors compared empirical Bayes preference weights for implementation strategies across clinicians, supervisors, agency executives, and payers.

Results

Preferences for implementation strategies overlapped among the stakeholders (N=357 survey respondents). Financial incentives were consistently ranked as most useful and performance feedback as the least useful for implementing EBPs. However, areas of divergence were identified. For example, payers preferred compensation for EBP delivery, whereas clinicians considered compensation for time spent on preparing for EBPs as equally useful.

Conclusions

The observed variation in stakeholder preferences for strategies to implement EBPs may shed light on why the ongoing shift from volume to value in behavioral health care has had mixed results.

Implementing Anti-Racism Interventions in Healthcare Settings: A Scoping Review

Racism towards Black, Indigenous and people of colour continues to exist in the healthcare system. This leads to profound harm for people who use and work within these settings. This is a scoping review to identify anti-racism interventions in outpatient healthcare settings. Searching the peer-reviewed and grey literature, articles were screened for inclusion by at least two independent reviewers. Synthesizing the socio-ecological levels of interventions with inductively identifying themes, a conceptual model for implementing anti-racism interventions in healthcare settings is presented. In total, 37 peer-reviewed articles were included in the review, with 12 empirical studies and 25 theoretical or conceptual papers. Six grey literature documents were also included. Healthcare institutions need to incorporate an explicit, shared language of anti-racism. Anti-racism action should incorporate leadership buy-in and commitment with dedicated resources, support and funding; a multi-level approach beginning with policy and organizational interventions; transparent accountability mechanisms for sustainable change; long-term meaningful partnerships with Black, Indigenous, and people of colour (i.e., racialized communities); and ongoing, mandatory, tailored staff education and training. Decision-makers and staff in healthcare settings have a responsibility to take anti-racism action and may improve the success and sustainability of their efforts by incorporating the foundational principles and strategies identified in this paper.

Querying stakeholders to inform comparative effectiveness research

Despite the growing recognition of the value of stakeholder engagement in research, there is limited guidance on effectively eliciting stakeholder views during the comparative effectiveness research (CER) process. This article outlines the potential role of each stakeholder (patient, provider, policymaker and payer) throughout the CER process and provides examples of practical questions that researchers can ask the four primary stakeholder groups at each step of the CER process. This guide aims to assist in the development of meaningful stakeholder-researcher shared decision-making to incorporate stakeholder views in the design, conduct and dissemination of patient-centered CER.

Building and executing a research agenda toward conducting implementation science in medical education

BACKGROUND

Implementation science (IS) is the study of methods that successfully integrate best evidence into practice. Although typically applied in healthcare settings to improve patient care and subsequent outcomes, IS also has immediate and practical applications to medical education toward improving physician training and educational outcomes. The objective of this article is to illustrate how to build a research agenda that focuses on applying IS principles in medical education.

APPROACH

We examined the literature to construct a rationale for using IS to improve medical education. We then used a generalizable scenario to step through a process for applying IS to improve team-based care.

PERSPECTIVES

IS provides a valuable approach to medical educators and researchers for making improvements in medical education and overcoming institution-based challenges. It encourages medical educators to systematically build upon the research outcomes of others to guide decision-making while evaluating the successes of best practices in individual environments and generate additional research questions and findings.

CONCLUSIONS

IS can act as both a driver and a model for educational research to ensure that best educational practices are easier and faster to implement widely.

Dissemination of an innovative mastery learning curriculum grounded in implementation science principles: a case study

Dissemination of a medical education innovation, such as mastery learning, from a setting where it has been used successfully to a new and different medical education environment is not easy. This article describes the uneven yet successful dissemination of a simulation-based mastery learning (SBML) curriculum on central venous catheter (CVC) insertion for internal medicine and emergency medicine residents across medical education settings. The dissemination program was grounded in implementation science principles. The article begins by describing implementation science which addresses the mechanisms of medical education and health care delivery. The authors then present a mastery learning case study in two phases: (1) the development, implementation, and evaluation of the SBML CVC curriculum at a tertiary care academic medical center; and (2) the dissemination of the SBML CVC curriculum to an academic community hospital setting. Contextual information about the drivers and barriers that affected the SBML CVC curriculum dissemination is presented. This work demonstrates that dissemination of mastery learning curricula, like all other medical education innovations, will fail without active educational leadership, personal contacts, dedication, hard work, rigorous measurement, and attention to implementation science principles. The article concludes by presenting a set of lessons learned about disseminating an SBML CVC curriculum across different medical education settings.

From Research to Practice: Which Research Strategy Contributes More to Clinical Excellence? Comparing High-Volume versus High-Quality Biomedical Research

The question when and to what extent academic research can benefit society is of great interest to policy-makers and the academic community. Physicians in university hospitals represent a highly relevant test-group for studying the link between research and practice because they engage in biomedical academic research while also providing medical care of measurable quality. Physicians' research contribution to medical practice can be driven by either high-volume or high-quality research productivity, as often pursuing one productivity strategy excludes the other. To empirically examine the differential contribution to medical practice of the two strategies, we collected secondary data on departments across three specializations (Cardiology, Oncology and Orthopedics) in 50 U.S.-based university hospitals served by 4,330 physicians. Data on volume and quality of biomedical research at each department was correlated with publicly available ratings of departments' quality of care, demonstrating that high-quality research has significantly greater contribution to quality of care than high-volume research.

A model to support shared decision making in electronic health records systems

Shared decision making (SDM) is an approach to medical care based on collaboration between provider and patient, with both sharing in medical decisions. When patients' values and preferences are incorporated in decision making, care is more appropriate, ethically sound, and often lower in cost. However, SDM is difficult to implement in routine practice because of the time required for SDM methods, the lack of integration of SDM approaches into electronic health record (EHR) systems, and absence of explanatory mechanisms for providers on the results of patients' use of decision aids. This article discusses potential solutions, including the concept of a "personalize button" for EHRs. Leveraging a 4-phase clinical model for SDM, this article describes how computer decision support (CDS) technologies integrated into EHRs can help ensure that health care is delivered in a way that is respectful of those preferences. The architecture described herein, called CDS for SDM, is built on recognized standards that are currently integrated into certification requirements for EHRs as part of meaningful use regulations. While additional work is needed on modeling of preferences and on techniques for rapid communication models of preferences to clinicians, unless EHRs are redesigned to support SDM around and during clinical encounters, they are likely to continue to be an unintended barrier to SDM. With appropriate development, EHRs could be a powerful tool to promote SDM by reminding providers of situations for SDM and monitoring ongoing care to ensure treatments are consistent with patients' preferences.

Evidence-based stillbirth prevention strategies: combining empirical and theoretical paradigms to inform health planning and decision-making

INTRODUCTION

A global health project undertaken in Qatar on the Arabian Peninsula immersed undergraduate nursing students in hands-on learning to address the question: What strategies are effective in preventing stillbirth? Worldwide stillbirth estimates of 2.6 million per year and the high rate in the Eastern Mediterranean Region of 27 per 1,000 total live births provided the stimulus for this inquiry.

METHODS

We used a dual empirical and theoretical approach that combined the principles of evidence-based practice and population health planning. Students were assisted to translate pre-appraised literature based on the 6S hierarchical pyramid of evidence. The PRECEDE-PROCEED (P-P) model served as an organizing template to assemble data extracted from the appraisal of 21 systematic literature reviews ± meta-analyses, 2 synopses of synthesized reports, and 9 individual studies summarizing stillbirth prevention strategies in low, middle, and high income countries. Consistent with elements of the P-P model, stillbirth prevention strategies were classified as social, epidemiological, educational, ecological, administrative, or policy.

RESULTS

Ten recommendations with clear evidence of effectiveness in preventing stillbirth in low, middle, or high income countries were identified. Several other promising interventions were identified with weak, uncertain, or inconclusive evidence. These require further rigorous testing.

LINKING EVIDENCE TO ACTION

Two complementary paradigms--evidence-based practice and an ecological population health program planning model--helped baccalaureate nursing students transfer research evidence into useable knowledge for practice. They learned the importance of comprehensive assessments and evidence-informed interventions. The multidimensional elements of the P-P model sensitized students to the complex interrelated factors influencing stillbirth and its prevention.

Dissemination and implementation of comparative effectiveness evidence: key informant interviews with Clinical and Translational Science Award institutions

AIM

To identify ongoing practices and opportunities for improving national comparative effectiveness research (CER) translation through dissemination and implementation (D&I) via NIH-funded Clinical and Translational Science Award (CTSA) institutions.

MATERIALS & METHODS

Key informant interviews were conducted with 18 CTSA grantees sampled to represent a range of D&I efforts.

RESULTS & CONCLUSIONS

The institutional representatives endorsed fostering CER translation nationally via the CTSA Consortium. However, five themes emerged from the interviews as barriers to CER D&I: lack of institutional awareness, insufficient capacity, lack of established D&I methods, confusion among stakeholders about what CER actually is and limited funding opportunities. Interviewees offered two key recommendations to improve CER translation: development of a centralized clearing house to facilitate the diffusion of CER D&I resources and methods across CTSA institutions; and formalization of the national CTSA network to leverage existing community engagement relationships and resources for the purpose of adapting and disseminating robust CER evidence locally with providers, patients and healthcare systems.

Optimizing health for complex adults in primary care: current challenges and a way forward

As the population ages, the quantity and complexity of comorbidities only increases in the primary care setting. Health systems strive to improve quality of care and enhance cost savings, but current administrative and payment systems do not easily support the implementation of existing evidence and best practices for multimorbid adults in most primary care offices. This perspectives piece sets forth a research agenda in the area of implementation science at the intersection of geriatrics and general internal medicine. We challenge academic medical centers, medical societies, journals, and funders to actively value and support investigation in this area as much as traditional research pathways.

A critical review of simulation-based mastery learning with translational outcomes

OBJECTIVES

This article has two objectives. Firstly, we critically review simulation-based mastery learning (SBML) research in medical education, evaluate its implementation and immediate results, and document measured downstream translational outcomes in terms of improved patient care practices, better patient outcomes and collateral effects. Secondly, we briefly address implementation science and its importance in the dissemination of innovations in medical education and health care.

METHODS

This is a qualitative synthesis of SBML with translational (T) science research reports spanning a period of 7 years (2006-2013). We use the 'critical review' approach proposed by Norman and Eva to synthesise findings from 23 medical education studies that employ the mastery learning model and measure downstream translational outcomes.

RESULTS

Research in SBML in medical education has addressed a range of interpersonal and technical skills. Measured outcomes have been achieved in educational laboratories (T1), and as improved patient care practices (T2), patient outcomes (T3) and collateral effects (T4).

CONCLUSIONS

Simulation-based mastery learning in medical education can produce downstream results. Such results derive from integrated education and health services research programmes that are thematic, sustained and cumulative. The new discipline of implementation science holds promise to explain why medical education innovations are adopted slowly and how to accelerate innovation dissemination.

Engaging hard-to-reach patients in patient-centered outcomes research

Aims: This study aimed to identify methods to engage hard-to-reach patients in the research process. Materials & methods: With funding from the Patient-Centered Outcomes Research Institute (Washington, DC, USA), the University of Maryland (MD, USA) conducted 20 focus groups and one individual interview. The sample consisted of six groups of hard-to-reach patients, two groups of healthcare providers who work with hard-to-reach patients and two groups of surrogates of hard-to-reach patients. Results & conclusion: In order to make patient-centered outcomes research more meaningful to patients and their caregivers, patient-centered outcomes research should be conducted with a focus on building and maintaining trust, which is achieved via pre-engagement with communities and continuous engagement of study participants and their communities.

Reducing quality-of-care disparities in childhood asthma: La Red de Asma Infantil intervention in San Juan, Puerto Rico

BACKGROUND AND OBJECTIVE

Although children living in Puerto Rico have the highest asthma prevalence of all US children, little is known regarding the quality-of-care disparities they experience nor the adaptability of existing asthma evidence-based interventions to reduce these disparities. The objective of this study was to describe our experience in reducing quality-of-care disparities among Puerto Rican children with asthma by adapting 2 existing evidence-based asthma interventions.

METHODS

We describe our experience in adapting and implementing 2 previously tested asthma evidence-based interventions: the Yes We Can program and the Inner-City Asthma Study intervention. We assessed the feasibility of combining key components of the 2 interventions to reduce asthma symptoms and estimated the potential cost savings associated with reductions in asthma-related hospitalizations and emergency department visits. A total of 117 children with moderate and severe asthma participated in the 12-month intervention in 2 housing projects in San Juan, Puerto Rico. A community-academic team with the necessary technical and cultural competences adapted and implemented the intervention.

RESULTS

Our case study revealed the feasibility of implementing the combined intervention, henceforth referred to as La Red intervention, in the selected Puerto Rican communities experiencing a disproportionately high level of asthma burden. After 1-year follow-up, La Red intervention significantly reduced asthma symptoms and exceeded reductions of the original interventions. Asthma-related hospitalizations and emergency department use, and their associated high costs, were also significantly reduced.

CONCLUSIONS

Asthma evidence-based interventions can be adapted to improve quality of care for children with asthma in a different cultural community setting.

Translational educational research: a necessity for effective health-care improvement

Medical education research contributes to translational science (TS) when its outcomes not only impact educational settings, but also downstream results, including better patient-care practices and improved patient outcomes. Simulation-based medical education (SBME) has demonstrated its role in achieving such distal results. Effective TS also encompasses implementation science, the science of health-care delivery. Educational, clinical, quality, and safety goals can only be achieved by thematic, sustained, and cumulative research programs, not isolated studies. Components of an SBME TS research program include motivated learners, curriculum grounded in evidence-based learning theory, educational resources, evaluation of downstream results, a productive research team, rigorous research methods, research resources, and health-care system acceptance and implementation. National research priorities are served from translational educational research. National funding priorities should endorse the contribution and value of translational education research.

Comparative effectiveness research: nursing science and health care delivery

Comparative effectiveness research (CER) is an important foundation in the development of scientific evidence that can assist patients, clinicians, and policy makers in making decisions that improve patient and system outcomes, including cost. CER is a part of what is now called dissemination and implementation science, which focuses on translating knowledge into practice by facilitating stakeholder access to more interpretable findings. CER has evolved from a rich history that aims to compare the effectiveness of select clinical treatments, approaches, or programs. This article describes the development of CER, approaches to designing and analyzing this research, and resources useful in generating such knowledge by nurse scientists.

Differences in Perceptions of Patient Safety Culture between Charge and Noncharge Nurses: Implications for Effectiveness Outcomes Research

The implementation of evidence-based practice guidelines can be influenced by nurses' perceptions of the organizational safety culture. Shift-by-shift management of each nursing unit is designated to a subset of staff nurses (charge nurses), whom are often recruited as champions for change. The findings indicate that compared to charge nurses, noncharge nurses were more positive about overall perceptions of safety (P = .05) and teamwork (P < .05). Among charge nurses, significant differences were observed based on the number of years' experience in charge: perception of teamwork within units [F(3, 365) = 3.52, P < .01]; overall perceptions of safety, [F(3, 365) = 4.20, P < .05]; safety grade for work area [F(3, 360) = 2.61, P < .05]; number of events reported within the last month [F(3, 362) = 3.49, P < .05]. These findings provide important insights to organizational contextual factors that may impact effectiveness outcomes research in the future.

Getting to value in neurological care: a roadmap for academic neurology

Academic neurology is undergoing transformational changes. The public investment in biomedical research and clinical care is enormous and there is a growing perception that the return on this huge investment is insufficient. Hospitals, departments, and individual neurologists should expect more scrutiny as information about their quality of care and financial relationships with industry are increasingly reported to the public. There are unprecedented changes occurring in the financing and delivery of health care and research that will have profound impact on the mission and operation of academic departments of neurology. With the passage of the Patient Protection and Affordable Care Act (PPACA) there will be increasing emphasis on research that demonstrates value and includes the patient's perspective. Here we review neurological investigations of our clinical and research enterprises that focus on quality of care and comparative effectiveness, including cost-effectiveness. By highlighting progress made and the challenges that lie ahead, we hope to create a clinical, educational, and research roadmap for academic departments of neurology to thrive in today's increasingly regulated environment.

The implications of comparative effectiveness research for academic medicine

With growing constraints on government spending, policy makers are investing in comparative effectiveness research (CER) to attempt to bring the power of science to bear on the problems of suboptimal outcomes and high cost in the U.S. health care system. This commitment of resources to CER reflects confidence that better evidence can help clinicians and patients make better decisions, consistent with the long tradition of medical schools' and teaching hospitals' use of science to inform medical care. Thus, CER offers a great opportunity, albeit with some considerable challenges, for academic medicine to play a central role in comprehensive health care reform. Certainly, many scientists conducting CER will learn their methodological rigor in the training programs of academic health centers. Numerous new CER research teams will be needed, establishing effective partnerships far outside the walls of the traditional academic setting. And the clinicians interpreting the medical literature and applying the insights from CER to the unique problems of individual patients will need to learn this evidence-based, patient-centered care from the educators, mentors, and role models at U.S. medical and other health science schools and teaching hospitals. Achieving this will require investment in research infrastructure, adaptations of institutional culture, development of new disciplines and research methods, establishment of new collaborations, training of new faculty, and the expansion and refocusing of educational capacity. By successfully responding to this challenge, academic medicine can further strengthen its long-standing commitment to the scientific practice of medicine and the use of evidence in patient-centered, personalized care.