Developing an instrument for evaluating implementation of clinical practice guidelines: a test-retest study

Action research improved general prerequisites for evidence-based practice

The present study was part of an action research project that was performed to implement a clinical pathway for patients on mechanical ventilation and simultaneously explore the implementation process in a Swedish intensive care unit. The aim of this questionnaire study was to evaluate whether an action research methodology could affect the general prerequisites for evidence-based practice (EBP). Informed by the Promoting Action on Research Implementation in Health Services (PARIHS) framework, the study included registered nurses, assistant nurses and anesthesiologists in the unit at start of the project (n = 50) and at follow-up (n = 44). Data was collected with the Evaluation Before Implementation Questionnaire and the Attitudes towards Guidelines Scale.

The results revealed that the general prerequisites for EBP in the setting improved. Compared to baseline measurements, the staff at follow-up conversed significantly more about the importance of the patients’ experiences, research utilization, context and facilitation, while changes with respect to clinical experiences were not significant. The attitudes towards guidelines were perceived as positive at baseline as well as at follow-up and did not significantly change.

Longer professional experience was associated with a slightly lower probability of perceiving that the importance of research utilization was discussed and reflected upon, while belonging to a profession with longer education was associated with a higher probability of this perception. Compared to registered nurses and assistant nurses, the anesthesiologists perceived, to a greater extent, that the importance of clinical experience was discussed and reflected upon in the setting, while there was no significant association with the length of professional experience and/or specific professions regarding the other components.

In conclusion, using action research to implement a clinical pathway methodology seems to set in motion various mechanisms that improve some but not all prerequisites that, according to the PARIHS framework, are advantageous for EBP.

The use of the PARIHS framework in implementation research and practice-a citation analysis of the literature

BACKGROUND

The Promoting Action on Research Implementation in Health Services (PARIHS) framework was developed two decades ago and conceptualizes successful implementation (SI) as a function (f) of the evidence (E) nature and type, context (C) quality, and the facilitation (F), [SI = f (E,C,F)]. Despite a growing number of citations of theoretical frameworks including PARIHS, details of how theoretical frameworks are used remains largely unknown. This review aimed to enhance the understanding of the breadth and depth of the use of the PARIHS framework.

METHODS

This citation analysis commenced from four core articles representing the key stages of the framework's development. The citation search was performed in Web of Science and Scopus. After exclusion, we undertook an initial assessment aimed to identify articles using PARIHS and not only referencing any of the core articles. To assess this, all articles were read in full. Further data extraction included capturing information about where (country/countries and setting/s) PARIHS had been used, as well as categorizing how the framework was applied. Also, strengths and weaknesses, as well as efforts to validate the framework, were explored in detail.

RESULTS

The citation search yielded 1613 articles. After applying exclusion criteria, 1475 articles were read in full, and the initial assessment yielded a total of 367 articles reported to have used the PARIHS framework. These articles were included for data extraction. The framework had been used in a variety of settings and in both high-, middle-, and low-income countries. With regard to types of use, 32% used PARIHS in planning and delivering an intervention, 50% in data analysis, 55% in the evaluation of study findings, and/or 37% in any other way. Further analysis showed that its actual application was frequently partial and generally not well elaborated.

CONCLUSIONS

In line with previous citation analysis of the use of theoretical frameworks in implementation science, we also found a rather superficial description of the use of PARIHS. Thus, we propose the development and adoption of reporting guidelines on how framework(s) are used in implementation studies, with the expectation that this will enhance the maturity of implementation science.

Perceived enabling factors and barriers for the implementation of improvements in health care in order to achieve patient-centred care: A case report from Sweden

RATIONALE, AIM, AND OBJECTIVES

More knowledge is needed regarding the complex factors and perceptions that enable the implementation of change in health care. The study aimed to examine the enabling factors and barriers encountered in the implementation of improvements in health care in order to achieve patient-centred care (PCC) and to study if there was a correlation in the extent the improvements were perceived to be implemented and the preconditions that were considered to affect them.

METHODS

Using a mixed method design, data were gathered via a questionnaire and individual interviews with health care personnel, clinic managers, and first-line managers. The data collection and analyses were based on the framework for Promoting Action on Research Implementation in Health Services (PARiHS). Correlations between PCC improvements and preconditions for improvements were performed.

RESULTS

A high level of involvement, knowledge, and adequate resources were considered important to achieve an implementation of PCC with joint responsibility. Leadership and management need to be explicit and promote continuous follow-up and feedback. Preconditions for improvement had a linear correlation with the perceived level of implementation. Knowledge-related preconditions had greatest impact on implementation.

CONCLUSIONS

The PARiHS framework was appropriate to use since the three components of evidence, context, and facilitation present different important preconditions in the implementation process. Evidence was the highest rated contributor since evidence-based practices in health care are necessary. It is vital that the important role of the context and facilitators is acknowledged in the implementation process to enable a successful implementation of change. There is a need to incorporate a clear strategy involving all levels in the organization. Furthermore, leaders play an important role in the implementation by facilitating communication and support and by having trust in facilitators and health care personnel. The results are applicable to other interventions implementing change in health care.

Improved postpartum care after a participatory facilitation intervention in Dar es Salaam, Tanzania: a mixed method evaluation

Background: In order to improve the health and survival of mothers/newborns, the quality and attendance rates of postpartum care (PPC) must be increased, particularly in low-resource settings.

Objective: To describe outcomes of a collegial facilitation intervention to improve PPC in government-owned health institutions in a low-resource suburb in Dar es Salaam, Tanzania.

Methods: A before-and-after evaluation of an intervention and comparison group was conducted using mixed methods (focus group discussions, questionnaires, observations, interviews, and field-notes) at health institutions. Maternal and child health aiders, enrolled nurse midwives, registered nurse midwives, and medical and clinical officers participated. A collegial facilitation intervention was conducted and healthcare providers were organized in teams to improve PPC at their workplaces. Facilitators defined areas of improvement with colleagues and met regularly with a supervisor for support.

Results: The number of mothers visiting the institution for PPC increased in the intervention group. Some care actions were noted in more than 80% of the observations and mothers reported high satisfaction with care. In the comparison group, PPC continued to be next to non-existent. The healthcare providers’ knowledge increased in both groups but was higher in the intervention group. The t-test showed a significant difference in knowledge between the intervention and comparison groups and between before and after the intervention in both groups. The difference of differences for knowledge was 1.3. The providers perceived the intervention outcomes to include growing professional confidence/knowledge, improved PPC quality, and mothers’ positive response. The quality grading was based on the national guidelines and involved nine experts and showed that none of the providers reached the level of good quality of care.

Conclusions: The participatory facilitation intervention contributed to improved quality of PPC, healthcare providers’ knowledge and professional confidence, awareness of PPC among mothers, and increased PPC attendance.

Developing a workbook to support the contextualisation of global health systems guidance: a case study identifying steps and critical factors for success in this process at WHO

BACKGROUND

Global guidance can help countries strengthen their health systems to deliver effective interventions to their populations. However, to have an impact, guidance needs to be contextualised or adapted to local settings; this process includes consideration of health system arrangements and political system factors. To date, methods to support contextualisation do not exist. In response, a workbook was designed to provide specific methods and strategies to enable the contextualisation of WHO's 'Optimizing health worker roles to improve maternal and newborn health' (OptimizeMNH) guidance at the national or subnational level. The objective of this study was to describe the process of developing the workbook and identify key steps of the development process, barriers that arose and facilitators that helped overcome some of these barriers.

METHODS

A qualitative single case study design was carried out. Interviews, documents and a reflexive journal were used. Constant comparison and an edit-style of organisation were used during data analysis to develop concepts, themes, subthemes and relationships among them.

RESULTS

Thirteen interviews were conducted and 52 documents were reviewed. Three main steps were identified in the process of developing the workbook for health systems guidance contextualisation, namely (1) determining the need for and gaining approval to develop the workbook, (2) developing the workbook (taking on the task, creating the structure of the workbook, operationalising its components, undergoing approval processes and editing it), and (3) implementing the workbook both at the WHO level and at the national/subnational level. Five barriers and/or facilitators emerged relevant to each step, namely (1) having well-placed and credible champions, (2) creating and capitalising on opportunities, (3) finding the right language to engage various actors and obtain buy-in, (4) obtaining and maintaining meaningful buy-in, and (5) ensuring access to resources.

CONCLUSIONS

Understanding the key steps and the critical factors involved in the process of developing the workbook could help in the planning of similar and other tools aimed to support the implementation of WHO guidance. A plan for dissemination and implementation needs to be addressed during the preparation of these tools.

Development and psychometric validation of a questionnaire to evaluate nurses' adherence to recommendations for preventing pressure ulcers (QARPPU)

AIM OF THE STUDY

The main objective of this work is the development and psychometric validation of an instrument to evaluate nurses' adherence to the main recommendations issued for preventing pressure ulcers.

MATERIAL AND METHODS

An instrument was designed based on the main recommendations for the prevention of pressure ulcers published in various clinical practice guides. Subsequently, it was proceeded to evaluate the face and content validity of the instrument by an expert group. It has been applied to 249 Spanish nurses took part in a cross-sectional study to obtain a psychometric evaluation (reliability and construct validity) of the instrument. The study data were compiled from June 2015 to July 2016.

RESULTS

From the results of the psychometric analysis, a final 18-item, 4-factor questionnaire was derived, which explained 60.5% of the variance and presented the following optimal indices of fit (CMIN/DF: 1.40 p < 0.001; GFI: 0.93; NFI: 0.92; CFI: 0.98; TLI: 0.97; RMSEA: 0.04 (90% CI 0.025-0.054).

CONCLUSIONS

The results obtained show that the instrument presents suitable psychometric properties for evaluating nurses' adherence to recommendations for the prevention of pressure ulcers.

To Assess Prerequisites Before an Implementation Strategy in an Orthopaedic Department in Sweden

Promoting Action on Research Implementation in Health Services (PARiHS) asserts that the success of knowledge implementation relates to multiple factors in a complex and dynamic way, and therefore the effects of implementation strategies vary by method and context. An instrument based on the PARiHS framework was developed to help assess critical factors influencing implementation strategies so that strategies can be tailored to promote implementation.The purpose of this study was to use the Evaluation Before Implementation Questionnaire (EBIQ), to describe staff perceptions in one orthopaedic department, and to investigate differences between wards.Staff members in four different wards at one orthopaedic department at a university hospital in Sweden were invited to complete a questionnaire related to planning for the implementation of a clinical practice guideline. The 23 items in the EBIQ were expected to capture staff perceptions about the evidence, context, and facilitation factors that influence the implementation process. Descriptive statistics and differences between wards were analyzed. Although the overall response rate was low (n = 49), two of the four wards accounted for most of the completed questionnaires (n = 25 and n = 12, respectively), enabling a comparison of these wards. We found significant differences between respondents' perceptions at the two wards in six items regarding context and facilitation in terms of receptiveness to change, forms of leadership, and evaluation and presence of feedback and facilitators.The EBIQ instrument requires further testing, but there appears to be initial support for pre-implementation use of the EBIQ as a means to enhance planning for implementation.

"If really we are committed things can change, starting from us": Healthcare providers' perceptions of postpartum care and its potential for improvement in low-income suburbs in Dar es Salaam, Tanzania

OBJECTIVE

To explore healthcare providers' perceptions of the current postpartum care (PPC) practice and its potential for improvement at governmental health institutions in low-resource suburbs in Dar es Salaam, Tanzania.

DESIGN

Qualitative design, using focus group discussions (8) and qualitative content analysis.

SETTING

Healthcare institutions (8) at three levels of governmental healthcare in Ilala and Temeke suburbs, Dar es Salaam.

PARTICIPANTS

Registered, enrolled and trained nurse-midwives (42); and medical and clinical officers (13).

RESULTS

The healthcare providers perceived that PPC was suboptimal and that they could have prevented maternal deaths. PPC was fragmented at understaffed institutions, lacked guidelines and was organized in a top-down structure of leadership. The participants called for improvement of: organization of space, time, resources, communication and referral system; providers' knowledge; and supervision and feedback. Their motivation to enhance PPC quality was high.

KEY CONCLUSIONS

The HCP awareness of the suboptimal quality of PPC, its potential for promoting health and their willingness to engage in improving care are promising for the implementation of interventions to improve quality of care. Provision of guidelines, sensitization of providers to innovate and maximize utilization of existing resources, and supportive supervision and feedback are likely to contribute to the sustainability of any improvement.

Development and Initial Validation of a New Practice Context Assessment Tool for Ambulatory Practices Engaged in Quality Improvement

Variable success with quality improvement (QI) efforts in ambulatory care is often attributed to differences in local contexts. Identifying and addressing patient-, practice-, or community-level contextual factors might improve implementation of QI projects. The authors developed and validated a framework for a Practice Context Assessment (PCA), and then created the PCA instrument to glean insights from staff on contextual factors and distributed it at 10 ambulatory practice sites. The PCA framework showed acceptable expert-assessed content validity, with content validity index scores ranging from 0.74 (community engagement) to 0.97 (leadership). The PCA instrument comprised several scales grouped into 7 domains with Cronbach α scores from 0.83 (leadership) to 0.95 (patient and family engagement). The PCA framework provides a valid construct to help ambulatory practices understand contextual issues that might influence QI projects. A revised version of the PCA instrument is now ready for further testing.

Response process and test-retest reliability of the Context Assessment for Community Health tool in Vietnam

BACKGROUND

The recently developed Context Assessment for Community Health (COACH) tool aims to measure aspects of the local healthcare context perceived to influence knowledge translation in low- and middle-income countries. The tool measures eight dimensions (organizational resources, community engagement, monitoring services for action, sources of knowledge, commitment to work, work culture, leadership, and informal payment) through 49 items.

OBJECTIVE

The study aimed to explore the understanding and stability of the COACH tool among health providers in Vietnam.

DESIGNS

To investigate the response process, think-aloud interviews were undertaken with five community health workers, six nurses and midwives, and five physicians. Identified problems were classified according to Conrad and Blair's taxonomy and grouped according to an estimation of the magnitude of the problem's effect on the response data. Further, the stability of the tool was examined using a test-retest survey among 77 respondents. The reliability was analyzed for items (intraclass correlation coefficient (ICC) and percent agreement) and dimensions (ICC and Bland-Altman plots).

RESULTS

In general, the think-aloud interviews revealed that the COACH tool was perceived as clear, well organized, and easy to answer. Most items were understood as intended. However, seven prominent problems in the items were identified and the content of three dimensions was perceived to be of a sensitive nature. In the test-retest survey, two-thirds of the items and seven of eight dimensions were found to have an ICC agreement ranging from moderate to substantial (0.5-0.7), demonstrating that the instrument has an acceptable level of stability.

CONCLUSIONS

This study provides evidence that the Vietnamese translation of the COACH tool is generally perceived to be clear and easy to understand and has acceptable stability. There is, however, a need to rephrase and add generic examples to clarify some items and to further review items with low ICC.

Exploring How Knowledge Translation Can Improve Sustainability of Community-based Health Initiatives for People with Intellectual/Developmental Disabilities

Community-based health initiatives (CBHI) play an important role in maintaining the health, function and participation of people with intellectual/developmental disabilities (I/DD) living in the community. However, implementation and long-term sustainability of CBHI is challenging. The Promoting Action on Research Implementation in Health Services (PARiHS) is a knowledge translation (KT) framework that is particularly relevant to intellectual/developmental disabilities research as it identifies the barriers and facilitators of implementation and action plans. This framework provides a foundation for understanding how KT can be used to aid the implementation and sustainability of CBHI for people with intellectual/developmental disabilities. The following study explores how KT - specifically the PARiHS framework - can be used to help sustain CBHI for people with intellectual/developmental disabilities.

Validity and reliability of the "German Utilization Questionnaire-Dissemination and Use of Research" to measure attitude, availability, and support toward implementation of research in nursing practice

BACKGROUND AND PURPOSE

In nursing practice, research results have to undergo a systematic process of transformation. Currently in Austria, there is no empirical data available concerning the actual implementation of research results. An English validated questionnaire was translated into German and tested for validity and reliability.

METHOD

A survey of 178 registered nurses (n = 178) was conducted in a multicenter, quantitative, cross-sectional study in Austria in 2011.

RESULTS

Cronbach's alpha values (.82-.92) were calculated for 4 variables ("use," "attitude," "availability," "support") after the reduction of 7 irrelevant items. Exploratory factor analysis was calculated with Kaiser-Meyer-Olkin (KMO) ranging from .78 to .92; the total variance ranged from 46% to 56%.

CONCLUSION

A validated German questionnaire concerning the implementation of research results is now available for the nursing practice.

Access to care for vulnerable veterans with hepatitis C: a hybrid conceptual framework and a case study to guide translation

The Veterans Health Administration (VHA) is the largest single provider of medical care to people with hepatitis C (HCV) in the USA. Given the advent of promising new HCV therapies, the VHA is now faced with a large number of chronically HCV-infected veterans with concomitant psychiatric or substance use comorbid conditions who will need to either be retreated or newly treated for HCV or will require management for chronic liver disease. There is a critical need in the VHA for behavioral medicine and hepatology specialists, along with infectious disease and primary care providers with an interest in hepatitis C, to provide coordinated care for these complex patients. The VHA Health Services Research and Development Service has advocated for the application of strong implementation science theories and methods to translate new models of healthcare delivery in clinical practice. To inform the delivery and evaluation of integrated behavioral medicine and specialty care for vulnerable patient populations, we sought to develop an enriched framework which incorporates implementation science theory and strong conceptual models for access to care. In this paper, we present a hybrid conceptual framework that accomplishes this goal. To illustrate how this hybrid model could inform the translation of a novel method of healthcare delivery, we provide a case study of a VHA initiative to improve access to integrated behavioral medicine and specialty care among veterans with HCV.

Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures

BACKGROUND

Two of the current methodological barriers to implementation science efforts are the lack of agreement regarding constructs hypothesized to affect implementation success and identifiable measures of these constructs. In order to address these gaps, the main goals of this paper were to identify a multi-level framework that captures the predominant factors that impact implementation outcomes, conduct a systematic review of available measures assessing constructs subsumed within these primary factors, and determine the criterion validity of these measures in the search articles.

METHOD

We conducted a systematic literature review to identify articles reporting the use or development of measures designed to assess constructs that predict the implementation of evidence-based health innovations. Articles published through 12 August 2012 were identified through MEDLINE, CINAHL, PsycINFO and the journal Implementation Science. We then utilized a modified five-factor framework in order to code whether each measure contained items that assess constructs representing structural, organizational, provider, patient, and innovation level factors. Further, we coded the criterion validity of each measure within the search articles obtained.

RESULTS

Our review identified 62 measures. Results indicate that organization, provider, and innovation-level constructs have the greatest number of measures available for use, whereas structural and patient-level constructs have the least. Additionally, relatively few measures demonstrated criterion validity, or reliable association with an implementation outcome (e.g., fidelity).

DISCUSSION

In light of these findings, our discussion centers on strategies that researchers can utilize in order to identify, adapt, and improve extant measures for use in their own implementation research. In total, our literature review and resulting measures compendium increases the capacity of researchers to conceptualize and measure implementation-related constructs in their ongoing and future research.

Swedish translation, adaptation and psychometric evaluation of the Context Assessment Index (CAI)

BACKGROUND

The strength of and relationship between the fundamental elements context, evidence and facilitation of the PARIHS framework are proposed to be key for successful implementation of evidence into healthcare practice. A better understanding of the presence and strength of contextual factors is assumed to enhance the opportunities of adequately developing an implementation strategy for a specific setting. A tool for assessing context-The Context Assessment Index (CAI)-was developed and published 2009. A Swedish version of the instrument was developed and evaluated among registered nurses. This work forms the focus of this paper.

PURPOSE

The purpose of this study was to translate the CAI into Swedish, adapt the instrument for use in Swedish healthcare practice and assess its psychometric properties.

METHODS

The instrument was translated and back-translated to English. The feasibility of items and response scales were evaluated through think aloud interviews with clinically active nurses. Psychometric properties were evaluated in a sample of registered nurses (n = 373) working in a variety of healthcare organisations in the Stockholm area. Item and factor analyses and Cronbach's alpha were computed to evaluate internal structure and internal consistency.

RESULT

Sixteen items were modified based on the think aloud interviews and to adapt the instrument for use in acute care. A ceiling effect was observed for many items and the originally identified 37 item five-factor model was not confirmed. Item analyses showed an overlap between factors and indicated a one-dimensional scale.

DISCUSSION

The Swedish version of the CAI has a wider application than the original instrument. This might have contributed to the differences in factor structure. Different opportunities for further development of the scale are discussed.

CONCLUSIONS

Further evaluation of the psychometric properties of the CAI is required.

FIRE (Facilitating Implementation of Research Evidence): a study protocol

BACKGROUND

Research evidence underpins best practice, but is not always used in healthcare. The Promoting Action on Research Implementation in Health Services (PARIHS) framework suggests that the nature of evidence, the context in which it is used, and whether those trying to use evidence are helped (or facilitated) affect the use of evidence. Urinary incontinence has a major effect on quality of life of older people, has a high prevalence, and is a key priority within European health and social care policy. Improving continence care has the potential to improve the quality of life for older people and reduce the costs associated with providing incontinence aids.

OBJECTIVES

This study aims to advance understanding about the contribution facilitation can make to implementing research findings into practice via: extending current knowledge of facilitation as a process for translating research evidence into practice; evaluating the feasibility, effectiveness, and cost-effectiveness of two different models of facilitation in promoting the uptake of research evidence on continence management; assessing the impact of contextual factors on the processes and outcomes of implementation; and implementing a pro-active knowledge transfer and dissemination strategy to diffuse study findings to a wide policy and practice community.

SETTING AND SAMPLE

Four European countries, each with six long-term nursing care sites (total 24 sites) for people aged 60 years and over with documented urinary incontinence

METHODS AND DESIGN

Pragmatic randomised controlled trial with three arms (standard dissemination and two different programmes of facilitation), with embedded process and economic evaluation. The primary outcome is compliance with the continence recommendations. Secondary outcomes include proportion of residents with incontinence, incidence of incontinence-related dermatitis, urinary tract infections, and quality of life. Outcomes are assessed at baseline, then at 6, 12, 18, and 24 months after the start of the facilitation interventions. Detailed contextual and process data are collected throughout, using interviews with staff, residents and next of kin, observations, assessment of context using the Alberta Context Tool, and documentary evidence. A realistic evaluation framework is used to develop explanatory theory about what works for whom in what circumstances.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN11598502.

Barriers and enablers to implementation of a New Zealand-wide guideline for assessment and management of cardiovascular risk in primary health care: a template analysis

AIM

The aim of this study was to identify the enablers and barriers to guideline implementation in a primary healthcare setting by employing the Promoting Action on Research Implementation in Health Services (PARIHS) framework as a template for data analysis and interpretation.

BACKGROUND

The use of clinical practice guidelines is pivotal to improving health outcomes. However, the implementation of guidelines into practice is complex, unpredictable, and, in spite of much investigation, remains resistant to explanation of what works and why. Optimising the enablers and minimising the barriers to implementation of a guideline for reducing cardiovascular disease risk has the potential to significantly reduce the burden of disease. The PARIHS framework has been successfully applied in a number of clinical contexts and has been found useful in illuminating the barriers and enablers to evidence implementation.

APPROACH

This qualitative study involved focus groups with 20 primary healthcare nurses, 4 general practitioners, 5 managers, and individual interviews with 3 funder/planners who discussed their contribution to the use of a guideline for the assessment and management of cardiovascular risk. Template analysis based on the PARIHS framework was applied to semi-structured narrative data to provide an in-depth analysis of the barriers and enablers to implementation of the guideline.

CONCLUSIONS

The lack of facilitation of the guideline into practice was a major barrier to implementation. Implementation plans that address the concerns and complexities of everyday practice are an essential aspect of guideline development. The PARIHS framework was found to be comprehensive and accommodating of the complexity of everyday practice associated with guideline implementation in primary health care. The pertinence of the framework confirms its usefulness as a tool to guide implementation.

A Guide for applying a revised version of the PARIHS framework for implementation

Background

Based on a critical synthesis of literature on use of the Promoting Action on Research Implementation in Health Services (PARIHS) framework, revisions and a companion Guide were developed by a group of researchers independent of the original PARIHS team. The purpose of the Guide is to enhance and optimize efforts of researchers using PARIHS in implementation trials and evaluations.

Methods

Authors used a planned, structured process to organize and synthesize critiques, discussions, and potential recommendations for refinements of the PARIHS framework arising from a systematic review. Using a templated form, each author independently recorded key components for each reviewed paper; that is, study definitions, perceived strengths/limitations of PARIHS, other observations regarding key issues and recommendations regarding needed refinements. After reaching consensus on these key components, the authors summarized the information and developed the Guide.

Results

A number of revisions, perceived as consistent with the PARIHS framework's general nature and intent, are proposed. The related Guide is composed of a set of reference tools, provided in Additional files. Its core content is built upon the basic elements of PARIHS and current implementation science.

Conclusions

We invite researchers using PARIHS for targeted evidence-based practice (EBP) implementations with a strong task-orientation to use this Guide as a companion and to apply the revised framework prospectively and comprehensively. Researchers also are encouraged to evaluate its use relative to perceived strengths and issues. Such evaluations and critical reflections regarding PARIHS and our Guide could thereby promote the framework's continued evolution.

Predicting implementation from organizational readiness for change: a study protocol

Background

There is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment.

Objectives

Our objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias.

Methods and Design

We will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique.

Discussion

We propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs.

A critical synthesis of literature on the promoting action on research implementation in health services (PARIHS) framework

Background

The Promoting Action on Research Implementation in Health Services framework, or PARIHS, is a conceptual framework that posits key, interacting elements that influence successful implementation of evidence-based practices. It has been widely cited and used as the basis for empirical work; however, there has not yet been a literature review to examine how the framework has been used in implementation projects and research. The purpose of the present article was to critically review and synthesize the literature on PARIHS to understand how it has been used and operationalized, and to highlight its strengths and limitations.

Methods

We conducted a qualitative, critical synthesis of peer-reviewed PARIHS literature published through March 2009. We synthesized findings through a three-step process using semi-structured data abstraction tools and group consensus.

Results

Twenty-four articles met our inclusion criteria: six core concept articles from original PARIHS authors, and eighteen empirical articles ranging from case reports to quantitative studies. Empirical articles generally used PARIHS as an organizing framework for analyses. No studies used PARIHS prospectively to design implementation strategies, and there was generally a lack of detail about how variables were measured or mapped, or how conclusions were derived. Several studies used findings to comment on the framework in ways that could help refine or validate it. The primary issue identified with the framework was a need for greater conceptual clarity regarding the definition of sub-elements and the nature of dynamic relationships. Strengths identified included its flexibility, intuitive appeal, explicit acknowledgement of the outcome of 'successful implementation,' and a more expansive view of what can and should constitute 'evidence.'

Conclusions

While we found studies reporting empirical support for PARIHS, the single greatest need for this and other implementation models is rigorous, prospective use of the framework to guide implementation projects. There is also need to better explain derived findings and how interventions or measures are mapped to specific PARIHS elements; greater conceptual discrimination among sub-elements may be necessary first. In general, it may be time for the implementation science community to develop consensus guidelines for reporting the use and usefulness of theoretical frameworks within implementation studies.

Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework

BACKGROUND

The Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation.

METHODS

We conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales.

RESULTS

Cronbach's alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources in general (from the context scale), and one clinical champion role (from the facilitation scale).

CONCLUSION

We find general support for the reliability and factor structure of the ORCA. However, there was poor reliability among measures of evidence, and factor analysis results for measures of general resources and clinical champion role did not conform to the PARIHS framework. Additional validation is needed, including criterion validation.