A survey of 92 quality improvement projects

Publicly reported quality-of-care measures influenced Wisconsin physician groups to improve performance

Public reporting of how physicians and hospitals perform on certain quality of care measures is increasingly common, but little is known about whether such disclosures have an impact on the quality of care delivered to patients. We analyzed fourteen publicly reported quality of ambulatory care measures from 2004 to 2009 for the Wisconsin Collaborative for Healthcare Quality, a voluntary consortium of physician groups. We also fielded a survey of the collaborative's members and analyzed Medicare billing data to independently compare members' performance to that of providers in the rest of Wisconsin, neighboring states, and the rest of the United States. We found that physician groups in the collaborative improved their performance during the study period on many measures, such as cholesterol control and breast cancer screening. Physician groups reported on the survey that publicly reported performance data motivated them to act on some, but not all, of the quality measures. Our study suggests that large group practices will engage in quality improvement efforts in response to public reporting, especially when comparative performance is displayed, as it was in this case on the collaborative's website.

Measuring team factors thought to influence the success of quality improvement in primary care: a systematic review of instruments

BACKGROUND

Measuring team factors in evaluations of Continuous Quality Improvement (CQI) may provide important information for enhancing CQI processes and outcomes; however, the large number of potentially relevant factors and associated measurement instruments makes inclusion of such measures challenging. This review aims to provide guidance on the selection of instruments for measuring team-level factors by systematically collating, categorizing, and reviewing quantitative self-report instruments.

METHODS

DATA SOURCES

We searched MEDLINE, PsycINFO, and Health and Psychosocial Instruments; reference lists of systematic reviews; and citations and references of the main report of instruments.

STUDY SELECTION

To determine the scope of the review, we developed and used a conceptual framework designed to capture factors relevant to evaluating CQI in primary care (the InQuIRe framework). We included papers reporting development or use of an instrument measuring factors relevant to teamwork. Data extracted included instrument purpose; theoretical basis, constructs measured and definitions; development methods and assessment of measurement properties. Analysis and synthesis: We used qualitative analysis of instrument content and our initial framework to develop a taxonomy for summarizing and comparing instruments. Instrument content was categorized using the taxonomy, illustrating coverage of the InQuIRe framework. Methods of development and evidence of measurement properties were reviewed for instruments with potential for use in primary care.

RESULTS

We identified 192 potentially relevant instruments, 170 of which were analyzed to develop the taxonomy. Eighty-one instruments measured constructs relevant to CQI teams in primary care, with content covering teamwork context (45 instruments measured enabling conditions or attitudes to teamwork), team process (57 instruments measured teamwork behaviors), and team outcomes (59 instruments measured perceptions of the team or its effectiveness). Forty instruments were included for full review, many with a strong theoretical basis. Evidence supporting measurement properties was limited.

CONCLUSIONS

Existing instruments cover many of the factors hypothesized to contribute to QI success. With further testing, use of these instruments measuring team factors in evaluations could aid our understanding of the influence of teamwork on CQI outcomes. Greater consistency in the factors measured and choice of measurement instruments is required to enable synthesis of findings for informing policy and practice.

Measuring organizational and individual factors thought to influence the success of quality improvement in primary care: a systematic review of instruments

Background

Continuous quality improvement (CQI) methods are widely used in healthcare; however, the effectiveness of the methods is variable, and evidence about the extent to which contextual and other factors modify effects is limited. Investigating the relationship between these factors and CQI outcomes poses challenges for those evaluating CQI, among the most complex of which relate to the measurement of modifying factors. We aimed to provide guidance to support the selection of measurement instruments by systematically collating, categorising, and reviewing quantitative self-report instruments.

Methods

Data sources: We searched MEDLINE, PsycINFO, and Health and Psychosocial Instruments, reference lists of systematic reviews, and citations and references of the main report of instruments. Study selection: The scope of the review was determined by a conceptual framework developed to capture factors relevant to evaluating CQI in primary care (the InQuIRe framework). Papers reporting development or use of an instrument measuring a construct encompassed by the framework were included. Data extracted included instrument purpose; theoretical basis, constructs measured and definitions; development methods and assessment of measurement properties. Analysis and synthesis: We used qualitative analysis of instrument content and our initial framework to develop a taxonomy for summarising and comparing instruments. Instrument content was categorised using the taxonomy, illustrating coverage of the InQuIRe framework. Methods of development and evidence of measurement properties were reviewed for instruments with potential for use in primary care.

Results

We identified 186 potentially relevant instruments, 152 of which were analysed to develop the taxonomy. Eighty-four instruments measured constructs relevant to primary care, with content measuring CQI implementation and use (19 instruments), organizational context (51 instruments), and individual factors (21 instruments). Forty-one instruments were included for full review. Development methods were often pragmatic, rather than systematic and theory-based, and evidence supporting measurement properties was limited.

Conclusions

Many instruments are available for evaluating CQI, but most require further use and testing to establish their measurement properties. Further development and use of these measures in evaluations should increase the contribution made by individual studies to our understanding of CQI and enhance our ability to synthesise evidence for informing policy and practice.

Perceptions of changes in practice following peer review in the National Chronic Obstructive Pulmonary Disease Resources and Outcomes Project

PURPOSE

The purpose of this paper is to examine perceptions of local service change and concepts of change amongst participants in a UK nationwide randomised controlled trial of informal, structured, reciprocated, multidisciplinary peer review with feedback to promote quality improvement: the National Chronic Obstructive Pulmonary Disease Resources and Outcomes Project (NCROP).

DESIGN/METHODOLOGY/APPROACH

The paper takes the form of a qualitative study, involving semi-structured interviews with 43 hospital respiratory consultants, nurses and general managers at 24 intervention and 11 control NCROP sites. Thematic analysis resulted in adoption of Joss and Kogan's quality indicators as an analytic framework.

FINDINGS

The paper finds that peer review was associated with positive changes, which may lead to sustained service improvement. Differences existed in perceptions of change among clinicians and between clinicians and managers. "Generic changes" (e.g. changes in interpersonal relations or cultural changes), were often not perceived as change.

RESEARCH LIMITATIONS/IMPLICATIONS

The study highlights the significance of generic change in evaluations of change processes. Most participants were clinicians limiting inter-professional comparisons. Some clinical staff failed to recognise changes they accomplished or their significance, perceiving change differently to others within their professional group. These findings have implications for policy and research. They should be considered when developing frameworks for assessing quality improvements and staff engagement with change.

ORIGINALITY/VALUE

This is the first qualitative study exploring participants' experience of peer review for quality improvement in healthcare. The study adds to previous research into UK health service improvement, which has had a more restricted focus on inter-professional differences.

What is the value and impact of quality and safety teams? A scoping review

BACKGROUND

The purpose of this study was to conduct a scoping review of the literature about the establishment and impact of quality and safety team initiatives in acute care.

METHODS

Studies were identified through electronic searches of Medline, Embase, CINAHL, PsycINFO, ABI Inform, Cochrane databases. Grey literature and bibliographies were also searched. Qualitative or quantitative studies that occurred in acute care, describing how quality and safety teams were established or implemented, the impact of teams, or the barriers and/or facilitators of teams were included. Two reviewers independently extracted data on study design, sample, interventions, and outcomes. Quality assessment of full text articles was done independently by two reviewers. Studies were categorized according to dimensions of quality.

RESULTS

Of 6,674 articles identified, 99 were included in the study. The heterogeneity of studies and results reported precluded quantitative data analyses. Findings revealed limited information about attributes of successful and unsuccessful team initiatives, barriers and facilitators to team initiatives, unique or combined contribution of selected interventions, or how to effectively establish these teams.

CONCLUSIONS

Not unlike systematic reviews of quality improvement collaboratives, this broad review revealed that while teams reported a number of positive results, there are many methodological issues. This study is unique in utilizing traditional quality assessment and more novel methods of quality assessment and reporting of results (SQUIRE) to appraise studies. Rigorous design, evaluation, and reporting of quality and safety team initiatives are required.

System characteristics of healthcare organizations conducting successful improvements

PURPOSE

In a previous study, based on a survey to all clinical department and primary care center managers in Sweden, it was concluded that the prevailing general improvement strategy is characterized by: drivers for improvement are staff needs; patients and data are not as important; improvements mainly focus on administrative routines and stress management; improvements are mainly reached, by writing guidelines, and conducting meetings; the majority of managers perceive outcomes from this strategy as successful. The purpose of current research in this paper is to investigate whether there is any other improvement strategy at play in Swedish health care.

DESIGN/METHODOLOGY/APPROACH

Data from the study of all Swedish managers were stratified into two populations based on an instrument predicting successful improvement. One population represented organizations with exceptionally high probability of successful imrpovement and remaining organizations represented the general improvement strategy.

FINDINGS

The paper finds that organizations with high probability for successful change differed from the comparison population at the p = 0.05 level in many of the surveyed characteristics. They put emphasis on patient focus, measuring outcomes, feedback of data, interorganizational collaboration, learning and knowledge, communication/information, culture, and development of administration and management. Thus these organizations center their attention towards behavioral changes supported by data.

PRACTICAL IMPLICATIONS

Organizations predicted to conduct successful improvement apply comprehensive improvement strategies as suggested in the literature. Such actions are part of the Patient Centered Task Alignment strategy and it is suggested that this concept has managerial implications as well, as it might be useful in further studies on improvement work in health care.

ORIGINALITY/VALUE

This paper provides empirically based findings on a successful improvement strategy that can aid research-informed policy decisions on organizational improvement strategies.

Toward stronger evidence on quality improvement. Draft publication guidelines: the beginning of a consensus project

In contrast with the primary goals of science, which are to discover and disseminate new knowledge, the primary goal of improvement is to change performance. Unfortunately, scholarly accounts of the methods, experiences, and results of most medical quality improvement work are not published, either in print or electronic form. In our view this failure to publish is a serious deficiency: it limits the available evidence on efficacy, prevents critical scrutiny, deprives staff of the opportunity and incentive to clarify thinking, slows dissemination of established improvements, inhibits discovery of innovations, and compromises the ethical obligation to return valuable information to the public.The reasons for this failure are many: competing service responsibilities of and lack of academic rewards for improvement staff; editors' and peer reviewers' unfamiliarity with improvement goals and methods; and lack of publication guidelines that are appropriate for rigorous, scholarly improvement work. We propose here a draft set of guidelines designed to help with writing, reviewing, editing, interpreting, and using such reports. We envisage this draft as the starting point for collaborative development of more definitive guidelines. We suggest that medical quality improvement will not reach its full potential unless accurate and transparent reports of improvement work are published frequently and widely.