Consumers and hospital use: the HCFA "death list"

Hospital Compare and Hospital Choice: Public Reporting and Hospital Choice by Hip Replacement Patients in Texas

The Centers for Medicare & Medicaid Services publicizes comparative performance data on Hospital Compare, a website maintained to support consumer decision making. Given the agency’s goal, this study investigates the relationship between public reporting and hospital choices of hip replacement patients in Texas. Estimating individual-level valuations of provider characteristics allowing for heterogeneity across patients, we find consumer selections and hospitals’ displayed performance vary together in time. Comparing associations involving public reporting with those associated with more readily observable hospital attributes, we conclude relationships coinciding with release of comparative performance data are modest, but not inconsequential. Our use of an empirical strategy novel for evaluation of public reporting has methodological implications, while the study’s affirmative result is of potential interest to policy makers and administrators.

Functionality and feedback: a realist synthesis of the collation, interpretation and utilisation of patient-reported outcome measures data to improve patient care

Background

The feedback of patient-reported outcome measures (PROMs) data is intended to support the care of individual patients and to act as a quality improvement (QI) strategy.

Objectives

To (1) identify the ideas and assumptions underlying how individual and aggregated PROMs data are intended to improve patient care, and (2) review the evidence to examine the circumstances in which and processes through which PROMs feedback improves patient care.

Design

Two separate but related realist syntheses: (1) feedback of aggregate PROMs and performance data to improve patient care, and (2) feedback of individual PROMs data to improve patient care.

Interventions

Aggregate – feedback and public reporting of PROMs, patient experience data and performance data to hospital providers and primary care organisations. Individual – feedback of PROMs in oncology, palliative care and the care of people with mental health problems in primary and secondary care settings.

Main outcome measures

Aggregate – providers’ responses, attitudes and experiences of using PROMs and performance data to improve patient care. Individual – providers’ and patients’ experiences of using PROMs data to raise issues with clinicians, change clinicians’ communication practices, change patient management and improve patient well-being.

Data sources

Searches of electronic databases and forwards and backwards citation tracking.

Review methods

Realist synthesis to identify, test and refine programme theories about when, how and why PROMs feedback leads to improvements in patient care.

Results

Providers were more likely to take steps to improve patient care in response to the feedback and public reporting of aggregate PROMs and performance data if they perceived that these data were credible, were aimed at improving patient care, and were timely and provided a clear indication of the source of the problem. However, implementing substantial and sustainable improvement to patient care required system-wide approaches. In the care of individual patients, PROMs function more as a tool to support patients in raising issues with clinicians than they do in substantially changing clinicians’ communication practices with patients. Patients valued both standardised and individualised PROMs as a tool to raise issues, but thought is required as to which patients may benefit and which may not. In settings such as palliative care and psychotherapy, clinicians viewed individualised PROMs as useful to build rapport and support the therapeutic process. PROMs feedback did not substantially shift clinicians’ communication practices or focus discussion on psychosocial issues; this required a shift in clinicians’ perceptions of their remit.

Strengths and limitations

There was a paucity of research examining the feedback of aggregate PROMs data to providers, and we drew on evidence from interventions with similar programme theories (other forms of performance data) to test our theories.

Conclusions

PROMs data act as ‘tin openers’ rather than ‘dials’. Providers need more support and guidance on how to collect their own internal data, how to rule out alternative explanations for their outlier status and how to explore the possible causes of their outlier status. There is also tension between PROMs as a QI strategy versus their use in the care of individual patients; PROMs that clinicians find useful in assessing patients, such as individualised measures, are not useful as indicators of service quality.

Future work

Future research should (1) explore how differently performing providers have responded to aggregate PROMs feedback, and how organisations have collected PROMs data both for individual patient care and to improve service quality; and (2) explore whether or not and how incorporating PROMs into patients’ electronic records allows multiple different clinicians to receive PROMs feedback, discuss it with patients and act on the data to improve patient care.

Study registration

This study is registered as PROSPERO CRD42013005938.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.

Information disclosure and peer effects in the use of antibiotics

Mandatory information disclosure may allow sellers to observe and respond to other sellers' attributes (seller peer effects) as well as informing consumers of the sellers' attributes (consumer learning effect). Using the data from mandatory information disclosure of antibiotic prescription rates for the common cold in Korea, this paper shows that while average prescription rates decreased after the disclosure, more than 30% of the clinics increased their antibiotic prescriptions. Moreover, clinics that were prescribing relatively fewer antibiotics than other local clinics before the disclosure requirement were more likely to increase their prescription rate. The average prescription rates also declined less in markets with stronger clinic competition. These results are consistent with seller peer effects.

Rankings versus reality in pancreatic cancer surgery: a real-world comparison

BACKGROUND

Patients are increasingly confronted with systems for rating hospitals. However, the correlations between publicized ratings and actual outcomes after pancreatectomy are unknown.

METHODS

The Massachusetts Division of Health Care Finance and Policy Hospital Inpatient Discharge Database was queried to identify pancreatic cancer resections carried out during 2005-2009. Hospitals performing fewer than 10 pancreatic resections in the 5-year period were excluded. Primary outcomes included mortality, complications, median length of stay (LoS) and a composite outcomes score (COS) combining primary outcomes. Ranks were determined and compared for: (i) volume, and (ii) ratings identified from consumer-directed hospital ratings including the US News & World Report (USN), Consumer Reports, Healthgrades and Hospital Compare. An inter-rater reliability analysis was performed and correlation coefficients (r) between outcomes and ratings, and between rating systems were calculated.

RESULTS

Eleven hospitals in which a total of 804 pancreatectomies were conducted were identified. Surgical volume correlated with overall outcome, but was not the strongest indicator. The highest correlation referred to that between USN rank and overall outcome. Mortality was most strongly correlated with Healthgrades ratings (r = 0.50); however, Healthgrades ratings demonstrated poorer correlations with all other outcomes. Consumer Reports ratings showed inverse correlations.

CONCLUSIONS

The plethora of publicly available hospital ratings systems demonstrates heterogeneity. Volume remains a good but imperfect indicator of surgical outcomes. Further systematic investigation into which measures predict quality outcomes in pancreatic cancer surgery will benefit both patients and providers.

Impact of the Iranian hospital grading system on hospitals' adherence to audited standards: an examination of possible mechanisms

INTRODUCTION

All Iranian hospitals have been subject to a grading system which determines the payments they can charge. We examined all possible pathways through which the grading system could influence hospitals' adherence to audited standards.

METHODS

Using a mixed methods study we examined five stakeholder groups: hospital staff, patients, general practitioners, health insurance organisations and surveyor organisations. Data were collected via semi-structured interviews, a questionnaire survey, observation and documentary analysis.

FINDINGS

Patients and general practitioners were generally unaware of the hospital grading. Hospital staff and insurance organisations were informed, but this was not found to influence the hospital staff's choice of where to work nor the insurance organisations contracting behaviour. The grading system was criticised for the performance standards' validity and the validity of hospitals' awarded results. Hospitals responded to financial and reputational incentives for achieving better grades, although gaming and misrepresentation was also reported.

CONCLUSION

Pay-for-performance was the main influential factor in shaping hospitals' adherence to audit standards. Other potential mechanisms for influencing hospital behaviour, the selection mechanism and intrinsic motives, were not found to be sufficient to affect hospital behaviour.

Public reporting in Germany: the content of physician rating websites

BACKGROUND

Physician rating websites (PRWs) are gaining in popularity among patients seeking quality information about physicians. However, little knowledge is available about the quantity and type of information provided on the websites.

OBJECTIVE

To determine and structure the quantity and type of information about physicians in the outpatient sector provided on German-language physician rating websites.

METHODS

In a first step, we identified PRWs through a systematic internet search using German keywords from a patient´s perspective in the two search engines Google and Yahoo. Afterwards, information about physicians available on the websites was collected and categorised according to Donabedian´s structure/process/outcome model. Furthermore, we investigated whether the information was related to the physician himself/ herself or to the practice as a whole.

RESULTS

In total, eight PRWs were detected. Our analysis turned up 139 different information items on eight websites; 67 are related to the structural quality, 4 to process quality, 5 to outcomes, and 63 to patient satisfaction/experience. In total, 37% of all items focus specifically on the physician and 63% on the physician's practice. In terms of the total amount of information provided on the PRWs, results range from 61 down to 13.5 items.

CONCLUSIONS

A broad range of information is available on German PRWs. While structural information can give a detailed overview of the financial, technical and human resources of a practice, other outcome measures have to be interpreted with caution. Specifically, patient satisfaction results are not risk-adjusted, and thus, are not appropriate to represent a provider's quality of care. Consequently, neither patients nor physicians should yet use the information provided to make their final decision for or against an individual physician.

State-sponsored public reporting of hospital quality: results are hard to find and lack uniformity

The prevalence of state public reporting initiatives that focus on hospital quality is not known. We systematically reviewed state-sponsored public reporting programs that focused on clinical aspects of hospital quality and performance for adults, by surveying the fifty US states and the District of Columbia. We found that although identifying information about programs was frequently a challenge, twenty-five states had programs that reported about hospital quality. Information varied considerably from state to state, by health condition and by process and outcome measures reported. We examine the implications of these findings for future state initiatives.

Effect of the Iranian hospital grading system on patients' and general practitioners' behaviour: an examination of awareness, belief and choice

There is considerable international interest in the use of performance measurement and their public release in order to improve the quality of care. However, few studies have assessed stakeholders' awareness and use of performance data. Iranian hospitals have been graded annually since 1998 and hospital hotel charges vary by grade, but this system has never been evaluated. We conducted a cross-sectional survey of 104 outpatients at eight Teheran hospitals and 103 general practitioners (GPs) to assess the awareness of and attitudes towards hospital grading system. Only 5.8% of patients (95% CI: 1.3-10.3%) and 11.7% of GPs (95% CI: 5.5-17.9%) were aware of grading results. Patients' awareness was positively associated with their education level (P = 0.016). No patient used the grading results for choosing a hospital and only one GP (1%, 95% CI: 0-2%) reported using hospital grade to influence referral decisions. Patients were more influenced by hospitals' public reputation and that of their specialists. GPs believed that the grading system did not reflect the quality of care in hospitals. When developing performance measurement systems, public release of data should be accompanied by evaluation of its impact on awareness and health-care choices.

Quality of care in inflammatory bowel disease: a review and discussion

The Institute of Medicine's publications To Err Is Human and Crossing the Quality Chasm publicized the widespread deficits in U.S. health care quality. Emerging studies continue to reveal deficits in the quality of adult and pediatric care, including subspecialty care. The inflammatory bowel diseases (IBD) Crohn's disease and ulcerative colitis require diligent, long-term management and attention to their impact on intestinal and extraintestinal organ systems. Although the quality of IBD care has not been prospectively or comprehensively evaluated in the United States, several small studies have demonstrated significant variation in care. As variation may indicate underuse, overuse, or misuse of medical services, such variation suggests a clear need for translating evidence-based practices into the actual practice and follow-up provided for patients. This article reviews the history, rationale, and methods of quality measurement and improvement and identifies the unique challenges in adapting these general strategies to the care of the inflammatory bowel diseases.

Quality information and fragmented markets: patient responses to hospital volume thresholds

Over the last two decades, information dissemination policies to improve patient hospital choice have emerged. But during this same period, policy makers have also generally adopted a market-oriented approach vis-à-vis hospitals, with limited regulation of facility expansion and few restrictions on hospital mergers and ownership changes. These policies may be in tension, and this analysis examines whether there have been changes over time in patient responses to information about the value of high-volume hospitals and the degree to which hospital market changes may have limited these patient responses. The results indicate modest changes consistent with an increase in quality-seeking behavior for several services for which research indicates a volume-outcome relationship. At the same time, there are services for which trends have been moving in the opposite direction--toward greater local-care seeking--and changes for the remaining services have been fairly small. Even for services with a trend toward greater patient sensitivity to volume as a marker for quality, however, hospital market changes have reduced the change over time in high-volume hospital use. These results highlight some of the limitations of market-oriented strategies for increasing patient use of high-quality hospitals.

Quality of health care in the United States: implications for pediatric inflammatory bowel disease

The Institute of Medicine's publications To Error is Human and Crossing the Quality Chasm publicized the widespread deficits in US health care quality. Emerging studies continue to reveal deficits in the quality of adult and pediatric care, including subspecialty care. In recent years, key stakeholders in the health care system including providers, purchasers, and the public have been applying various quality improvement methods to address these concerns. Lessons learned from these efforts in other pediatric conditions, including asthma, cystic fibrosis, neonatal intensive care, and liver transplantation may be applicable to the care of children with inflammatory bowel disease (IBD).This review is intended to be a primer on the quality of care movement in the United States, with a focus on pediatric IBD. In this article, we review the history, rationale, and methods of quality measurement and improvement, and we discuss the unique challenges in adapting these general strategies to pediatric IBD care.

Quality and Consumer Decision Making in the Market for Health Insurance and Health Care Services

This article reviews the literature relating quality to consumer choice of health plan or health care provider. Evidence suggests that consumers tend to choose better performing health plans and providers and are responsive to initiatives that provide quality information. The response to quality and quality information differs significantly among consumers and across population subgroups. As such the effect of quality information on choice is apparent in only a relatively small, though perhaps consequential, number of consumers. Despite the wealth of findings on the topic to date, the authors suggest directions for future work, including better assessment of the dynamic issues related to information release, as well as a better understanding of how the response to information varies across different groups of patients.

Assessing changes in high-volume hospital use: hospitals, payers, and aggregate volume trends

The relationship between higher procedure volumes and medical outcomes has generated recommendations for greater use of high-volume hospitals, with research and advocacy on this issue intensifying during the 1990s. Despite this interest, the trends presented here showed only limited changes between 1995 and 2002. For a number of services, less than half of patients received care at high-volume hospitals, and for several services, there was a surprising decline in the proportion at high-volume hospitals. Trends in the rate of high-volume hospital use appeared to be associated with trends in aggregate volume, at the same time that there were only modest changes in either patient use of high-volume hospitals or the number of hospitals offering these services. These trends suggested the importance of research on factors that affect patient choices, hospital decisions, and payer incentives concerning hospital use, especially in the context of declining aggregate procedure volumes.

Reducing avoidable deaths among veterans: directing private-sector surgical care to high-performance hospitals

OBJECTIVES

We quantified older (65 years and older) Veterans Health Administration (VHA) patients' use of the private sector to obtain 14 surgical procedures and assessed the potential impact of directing that care to high-performance hospitals.

METHODS

Using a merged VHA-Medicare inpatient database for 2000 and 2001, we determined where older VHA enrollees obtained 6 cardiovascular surgeries and 8 cancer resections and whether private-sector care was obtained in high- or low-performance hospitals (based on historical performance and determined 2 years in advance of the service year). We then modeled the mortality and travel burden effect of directing private-sector care to high-performance hospitals.

RESULTS

Older veterans obtained most of their procedures in the private sector, but that care was equally distributed across high- and low-performance hospitals. Directing private-sector care to high-performance hospitals could have led to the avoidance of 376 to 584 deaths, most through improved cardiovascular care outcomes. Using historical mortality to define performance would produce better outcomes with lower travel time.

CONCLUSIONS

Policy that directs older VHA enrollees' private-sector care to high-performance hospitals promises to reduce mortality for VHA's service population and warrants further exploration.

Characteristics of VA patients who use low-quality private-sector CABG centers in New York

Little is known about the quality of care that veterans obtain in the private sector. To explore this issue, we identified the hospital in which 4,008 veterans enrolled in the Veterans Administration (VA) obtained coronary artery bypass graft (CABG) surgery in the New York private sector between 1997 and 2000. We used published risk-adjusted mortality rates to assign New York CABG centers to performance quintiles. VA patients with the lowest incomes were 2.4 times more likely than those in the highest incomes to use the highest-mortality hospitals (95 percent CI: 2.0-3.0). Compared with white patients, black (OR 1.8; 95 percent CI: 1.2-2.8) and Hispanic VA patients (OR 1.6; 95 percent CI: 0.9-2.8) were more likely to use the highest-mortality hospitals. About one third of patients using the highest-mortality hospitals lived closer to low-mortality hospitals. Efforts to direct VA patients' care to high-performance hospitals could improve outcomes and reduce racial and ethnic disparities in care.

Measuring Performance For Treating Heart Attacks And Heart Failure: The Case For Outcomes Measurement

To complement the current process measures for treating patients with heart attacks and with heart failure, which target gaps in quality but do not capture patient outcomes, the Centers for Medicare and Medicaid Services (CMS) has proposed the public reporting of hospital-level thirty-day mortality for these conditions in 2007. We present the case for including measurements of outcomes in the assessment of hospital performance, focusing on the care of patients with heart attacks and with heart failure. Recent developments in the methodology and standards for outcomes measurement have laid the groundwork for incorporating outcomes into performance monitoring efforts for these conditions.

Veterans Health Administration patients' use of the private sector for coronary revascularization in New York: opportunities to improve outcomes by directing care to high-performance hospitals

OBJECTIVE

We sought to quantify Veterans Health Administration (VA) patients' utilization of coronary revascularization in the private sector and to assess the potential impact of directing this care to high-performance hospitals.

METHODS

Using VA and New York State administrative and clinical databases, we conducted a retrospective cohort study examining residents of New York State who were enrolled in the VA and underwent either coronary artery bypass graft (CABG) surgery or percutaneous coronary intervention (PCI) in 1999 or 2000 (n=6562) in either the VA or the private sector. We first calculated the proportion of revascularizations obtained in the VA and the private sector. We then identified the private sector hospitals in which these men obtained revascularizations and determined potential changes in mortality and travel burden associated with directing private sector care to high performance hospitals.

RESULTS

VA patients in New York were much more likely to undergo revascularization in the private sector than in VA hospitals: 83% of CABGs (2341/2829) and 87% of PCIs (4054/4665) were obtained in the private sector. Private sector utilization was distributed evenly across high- and low-mortality hospitals. Directing private-sector CABG surgery to high-performance hospitals could have reduced expected mortality by 24% (from 2.3% to 1.7%) and would only increase median travel time from 21 to 30 minutes. The benefit of redirecting PCI care is minimal.

CONCLUSIONS

For high-mortality procedures that veterans frequently obtain in the private sector, like CABG, directing care to high-performance hospitals may be an effective way to improve outcomes for veterans.

Do report cards influence hospital choice? The case of kidney transplantation

The national program to report hospital-level outcomes for transplantation has been in place since 1991, yet it has not been addressed in the existing literature on hospital report cards. We study the impact of reported outcomes on demand at kidney transplant centers. Using a negative binomial regression with hospital fixed effects, we estimate the number of patients choosing each center as a function of reported outcomes. Parameters are identified by the within-hospital variation in outcomes over five successive report cards. We find some evidence that report cards influence younger and college-educated patients, but, overall, report cards do not affect demand.

Do Well-Publicized Risk-Adjusted Outcomes Reports Affect Hospital Volume?

BACKGROUND

Report cards on hospital performance are common but have uncertain impact.

OBJECTIVES

The objective of this study was to determine whether hospitals recognized as performance outliers experience volume changes after publication of a report card. Secondary objectives were to test whether favorable outliers attract more patients with related conditions, or from outside their catchment areas; and whether disadvantaged groups are less responsive to report cards.

STUDY DESIGN

We used a time-series analysis using linear and autoregressive models.

SUBJECTS

We studied patients admitted to nonfederal hospitals designated as outliers in reports on coronary bypass surgery (CABG) mortality in New York, acute myocardial infarction (AMI) mortality in California, and postdiskectomy complications in California.

MEASURES

We studied observed versus expected hospital volume for topic and related conditions and procedures, by month/quarter after a report card, with and without stratification by age, race/ethnicity, insurance, and catchment area. Potential confounders included statewide prevalence, prereport hospital volume and market share, and unrelated volume.

RESULTS

In California, low-mortality and high-mortality outliers did not experience changes in AMI volume after adjusting for autocorrelation. Low-complication outliers for lumbar diskectomy experienced slightly increased volume in autoregressive models. No other cohorts demonstrated consistent trends. In New York, low-mortality outliers experienced significantly increased CABG volume in the first month after publication, whereas high-mortality outliers experienced decreased volume in the second month. The strongest effects were among white patients and those with HMO coverage in California, and among white or other patients and those with Medicare in New York.

CONCLUSIONS

Volume effects were modest, transient, and largely limited to white Medicare patients in New York.

Urologists on a tightrope--do we have a net?

PURPOSE

The purpose of this review is to analyze the current health care environment and its impact on urological practice.

MATERIALS AND METHODS

The medical and lay literature as it pertains to the socioeconomics of health care was reviewed.

RESULTS

Analysis of the political and economic factors that influence the delivery of health care today reveals alarming realities. More than 40 million Americans remain uninsured, and with a retrenched economy that number is likely to increase. Neither government nor the private sector has been either willing or able to address the health care problem in a coherent or comprehensive way. As the population ages, the Medicare and Medicaid programs will become further stressed. Employers are increasingly unwilling to finance the health care expenses of their employees. Academic medical centers are facing unique exigencies that, if left uncorrected, will jeopardize the future training of physicians.

CONCLUSIONS

In the current environment of a depressed economy, further proposed tax cuts and increased military spending it appears inevitable that the economic restraints on medical care will increase substantially in the foreseeable future.

Public disclosure of comparative clinical performance data: lessons from the Scottish experience

There is growing international interest in making information available on the clinical quality and performance of health care providers. In the United States of America, where public reporting is most advanced, comparative performance information in the form of 'report cards', 'provider profiles' and 'consumer reports' has been published for over a decade. In Europe, Scotland has been at the forefront of releasing clinical performance data and has disseminated such information since 1994. This paper reviews the Scottish experience of public disclosure and distils the key lessons for other countries seeking to implement similar programmes. It is based on the findings of the first empirical evaluation of a national clinical reporting initiative outside the United States. The study examined the impact of publication of Scottish (CRAG) clinical outcome indicators on four key stakeholder groups: health care providers, regional government health care purchasers, general practitioners and consumer advocacy agencies. We conclude that those responsible for developing clinical reporting systems should not only pay close attention to developing technically valid and professionally credible data which are tailored to the information needs of different end users, but should also focus on developing a suitable incentive structure and organizational environment that fosters the constructive use of such information.

Quality report cards and nursing home quality

PURPOSE

This study examined the potential role that publicly disseminated quality report cards can play in improving quality of care in nursing homes.

DESIGN AND METHODS

We review the literature and the experience gained over the last two decades with report cards for hospitals, physicians, and health plans, and consider the issues that are of particular importance in the context of nursing home care.

RESULTS

Experience with report cards in other areas of the health care system suggests that nursing home quality reports may have a role to play in informing consumers' choices and providing incentives for quality improvement. Their impact may, however, not be large. Methodological issues that may limit the accuracy of quality indicators and issues related to the design and comprehension of the information by consumers are discussed.

IMPLICATIONS

Quality report cards should be viewed as one of several options to ensure higher quality nursing home care.

Using performance measurement to drive improvement: a road map for change

BACKGROUND

Performance measures and reporting have not been adopted throughout the US health care system despite their central role in encouraging increased participation by consumers in decision-making. Understanding whether the failure of measurement and reporting to diffuse throughout the health system can be overcome is critical for determining future policy in this area.

OBJECTIVES

To create a conceptual framework for analyzing the current rate of adoption and evaluating alternatives for accelerating adoption, and to recommend a set of concrete steps that can be taken to increase the use of performance measurement and reporting.

RESEARCH DESIGN

Review of three theoretic models (Rogers, Prochaska/DiClemente, Gladwell), examination of the literature on previous experiences with quality measurement and reporting, and interviews with select stakeholders.

FINDINGS

The three theoretic models provide a valuable framework for understanding why the use of performance measures is stalled ("the circle of unaccountability") and for generating ideas about concrete steps that could be taken to accelerate adoption. Six steps are recommended: (1) raise public awareness, (2) redesign measures and reports, (3) make the delivery of information timely, (4) require public reporting, (5) develop and implement systems to reward quality, and (6) actively court leaders.

CONCLUSIONS

The recommended six steps are interconnected; action on all will be required to drive significant acceleration in rates of adoption of performance measurement and reporting. Leadership and coordination are necessary to ensure these steps are taken and that they work in concert with one another.

Attitudes to the public release of comparative information on the quality of general practice care: qualitative study

OBJECTIVES

To examine the attitudes of service users, general practitioners, and clinical governance leads based in primary care trusts to the public dissemination of comparative reports on quality of care in general practice, to guide the policy and practice of public disclosure of information in primary care.

DESIGN

Qualitative focus group study using mock quality report cards as prompts for discussion.

SETTING

12 focus groups held in an urban area in north west England and a semirural area in the south of England.

PARTICIPANTS

35 service users, 24 general practitioners, and 18 clinical governance leads.

RESULTS

There was general support for the principle of publishing comparative information, but all three stakeholder groups expressed concerns about the practical implications. Attitudes were strongly influenced by experience of comparative reports from other sectors-for example, school league tables. Service users distrusted what they saw as the political motivation driving the initiative, expressed a desire to "protect" their practices from political and managerial interference, and were uneasy about practices being encouraged to compete against each other. General practitioners focused on the unfairness of drawing comparisons from current data and the risks of "gaming" the results. Clinical governance leads thought that public disclosure would damage their developmental approach to implementing clinical governance. The initial negative response to the quality reports seemed to diminish on reflection.

CONCLUSIONS

Despite support for the principle of greater openness, the planned publication of information about quality of care in general practice is likely to face considerable opposition, not only from professional groups but also from the public. A greater understanding of the practical implications of public reporting is required before the potential benefits can be realised.

Reporting health care performance: learning from the past, prospects for the future

In the USA, where public reporting of data on clinical performance is most advanced, comparative performance information, in the form of 'report cards', 'provider profiles' and 'physician profiling', has been published for over a decade. Many other countries are now following a similar route and are seeking to develop comparative data on health care performance. Notwithstanding the idiosyncratic nature of US health care, and the implications this has for the generalizability of findings from the USA to other countries, it is pertinent to ask what other countries can learn from the US experience. Based on a series of structured interviews with leading experts on the US health system, this article draws out the key lessons for other countries as they develop similar policies in this area. This paper highlights three concerns that have dominated the development of adequate measures in the USA, and that require consideration when developing similar schemes elsewhere. Firstly, the need to develop indicators with sound metric properties - high in validity and meaningfulness, and appropriately risk-adjusted. Secondly, the need to involve all stakeholders in the design of indicators, and a requirement that those measures be adapted to different audiences. Thirdly, a need to understand the needs of end users and to engage with them in partnerships to increase the attention paid to measurement. This study concludes that the greatest challenge is posed by the desire to make comparative performance data more influential in leveraging performance improvement. Simply collecting, processing, analysing and disseminating comparative data is an enormous logistical and resource-intensive task, yet it is insufficient. Any national strategy emphasizing comparative data must grapple with how to engage the serious attention of those individuals and organizations to whom change is to be delivered.

Cardiac surgery report cards: comprehensive review and statistical critique

Public report cards and confidential, collaborative peer education represent distinctly different approaches to cardiac surgery quality assessment and improvement. This review discusses the controversies regarding their methodology and relative effectiveness. Report cards have been the more commonly used approach, typically as a result of state legislation. They are based on the presumption that publication of outcomes effectively motivates providers, and that market forces will reward higher quality. Numerous studies have challenged the validity of these hypotheses. Furthermore, although states with report cards have reported significant decreases in risk-adjusted mortality, it is unclear whether this improvement resulted from public disclosure or, rather, from the development of internal quality programs by hospitals. An additional confounding factor is the nationwide decline in heart surgery mortality, including states without quality monitoring. Finally, report cards may engender negative behaviors such as high-risk case avoidance and "gaming" of the reporting system, especially if individual surgeon results are published. The alternative approach, continuous quality improvement, may provide an opportunity to enhance performance and reduce interprovider variability while avoiding the unintended negative consequences of report cards. This collaborative method, which uses exchange visits between programs and determination of best practice, has been highly effective in northern New England and in the Veterans Affairs Administration. However, despite their potential advantages, quality programs based solely on confidential continuous quality improvement do not address the issue of public accountability. For this reason, some states may continue to mandate report cards. In such instances, it is imperative that appropriate statistical techniques and report formats are used, and that professional organizations simultaneously implement continuous quality improvement programs. The statistical methodology underlying current report cards is flawed, and does not justify the degree of accuracy presented to the public. All existing risk-adjustment methods have substantial inherent imprecision, and this is compounded when the results of such patient-level models are aggregated and used inappropriately to assess provider performance. Specific problems include sample size differences, clustering of observations, multiple comparisons, and failure to account for the random component of interprovider variability. We advocate the use of hierarchical or multilevel statistical models to address these concerns, as well as report formats that emphasize the statistical uncertainty of the results.

Consumer reports in health care: do they make a difference?

The public release of health care-quality data into more formalized consumer health report cards is intended to educate consumers, improve quality of care, and increase competition in the marketplace The purpose of this review is to evaluate the evidence on the impact of consumer report cards on the behavior of consumers, providers, and purchasers. Studies were selected by conducting database searches in Medline and Healthstar to identify papers published since 1995 in peer-review journals pertaining to consumer report cards on health care. The evidence indicates that consumer report cards do not make a difference in decision making, improvement of quality, or competition. The research to date suggests that perhaps we need to rethink the entire endeavor of consumer report cards. Consumers desire information that is provider specific and may be more likely to use information on rates of errors and adverse outcomes. Purchasers may be in a better position to understand and use information about health plan quality to select high-quality plans to offer consumers and to design premium contributions to steer consumers, through price, to the highest-quality plans.

Publicly disclosed information about the quality of health care: response of the US public

Public disclosure of information about the quality of health plans, hospitals, and doctors continues to be controversial. The US experience of the past decade suggests that sophisticated quality measures and reporting systems that disclose information on quality have improved the process and outcomes of care in limited ways in some settings, but these efforts have not led to the "consumer choice" market envisaged. Important reasons for this failure include limited salience of objective measures to consumers, the complexity of the task of interpretation, and insufficient use of quality results by organised purchasers and insurers to inform contracting and pricing decisions. Nevertheless, public disclosure may motivate quality managers and providers to undertake changes that improve the delivery of care. Efforts to measure and report information about quality should remain public, but may be most effective if they are targeted to the needs of institutional and individual providers of care.

The impact of quality report cards on choice of physicians, hospitals, and HMOs: a midcourse evaluation

BACKGROUND

Increasing competition in health care markets and ongoing pressures to contain costs raise concerns about possible deterioration in the quality of medical care. Publicly disseminated quality report cards are designed to inform consumers' choice of providers and health plans, thus counteracting incentives to provide low-quality care and improving the functioning of health care markets.

METHODS

This article reviews and evaluates the published evidence on the impact of quality report cards on patients' choice of health care providers and health plans.

RESULTS

Studies found only minimal effect of quality report cards on patient referral choices. These findings can be explained by several study design issues and by the economic forces governing health care markets. They cannot be construed to imply that quality report cards are not effective.

DISCUSSION

Whether report cards are effective or not is still an unanswered question. Further efforts to improve the information contained in report cards and to make them more understandable could increase their effectiveness.

Research evidence on the validity of risk-adjusted mortality rate as a measure of hospital quality of care

For more than 10 years, reports comparing quality of care in hospitals have been disseminated to the public. The most commonly used measure in these reports is hospital mortality rate. Despite the resources devoted to analyzing and disseminating mortality data, little attention has been given to the question of their validity as a quality measure. In this article, the authors synthesize findings from 18 articles identified as providing information relevant to this issue. From this review, the authors find evidence that poor quality care increases patients' risk of mortality and that, on average, quality of care provided in hospitals identified as high—mortality rate outliers is poorer than that provided in low—mortality rate outlier hospitals. Nevertheless, a clear conclusion from these studies is that when used as a measure of quality for individual hospitals, risk-adjusted mortality rates are seriously inaccurate. Publication of hospital mortality rates misinforms the public about hospital quality.

Quality of care information makes a difference: an analysis of market share and price changes after publication of the New York State Cardiac Surgery Mortality Reports

OBJECTIVES

Quality report cards are becoming increasingly more common and receive much publicity. They can have significant impact on competition among providers, costs, and quality of health care. The authors test the hypotheses that hospitals and surgeons with better outcomes reported in the NYS Cardiac Surgery Reports experience a relative increase in their market share and prices.

METHODS

Information from the New York State Cardiac Surgery Reports was linked with physicians' claims submitted to Medicare and was used to calculate market shares and average prices for hospitals and physicians performing CABG surgeries. Regression models were estimated to test hypotheses. All 30 hospitals offering coronary artery bypass graft (CABG) were studied as well as a majority of surgeons (114 or approximately 80%) performing CABG surgery in New York State during the 1990-1993 period.

RESULTS

Findings indicate that hospitals and physicians with better outcomes experienced higher rates of growth in market shares. Physicians with better outcomes also had higher rates of growth in charges for this procedure.

CONCLUSIONS

Patients (and referring physicians) seem to respond to information about quality of individual surgeons and hospitals as expected. The magnitude of the association between reported mortality and market shares varies geographically, potentially reflecting differences in sociodemographic characteristics. The association tends to decline over time, suggesting that it is primarily due to "new" information.

The California Hospital Outcomes Project: how useful is California's report card for quality improvement?

BACKGROUND

Hospital report cards have proliferated in the 1990s but remain controversial because risk-adjusted outcomes measures are complex and have uncertain validity. Despite this controversy, little is known about their value and impact.

METHODS

A two-stage survey of hospital leaders in California was undertaken in September 1996 and July 1997 to explore how the 1996 reports and data from the California Hospital Outcomes Project (CHOP) were used to improve organizations' performance. In the first stage, a questionnaire was mailed to the chief executive officer of each hospital in the report. In the second stage, a stratified random sample of the respondents who indicated a willingness to provide further information was interviewed.

RESULTS

Thirty-nine interviews were completed, representing 87% yield after replacing informants who failed to return six messages. About three-quarters of the interviewees found some aspect of the CHOP report to be useful, especially for benchmarking performance, improving ICD-9-CM (International Classification of Diseases, 9th Revision, Clinical Modification) coding, and educating physicians about documentation and clinical pathways. The most common criticisms were that the reports were not timely and described death rates without providing practical information about the process of care.

DISCUSSION

Although the 1996 CHOP reports and data were widely disseminated within hospitals, most reported uses did not directly affect the process of care for patients with acute myocardial infarction. This finding reflects two critical weaknesses of the project--nontimely data and lack of information about the process of care. Nevertheless, hospital quality managers recognize that public report cards are here to stay, and some carefully studied their outcomes data to identify areas for improvement.

Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists

BACKGROUND

Reports on the comparative performance of physicians are becoming increasingly common. Little is known, however, about the credibility of these reports with target audiences or their influence on the delivery of medical services.

METHODS

Since 1992, Pennsylvania has published the Consumer Guide to Coronary Artery Bypass Graft Surgery, which lists annual risk-adjusted mortality rates for all hospitals and surgeons providing such surgery in the state. In 1995, we surveyed a randomly selected sample of 50 percent of Pennsylvania cardiologists and cardiac surgeons to find out whether they were aware of the guide and, if so, to determine their views on its usefulness, limitations, and influence on providers.

RESULTS

Eighty-two percent of the cardiologists and all the cardiac surgeons were aware of the guide. Only 10 percent of these respondents reported that its mortality rates were "very important" in assessing the performance of a cardiothoracic surgeon. Less than 10 percent reported discussing the guide with more than 10 percent of their patients who were candidates for a coronary-artery bypass graft (CABG). Eighty-seven percent of the cardiologists reported that the guide had a minimal influence or none on their referral recommendations. For both groups, the most important limitations of the guide were the absence of indicators of quality other than mortality (cited by 78 percent), inadequate risk adjustment (79 percent), and the unreliability of data provided by hospitals and surgeons (53 percent). Fifty-nine percent of the cardiologists reported increased difficulty in finding surgeons willing to perform CABG surgery in severely ill patients who required it, and 63 percent of the cardiac surgeons reported that they were less willing to operate on such patients.

CONCLUSIONS

The Consumer Guide to Coronary Artery Bypass Graft Surgery has limited credibility among cardiovascular specialists. It has little influence on referral recommendations and may introduce a barrier to care for severely ill patients. If publicly released performance reports are intended to guide the choice of providers without impeding access to medical care, a collaborative process involving physicians may enhance the credibility and usefulness of the reports.

A consumer guide for marketing medical services: one institution's experience

This paper describes the Cleveland Clinic Foundation's experience with the development and implementation of a 20-page quality indicator consumer guide for patients with or at risk of developing coronary artery disease. The guide, which provides six "quality indicators," was designed to enable patients to evaluate and compare quality-related information when choosing a provider. Design elements for the guide included a user-friendly format to offset the amount of information consumers are asked to absorb. Data on inquiries showed that the majority were women (53%) and adults under the age of 65 years (57%). Although the media criticized the guide as a marketing tool, it represents an effort to educate consumers about the importance of research when choosing a provider.