Comparison of clinical indicators for performance measurement of health care quality: a cautionary note

Hospital diagnostic aggregation and risk-adjusted quality

OBJECTIVE

To demonstrate the importance of diagnostic aggregation when assessing hospitals.

DATA SOURCES

Patient data from the Victorian Admitted Episodes Database (VAED), 1999/2000 to 2004/2005. Financial statements from public hospitals, 2002/2003 to 2004/2005.

STUDY DESIGN

Risk-adjusted quality computed for each hospital using two aggregation levels. Each is then used to estimate the relationship between hospital efficiency and quality using two-stage DEA/Tobit model by Wilson and Simar (2006).

DATA COLLECTION

Selected variables from the VAED were obtained from the Department of Health in Victoria, then linked anonymously with financial statements.

PRINCIPAL FINDINGS

Hospital quality and, in some cases, its relationship with efficiency differs depending on aggregations.

CONCLUSIONS

Patient risk adjustment should be conducted using more than one aggregation level whenever possible.

Diabetic retinopathy care--an international quality comparison

PURPOSE

The purpose of this paper is to study the quality indicator appropriateness and use it for international quality comparison on diabetic retinopathy (DR) patient care process in one American and one Dutch eye hospital.

DESIGN/METHODOLOGY/APPROACH

A 17-item DR quality indicator set was composed based on a literature review and systematically applied in two hospitals. Qualitative analysis entailed document study and 12 semi-structured face-to-face interviews with ophthalmologists, managers, and board members of the two hospitals.

FINDINGS

While the medical-clinical approach to DR treatment in both hospitals was similar, differences were found in quality of care perception and operationalization. Neither hospital systematically used outcome indicators for DR care. On the process level, the authors found larger differences. Similarities and differences were found in the structure of both hospitals. The hospitals' particular contexts influenced the interpretation and use of quality indicators.

PRACTICAL IMPLICATIONS

Although quality indicators and quality comparison between hospitals are increasingly used in international settings, important local differences influence their application. Context should be taken into account. Since that context is locally bound and directly linked to hospital setting, caution should be used interpreting the results of quality comparison studies.

ORIGINALITY/VALUE

International quality comparison is increasingly suggested as a useful way to improve healthcare. Little is known, however, about the appropriateness and use of quality indicators in local hospital care practices.

Clinical performance indicators: intolerance for variety?

PURPOSE

The performance of NHS U.K. hospitals is under continuous scrutiny as they are constantly under pressure to perform well. A recent document published by an independent body has recommended a host of clinical indicators to assess non-financial performance of hospitals. This study aims to critically analyse the performance of a single U.K. hospital against several of these recommended indicators.

DESIGN/METHODOLOGY/APPROACH

Data presented to the Hospital Trust Board for 12 months were used for this study. Previous years' data were used wherever available.

FINDINGS

Based on data analysis, this hospital's performance is extremely difficult to calculate. The indicators use complex ratios and due to lack of standardisation, the hospital performance could be interpreted as better, worse or indifferent.

RESEARCH LIMITATIONS/IMPLICATIONS

This study analyses most of the recommended indicators. Literature review did not reveal a similar analysis of another hospital against these indicators which precludes comparison.

PRACTICAL IMPLICATIONS

This study highlights the difficulty in comparing the performance of hospitals due to the inherent lack of consistency. Therefore it is apparent that any reward-rebuke system linked to performance should interpret the data with caution. It is therefore suggested that easy to control single value activities and standardised routine activities could be used to measure hospital performance. Alternatively, the hospital could compare with its own statistics from previous years.

ORIGINALITY/VALUE

Literature acknowledges the difficulties in measuring clinical performance. This paper elucidates these difficulties applied to the NHS and suggests alternatives.

[QUALIFY--a tool for assessing quality indicators]

Quality indicators are used world-wide to control the quality of health care. To be effective these indicators have to meet quality criteria themselves. But until now there has been no fully operational method for assessing indicators. Using a systematic review of existing criteria and scientific analyses the criteria set QUALIFY was developed. Particular characteristics of QUALIFY include clear definitions of all criteria, consistent information as the basis of assessment and a standardized approach throughout the assessment process. QUALIFY comprises 20 criteria which are assigned to the three categories relevance, scientific soundness and feasibility. It was tested during the assessment of 55 indicators and served as the methodological basis for selecting those quality indicators that German hospitals are required to report publicly.