Clinical performance indicators: intolerance for variety?

A comprehensive map of the evidence on the performance evaluation indicators of public hospitals: a scoping study and best fit framework synthesis

INTRODUCTION

Key performance indicators are essential navigation tools for hospitals. They provide managers with valid information enabling them to identify institutional strengths and weaknesses and improve managerial performance. In this study, the synthesis of evidence relating to hospital performance indicators was carried out by means of a field review and the indicators were analyzed through the Best Fit Method.

METHODS

The five-step approach of Arksey and O'Malley was used as follows: selection of the research question; search for related studies; selection and refinement of the studies; synthesis and tabulation of key information; derivation of the related summary and report. Applying the Best Fit Framework Synthesis Method, the initial themes and subthemes were created and a model of public hospitals performance evaluation finally generated.

RESULTS

Forty-nine studies were considered eligible to form part of the synthesis. The final model included the efficiency/productivity, effectiveness and financial themes. The efficiency/productivity sub-themes incorporated human resources indicators, hospital beds, costs, operating room productivity, emergency rooms, ICU, radiology, labs, technology and equipment productivity. Other sub-themes relate to general indicators such as BOR, ALS, number of outpatients and hospitalized patients. Financial themes included profit, revenue, cash flow, cost, investment, assets, debt and liquidity. Concerning effectiveness, the indicators were categorized in terms of access (equity), safety, quality and responsiveness. The accountability indicators were classified into patient-centeredness, staff orientation, and social responsibility.

CONCLUSION

Hospital performance management is a multi-dimensional issue, each dimension having its own significance. Based on the evidence, indicators are dependent on the evaluation model employed, the evaluation objective, and the views of executive managers and participants in the study. Selection of the most appropriate indicators is therefore key to a comprehensive performance evaluation system.

Is the readmission rate a valid quality indicator? A review of the evidence

INTRODUCTION

Hospital readmission rates are increasingly used for both quality improvement and cost control. However, the validity of readmission rates as a measure of quality of hospital care is not evident. We aimed to give an overview of the different methodological aspects in the definition and measurement of readmission rates that need to be considered when interpreting readmission rates as a reflection of quality of care.

METHODS

We conducted a systematic literature review, using the bibliographic databases Embase, Medline OvidSP, Web-of-Science, Cochrane central and PubMed for the period of January 2001 to May 2013.

RESULTS

The search resulted in 102 included papers. We found that definition of the context in which readmissions are used as a quality indicator is crucial. This context includes the patient group and the specific aspects of care of which the quality is aimed to be assessed. Methodological flaws like unreliable data and insufficient case-mix correction may confound the comparison of readmission rates between hospitals. Another problem occurs when the basic distinction between planned and unplanned readmissions cannot be made. Finally, the multi-faceted nature of quality of care and the correlation between readmissions and other outcomes limit the indicator's validity.

CONCLUSIONS

Although readmission rates are a promising quality indicator, several methodological concerns identified in this study need to be addressed, especially when the indicator is intended for accountability or pay for performance. We recommend investing resources in accurate data registration, improved indicator description, and bundling outcome measures to provide a more complete picture of hospital care.

Advancing the use of performance evaluation in health care

PURPOSE

The purpose of this paper is to develop a framework for health care performance evaluation that enables decision makers to identify areas indicative of corrective actions. The framework should provide information on strategic pro-/regress in an operational context that justifies the need for organizational adjustments.

DESIGN/METHODOLOGY/APPROACH

The study adopts qualitative methods for constructing the framework, subsequently implementing the framework in a Danish magnetic resonance imaging (MRI) unit. Workshops and interviews form the basis of the qualitative construction phase, and two internal and five external databases are used for a quantitative data collection.

FINDINGS

By aggregating performance outcomes, collective measures of performance are achieved. This enables easy and intuitive identification of areas not strategically aligned. In general, the framework has proven helpful in an MRI unit, where operational decision makers have been struggling with extensive amounts of performance information.

RESEARCH LIMITATIONS/IMPLICATIONS

The implementation of the framework in a single case in a public and highly political environment restricts the generalizing potential. The authors acknowledge that there may be more suitable approaches in organizations with different settings.

PRACTICAL IMPLICATIONS

The strength of the framework lies in the identification of performance problems prior to decision making. The quality of decisions is directly related to the individual decision maker. The only function of the framework is to support these decisions.

ORIGINALITY/VALUE

The study demonstrates a more refined and transparent use of performance reporting by combining strategic weight assignment and performance aggregation in hierarchies. In this way, the framework accentuates performance as a function of strategic progress or regress, thus assisting decision makers in exerting operational effort in pursuit of strategic alignment.

Diabetic retinopathy care--an international quality comparison

PURPOSE

The purpose of this paper is to study the quality indicator appropriateness and use it for international quality comparison on diabetic retinopathy (DR) patient care process in one American and one Dutch eye hospital.

DESIGN/METHODOLOGY/APPROACH

A 17-item DR quality indicator set was composed based on a literature review and systematically applied in two hospitals. Qualitative analysis entailed document study and 12 semi-structured face-to-face interviews with ophthalmologists, managers, and board members of the two hospitals.

FINDINGS

While the medical-clinical approach to DR treatment in both hospitals was similar, differences were found in quality of care perception and operationalization. Neither hospital systematically used outcome indicators for DR care. On the process level, the authors found larger differences. Similarities and differences were found in the structure of both hospitals. The hospitals' particular contexts influenced the interpretation and use of quality indicators.

PRACTICAL IMPLICATIONS

Although quality indicators and quality comparison between hospitals are increasingly used in international settings, important local differences influence their application. Context should be taken into account. Since that context is locally bound and directly linked to hospital setting, caution should be used interpreting the results of quality comparison studies.

ORIGINALITY/VALUE

International quality comparison is increasingly suggested as a useful way to improve healthcare. Little is known, however, about the appropriateness and use of quality indicators in local hospital care practices.

A theoretical framework for holistic hospital management in the Japanese healthcare context

OBJECTIVE

This paper develops a conceptual framework for performance measurement as a pilot study on holistic hospital management in the Japanese healthcare context.

METHODS

We primarily used two data sources as well as expert statements obtained through interviews: a systematic review of literature and a questionnaire survey to healthcare experts. The systematic survey searched PubMed and PubMed Central, and 24 relevant papers were elicited. The expert questionnaire asked respondents to rate the degree of "usefulness" for each of 66 indicators on a three-point scale.

RESULTS

Applying the theoretical framework, a minimum set of performance indicators was selected for holistic hospital management, which well fit the healthcare context in Japan. This indicator set comprised 35 individual indicators and several factors measured through questionnaire surveys. The indicators were confirmed by expert judgments from viewpoints of face, content and construct validities as well as their usefulness.

CONCLUSION

A theoretical framework of performance measurement was established from primary healthcare stakeholders' perspectives. Performance indicators were largely divided into healthcare outcomes and performance shaping factors. Indicators in the former category may be applied for the detection of operational problems, while their latent causes can be effectively addressed by the latter category in terms of process, structure and culture/climate within the organization.

Prioritizing public- private partnership models for public hospitals of iran based on performance indicators

BACKGROUND

The present study was conducted to scrutinize Public- Private Partnership (PPP) models in public hospitals of different countries based on performance indicators in order to se-lect appropriated models for Iran hospitals.

METHODS

In this mixed (quantitative-qualitative) study, systematic review and expert panel has been done to identify varied models of PPP as well as performance indicators. In the second step we prioritized performance indicator and PPP models based on selected performance indicators by Analytical Hierarchy process (AHP) technique. The data were analyzed by Excel 2007 and Expert Choice11 software's.

RESULTS

In quality - effectiveness area, indicators like the rate of hospital infections (100%), hospital accidents prevalence rate (73%), pure rate of hospital mortality (63%), patient satisfaction percentage (53%), in accessibility equity area indicators such as average inpatient waiting time (100%) and average outpatient waiting time (74%), and in financial - efficiency area, indicators including average length of stay (100%), bed occupation ratio (99%), specific income to total cost ratio (97%) have been chosen to be the most key performance indicators. In the pri¬oritization of the PPP models clinical outsourcing, management, privatization, BOO (build, own, operate) and non-clinical outsourcing models, achieved high priority for various performance in¬dicator areas.

CONCLUSION

This study had been provided the most common PPP options in the field of public hospitals and had gathered suitable evidences from experts for choosing appropriate PPP option for public hospitals. Effect of private sector presence in public hospital performance, based on which PPP options undertaken, will be different.

A Contingency Approach to the Strategic Management of Intangible Resources

Not with standing the increasing awareness of the importance of intangible resources for the success of organisations, there is a paucity of research investigating the impact of contingent factors on the strategic management of such resources. This research therefore aims to address this issue by investigating the contingencies which come into play in hospital organisations. It does so by using a case study approach to examine the impact of industry-specific and organisation-specific factors on management activities. It was found that factors such as task complexity and staff profile play a significant role in regulated management contexts making the adjustment of existing approaches an indispensable task for general managers who aim to achieve sustainable competitive advantage. The paper contributes to various disciplines. For example, it contributes to the literature on resource management, by proposing a framework for the effective management of intangible resources in the light of the impact of contingent factors.

Understanding and benchmarking health service achievement of policy goals for chronic disease

BACKGROUND

Key challenges in benchmarking health service achievement of policy goals in areas such as chronic disease are: 1) developing indicators and understanding how policy goals might work as indicators of service performance; 2) developing methods for economically collecting and reporting stakeholder perceptions; 3) combining and sharing data about the performance of organizations; 4) interpreting outcome measures; 5) obtaining actionable benchmarking information. This study aimed to explore how a new Boolean-based small-N method from the social sciences-Qualitative Comparative Analysis or QCA-could contribute to meeting these internationally shared challenges.

METHODS

A 'multi-value QCA' (MVQCA) analysis was conducted of data from 24 senior staff at 17 randomly selected services for chronic disease, who provided perceptions of 1) whether government health services were improving their achievement of a set of statewide policy goals for chronic disease and 2) the efficacy of state health office actions in influencing this improvement. The analysis produced summaries of configurations of perceived service improvements.

RESULTS

Most respondents observed improvements in most areas but uniformly good improvements across services were not perceived as happening (regardless of whether respondents identified a state health office contribution to that improvement). The sentinel policy goal of using evidence to develop service practice was not achieved at all in four services and appears to be reliant on other kinds of service improvements happening.

CONCLUSIONS

The QCA method suggested theoretically plausible findings and an approach that with further development could help meet the five benchmarking challenges. In particular, it suggests that achievement of one policy goal may be reliant on achievement of another goal in complex ways that the literature has not yet fully accommodated but which could help prioritize policy goals. The weaknesses of QCA can be found wherever traditional big-N statistical methods are needed and possible, and in its more complex and therefore difficult to empirically validate findings. It should be considered a potentially valuable adjunct method for benchmarking complex health policy goals such as those for chronic disease.