The design and analysis of hospital utilization studies

Reliability and Validity of the Chinese Version Appropriateness Evaluation Protocol

Objective

To adapt the Appropriateness Evaluation Protocol (AEP) to the specific settings of health care in China and to validate the Chinese version AEP (C-AEP).

Methods

Forward and backward translations were carried out to the original criteria. Twenty experts participated in the consultancy to form a preliminary version of the C-AEP. To ensure applicability, tests of reliability and validity were performed on 350 admissions and 3,226 hospital days of acute myocardial infraction patients and total hip replacement patients in two tertiary hospitals by two C-AEP reviewers and two physician reviewers. Overall agreement, specific agreement, and Cohen’s Kappa were calculated to compare the concordance of decisions between pairs of reviewers to test inter-rater reliability and convergent validity. The use of “overrides” and opinions of experts were recorded as measurements of content validity. Face validity was tested through collecting perspectives of nonprofessionals. Sensitivity, specificity, and predictive values were also reported.

Results

There are 14 admission and 24 days of care criteria in the initial version of C-AEP. Kappa coefficients indicate substantial agreement between reviewers: with regard to inter-rater reliability, Kappa (κ) coefficients are 0.746 (95% confidence interval [CI] 0.644–0.834) and 0.743 (95% CI 0.698–0.767) of admission and hospital days, respectively; for convergent validity, the κ statistics are 0.678 (95% CI 0.567–0.778) and 0.691 (95% CI 0.644–0.717), respectively. Overrides account for less than 2% of all judgments. Content validity and face validity were confirmed by experts and nonprofessionals, respectively. According to the C-AEP reviewers, 18.3% of admissions and 28.5% of inpatient days were deemed inappropriate.

Conclusions

The C-AEP is a reliable and valid screening tool in China’s tertiary hospitals. The prevalence of inappropriateness is substantial in our research. To reduce inappropriate utilization, further investigation is needed to elucidate the reasons and risk factors for this inappropriateness.

Effectiveness of feedback to physicians in reducing inappropriate use of hospitalization: a study in a Spanish hospital

OBJECTIVES

To evaluate the effectiveness of feedback to medical staff in reducing inappropriate hospital days, particularly those attributable to conservative medical discharge policies.

DESIGN

Quasi-experimental pre-test/post-test with non-equivalent control group.

SETTING

A publicly funded hospital in industrial belt in Barcelona (Spain), serving a predominantly urban population of 100,000.

STUDY PARTICIPANTS

Two non-equivalent groups: control group (surgery department) and intervention group (internal medicine department).

INTERVENTION

Meetings between hospital management and medical staff of the intervention group to inform clinicians of percentages and reasons for inappropriate stays in their departments.

MAIN OUTCOME MEASURES

Total inappropriate hospital days and percentage attributable to physicians, measured with the Appropriateness Evaluation Protocol before, during, and after intervention.

RESULTS

There were no relevant differences in the characteristics of the populations whose stays were reviewed during each of the periods. The total number of inappropriate stays and the percentage attributable to the doctor in the control group did not show any differences between the periods. In the intervention group, inappropriate stays attributable to the doctor decreased from 35.9% in the period to intervention to 27.7% during intervention (relative drop of 22.8%; P < 0.01), and rose to 32.7% after intervention. Differences in total inappropriate days were not significant.

CONCLUSIONS

Providing physicians with feedback about percentage of inappropriate hospital days produced a significant reduction in the number of inappropriate stays attributable to the doctor, although the impact on overall inappropriate stays is inconclusive.