Selecting disease-outcome pairs for monitoring the quality of hospital care

A Framework Proposal for Quality and Safety Measurement in Gynecologic Emergency Care

Potentially life-threatening gynecologic emergencies are high-risk conditions that may serve as a useful framework to improve quality and safety in emergency care.

OBJECTIVE:

To define and assess the prevalence of potentially life-threatening gynecologic emergencies among women presenting for acute pelvic pain for the purpose of developing measures to audit quality of care in emergency departments.

METHODS:

We conducted a mixed-methods multicenter study at gynecologic emergency departments in France and Belgium. A modified Delphi procedure was first conducted in 2014 among health care professionals to define relevant combinations of potentially life-threatening conditions and near misses in the field of gynecologic emergency care. A prospective case–cohort study in the spring of 2015 then assessed the prevalence of these potentially life-threatening emergencies and near misses among women of reproductive age presenting for acute pelvic pain. Women in the case group were identified at 21 participating centers. The control group consisted of a sample of women hospitalized for acute pelvic pain not caused by a potentially life-threatening condition and a 10% random sample of outpatients.

RESULTS:

Eight gynecologic emergencies and 17 criteria for near misses were identified using the Delphi procedure. Among the 3,825 women who presented for acute pelvic pain, 130 (3%) were considered to have a potentially life-threatening condition. The most common diagnoses were ectopic pregnancies with severe bleeding (n=54; 42%), complex pelvic inflammatory disease (n=30; 23%), adnexal torsion (n=20; 15%), hemorrhagic miscarriage (n=15; 12%), and severe appendicitis (n=6; 5%). The control group comprised 225 hospitalized women and 381 outpatients. Diagnostic errors occurred more frequently among women with potentially life-threatening emergencies than among either hospitalized (odds ratio [OR] 1.7, 95% CI 1.1–2.7) or outpatient (OR 14.7, 95% CI 8.1–26.8) women in the control group. Of the women with potentially life-threatening conditions, 26 met near-miss criteria compared with six with not potentially life-threatening conditions (OR 25.6, 95% CI 10.9–70.7).

CONCLUSIONS:

Potentially life-threatening gynecologic emergencies are high-risk conditions that may serve as a useful framework to improve quality and safety in emergency care.

Validation of data and indicators in the Danish Cholecystectomy Database

OBJECTIVE

In The Danish Cholecystectomy Database (DCD), quality indicators are derived from clinical data in combination with administrative data from the National Patient Registry. The indicators 'Length of postoperative stay < or =1 day and no readmission', 'Length of stay (LOS) >3 days and/or readmission', 'Additional procedures within 30 days', 'Reconstructive bile duct surgery', 'Other surgery of the bile duct' and 'Death within 30 days' are all derived from administrative data. This study investigates the validity of the administrative data and evaluates the association between these indicators and postoperative complications.

RESEARCH DESIGN AND SUBJECTS

Data from 1360 medical records of patients undergoing cholecystectomy were compared with the relevant administrative data from the National Patient Registry. The medical records served as the 'gold standard'. The association between the individual indicators and the occurrence of a postoperative complication was assessed.

MEASURES

Validation of administrative data against the gold standard was done by the calculation of per cent agreement (including kappa-values) sensitivity/specificity and predictive values. The association between indicators and complications was analysed with crude event rates and odds ratios.

RESULTS

The validity of the administrative data was excellent (97.1-100% agreement, kappa = 0.73-1.00). All of the indicators except 'Other bile duct surgery' were significantly associated with postoperative complications. A subdivision of some indicators strengthened the associations.

CONCLUSIONS

The DCD is a valid method for monitoring the quality of cholecystectomy in Denmark.

Validation of the potentially avoidable hospital readmission rate as a routine indicator of the quality of hospital care

BACKGROUND

The hospital readmission rate has been proposed as an important outcome indicator computable from routine statistics. However, most commonly used measures raise conceptual issues.

OBJECTIVES

We sought to evaluate the usefulness of the computerized algorithm for identifying avoidable readmissions on the basis of minimum bias, criterion validity, and measurement precision.

RESEARCH DESIGN AND SUBJECTS

A total of 131,809 hospitalizations of patients discharged alive from 49 hospitals were used to compare the predictive performance of risk adjustment methods. A subset of a random sample of 570 medical records of discharge/readmission pairs in 12 hospitals were reviewed to estimate the predictive value of the screening of potentially avoidable readmissions.

MEASURES

Potentially avoidable readmissions, defined as readmissions related to a condition of the previous hospitalization and not expected as part of a program of care and occurring within 30 days after the previous discharge, were identified by a computerized algorithm. Unavoidable readmissions were considered as censored events.

RESULTS

A total of 5.2% of hospitalizations were followed by a potentially avoidable readmission, 17% of them in a different hospital. The predictive value of the screen was 78%; 27% of screened readmissions were judged clearly avoidable. The correlation between the hospital rate of clearly avoidable readmission and all readmissions rate, potentially avoidable readmissions rate or the ratio of observed to expected readmissions were respectively 0.42, 0.56 and 0.66. Adjustment models using clinical information performed better.

CONCLUSION

Adjusted rates of potentially avoidable readmissions are scientifically sound enough to warrant their inclusion in hospital quality surveillance.

Readmission rate as an indicator of hospital performance: The case of Spain

Objectives:Hospital readmission rate is currently used as a quality of care indicator, although its validity has not been established. Our aims were to identify the frequency and characteristics of potential avoidable readmissions and to compare the assessment of quality of care derived from readmission rate with other measure of quality (judgment of experts).

Methods:Design: cross-sectional observational study; Setting: acute care hospital located in Marbella, South of Spain; Study participants: random sample of patients readmitted at the hospital within six months from discharge (n=363); Interventions: review of clinical records by a pair of observers to assess the causes of readmissions and their potential avoidability; Main measures: logistic regression analysis to identify the variables from the databases of hospital discharges which are related to avoidability of readmissions. Determination of sensitivity and specificity of different definitions of readmission rate to detect avoidable situations.

Results:Nineteen percent of readmissions were considered potentially avoidable. Variables related to readmission avoidability were (i) time elapsed between index admission and readmission and (ii) difference in diagnoses of both episodes. None of the definitions of readmission rate used in this study provided adequate values of sensitivity and specificity in the identification of potentially avoidable readmissions.

Conclusions:Most readmissions in our hospital were unavoidable. Thus, readmission rate might not be considered a valid indicator of quality of care.

Are diagnosis specific outcome indicators based on administrative data useful in assessing quality of hospital care?

BACKGROUND

Hospital performance reports based on administrative data should distinguish differences in quality of care between hospitals from case mix related variation and random error effects. A study was undertaken to determine which of 12 diagnosis-outcome indicators measured across all hospitals in one state had significant risk adjusted systematic (or special cause) variation (SV) suggesting differences in quality of care. For those that did, we determined whether SV persists within hospital peer groups, whether indicator results correlate at the individual hospital level, and how many adverse outcomes would be avoided if all hospitals achieved indicator values equal to the best performing 20% of hospitals.

METHODS

All patients admitted during a 12 month period to 180 acute care hospitals in Queensland, Australia with heart failure (n = 5745), acute myocardial infarction (AMI) (n = 3427), or stroke (n = 2955) were entered into the study. Outcomes comprised in-hospital deaths, long hospital stays, and 30 day readmissions. Regression models produced standardised, risk adjusted diagnosis specific outcome event ratios for each hospital. Systematic and random variation in ratio distributions for each indicator were then apportioned using hierarchical statistical models.

RESULTS

Only five of 12 (42%) diagnosis-outcome indicators showed significant SV across all hospitals (long stays and same diagnosis readmissions for heart failure; in-hospital deaths and same diagnosis readmissions for AMI; and in-hospital deaths for stroke). Significant SV was only seen for two indicators within hospital peer groups (same diagnosis readmissions for heart failure in tertiary hospitals and inhospital mortality for AMI in community hospitals). Only two pairs of indicators showed significant correlation. If all hospitals emulated the best performers, at least 20% of AMI and stroke deaths, heart failure long stays, and heart failure and AMI readmissions could be avoided.

CONCLUSIONS

Diagnosis-outcome indicators based on administrative data require validation as markers of significant risk adjusted SV. Validated indicators allow quantification of realisable outcome benefits if all hospitals achieved best performer levels. The overall level of quality of care within single institutions cannot be inferred from the results of one or a few indicators.

Measuring potentially avoidable hospital readmissions

The objectives of this study were to develop a computerized method to screen for potentially avoidable hospital readmissions using routinely collected data and a prediction model to adjust rates for case mix. We studied hospital information system data of a random sample of 3,474 inpatients discharged alive in 1997 from a university hospital and medical records of those (1,115) readmitted within 1 year. The gold standard was set on the basis of the hospital data and medical records: all readmissions were classified as foreseen readmissions, unforeseen readmissions for a new affection, or unforeseen readmissions for a previously known affection. The latter category was submitted to a systematic medical record review to identify the main cause of readmission. Potentially avoidable readmissions were defined as a subgroup of unforeseen readmissions for a previously known affection occurring within an appropriate interval, set to maximize the chance of detecting avoidable readmissions. The computerized screening algorithm was strictly based on routine statistics: diagnosis and procedures coding and admission mode. The prediction was based on a Poisson regression model. There were 454 (13.1%) unforeseen readmissions for a previously known affection within 1 year. Fifty-nine readmissions (1.7%) were judged avoidable, most of them occurring within 1 month, which was the interval used to define potentially avoidable readmissions (n = 174, 5.0%). The intra-sample sensitivity and specificity of the screening algorithm both reached approximately 96%. Higher risk for potentially avoidable readmission was associated with previous hospitalizations, high comorbidity index, and long length of stay; lower risk was associated with surgery and delivery. The model offers satisfactory predictive performance and a good medical plausibility. The proposed measure could be used as an indicator of inpatient care outcome. However, the instrument should be validated using other sets of data from various hospitals.

Factors associated with unplanned readmissions following cardiac surgery

Cardiac surgery patients are at risk for unplanned readmissions due to the various complications they may experience following surgery. The purpose of this report is to critically review the literature related to predictors of unplanned readmissions of cardiac surgery patients following discharge from the hospital. A literature review was conducted from 1989 to 1999 using MEDLINE and CINAHL, with the following key words: cardiac surgery, coronary artery bypass surgery, recovery, and readmission. The literature revealed that gender and race do have an effect on how well a patient will recover following cardiac surgery. It was also found that patients with longer lengths of stay due to complications were at greater risk for readmission following discharge from the hospital. There was no evidence that decreased length of stay for this patient group led to a greater number of readmissions. Implications for nurses include the need for improved coordination of patient care and implementation of effective discharge planning in high-risk patients. Additional research is needed to develop interventions to decrease readmissions of women and African Americans and other racial groups specific to their particular risk factors for readmission following cardiac surgery.

Unplanned readmissions of patients with congestive heart failure: do they reflect in-hospital quality of care or patient characteristics?

PURPOSE

To determine if early unplanned readmissions of patients hospitalized for heart failure are associated with suboptimal in-hospital care or with the clinical and demographic characteristics of the patient.

SUBJECT AND METHODS

We performed a case-control study among patients discharged with a principal diagnosis of heart failure. Cases included all patients unexpectedly readmitted within 31 days of discharge; controls were randomly selected from among those not readmitted. Quality of care was measured using explicit criteria reflecting the admission work-up, evaluation and treatment, and readiness for discharge.

RESULTS

Ninety-one cases and 351 controls were included. There was no significant association between early unplanned readmissions and the scores for quality of the admission work-up or evaluation and treatment during the stay. There was a significant association between readiness for discharge and subsequent early readmission: for each 10% decrease in the proportion of fulfilled criteria, the odds of readmission increased by 14% (95% confidence interval [CI] 1. 01 to 1.28, P = 0.04) for all-cause readmissions and by 19% (95% CI: 1.04 to 1.36, P = 0.01) for heart-failure-related readmissions. In a multiple logistic regression model, previous diagnosis of heart failure (odds ratio [OR] = 2.9, 95% CI: 1.7 to 4.8, P <0.001), age (OR = 3.3, 95% CI: 1.3 to 8.5, P = 0.01 for patients aged 65 to 79 years and OR = 4.1, 95% CI: 1.6 to 11, P = 0.004 for patients aged 80 years and older), and history of cardiac revascularization (OR = 2.1, 95% CI: 1.2 to 3.9, P = 0.01) showed a stronger association with early unplanned all-cause readmissions than the readiness-for-discharge score (OR = 1.16, 95% CI: 1.02 to 1.31, P = 0.02). Similar findings were seen for heart failure-related readmissions.

CONCLUSIONS

Among patients with heart failure, early unplanned readmissions were not associated with suboptimal admission work-up or evaluation and treatment but were weakly associated with readiness for discharge. However, they were strongly associated with the patients' clinical and demographic characteristics.

Prediction of readmissions after CABG using detailed follow-up data: the Israeli CABG Study (ISCAB)

OBJECTIVE

To use detailed pre-discharge follow-up data to predict readmissions within 3 months after Coronary Artery Bypass Grafting (CABG).

SETTINGS AND DESIGN

A prospective nationwide study (ISCAB) of 4,835 patients undergoing isolated CABG in Israel in 1994. Survivors of the initial hospitalization were candidates for the readmission study.

METHODS

Patient information was prospectively collected from preoperative interviews and hospital follow-up. Readmissions' data were obtained from the National Hospital Admission Registry. Logistic and multinomial models were constructed for total and cause-specific readmissions, respectively.

RESULTS

Of CABG survivors, 1,094 (24.1%) were rehospitalized within 3 months of the original surgery. Significant multivariate predictors of total readmissions included the following: preoperative co-morbidities; operative factors; immediate post-operative complications and socio-demographic characteristics as well as provider characteristics. However, the logistic model had low predictive power (c-statistic = 0.65). The heterogeneous reasons for readmissions were classified into specific serious cardiac diagnoses (19.0%), other cardiac reasons (35.4%), specific infections at the site of the operation (10.2%), other infections (7.3%), and various other reasons (23.0%). The multinomial model for cause-specific readmissions caused by either serious cardiac reasons or wound infection had a higher predictive value (c-statistics of 0.75, 0.72, respectively).

CONCLUSIONS

Total readmissions after CABG in Israel were difficult to predict, even with an extensive pre-discharge follow-up data. We propose that reasons for readmission vary from true emergencies to nonspecific causes, with the latter related to a lack of support services in the community. We suggest that cause-specific rehospitalizations could be a better outcome for evaluating quality of care.

Comparison between planned and unplanned readmissions to a department of internal medicine

The objective of this study was to assess the respective frequency of planned and unplanned early readmissions after discharge from an internal medicine department, and to identify and compare risk factors for these two types of readmissions. Readmissions within 31 days of discharge were identified as planned or unplanned based on analysis of discharge summaries. Time-failure methods were used to describe the risk of readmissions over time and to assess relationships between patient and index stay characteristics and risk of readmission. Of 5828 patients discharged alive, 730 (12.5%) were readmitted within 31 days. There were slightly more planned than unplanned readmissions (393 vs. 337). The difference in time-to-event functions was significant (P=0.04). The risk of planned readmission was increased for men, younger patients, and for patients discharged with a diagnosis of coronary heart disease, cardiac arrhythmia, and neoplastic disease. Increased risk of unplanned readmission was associated with index length of stay longer than 3 days, an increased number of comorbidities, and with a diagnosis of neoplastic disease. Planned readmissions constitute more than half of early readmissions to our internal medicine department. Therefore, a crude readmission rate is unlikely to be a useful indicator of quality of care. Several patient characteristics influence the risk of unplanned readmission, suggesting that case-mix adjustments are necessary when readmission rates are compared between institutions or tracked over time.

Application of an analytic model to early readmission rates within the Department of Veterans Affairs

OBJECTIVES

Adverse outcome rates are increasingly used as yardsticks for the quality of hospital care. However, the validity of many outcome studies has been undermined by the application of one outcome to all patients in large, diagnostically diverse populations, many of which lack evidence of a link between antecedent process of care and the rate of the outcome, the underlying assumption of the analysis.

METHODS

To address this analytic problem, the authors developed a model that improves the ability to identify quality problems because it selects diseases for which there are processes of care known to affect the outcome of interest. Thus, for these diseases, the outcome is most likely to be causally related to the antecedent care. In this study of hospital readmissions, risk-adjusted models were created for 17 disease categories with strong links between process and outcome. Using these models, we identified outlier hospitals.

RESULTS

The authors hypothesized that if the model improved on identifying hospitals with quality of care problems, then outlier status would not be random. That is, hospitals found to have extreme rates in one year would be more likely to have extreme rates in subsequent years, and hospitals with extreme rates in one condition would be more likely to have extreme rates in related disease categories. It was hypothesized further that the correlation of outlier status across time and across diseases would be stronger in the 17 disease categories selected by the model than in 10 comparison disease categories with weak links between process and outcome.

CONCLUSIONS

The findings support all these hypotheses. Although the present study shows that the model selects disease-outcome pairs where hospital outlier status is not random, the causal factors leading to outlier status could include (1) systematic unmeasured patient variation, (2) practice pattern variation that, although stable with time, is not indicative of substandard care, or (3) true quality-of-care problems. Primary data collection must be done to determine which of these three factors is most causally related to hospital outlier status.

Correlates of early hospital readmission or death in patients with congestive heart failure

Among patients with heart failure who survive an admission to the hospital, those who are readmitted or die soon after discharge may warrant special attention. Therefore, we prospectively followed 257 patients admitted nonelectively to an urban university hospital, with a complaint of shortness of breath or fatigue and evidence of congestive heart failure on admission chest radiograph, who were discharged alive. Through survey of patients and families, review of the hospital computer system, and a search of the National Death Index, we recorded death and hospital readmission. Within 60 days of discharge, 13 patients (5%) died and 82 (32%) died or were readmitted to the hospital. Using Cox proportional-hazards modeling, the multivariable correlates of readmission or death were single marital status (adjusted hazard ratio [HR] 2.1, 95% confidence interval [CI] 1.3 to 3.3), Charlson Comorbidity Index score (HR 1.3 per point to maximum 4 points, 95% CI 1.1 to 1.6), admission systolic blood pressure of < or = 100 mm Hg (HR 2.8, 95% CI 1.6 to 5.0), and absence of new ST-T-wave changes on the initial electrocardiogram (HR 1.9, 95% CI 1.1 to 3.3). Self-reported patient compliance and clinical instability at discharge were not correlates. Almost all patients stratified by these factors had at least a 25% risk of readmission or death. Our independent correlates of readmission or death support the importance of both medical and social factors in the pathway to clinical decline. However, we could not reliably identify a truly low-risk group. Interventions to decrease early readmission or death among patients with heart failure should target both medical management and the adequacy of social support, and probably need to be applied to all admitted patients.

Modeling case mix adjustment of stroke rehabilitation outcomes

Case mix adjustment models for long-term stroke rehabilitation outcomes should be developed (1) to facilitate equitable comparisons of outcomes across treatment settings, thereby reducing disincentives for treating complex cases, (2) to improve triage into the most appropriate level of rehabilitative care after discharge from acute care, and (3) to confirm that case mix factors are equated in treatment effectiveness studies and by random assignment across conditions in clinical trials. Case mix adjustment is necessary for valid quality improvement processes. A conceptual model of case mix adjustment of long-term rehabilitation outcomes is presented that (1) is diagnosis-specific, (2) includes demographic variables as important case mix factors, (3) encompasses triage into rehabilitation as well as treatment processes as aspects of quality of rehabilitative care, (4) contains outcomes measuring functional status as well as mortality and morbidity, and (5) keys timing of outcomes to onset of conditions requiring rehabilitation rather than discharge from rehabilitation. The number of potential interactions among case mix indicators requires a sophisticated analytic framework. Random factors in the model illustrate that case mix adjustment can never be perfect. Nevertheless, it is essential. A brief review of the stroke literature on prediction of long-term outcomes suggests that additional work is needed to specify relevant case mix indicators.

Skill mix literature review

Administrators are currently being challenged to maintain high quality patient care in the face of shrinking health care resources. The introduction of different skill mix ratios has been suggested as one way to help manage health care costs. This paper briefly reviews the literature and research data on skill mix, discussing the relevant issues and identifying the positive and negative implications of this approach. It concludes with suggestions for further research.

A conceptual framework for the study of early readmission as an indicator of quality of care

Despite the perennial popularity of readmission as an indicator of the quality of hospital care, the empiric evidence linking it to process-of-care problems during the prior hospitalization is inconsistent. We devised a conceptual model for the use of unscheduled readmission within 31 days as an indicator of the quality of medical-surgical inpatient care for adults, and then conducted a systematic review of the readmission literature to determine the extent to which the evidence supports the proposed relationships. A fairly complex web of relationships influences the association between the process of inpatient care and early readmission. From the evidence to date, it is impossible to say with confidence that early readmission is or is not a valid and useful quality indicator. In most negative studies, the absence of an association appears to be explainable on the basis of improper study design, omission of important variables, or mis-specification of variables. Variables intervening between or confounding the relationship of the process of inpatient care to early readmission have received inadequate attention in past work. Investigators can use the proposed model and literature review to ensure their work advances the field and puts the hypothesis that early readmission is a valid quality indicator to a rigorous test. This matter has a certain urgency in view of the vast amount of resources that providers and payers devote to monitoring readmission rates and reviewing readmissions.