Accuracy of hospital report cards based on administrative data

Use of Angiotensin-Converting Enzyme Inhibitors and Angiotensin Receptor Blockers for Geriatric Ischemic Stroke Patients: Are the Rates Right?

BACKGROUND

Our objective is to estimate the effects associated with higher rates of renin-angiotensin system antagonists, angiotensin-converting enzyme inhibitors and angiotensin receptor blockers (ACEI/ARBs), in secondary prevention for geriatric (aged >65 years) patients with new ischemic strokes by chronic kidney disease (CKD) status.

METHODS AND RESULTS

The effects of ACEI/ARBs on survival and renal risk were estimated by CKD status using an instrumental variable (IV) estimator. Instruments were based on local area variation in ACEI/ARB use. Data abstracted from charts were used to assess the assumptions underlying the instrumental estimator. ACEI/ARBs were used after stroke by 45.9% and 45.2% of CKD and non-CKD patients, respectively. ACEI/ARB rate differences across local areas grouped by practice styles were nearly identical for CKD and non-CKD patients. Higher ACEI/ARB use rates for non-CKD patients were associated with higher 2-year survival rates, whereas higher ACEI/ARB use rates for patients with CKD were associated with lower 2-year survival rates. While the negative survival estimates for patients with CKD were not statistically different from zero, they were statistically lower than the estimates for non-CKD patients. Confounders abstracted from charts were not associated with the instrumental variable used.

CONCLUSIONS

Higher ACEI/ARB use rates had different survival implications for older ischemic stroke patients with and without CKD. ACEI/ARBs appear underused in ischemic stroke patients without CKD as higher use rates were associated with higher 2-year survival rates. This conclusion is not generalizable to the ischemic stroke patients with CKD, as higher ACEI/ARBS use rates were associated with lower 2-year survival rates that were statistically lower than the estimates for non-CKD patients.

The National Cardiovascular Data Registry Voluntary Public Reporting Program: An Interim Report From the NCDR Public Reporting Advisory Group

Public reporting of health care data continues to proliferate as consumers and other stakeholders seek information on the quality and outcomes of care. Medicare's Hospital Compare website, the U.S. News & World Report hospital rankings, and several state-level programs are well known. Many rely heavily on administrative data as a surrogate to reflect clinical reality. Clinical data are traditionally more difficult and costly to collect, but more accurately reflect patients' clinical status, thus enhancing the validity of quality metrics. We describe the public reporting effort being launched by the American College of Cardiology and partnering professional organizations using clinical data from the National Cardiovascular Data Registry (NCDR) programs. This hospital-level voluntary effort will initially report process of care measures from the percutaneous coronary intervention (CathPCI) and implantable cardioverter-defibrillator (ICD) registries of the NCDR. Over time, additional process, outcomes, and composite performance metrics will be reported.

Risk factors and outcomes associated with new-onset atrial fibrillation during acute respiratory distress syndrome

PURPOSE

Outcomes and risk factors associated with new-onset atrial fibrillation (AF) during acute respiratory distress syndrome (ARDS) are unclear. We investigated mortality and risk factors associated with new-onset AF during ARDS.

MATERIALS AND METHODS

We obtained data from the ARDS Network Albuterol for Treatment of Acute Lung Injury trial, which prospectively identified new-onset AF among patients with ARDS as an adverse event. We determined Acute Physiology and Chronic Health Evaluation III-adjusted associations between new-onset AF and 90-day mortality. We also examined associations between new-onset AF and markers of inflammation (interleukin 6 and interleukin 8), myocardial injury (troponin I), autonomic activation (epinephrine), and atrial stretch (central venous pressure) as well as other clinical characteristics.

MEASUREMENTS AND MAIN RESULTS

Of 282 patients (mean age, 51.6 years; 45% women; 77% white) enrolled in Albuterol for Treatment of Acute Lung Injury, 28 (10%) developed new-onset AF during the study. We did not identify associations between new-onset AF and baseline central venous pressure, plasma levels of troponin I, epinephrine, interleukin 6, or interleukin 8. New-onset AF during ARDS was associated with increased 90-day mortality (new-onset AF, 43% vs no new-onset AF, 19%; Acute Physiology and Chronic Health Evaluation-adjusted odds ratio, 3.09 [95% confidence interval, 1.24-7.72]; P = .02).

CONCLUSION

New-onset AF during ARDS is associated with increased mortality; however, its mechanisms require further study.

Procedure-based severity index for inpatients: development and validation using administrative database

Background

Risk adjustment is important in studies using administrative databases. Although utilization of diagnostic and therapeutic procedures can represent patient severity, the usability of procedure records in risk adjustment is not well-documented. Therefore, we aimed to develop and validate a severity index calculable from procedure records.

Methods

Using the Japanese nationwide Diagnosis Procedure Combination database of acute-care hospitals, we identified patients discharged between 1 April 2012 and 31 March 2013 with an admission-precipitating diagnosis of acute myocardial infarction, congestive heart failure, acute cerebrovascular disease, gastrointestinal hemorrhage, pneumonia, or septicemia. Subjects were randomly assigned to the derivation cohort or the validation cohort. In the derivation cohort, we used multivariable logistic regression analysis to identify procedures performed on admission day which were significantly associated with in-hospital death, and a point corresponding to regression coefficient was assigned to each procedure. An index was then calculated in the validation cohort as sum of points for performed procedures, and performance of mortality-predicting model using the index and other patient characteristics was evaluated.

Results

Of the 539 385 hospitalizations included, 270 054 and 269 331 were assigned to the derivation and validation cohorts, respectively. Nineteen significant procedures were identified from the derivation cohort with points ranging from −3 to 23, producing a severity index with possible range of −13 to 69. In the validation cohort, c-statistic of mortality-predicting model was 0.767 (95 % confidence interval: 0.764–0.770). The ω-statistic representing contribution of the index relative to other variables was 1.09 (95 % confidence interval: 1.03–1.17).

Conclusions

Procedure-based severity index predicted mortality well, suggesting that procedure records in administrative database are useful for risk adjustment.

Electronic supplementary material

The online version of this article (doi:10.1186/s12913-015-0889-x) contains supplementary material, which is available to authorized users.

Capturing diagnosis-timing in ICD-coded hospital data: recommendations from the WHO ICD-11 topic advisory group on quality and safety

PURPOSE

To develop a consensus opinion regarding capturing diagnosis-timing in coded hospital data.

METHODS

As part of the World Health Organization International Classification of Diseases-11th Revision initiative, the Quality and Safety Topic Advisory Group is charged with enhancing the capture of quality and patient safety information in morbidity data sets. One such feature is a diagnosis-timing flag. The Group has undertaken a narrative literature review, scanned national experiences focusing on countries currently using timing flags, and held a series of meetings to derive formal recommendations regarding diagnosis-timing reporting.

RESULTS

The completeness of diagnosis-timing reporting continues to improve with experience and use; studies indicate that it enhances risk-adjustment and may have a substantial impact on hospital performance estimates, especially for conditions/procedures that involve acutely ill patients. However, studies suggest that its reliability varies, is better for surgical than medical patients (kappa in hip fracture patients of 0.7-1.0 versus kappa in pneumonia of 0.2-0.6) and is dependent on coder training and setting. It may allow simpler and more precise specification of quality indicators.

CONCLUSIONS

As the evidence indicates that a diagnosis-timing flag improves the ability of routinely collected, coded hospital data to support outcomes research and the development of quality and safety indicators, the Group recommends that a classification of 'arising after admission' (yes/no), with permitted designations of 'unknown or clinically undetermined', will facilitate coding while providing flexibility when there is uncertainty. Clear coding standards and guidelines with ongoing coder education will be necessary to ensure reliability of the diagnosis-timing flag.

Do-not-resuscitate status and observational comparative effectiveness research in patients with septic shock*

OBJECTIVES

To assess the importance of including do-not-resuscitate status in critical care observational comparative effectiveness research.

DESIGN

Retrospective analysis.

SETTING

All California hospitals participating in the 2007 California State Inpatient Database, which provides do-not-resuscitate status within the first 24 hours of admission.

PATIENTS

Septic shock present at admission.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

We investigated the association of early do-not-resuscitate status with in-hospital mortality among patients with septic shock. We also examined the strength of confounding of do-not-resuscitate status on the association between activated protein C therapy and mortality, an association with conflicting results between observational and randomized studies. We identified 24,408 patients with septic shock; 19.6% had a do-not-resuscitate order. Compared with patients without a do-not-resuscitate order, those with a do-not-resuscitate order were significantly more likely to be older (75 ± 14 vs 67 ± 16 yr) and white (62% vs 53%), with more acute organ failures (1.44 ± 1.15 vs 1.38 ± 1.15), but fewer inpatient interventions (1.0 ± 1.0 vs 1.4 ± 1.1). Adding do-not-resuscitate status to a model with 47 covariates improved mortality discrimination (c-statistic, 0.73-0.76; p < 0.001). Addition of do-not-resuscitate status to a multivariable model assessing the association between activated protein C and mortality resulted in a 9% shift in the activated protein C effect estimate toward the null (odds ratio from 0.78 [95% CI, 0.62-0.99], p = 0.04, to 0.85 [0.67-1.08], p = 0.18).

CONCLUSIONS

Among patients with septic shock, do-not-resuscitate status acts as a strong confounder that may inform past discrepancies between observational and randomized studies of activated protein C. Inclusion of early do-not-resuscitate status into more administrative databases may improve observational comparative effectiveness methodology.

The Dutch hospital standardised mortality ratio (HSMR) method and cardiac surgery: benchmarking in a national cohort using hospital administration data versus a clinical database

Objective

To compare the accuracy of data from hospital administration databases and a national clinical cardiac surgery database and to compare the performance of the Dutch hospital standardised mortality ratio (HSMR) method and the logistic European System for Cardiac Operative Risk Evaluation, for the purpose of benchmarking of mortality across hospitals.

Methods

Information on all patients undergoing cardiac surgery between 1 January 2007 and 31 December 2010 in 10 centres was extracted from The Netherlands Association for Cardio-Thoracic Surgery database and the Hospital Discharge Registry. The number of cardiac surgery interventions was compared between both databases. The European System for Cardiac Operative Risk Evaluation and hospital standardised mortality ratio models were updated in the study population and compared using the C-statistic, calibration plots and the Brier-score.

Results

The number of cardiac surgery interventions performed could not be assessed using the administrative database as the intervention code was incorrect in 1.4–26.3%, depending on the type of intervention. In 7.3% no intervention code was registered. The updated administrative model was inferior to the updated clinical model with respect to discrimination (c-statistic of 0.77 vs 0.85, p<0.001) and calibration (Brier Score of 2.8% vs 2.6%, p<0.001, maximum score 3.0%). Two average performing hospitals according to the clinical model became outliers when benchmarking was performed using the administrative model.

Conclusions

In cardiac surgery, administrative data are less suitable than clinical data for the purpose of benchmarking. The use of either administrative or clinical risk-adjustment models can affect the outlier status of hospitals. Risk-adjustment models including procedure-specific clinical risk factors are recommended.

Thirty-day complications after laparoscopic or open cholecystectomy: a population-based cohort study in Italy

OBJECTIVE

The objective of the study is to evaluate short-term complications after laparoscopic (LC) or open cholecystectomy (OC) in patients with gallstones by using linked hospital discharge data.

DESIGN

Population-based cohort study.

SETTING

Data were obtained from the Regional Hospital Discharge Registry Lazio Region in Central Italy (around 5 million inhabitants) in 2007-2008.

PARTICIPANTS

All patients admitted to hospitals of Lazio with symptomatic gallstones (International Classification of disease, 9th Revision, Clinical Modification (ICD-9-CM)=574) who underwent LC (ICD-9-CM 51.23) or OC (ICD-9-CM 51.22).

OUTCOME MEASURES

(1)'30-day surgical-related complications' defined as any complication of the biliary tract (including postoperative infection, haemorrhage or haematoma or seroma complicating a procedure, persistent postoperative fistula, perforation of bile duct and disruption of wound). (2) '30-day systemic complications' defined as any complications of other organs (including sepsis, infections from other organs, major cardiovascular events and selected adverse events).

RESULTS

13 651 patients were included; 86.1% had LC, 13.9% OC. 2.0% experienced surgical-related complications (SRC), 2.1% systemic complications (SC). The OR of complications after LC versus OC was 0.60 (p<0.001) for SRC and 0.52 (p<0.001) for SC. In relation to SRC, the advantage of LC was consistent across age categories, severity of gallstones and previous upper abdominal surgery, whereas there was no advantage among people with emergency admission (OR=0.94, p=0.764). For SC, no significant advantage of LC was seen among very old people (OR=0.99, p=0.975) and among those with previous upper abdominal surgery (OR=0.86, p=0.905).

CONCLUSIONS

This large observational study confirms that LC is more effective than OC with respect to 30-day complications. Population-based linkage of administrative datasets can enlarge evidence of treatment benefits in clinical practice.

Assessing the accuracy of profiling methods for identifying top providers: performance of mental health care providers

Provider profiling as a means to describe and compare the performance of health care professionals has gained momentum in the past decade. As a key component of pay-for-performance programs profiling has been increasingly used to identify top-performing providers. However, rigorous examination of the performance of statistical methods for profiling when used to classify top-performing providers is lacking. The objective of this study was to compare the classification accuracy of three methods for identifying providers exceeding performance thresholds and to analyze data on satisfaction with mental health care providers at Group Health Cooperative using these methods. Questionnaire data on patient satisfaction with mental health care providers at Group Health Cooperative was collected between April 2008 and January 2010. Simulated data were used to compare the classification accuracy of alternative statistical methods. We evaluated sensitivity, specificity, and root mean squared error of alternative statistical methods using simulated data. For Group Health providers, we compared agreement of alternative approaches to classification. We found that when between-provider variability in performance was low, all three methods exhibited poor classification accuracy. When used to evaluate mental health care provider performance, we found substantial uncertainty in the estimates and poor agreement across methods. Based on these findings, we recommend providing uncertainty estimates for provider rankings and caution against the use of any classification method when between-provider variability is low.

Risk-adjusted mortality: problems and possibilities

The ratio of observed-to-expected deaths is considered a measure of hospital quality and for this reason will soon become a basis for payment. However, there are drivers of that metric more potent than quality: most important are medical documentation and patient acuity. If hositals underdocument and therefore do not capture the full “expected mortality” they may be tempted to lower their observed/expected ratio by reducing “observed mortality” through limiting access to the very ill. Underdocumentation occurs because hospitals do not recognize, and therefore cannot seek to confirm, specific comorbidities conferring high mortality risk. To help hospitals identify these comorbidities, this paper describes an easily implemented spread-sheet for evaluating comorbid conditions associated, in any particular hospital, with each discharge. This method identifies comorbidities that increase in frequency as mortality risk increases within each diagnostic grouping. The method is inductive and therefore independent of any particular risk-adjustment technique.

Incident stroke and mortality associated with new-onset atrial fibrillation in patients hospitalized with severe sepsis

CONTEXT

New-onset atrial fibrillation (AF) has been reported in 6% to 20% of patients with severe sepsis. Chronic AF is a known risk factor for stroke and death, but the clinical significance of new-onset AF in the setting of severe sepsis is uncertain.

OBJECTIVE

To determine the in-hospital stroke and in-hospital mortality risks associated with new-onset AF in patients with severe sepsis.

DESIGN AND SETTING

Retrospective population-based cohort of California State Inpatient Database administrative claims data from nonfederal acute care hospitals for January 1 through December 31, 2007.

PATIENTS

Data were available for 3,144,787 hospitalized adults. Severe sepsis (n = 49,082 [1.56%]) was defined by validated International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) code 995.92. New-onset AF was defined as AF that occurred during the hospital stay, after excluding AF cases present at admission.

MAIN OUTCOME MEASURES

A priori outcome measures were in-hospital ischemic stroke (ICD-9-CM codes 433, 434, or 436) and mortality.

RESULTS

Patients with severe sepsis were a mean age of 69 (SD, 16) years and 48% were women. New-onset AF occurred in 5.9% of patients with severe sepsis vs 0.65% of patients without severe sepsis (multivariable-adjusted odds ratio [OR], 6.82; 95% CI, 6.54-7.11; P < .001). Severe sepsis was present in 14% of all new-onset AF in hospitalized adults. Compared with severe sepsis patients without new-onset AF, patients with new-onset AF during severe sepsis had greater risks of in-hospital stroke (75/2896 [2.6%] vs 306/46,186 [0.6%] strokes; adjusted OR, 2.70; 95% CI, 2.05-3.57; P < .001) and in-hospital mortality (1629 [56%] vs 18,027 [39%] deaths; adjusted relative risk, 1.07; 95% CI, 1.04-1.11; P < .001). Findings were robust across 2 definitions of severe sepsis, multiple methods of addressing confounding, and multiple sensitivity analyses.

CONCLUSION

Among patients with severe sepsis, patients with new-onset AF were at increased risk of in-hospital stroke and death compared with patients with no AF and patients with preexisting AF.

Development of a validation algorithm for 'present on admission' flagging

BACKGROUND

The use of routine hospital data for understanding patterns of adverse outcomes has been limited in the past by the fact that pre-existing and post-admission conditions have been indistinguishable. The use of a 'Present on Admission' (or POA) indicator to distinguish pre-existing or co-morbid conditions from those arising during the episode of care has been advocated in the US for many years as a tool to support quality assurance activities and improve the accuracy of risk adjustment methodologies. The USA, Australia and Canada now all assign a flag to indicate the timing of onset of diagnoses. For quality improvement purposes, it is the 'not-POA' diagnoses (that is, those acquired in hospital) that are of interest.

METHODS

Our objective was to develop an algorithm for assessing the validity of assignment of 'not-POA' flags. We undertook expert review of the International Classification of Diseases, 10th Revision, Australian Modification (ICD-10-AM) to identify conditions that could not be plausibly hospital-acquired. The resulting computer algorithm was tested against all diagnoses flagged as complications in the Victorian (Australia) Admitted Episodes Dataset, 2005/06. Measures reported include rates of appropriate assignment of the new Australian 'Condition Onset' flag by ICD chapter, and patterns of invalid flagging.

RESULTS

Of 18,418 diagnosis codes reviewed, 93.4% (n = 17,195) reflected agreement on status for flagging by at least 2 of 3 reviewers (including 64.4% unanimous agreement; Fleiss' Kappa: 0.61). In tests of the new algorithm, 96.14% of all hospital-acquired diagnosis codes flagged were found to be valid in the Victorian records analysed. A lower proportion of individual codes was judged to be acceptably flagged (76.2%), but this reflected a high proportion of codes used <5 times in the data set (789/1035 invalid codes).

CONCLUSION

An indicator variable about the timing of occurrence of diagnoses can greatly expand the use of routinely coded data for hospital quality improvement programmes. The data-cleaning instrument developed and tested here can help guide coding practice in those health systems considering this change in hospital coding. The algorithm embodies principles for development of coding standards and coder education that would result in improved data validity for routine use of non-POA information.

Shop for quality or volume? Volume, quality, and outcomes of coronary artery bypass surgery

BACKGROUND

Care from high-volume centers or surgeons has been associated with lower mortality rates in coronary artery bypass surgery, but how volume and quality of care relate to each other is not well understood.

OBJECTIVE

To determine how volume and differences in quality of care influence outcomes after coronary artery bypass surgery.

DESIGN

Observational cohort.

SETTING

164 hospitals in the United States.

PATIENTS

81,289 patients 18 years or older who had coronary artery bypass grafting from 1 October 2003 to 1 September 2005.

MEASUREMENTS

Hospital and surgeon case volumes were estimated by using a data set. Quality measures were defined by whether patients received specific medications and by counting the number of measures missed. Hierarchical models were used to estimate effects of volume and quality on death and readmission up to 30 days.

RESULTS

After adjustment for clinical factors, lowest surgeon volume and highest hospital volume were associated with higher mortality rates and lower readmission risk, respectively. Patients who did not receive aspirin (odds ratio, 1.89 [95% CI, 1.65 to 2.16) or beta-blockers (odds ratio, 1.29 [CI, 1.12 to 1.49]) had higher odds for death, after adjustment for clinical risk factors and case volume. Adjustment for individual quality measures did not alter associations between volume and readmission or death. However, if no quality measures were missed, mortality rates at the lowest-volume centers (adjusted mortality rate, 1.05% [CI, 0.81% to 1.29%]) and highest-volume centers (adjusted mortality rate, 0.98% [CI, 0.72% to 1.25%]) were similar.

LIMITATION

Because administrative data were used, the quality measures may not replicate measures collected through chart abstraction.

CONCLUSION

Maximizing adherence to quality measures is associated with improved mortality rates, independent of hospital or surgeon volume.

PRIMARY FUNDING SOURCE

California HealthCare Foundation.

National release of the nursing home quality report cards: implications of statistical methodology for risk adjustment

OBJECTIVE

To determine how alternative statistical risk-adjustment methods may affect the quality measures (QMs) in nursing home (NH) report cards.

DATA SOURCES/STUDY SETTINGS

Secondary data from the national Minimum Data Set files of 2004 and 2005 that include 605,433 long-term residents in 9,336 facilities.

STUDY DESIGN

We estimated risk-adjusted QMs of decline in activities of daily living (ADL) functioning using classical, fixed-effects, and random-effects logistic models. Risk-adjusted QMs were compared with each other, and with the published QM (unadjusted) in identifying high- and low-quality facilities by either the rankings or 95 percent confidence intervals of QMs.

PRINCIPAL FINDINGS

Risk-adjusted QMs showed better overall agreement (or convergent validity) with each other than did the unadjusted versus each adjusted QM; the disagreement rate between unadjusted and adjusted QM can be as high as 48 percent. The risk-adjusted QM derived from the random-effects shrinkage estimator deviated nonrandomly from other risk-adjusted estimates in identifying the best 10 percent facilities using rankings.

CONCLUSIONS

The extensively risk-adjusted QMs of ADL decline, even when estimated by alternative statistical methods, show higher convergent validity and provide more robust NH comparisons than the unadjusted QM. Outcome rankings based on ADL decline tend to show lower convergent validity when estimated by the shrinkage estimator rather than other statistical methods.

Impact of date stamping on patient safety measurement in patients undergoing CABG: experience with the AHRQ Patient Safety Indicators

Background

The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) provide information on hospital risk-adjusted rates for potentially preventable adverse events. Although designed to work with routine administrative data, it is unknown whether the PSIs can accurately distinguish between complications and pre-existing conditions. The objective of this study is to examine whether the AHRQ PSIs accurately measure hospital complication rates, using the data with present-on-admission (POA) codes to distinguish between complications and pre-existing conditions

Methods

Retrospective cohort study of patients undergoing isolated CABG surgery in California conducted using the 1998–2000 California State Inpatient Database. We calculated the positive predictive value of selected AHRQ PSIs using information from the POA as the gold standard, and the intra-class correlation coefficient to assess the level of agreement between the hospital risk-adjusted PSI rates with and without the information contained in the POA modifier.

Results

The false positive error rate, defined as one minus the positive predictive value, was greater than or equal to 20% for four of the eight PSIs examined: decubitus ulcer, failure-to-rescue, postoperative physiologic and metabolic derangement, and postoperative pulmonary embolism or deep venous thrombosis. Pairwise comparison of the hospital risk-adjusted PSI rates, with and without POA information, demonstrated almost perfect agreement for five of the eight PSI's. For decubitus ulcer, failure-to-rescue, and postoperative pulmonary embolism or DVT, the intraclass-correlation coefficient ranged between 0.63 to 0.79.

Conclusion

For some of the AHRQ Patient Safety Indicators, there are significant differences in the risk-adjusted rates of adverse events depending on whether the POA indicator is used to distinguish between pre-existing conditions and complications. The use of the POA indicator will increase the accuracy of the AHRQ PSIs as measures of adverse outcomes.

Using administrative data for longitudinal substance abuse research

The utilization of administrative data in substance abuse research has become more widespread than ever. This selective review synthesizes recent extant research from 31 articles to consider what has been learned from using administrative data to conduct longitudinal substance abuse research in four overlapping areas: (1) service access and utilization, (2) underrepresented populations, (3) treatment outcomes, and (4) cost analysis. Despite several notable limitations, administrative data contribute valuable information, particularly in the investigation of service system interactions and outcomes among substance abusers as they unfold and influence each other over the long term. This critical assessment of the advantages and disadvantages of using existing administrative data within a longitudinal framework should stimulate innovative thinking regarding future applications of administrative data for longitudinal substance abuse research purposes.

Does risk adjustment of the CMS quality measures for nursing homes matter?

BACKGROUND

The Centers for Medicare and Medicaid Services (CMS) publish a report card for nursing homes with 19 clinical quality measures (QMs). These measures include minimal risk adjustment.

OBJECTIVES

To develop QMs with more extensive risk adjustment and to investigate the impact on quality rankings.

RESEARCH DESIGN

Retrospective analysis of individual level data reported in the Minimum Data Set (MDS). Random effect logistic models were used to estimate risk adjustment models for 5 outcomes: pressure ulcers for high and low risk patients, physical restraints, and pain for long- and short-stay patients. These models were used to create 5 QMs with extended risk adjustment, enhanced QMs (EQMs). The EQMs were compared with the corresponding QMs.

SUBJECTS

All (17,469) nursing homes that reported MDS data in the period 2001-2005, and their 9.6 million residents.

MEASURES

QMs were compared with EQMs for all nursing homes in terms of agreement on outlier identification: Kappa, false positive and false negative error rates.

RESULTS

Kappa values ranged from 0.63 to 0.90. False positive and negative error rates ranged from 8% to 37%. Agreement between QMs and EQMs was better on high quality rather than on low quality.

CONCLUSIONS

More extensive risk adjustment changes quality ranking of nursing homes and should be considered as potential improvement to the current QMs. Other methodological issues related to construction of the QMs should also be investigated to determine if they are important in the context of nursing home care.