The minimum data set depression quality indicator: does it reflect differences in care processes?

Quality Indicators as Predictors of Future Inspection Performance in Ontario Nursing Homes

OBJECTIVES

There are several mechanisms for monitoring the quality of care in long-term care (LTC), including the use of quality indicators derived from resident assessments and formal inspections. The LTC inspection process is time and resource-intensive, and there may be opportunities to better target inspections. In this study, we aimed to examine whether quality indicators could predict future inspection performance in LTC homes across Ontario, Canada.

SETTING AND PARTICIPANTS

In total, 594 LTC homes across Ontario.

METHODS

Using a database compiling detailed inspection reports for the period from 2017 to 2018, we classified each home into 1 of 3 categories (in good standing, needing improvement, needing significant improvement). Machine learning techniques were used to examine whether publicly available Resident Assessment Instrument‒Minimum Data Set quality indicators for the period 2016‒2017 could predict facility classification based on inspection results.

RESULTS

After running a wide range of models, only a weak relationship was found between quality indicators and future inspection performance. The best-performing model was able to achieve a classification accuracy of 40.1%. Feature analysis was performed on the final model to identify which quality indicators were most indicative of predicted poor performance. Experiencing worsened pain, restraint use, and worsened pressure ulcers were correlated with homes predicted as needing significant improvement. Counterintuitively, improved physical functioning had an inverse relationship with homes predicted as being in good standing.

CONCLUSIONS AND IMPLICATIONS

Most quality indicators are poor predictors of inspection performance. Further work is required to explore the limited relationship between these 2 measures of LTC quality, and to identify other quality measures that may be useful as predictors of facilities facing difficulty in meeting quality standards.

Development of the psychometric property of a Minimum Data-Set-Based Depression Rating Scale for use in long-term care facilities in Taiwan

OBJECTIVES

Our aim is to develop the psychometric property of the Minimum Data-Set-Based Depression Rating Scale (MDS-DRS) to ensure its use to assess service needs and guide care plans for institutionalized residents.

METHODS

378 residents were recruited from the Haoran Senior Citizen Home in northern Taiwan. The MDS-DRS and GDS-SF were used to identify observable features of depression symptoms in the elderly residents.

RESULTS

A total of 378 residents participated in this study. The receiver operating characteristic (ROC) curve indicated that the MDS-DRS has a 43.3% sensitivity and a 90.6% specificity when screening for depression symptoms. The total variance, explained by the two factors 'sadness' and 'distress,' was 58.1% based on the factor analysis.

CONCLUSIONS

Reliable assessment tools for nurses are important because they allow the early detection of depression symptoms. The MDS-DRS items perform as well as the GDS-SF items in detecting depression symptoms. Furthermore, the MDS-DRS has the advantage of providing information to staff about care process implementation, which can facilitate the identification of areas that need improvement. Further research is needed to validate the use of the MDS-DRS in long-term care facilities.

Structure, process, and outcomes in skilled nursing facilities: understanding what happens to surgical patients when they cannot go home. A systematic review

BACKGROUND

The surgical population is aging, and greater numbers of surgical patients are being discharged to skilled nursing facilities. Post-acute care is a poorly understood but very important aspect of our healthcare system.

METHODS

This systematic review examines the current body of literature surrounding the structural, process of care, and outcomes measurements for patients in skilled nursing facilities. English language articles published between 1998 and 2011 that purposed to examine nursing facility structure, process of care, and/or outcomes were included.

RESULTS & CONCLUSIONS

Abstracts (2129) were screened and 102 articles were reviewed in full. Twenty-nine articles were included in the qualitative synthesis. The role of the care setting and care delivery in contributing to outcomes has not been well studied, and no strong conclusions can be made. This area of care currently represents a "black box" to practicing surgeons. An understanding of these factors maybe instrumental to determining future directions for research to maximize positive outcomes for these patients.

Practice sensitive quality indicators in RAI-MDS 2.0 nursing home data

BACKGROUND

In recent years, improving the quality of care for nursing home residents has generated a considerable amount of attention. In response, quality indicators (QIs), based on available evidence and expert consensus, have been identified within the Resident Assessment Instrument-Minimum Data Set 2.0 (RAI-MDS 2.0), and validated as proxy measures for quality of nursing home care. We sought to identify practice sensitive QIs; that is, those QIs believed to be the most sensitive to clinical practice.

METHOD

We enlisted two experts to review a list of 35 validated QIs and to select those that they believed to be the most sensitive to practice. We then asked separate groups of practicing physicians, nurses, and policy makers to (1) rank the items on the list for overall "practice sensitivity" and then, (2) to identify the domain to which the QI was most sensitive (nursing care, physician care, or policy maker).

RESULTS

After combining results of all three groups, pressure ulcers were identified as the most practice sensitive QI followed by worsening pain, physical restraint use, the use of antipsychotic medications without a diagnosis of psychosis, and indwelling catheters. When stratified by informant group, although the top five QIs stayed the same, the ranking of the 13 QIs differed by group.

CONCLUSIONS

In addition to identifying a reduced and manageable set of QIs for regular reporting, we believe that focusing on these 13 practice sensitive QIs provides both the greatest potential for improving resident function and slowing the trajectory of decline that most residents experience.

Psychiatric illness and resident assaults among veterans in long-term care facilities

This article describes a quality improvement program to reduce the prevalence of physical assaults in a university-affiliated, 234-bed Veterans Affairs (VA) long-term care (LTC) facility, which experienced a rise in the number of physical assaults to >4 per 1,000 bed days of care in four LTC units. Analysis of 55 events (29 patients) at this VA LTC site during 2007 revealed 19 resident assailants (8% total population), 10 victims, and 30% repeat events. Of the residents who exhibited assaultive behavior, 44% had dementia and 32% had schizophrenia as a major diagnosis. Following a process improvement plan, new occurrence assaultive behaviors declined from >4 to <1 per 1,000 bed days and remained low during 5-year follow up.

Antidepressant prescribing patterns in the nursing home: second-generation issues revisited

INTRODUCTION

The object of this study was to provide an updated evaluation of the quality of antidepressant management and prescribing patterns in nursing homes in the context of organizational and resident factors.

DESIGN

Pearson correlation and chi-square analyses were conducted using information gathered from random nursing home charts.

SETTING

Nursing home facilities in and around the Louisville, KY, metropolitan area (n = 10).

PARTICIPANTS

Chart reviews were randomly chosen for 20% of long term care resident records in participating homes (n = 209).

MEASUREMENTS

Demographic information, documentation of depression diagnoses, and antidepressant prescribing patterns were evaluated using the Quality of Depression Management and Antidepressant Prescribing rating scale and information found in the Minimum Data Set 2.0.

RESULTS

Of the sample, 59.8% was prescribed antidepressants at the time of the chart review; 205 chart reviews indicated the absence or presence of a depression diagnosis. For those with documented depression diagnoses (n = 126), nearly one-quarter were not prescribed antidepressants. Of 79 chart reviews indicating no depression diagnosis, nearly a third were receiving an antidepressant. Documentation related to changes in dosing, the presence or absence of side effects, or reasons for continuation were suboptimal.

CONCLUSION

Discrepancy between antidepressant prescribing and the presence/absence of depression diagnoses continue to exist for nursing home residents. The quality of antidepressant documentation in nursing home charts continues to be inadequate. Future research should aim to explore possible solutions to these discrepancies and deficiencies in documentation.

Top management leadership style and quality of care in nursing homes

PURPOSE

The purpose of this study was to examine the association of Nursing Home Administrator (NHA) leadership style and Director of Nursing (DON) leadership style with quality of care.

DESIGN AND METHODS

Leaders were categorized into 4 groups: consensus managers, consultative autocrats, shareholder managers, or autocrats. This leadership style assessment came from primary data collected from approximately 4,000 NHAs and DONs that was linked to quality information (i.e., Nursing Home Compare Quality Measures and 5-Star rating scores) and nursing home information (i.e., Online Survey, Certification, And Reporting data).

RESULTS

A consensus manager leadership style has a strong association with better quality. Top managers using this style solicit and act upon input from their employees. For NHAs exhibiting this leadership style, the coefficients on 5 of the 7 quality indicators are statistically significant, and all 7 are significant when the DON exhibits this style. When the NHA and DON both have a consensus manager leadership style, 6 of the 7 quality indicator coefficients are significantly associated with better quality.

IMPLICATIONS

The findings indicate that NHA and DON leadership style is associated with quality of care. Leadership strategies are amenable to change; thus, the findings of this study may be used to develop policies for promoting more effective leadership in nursing homes.

Temporal and Geographic variation in the validity and internal consistency of the Nursing Home Resident Assessment Minimum Data Set 2.0

Background

The Minimum Data Set (MDS) for nursing home resident assessment has been required in all U.S. nursing homes since 1990 and has been universally computerized since 1998. Initially intended to structure clinical care planning, uses of the MDS expanded to include policy applications such as case-mix reimbursement, quality monitoring and research. The purpose of this paper is to summarize a series of analyses examining the internal consistency and predictive validity of the MDS data as used in the "real world" in all U.S. nursing homes between 1999 and 2007.

Methods

We used person level linked MDS and Medicare denominator and all institutional claim files including inpatient (hospital and skilled nursing facilities) for all Medicare fee-for-service beneficiaries entering U.S. nursing homes during the period 1999 to 2007. We calculated the sensitivity and positive predictive value (PPV) of diagnoses taken from Medicare hospital claims and from the MDS among all new admissions from hospitals to nursing homes and the internal consistency (alpha reliability) of pairs of items within the MDS that logically should be related. We also tested the internal consistency of commonly used MDS based multi-item scales and examined the predictive validity of an MDS based severity measure viz. one year survival. Finally, we examined the correspondence of the MDS discharge record to hospitalizations and deaths seen in Medicare claims, and the completeness of MDS assessments upon skilled nursing facility (SNF) admission.

Results

Each year there were some 800,000 new admissions directly from hospital to US nursing homes and some 900,000 uninterrupted SNF stays. Comparing Medicare enrollment records and claims with MDS records revealed reasonably good correspondence that improved over time (by 2006 only 3% of deaths had no MDS discharge record, only 5% of SNF stays had no MDS, but over 20% of MDS discharges indicating hospitalization had no associated Medicare claim). The PPV and sensitivity levels of Medicare hospital diagnoses and MDS based diagnoses were between .6 and .7 for major diagnoses like CHF, hypertension, diabetes. Internal consistency, as measured by PPV, of the MDS ADL items with other MDS items measuring impairments and symptoms exceeded .9. The Activities of Daily Living (ADL) long form summary scale achieved an alpha inter-consistency level exceeding .85 and multi-item scale alpha levels of .65 were achieved for well being and mood, and .55 for behavior, levels that were sustained even after stratification by ADL and cognition. The Changes in Health, End-stage disease and Symptoms and Signs (CHESS) index, a summary measure of frailty was highly predictive of one year survival.

Conclusion

The MDS demonstrates a reasonable level of consistency both in terms of how well MDS diagnoses correspond to hospital discharge diagnoses and in terms of the internal consistency of functioning and behavioral items. The level of alpha reliability and validity demonstrated by the scales suggest that the data can be useful for research and policy analysis. However, while improving, the MDS discharge tracking record should still not be used to indicate Medicare hospitalizations or mortality. It will be important to monitor the performance of the MDS 3.0 with respect to consistency, reliability and validity now that it has replaced version 2.0, using these results as a baseline that should be exceeded.

Evidence-based nursing-sensitive indicators for patients hospitalized with depression in Thailand

The aim of this study was to develop and validate nursing-sensitive indicators for patients hospitalized with depression in Thailand. The initial draft, consisting of 12 categories with 37 subcategories, was then evaluated by experts in the US and Thailand. Hospital records were then utilized to evaluate the feasibility and efficacy of the indicators. The finalized instrument consisted of 11 categories with 43 items with a validity of .98 and internal consistency of .88. This is the first set of indicators developed to evaluate nursing-sensitivity for patients hospitalized with a diagnosis of depression in Thailand. Having nursing indicators for depressed patients provides nurses with concrete tools to evaluate their work with depressed patients, allowing these staff to assess their work in a very specific, methodical, and consistent manner. When problems are discovered, both the staff and administration can work to address these issues through training, procedural changes, and departmental shifts.

The Resident Assessment Instrument-Minimum Data Set 2.0 quality indicators: a systematic review

Background

The Resident Assessment Instrument-Minimum Data Set (RAI-MDS) 2.0 is designed to collect the minimum amount of data to guide care planning and monitoring for residents in long-term care settings. These data have been used to compute indicators of care quality. Use of the quality indicators to inform quality improvement initiatives is contingent upon the validity and reliability of the indicators. The purpose of this review was to systematically examine published and grey research reports in order to assess the state of the science regarding the validity and reliability of the RAI-MDS 2.0 Quality Indicators (QIs).

Methods

We systematically reviewed the evidence for the validity and reliability of the RAI-MDS 2.0 QIs. A comprehensive literature search identified relevant original research published, in English, prior to December 2008. Fourteen articles and one report examining the validity and/or reliability of the RAI-MDS 2.0 QIs were included.

Results

The studies fell into two broad categories, those that examined individual quality indicators and those that examined multiple indicators. All studies were conducted in the United States and included from one to a total of 209 facilities. The number of residents included in the studies ranged from 109 to 5758. One study conducted under research conditions examined 38 chronic care QIs, of which strong evidence for the validity of 12 of the QIs was found. In response to these findings, the 12 QIs were recommended for public reporting purposes. However, a number of observational studies (n = 13), conducted in "real world" conditions, have tested the validity and/or reliability of individual QIs, with mixed results. Ten QIs have been studied in this manner, including falls, depression, depression without treatment, urinary incontinence, urinary tract infections, weight loss, bedfast, restraint, pressure ulcer, and pain. These studies have revealed the potential for systematic bias in reporting, with under-reporting of some indicators and over-reporting of others.

Conclusion

Evidence for the reliability and validity of the RAI-MDS QIs remains inconclusive. The QIs provide a useful tool for quality monitoring and to inform quality improvement programs and initiatives. However, caution should be exercised when interpreting the QI results and other sources of evidence of the quality of care processes should be considered in conjunction with QI results.

The Taste for Regulation in Long-Term Care

Concerns about the quality of long-term care have resulted in an extensive array of regulations governing provider behavior. This article reports the results of a survey of 1,147 long-term care specialists on issues related to the government’s performance in assuring quality and improving care. With the exception of providers, more than half of specialists ranked the quality of the average nursing home as fair or poor; home health agencies and even assisted-living facilities fared only somewhat better. Yet despite the perceived ineffectiveness of the current regime, the majority of specialists expressed a general willingness to continue pursuing more stringent and enhanced enforcement and to proceed down the same path with assisted-living facilities. Furthermore, while most were not sanguine about public reporting, the majority favored pay-for-performance, even though both rely on the same information. In addition to constituency group affiliation, differences in views derived largely from respondents’ ideological predispositions.

The work of the RN Minimum Data Set coordinator in its organizational context

The Resident Assessment Instrument/Minimum Data Set (RAI/MDS) is the foundational clinical framework for nursing home care, functioning as both a clinical assessment instrument and an assessment process. An RN is mandated by statute to complete or coordinate the work associated with this framework. Using both focus groups and questionnaires, 24 RN MDS coordinators attending a national conference for MDS coordinators described their work in its organizational context. Shortell et al.'s continuous quality framework of structural, technical, cultural, and strategic organizational dimensions was used to categorize descriptive themes. Clinical implications of the study findings are summarized.

Nursing sensitive quality indicators for nursing home care: international review of literature, policy and practice

OBJECTIVES

To review nursing sensitive indicators used for nursing home care across seven nations with similar elder care (USA, Australia, Norway, New Zealand, England, Sweden and Denmark), and to evaluate their validity.

DESIGN

Systematic search in the literature and other sources to find descriptions of development and validity testing of national quality indicators.

DATA SOURCES

Papers from scientific databases, relevant websites, additional papers and reports, and personal communication with experts in the field. The material was included if it contributed to the description of each country's processes in defining nursing sensitive quality indicators for nursing home care, and the main focus was use, developing and/or testing of quality.

REVIEW METHODS

An overview of each country's utilization of nursing sensitive quality indicators was obtained. The evidence for the validity in development and testing procedures was analyzed using a set of evaluation criteria.

RESULTS

All countries, except Sweden, have nationally standardized assessment of the patient before admission to the nursing home. There is large variation in the way these data collection tools were developed and how the data is used. Only the USA has systematically developed quality indicators on the basis of resident assessments. Twenty-three indicators used nationally in USA, thirteen in Australia, four in Norway, three in New Zealand and three in England were selected for review, and were evaluated for their validity as described in the literature. All selected indicators had satisfactory face validity, and for the twenty-three indicators used in the USA there was evidence for reliability testing. None of the quality indicators met all the criteria for validity. Evidence that the quality indicators can demonstrate meaningful differences in care and that the information can be extracted with minimal extra efforts was not found. Thresholds for high or low quality were determined only for the US quality indicators.

CONCLUSIONS

There are concerns about the validity and reliability of nursing sensitive quality indicators for nursing home care. The indicator development is sparsely documented. It is recommended that the development of quality indicators follows a sound process and that extensive empirical testing of the indicators is done.

Nursing Home Quality as a Common Good

A long-standing assumption among economists is that nursing home quality is common across Medicaid and private-pay patients within a shared facility. However, there has been only limited empirical work addressing this issue. Using a unique individual level panel of residents of nursing homes from seven states, we exploit both within-facility and within-person variation in payer source and quality to examine this issue. We also test the robustness of these results across states with different Medicaid and private-pay rate differentials. Across various identification strategies, our results are consistent with the assumption of common quality across Medicaid and private-paying patients within facilities.

Prevention of unintentional weight loss in nursing home residents: a controlled trial of feeding assistance

OBJECTIVES

To determine the effects of a feeding assistance intervention on food and fluid intake and body weight.

DESIGN

Crossover controlled trial.

SETTING

Four skilled nursing homes (NHs).

PARTICIPANTS

Seventy-six long-stay NH residents at risk for unintentional weight loss.

INTERVENTION

Research staff provided feeding assistance twice per day during or between meals, 5 days per week for 24 weeks.

MEASUREMENTS

Research staff independently weighed residents at baseline and monthly during a 24-week intervention and 24-week control period. Residents' food and fluid intake and the amount of staff time spent providing assistance to eat was assessed for 2 days at baseline and 3 and 6 months during each 24-week period.

RESULTS

The intervention group showed a significant increase in estimated total daily caloric intake and maintained or gained weight, whereas the control group showed no change in estimated total daily caloric intake and lost weight over 24 weeks. The average amount of staff time required to provide the interventions was 42 minutes per person per meal and 13 minutes per person per between-meal snack, versus usual care, during which residents received, on average, 5 minutes of assistance per person per meal and less than 1 minute per person per snack.

CONCLUSION

Two feeding assistance interventions are efficacious in promoting food and fluid intake and weight gain in residents at risk for weight loss. Both interventions require more staff time than usual NH care. The delivery of snacks between meals requires less time than mealtime assistance and thus may be more practical to implement in daily NH care practice.

Impact of Medicaid reimbursement on mental health quality indicators

OBJECTIVE

To examine the relationship between the use of the Minimum Data Set (MDS) for determining Medicaid reimbursement to nursing facilities and the MDS Quality Indicators examining nursing facility residents' mental health.

DATA SOURCES

The 2004 National MDS facility Quality Indicator reports served as the dependent variables. Explanatory variables were based on the 2004 Online Survey Certification and Reporting system (OSCAR) and an examination of existing reports, a review of the State Medicaid Plans, and State Medicaid personnel.

STUDY DESIGN

Multilevel regression models were used to account for the hierarchical structure of the data.

DATA COLLECTION

MDS and OSCAR data were linked by facility identifiers and subsequently linked with state-level variables.

PRINCIPAL FINDINGS

The use of the MDS for determining Medicaid reimbursement was associated with higher (poorer) quality indicator values for all four mental health quality indicators examined. This effect was not found in four comparison quality indicators.

CONCLUSIONS

The findings indicate that documentation of mental health symptoms may be influenced by economic incentives. Policy makers should be cautioned from using these measures as the basis for decision making, such as with pay-for-performance initiatives.

Implementation of the resident assessment instrument/minimum data set in the nursing home as organization: implications for quality improvement in RN clinical assessment

The Resident Assessment Instrument/Minimum Data Set (RAI/MDS) used in nursing homes (NHs) participating in the Federal Medicare and Medicaid programs is a state-of-the-art, computerized clinical assessment instrument. RAI/MDS-derived data are essential, used for NH reimbursement, quality measurement, regulatory quality monitoring activities, and clinical care planning. Completing or coordinating the RAI/MDS, which may be conceived of as implementation, is a federally mandated responsibility of the RN involving clinical assessment, a core professional competency of any RN. How the RAI/MDS is implemented in each NH provides evidence of how each NH as an organization understands both the RAI/MDS process and its organizational level responsibility for promotion of RN competence in clinical assessment. Research literature related to RAI/MDS development, testing, and accuracy is used to identify what is known about organizational level implementation of the RAI/MDS. Evidence-based suggestions to enhance RN competence in RAI/MDS clinical assessments, given existing organizational barriers, are provided.

The relationship between psychiatric disorders and the ability to age in place in assisted living

OBJECTIVE

Residential care/assisted living (RC/AL) has become a popular long-term care option in the past decade, in part because these settings offer residents a more home-like environment than nursing homes (NHs) while still offering supervision and assistance to meet individuals' personal and healthcare needs. One of the goals of RC/AL is to facilitate residents' ability to age in place by providing access to needed services and thereby delaying NH admission. This article explores individual and facility-level characteristics associated with discharge from RC/AL to NH with particular attention to persons with a psychiatric disorder.

METHODS

A Cox proportional hazards model was used to examine the risk factors associated with discharge from RC/AL to NH for a nonrandom sample of 366 residents in 37 RC/AL facilities in one state.

RESULTS

Thirty-two percent of residents sampled had a psychiatric disorder. Residents with a psychiatric disorder were 1.78 times more likely to discharge to a NH. Other factors significantly associated with discharge from a RC/AL to NH included age (older), being married, more hospitalizations, for-profit ownership status, and part of a NH or continuing care retirement community.

CONCLUSION

This article identifies both facility and individual characteristics that increase the likelihood of RC/AL residents discharging to NHs. Given that one of the main findings is that persons with a psychiatric disorder are at increased risk of discharge to NH, there is a need for improved provision of services for this population in RC/ALs to reduce premature discharge to NHs and to support aging in place in RC/ALs.

Evaluating the Accuracy of Minimum Data Set Bed-Mobility Ratings Against Independent Performance Assessments: Systematic Error and Directions for Improvement

PURPOSE

The Minimum Data Set (MDS) Activities of Daily Living (ADL) bed-mobility item, which rates the staff-assistance level necessary for bed movement, is used to target scheduled repositioning interventions and to identify physical function changes in nursing home residents; however, accuracy of the item is uncertain. The purpose of this study was to evaluate the accuracy of the MDS ADL bed-mobility item as completed by nursing home nurses with independent performance assessments conducted by research staff.

DESIGN AND METHODS

A convenience sample of 197 long-stay residents from 26 California nursing homes participating in a larger project was used in this cross-sectional study to compare independent research-staff performance assessments (using graduated assistance protocols of residents' ability to move in bed) and nursing home nurse MDS bed-mobility ratings. Participants also wore movement monitors to verify performance assessments.

RESULTS

Poor agreement existed between the nursing home nurse MDS bed-mobility ratings and the research-staff performance assessments across all assistance levels (kappa range, kappa = 0.007, p =.918 to kappa = 0.484, p <.001), with better agreement seen in totally dependent participants and with fewer elapsed days between MDS ratings and performance assessments. The odds of nursing home nurse errors (underestimating or overestimating dependency) on the MDS bed-mobility item were 2.1 times higher for participants judged independent by research staff compared with participants judged as requiring physical assistance by research staff (95% confidence interval, 1.14-4.03) when adjusted for number of days between nurse MDS ratings and research-staff performance assessments.

IMPLICATIONS

Nursing home nurses overestimated resident dependency in bed mobility. The systematic inaccuracies in MDS bed-mobility ratings have implications for their use as a basis for targeting residents for repositioning programs and determining changes in residents' physical function. Performance assessments utilizing graduated assistance protocols are recommended as a method of improving the accuracy of MDS bed-mobility ratings.

Quality of care in nursing home organizations: Establishing a health services research agenda

Health services research has led to exciting new findings about the critical importance of the amount and type of nursing staff in nursing homes for improving the processes and outcomes of care. This paper reviews recent theoretical and research issues and outlines areas where research is needed. The nursing home research agenda for the future needs to concentrate on: (1) the relationship between structural measures of nursing (eg, staffing levels, education, turnover rates) and the outcomes and processes of care; (2) adequate processes of care and ways to improve the reliability of clinical outcome measures; (3) better ways to risk-adjust for resident characteristics; (4) the impact of nursing home characteristics (eg, ownership) and public policies (eg, reimbursement) on structural factors, processes, and outcomes; and (5) cost-effectiveness studies of nursing care at the organizational or system level.

The quality of the quality indicator of pain derived from the minimum data set

OBJECTIVE

To examine facility variation in data quality of the level of pain documented in the minimum data set (MDS) as a function of level of hospice enrollment in nursing homes (NHs).

DATA SOURCE

Clinical assessments on 3,469 nonhospice residents from 178 NHs were merged with On-line Survey Certification and Reporting data of 2000, Medicare Claims data of 2000 and the MDS of 2000-2002.

STUDY DESIGN

Using the same assessment protocol, NH staff and study nurses independently assessed 3,469 nonhospice residents. Study nurses' assessments being gold standard, we quantified and compared quality of NH staff's pain rating across NHs with high, medium, or low hospice use. Multilevel models were built to assess the effect of NH hospice use levels on the occurrence of false positive (FP) and false negative (FN) errors in NH-rated "severe pain."

PRINCIPAL FINDINGS

Of 178 NHs, 25 had medium and 41 high hospice use. NHs with higher hospice use had lower sensitivities. In multilevel analysis, we found a significant facility-level variation in the probability of FP and FN errors in facility-rated "severe pain." Resident characteristics only explained 4 and 0 percent of the facility variation in FP and FN, respectively; characteristics and locations (state) of NHs further explained 53 and 52 percent of the variance. After controlling for resident and NH characteristics, staff in NHs with medium or high hospice use were less likely to have FP or FN errors in their MDS documentation of pain than were staff in NHs with low or no hospice use.

CONCLUSIONS

The examination of data quality of pooled MDS data from multiple NHs is insufficient. Multilevel analysis is needed to elucidate sources of heterogeneity in the quality of MDS data across NHs. Facility characteristics, e.g., hospice use or NH location, are systematically associated with overrated/underrated pain and may bias pain quality indicator (QI) comparisons. To ensure the integrity of QI comparison in the NH setting, the government may need to institute regular audits of MDS data quality.