Reinventing VA Health Care

The role of organizational research in implementing evidence-based practice: QUERI Series

BACKGROUND

Health care organizations exert significant influence on the manner in which clinicians practice and the processes and outcomes of care that patients experience. A greater understanding of the organizational milieu into which innovations will be introduced, as well as the organizational factors that are likely to foster or hinder the adoption and use of new technologies, care arrangements and quality improvement (QI) strategies are central to the effective implementation of research into practice. Unfortunately, much implementation research seems to not recognize or adequately address the influence and importance of organizations. Using examples from the U.S. Department of Veterans Affairs (VA) Quality Enhancement Research Initiative (QUERI), we describe the role of organizational research in advancing the implementation of evidence-based practice into routine care settings.

METHODS

Using the six-step QUERI process as a foundation, we present an organizational research framework designed to improve and accelerate the implementation of evidence-based practice into routine care. Specific QUERI-related organizational research applications are reviewed, with discussion of the measures and methods used to apply them. We describe these applications in the context of a continuum of organizational research activities to be conducted before, during and after implementation.

RESULTS

Since QUERI's inception, various approaches to organizational research have been employed to foster progress through QUERI's six-step process. We report on how explicit integration of the evaluation of organizational factors into QUERI planning has informed the design of more effective care delivery system interventions and enabled their improved "fit" to individual VA facilities or practices. We examine the value and challenges in conducting organizational research, and briefly describe the contributions of organizational theory and environmental context to the research framework.

CONCLUSION

Understanding the organizational context of delivering evidence-based practice is a critical adjunct to efforts to systematically improve quality. Given the size and diversity of VA practices, coupled with unique organizational data sources, QUERI is well-positioned to make valuable contributions to the field of implementation science. More explicit accommodation of organizational inquiry into implementation research agendas has helped QUERI researchers to better frame and extend their work as they move toward regional and national spread activities.

Excessive antibiotic use in men with prostatitis

BACKGROUND

Prostatitis accounts for 2 million outpatient visits annually. The majority of prostatitis cases fit the definition of chronic pelvic pain syndrome, for which routine antibiotic use is not indicated.

METHODS

Inpatient, outpatient, and pharmacy datasets from the Veterans Health Administration were used to quantify the magnitude of antibiotic use attributable to chronic pelvic pain syndrome. Specifically, men with a diagnosis of infectious/acute prostatitis or a urinary tract infection were excluded, and the remaining men with a diagnosis of prostatitis were defined as having chronic pelvic pain syndrome.

RESULTS

The annual prevalence of chronic pelvic pain syndrome was 0.5%. Prescriptions for fluoroquinolone antibiotics were filled in 49% of men with a diagnosis of chronic pelvic pain syndrome compared with 5% in men without chronic pelvic pain syndrome. Men with chronic pelvic pain syndrome were more than 7 times more likely to receive a fluoroquinolone prescription independently of age, race/ethnicity, and comorbid conditions. Increased use of other antibiotics also was observed. High use was similar in men with either infectious/acute prostatitis or chronic pelvic pain syndrome.

CONCLUSION

Despite evidence that antibiotics are not effective in the majority of men with chronic pelvic pain syndrome, they were prescribed in 69% of men with this diagnosis. Some increased use is probably due to uncontrolled confounding by comorbid conditions or inaccurate diagnostic coding. However, a 7-fold higher rate of fluoroquinolone usage suggests that strategies to reduce unnecessary antibiotic use in men with prostatitis are warranted.

Trend and variation of prescription drug cost in the veterans health-care system

Using descriptive statistics, this paper revealed that the prescription drug cost as a percentage of total health-care cost in Department of Veterans Affairs (VA) health-care system has outpaced the national trend. Given the fact that the national drug expenditure is the most fast-growing component in the health-care expenditure, the drug cost trend in VA commands further assessment for its financial and clinical impact. Furthermore, by applying simple log linear regression, we analysed the geographic variation in prescription drug use in the VA health-care system. We found a 30% deviation from the predicted drug cost at medical centre level and 15% deviation at Network level. Although this variation is relatively small compared with the variation in other medical service use, reduction of the variation has significant clinical and financial implications. Since the method used in this study is easy to implement, this paper provides a practical tool for large health-care systems such as VA, States and health maintenance organizations to identify those hospitals that over- or under-prescribe drugs.

Overview of the VA Quality Enhancement Research Initiative (QUERI) and QUERI theme articles: QUERI Series

Background

Continuing challenges to timely adoption of evidence-based clinical practices in healthcare have generated intense interest in the development and application of new implementation methods and frameworks. These challenges led the United States (U.S.) Department of Veterans Affairs (VA) to create the Quality Enhancement Research Initiative (QUERI) in the late 1990s. QUERI's purpose was to harness VA's health services research expertise and resources in an ongoing system-wide effort to improve the performance of the VA healthcare system and, thus, quality of care for veterans. QUERI in turn created a systematic means of involving VA researchers both in enhancing VA healthcare quality, by implementing evidence-based practices, and in contributing to the continuing development of implementation science.

The efforts of VA researchers to improve healthcare delivery practices through QUERI and related initiatives are documented in a growing body of literature. The scientific frameworks and methodological approaches developed and employed by QUERI are less well described. A QUERI Series of articles in Implementation Science will illustrate many of these QUERI tools. This Overview article introduces both QUERI and the Series.

Methods

The Overview briefly explains the purpose and context of the QUERI Program. It then describes the following: the key operational structure of QUERI Centers, guiding frameworks designed to enhance implementation and related research, QUERI's progress and promise to date, and the Series' general content. QUERI's frameworks include a core set of steps for diagnosing and closing quality gaps and, simultaneously, advancing implementation science. Throughout the paper, the envisioned involvement and activities of VA researchers within QUERI Centers also are highlighted. The Series is then described, illustrating the use of QUERI frameworks and other tools designed to respond to implementation challenges.

Conclusion

QUERI's simultaneous pursuit of improvement and research goals within a large healthcare system may be unique. However, descriptions of this still-evolving effort, including its conceptual frameworks, methodological approaches, and enabling processes, should have applicability to implementation researchers in a range of health care settings. Thus, the Series is offered as a resource for other implementation research programs and researchers pursuing common goals in improving care and developing the field of implementation science.

Relationship between number of medical conditions and quality of care

BACKGROUND

There is emerging concern that the methods used to measure the quality of care unfairly penalize providers caring for patients with multiple chronic conditions. We therefore sought to study the relationship between the quality of care and the number of medical conditions a patient has.

METHODS

We assessed measurements of the quality of medical care received in three cohorts of community-dwelling adult patients in the Community Quality Index study, the Assessing Care of Vulnerable Elders study, and the Veterans Health Administration project (7680 patients in total). We analyzed the relationship between the quality of care that patients received, defined as the percentage of quality indicators satisfied among those for which patients were eligible, and the number of chronic medical conditions each patient had. We further explored the roles of characteristics of patients, use of health care (number of office visits and hospitalizations), and care provided by specialists as explanations for the observed relationship.

RESULTS

The quality of care increased as the number of medical conditions increased. Each additional condition was associated with an increase in the quality score of 2.2% (95% confidence interval [CI], 1.7 to 2.7) in the Community Quality Index cohort, of 1.7% (95% CI, 1.1 to 2.4) in the Assessing Care of Vulnerable Elders cohort, and of 1.7% (95% CI, 0.7 to 2.8) in the Veterans Health Administration cohort. The relationship between the quality of care and the number of conditions was little affected by adjustment for the difficulty of delivering the care recommended in a quality indicator and for the fact that, because of multiple conditions requiring the same care, a patient could be eligible to receive the same care process more than once. Adjustment for characteristics of patients, use of health care, and care provided by specialists diminished the relationship, but it remained positive.

CONCLUSIONS

The quality of care, measured according to whether patients were offered recommended services, increases as a patient's number of chronic conditions increases.

Change in health status and mortality as indicators of outcomes: comparison between the Medicare Advantage Program and the Veterans Health Administration

BACKGROUND

Comparing health outcomes with adequate methodology is central to performance assessments of health care systems. We compared the Medicare Advantage Program (MAP) and the Veterans Health Administration (VHA) with regard to changes in health status and mortality.

METHODS

We used the Death-Master-File for vital status and the Short-Form 36 to determine physical (PCS) and mental (MCS) health at baseline and at 2 years. We compared the probability of being alive with the same or better (than would be expected by chance) PCS (or MCS) at 2 years and mortality, while adjusting for case-mix. Given the geographic variations in MAP enrollment, we did a regional sub-analysis.

RESULTS

There were no significant differences in the probability of being alive with the same or better PCS except for the South (VHA 65.8% vs. MAP 62.5%, P = .0014). VHA patients had a slightly higher probability than MAP patients of being alive with the same or better MCS (71.8% vs. 70.1%, P = .002) but no significant regional variations. The hazard ratios for mortality in the MAP were higher than in the VHA across all regions.

CONCLUSION

With the use of appropriate methodology, we found small differences in 2-year health outcomes that favor the VHA.

Facility-level factors influencing chronic heart failure care process performance in a national integrated health delivery system

BACKGROUND

Gaps between evidence and practice in the care of patients with chronic heart failure (CHF) in the United States suggest major opportunities for improvement. However, the organizational factors and implementation approaches that influence adherence to national guidelines are poorly understood.

OBJECTIVES

The objectives of this study were to explore the degree to which providers in the Veterans Health Administration system adhere to CHF clinical practice guidelines, and to identify facility-level factors influencing adherence.

DESIGN

In a national cross-sectional study, facility quality managers were surveyed regarding quality improvement efforts, guideline implementation, and context. These data were linked to organizational structure data and provider adherence data from chart reviews. The unit of analysis was the facility. The data were adjusted for the average number of comorbidities per CHF patient. Multivariate logistic regression models were constructed to model factors affecting adherence to CHF guidelines.

SAMPLE

The sample consisted of 143 Veterans Administration Medical Centers with ambulatory care clinics.

RESULTS

The quality manager survey included data from 91% of facilities. Facility-level estimates of provider adherence measures were, on average, 85% or more for most measures. In multivariate analyses, facilities with higher levels of adherence were more likely to have: (1) providers who had been given a brief guideline summary, (2) providers receptive to the guidelines, (3) guideline-specific task forces to support implementation, and 4) a well-planned implementation process.

CONCLUSIONS

Healthcare organizations should adapt implementation to meet local conditions, including creating guideline-specific task forces, developing a well-planned implementation process, fostering provider buy-in, and providing guideline summaries to providers.

Measuring the quality of colorectal cancer screening: the importance of follow-up

PURPOSE

As evidence mounts for effectiveness, an increasing proportion of the United States population undergoes colorectal cancer screening. However, relatively little is known about rates of follow-up after abnormal results from initial screening tests. This study examines patterns of colorectal cancer screening and follow-up within the nation's largest integrated health care system: the Veterans Health Administration.

METHODS

We obtained information about patients who received colorectal cancer screening in the Veterans Health Administration from an existing quality improvement program and from the Veterans Health Administration's electronic medical record. Linking these data, we analyzed receipt of screening and follow-up testing after a positive fecal occult blood test.

RESULTS

A total of 39,870 patients met criteria for colorectal cancer screening; of these 61 percent were screened. Screening was more likely in patients aged 70 to 80 years than in those younger or older. Female gender (relative risk, 0.92; 95 percent confidence interval, 0.9-0.95), Black race (relative risk, 0.92; 95 percent confidence interval, 0.89-0.96), lower income, and infrequent primary care visits were associated with lower likelihood of screening. Of those patients with a positive fecal occult blood test (n = 313), 59 percent received a follow-up barium enema or colonoscopy. Patient-level factors did not predict receipt of a follow-up test.

CONCLUSIONS

The Veterans Health Administration rates for colorectal cancer screening are significantly higher than the national average. However, 41 percent of patients with positive fecal occult blood tests failed to receive follow-up testing. Efforts to measure the quality of colorectal cancer screening programs should focus on the entire diagnostic process.

Quality Improvement Strategies for Hypertension Management

BACKGROUND

Care remains suboptimal for many patients with hypertension.

PURPOSE

The purpose of this study was to assess the effectiveness of quality improvement (QI) strategies in lowering blood pressure.

DATA SOURCES

MEDLINE, Cochrane databases, and article bibliographies were searched for this study.

STUDY SELECTION

Trials, controlled before-after studies, and interrupted time series evaluating QI interventions targeting hypertension control and reporting blood pressure outcomes were studied.

DATA EXTRACTION

Two reviewers abstracted data and classified QI strategies into categories: provider education, provider reminders, facilitated relay of clinical information, patient education, self-management, patient reminders, audit and feedback, team change, or financial incentives were extracted.

DATA SYNTHESIS

Forty-four articles reporting 57 comparisons underwent quantitative analysis. Patients in the intervention groups experienced median reductions in systolic blood pressure (SBP) and diastolic blood pressure (DBP) that were 4.5 mm Hg (interquartile range [IQR]: 1.5 to 11.0) and 2.1 mm Hg (IQR: -0.2 to 5.0) greater than observed for control patients. Median increases in the percentage of individuals achieving target goals for SBP and DBP were 16.2% (IQR: 10.3 to 32.2) and 6.0% (IQR: 1.5 to 17.5). Interventions that included team change as a QI strategy were associated with the largest reductions in blood pressure outcomes. All team change studies included assignment of some responsibilities to a health professional other than the patient's physician.

LIMITATIONS

Not all QI strategies have been assessed equally, which limits the power to compare differences in effects between strategies.

CONCLUSION

QI strategies are associated with improved hypertension control. A focus on hypertension by someone in addition to the patient's physician was associated with substantial improvement. Future research should examine the contributions of individual QI strategies and their relative costs.

Are there gender differences in diabetes care among elderly Medicare enrolled veterans?

OBJECTIVE

To examine gender differences in diabetes care process measures and intermediate outcomes among veteran clinic users.

DESIGN

A retrospective cohort study using Veterans Health Administration (VHA) and Medicare files of VHA clinic users with diabetes. Diabetes care process measures were tests for hemoglobin A1c (HbA1c), low-density lipoprotein (LDL-C) values, and eye exams. Intermediate outcomes were HbA1c and LDL-C values below recommended thresholds. Chi-square tests and logistic regressions were used to assess gender differences.

PARTICIPANTS

Study population included 3,225 women and 231,922 men veterans with diabetes, enrolled in Medicare fee-for-service and alive at the end of fiscal year 2000.

RESULTS

Overall, there were no significant gender differences in HbA1c or LDL-C testing. However, women had higher rates in these process measures than men among the non-African American minorities. Women were more likely to have completed eye exams (odds ratio [OR]=1.11; 99% confidence interval [CI]=1.10, 1.23) but were less likely to have LDL-C under 130 mg/dL (OR=0.77; 99% CI=0.69, 0.87).

CONCLUSIONS

Among VHA patients with diabetes, clinically significant gender inequality was not apparent in most of diabetes care measures. However, there was evidence of better care among nonwhite and non-African American women than their male counterparts. Further research on interaction of race and gender on diabetes care is needed. This includes evaluation of integrated VHA women's health programs as well as cultural issues. Lower LDL-C control among women suggests areas of unmet needs for women and opportunities for future targeted quality improvement interventions at system and provider levels.

Changes in veterans' use of outpatient care from 1992 to 2000

OBJECTIVES

During the mid-1990s, the Veterans Health Administration (VHA) reorganized and placed greater emphasis on high-quality primary care. To determine whether the reorganization was associated with changes in patterns of out-patient VHA use, we sought to evaluate changes in characteristics of veterans who use VHA outpatient services between 1992 and 2000.

METHODS

We merged 2 waves of the National Survey of Veterans to determine changes in patterns of outpatient care use. We evaluated the extent to which veterans who received outpatient care received that care from the VHA.

RESULTS

The odds ratio for VHA-only outpatient care relative to non-VHA-only care in 2000 relative to 1992 was 1.75 (95% confidence interval [CI]=1.51, 2.04), and the odds ratio for dual relative to non-VHA-only care was 1.22 (95% CI=1.08, 1.37). Veterans who were older, had low incomes, and had no additional health insurance coverage were most likely to increase their use of VHA outpatient care.

CONCLUSIONS

Our results suggest that the VHA is increasingly serving veterans who have trouble accessing the private health care system.

Use of risk-adjusted change in health status to assess the performance of integrated service networks in the Veterans Health Administration

OBJECTIVE

Health outcome assessments have become an expectation of regulatory and accreditation agencies. We examined whether a clinically credible risk adjustment methodology for the outcome of change in health status can be developed for performance assessment of integrated service networks.

STUDY DESIGN

Longitudinal study.

SETTING

Outpatient.

STUDY PARTICIPANTS

Thirty-one thousand eight hundred and twenty-three patients from 22 Veterans Health Administration (VHA) integrated service networks were followed for 18 months.

MAIN MEASURE

The physical (PCS) and mental (MCS) component scales from the Veterans Rand 36-items Health Survey (VR-36) and mortality. The outcomes were decline in PCS (decline in PCS scores greater than -6.5 points or death) and MCS (decline in MCS scores greater than -7.9 points).

RESULTS

Four thousand three hundred and twenty-eight (13.6%) patients showed a decline in PCS scores greater than -6.5 points, 4322 (13.5%) had a decline in MCS scores by more than -7.9 points, and 1737 died (5.5%). Multivariate logistic regression models were used to adjust for case-mix. The models performed reasonably well in cross-validated tests of discrimination (c-statistics = 0.72 and 0.68 for decline in PCS and MCS, respectively) and calibration. The resulting risk-adjusted rates of decline in PCS and MCS and ranks of the networks differed considerably from unadjusted ratings.

CONCLUSION

It is feasible to develop clinically credible risk adjustment models for the outcomes of decline in PCS and MCS. Without adequate controls for case-mix, we could not determine whether poor patient outcomes reflect poor performance, sicker patients, or other factors. This methodology can help to measure and report the performance of health care systems.

Changes in characteristics of veterans using the VHA health care system between 1996 and 1999

BACKGROUND: The Department of Veterans Affairs' Veterans Health Administration (VHA) provides a health care safety net to veterans. This study examined changes in characteristics of veterans using the VHA health care system between 1996 and 1999 when VHA implemented major organizational changes to improve access of ambulatory care and to provide care to more veterans. METHODS: The study used two cross-sectional samples of the Medical Expenditures Panel Survey (MEPS), a national representative survey, in 1996 and 1999. The 1996 MEPS survey included 1,944 veterans and the 1999 MEPS survey included 1,974 veterans. There were 534 veterans and 740 veterans who used VHA services in 1996 and 1999, respectively. RESULTS: The proportion of veterans using the VHA system increased from 12.4% in 1996 to 14.6% in 1999. In both years, veterans were more likely to use VHA care if they were older, male, less educated, uninsured, unemployed, and in fair or poor health status. Only two variables, marital status and income, were different between the two years. Married veterans were more likely to use VHA care in 1999, but not in 1996. Veterans with higher incomes had greater odds of using VHA care in 1996, but there was no significant association between income and VHA use in 1999. CONCLUSION: Characteristics of VHA users did not fundamentally change despite the reorganization of VHA health care delivery system and changes in eligibility and enrollment policy. The VHA system maintains its safety net mission while attracting more veterans.

Homeland security and public health: role of the Department of Veterans Affairs, the US Department of Homeland Security, and implications for the public health community

The terrorist attacks of 11 September 2001 led to the largest US Government transformation since the formation of the Department of Defense following World War II. More than 22 different agencies, in whole or in part, and >170,000 employees were reorganized to form a new Cabinet-level Department of Homeland Security (DHS), with the primary mission to protect the American homeland. Legislation enacted in November 2002 transferred the entire Federal Emergency Management Agency and several Department of Health and Human Services (HHS) assets to DHS, including the Office of Emergency Response, and oversight for the National Disaster Medical System, Strategic National Stockpile, and Metropolitan Medical Response System. This created a potential separation of "health" and "medical" assets between the DHS and HHS. A subsequent presidential directive mandated the development of a National Incident Management System and an all-hazard National Response Plan. While no Department of Veterans Affairs (VA) assets were targeted for transfer, the VA remains the largest integrated healthcare system in the nation with important support roles in homeland security that complement its primary mission to provide care to veterans. The Emergency Management Strategic Healthcare Group (EMSHG) within the VA's medical component, the Veteran Health Administration (VHA), is the executive agent for the VA's Fourth Mission, emergency management. In addition to providing comprehensive emergency management services to the VA, the EMSHG coordinates medical back-up to the Department of Defense, and assists the public via the National Disaster Medical System and the National Response Plan. This article describes the VA's role in homeland security and disasters, and provides an overview of the ongoing organizational and operational changes introduced by the formation of the new DHS. Challenges and opportunities for public health are highlighted.

Introducing coordinated care (2): evaluation of design features and implementation processes implications for a preferred health system reform model

The study investigated why the goals of the Australian Coordinated Care trials for clients with complex care needs were not achieved. Significantly higher health service use and costs were incurred in the absence of clear evidence of improved client health outcomes. The validity of assumptions underpinning trial design and the success of implementation at each step in application of the model were examined. There were failures in both design and implementation. Many clients did not require care coordination. The funds pooling arrangements contributed to limited possibilities for service substitution and training of GP care coordinators was inadequate. Trial design did not focus on either clinical guidelines or consumer empowerment. Furthermore, the expectations of the overall national trial were unrealistic both in trial design and expected outcomes given the rigidities and realities of the Australian health care system. Broader system reform in the form of funds pooling and health services planning at the regional level, based on large populations, may be a more effective means to address problems of care coordination and an inflexible supply system.

The Health Status of Elderly Veteran Enrollees in the Veterans Health Administration

OBJECTIVES

To examine the health status of elderly veteran enrollees, stratified by age group, and compare with nonveteran populations.

DESIGN

Cross-sectional study.

SETTING

Outpatient.

PARTICIPANTS

A total of 1,406,049 veteran enrollees were surveyed, and 887,775 returned the questionnaire (63.1%). Of these, 663,729 (74%) were aged 65 and older.

MEASUREMENTS

Patient demographics, comorbid conditions, and health status, which was assessed using the Veterans 36-item short form (SF-36), a reliable and valid measure of health-related quality of life (HRQoL).

RESULTS

Elderly veteran enrollees are a group with poor health status across all scales of the Veterans SF-36. Significant decline in HRQoL was found in patients grouped by increasing age (65-74, 75-84, and > or =85). Of the Veterans SF-36 scales, the role physical and role emotional scales and physical functioning presented the largest decrements by age group. The elderly veteran enrollees had poorer health status than older people enrolled in Medicare managed care, ranging from 0.5 to 1 standard deviations worse.

CONCLUSION

Elderly veteran enrollees have substantial disease burden, as reflected by major impairments across multiple dimensions of HRQoL. These findings bear important implications for use of services, suggesting that the Veterans Health Administration will require considerable resources to provide care for its aging population.

Introducing co-ordinated care (1): a randomised trial assessing client and cost outcomes

A program of care co-ordination (CC) in Melbourne for individuals with a history of high use of in-patient services was evaluated. The intervention involved care planning by a general practitioner (GP) and graduated case management depending on client health status. Services were purchased from pooled funds of participating health care agencies. A randomised control trial of 2,742 participants demonstrated no significant differences between the intervention and usual care group for two quality of life measures, the SF-36 and the AQoL (assessment of quality of life), and no difference in mortality rates. Total resource usage in the CC group was substantially higher, principally due to the extra costs for care planning and case management and for administering the CC model. Results conform to the higher costs typically found in other CC trials, although the failure to demonstrate improved client outcomes is less often reported. The reasons for this failure, whether in trial design, implementation, or theoretical underpinnings are explored in a companion paper.

No racial differences in mortality found among veterans health administration out-patients

OBJECTIVE

Health care delivery systems that offer equal access to ambulatory care may hold promise for preventing and correcting racial disparities that exist in our health care system as a whole. We examined whether racial differences in mortality rates exist among patients receiving outpatient care within the Veterans Health Administration.

STUDY DESIGN AND SETTING

This study used data from the 1998 National Survey of Ambulatory Care Patients, a prospective monitoring system of patient outcomes. We used an outpatient care system in the Veterans Health Administration. We followed 25,172 Whites and 3,517 African-Americans for 48 months. The main study outcome measures were unadjusted and adjusted mortality rates over a 48-month period.

RESULTS

African-Americans had significantly lower unadjusted 48-month mortality rates than Whites (33 vs. 40 deaths per 1,000 person-year, hazard ratio, 0.84; 95% confidence interval [CI], 0.75-0.95). After risk adjustment, the mortality rates became similar for African-Americans and Whites (hazard ratio, 0.99; 95% CI, 0.89-1.09). These findings were consistent across all time points evaluated during the 48-month follow-up.

CONCLUSIONS

The lack of racial differences in mortality in patients receiving ambulatory care in the Veterans Health Administration is reassuring, given the emphasis on equal access within this health care system. This warrants further research to determine whether efforts to improve access in other settings have the potential to reduce racial disparities in health care.

Case-mix adjusting performance measures in a veteran population: pharmacy- and diagnosis-based approaches

OBJECTIVE

To compare the rankings for health care utilization performance measures at the facility level in a Veterans Health Administration (VHA) health care delivery network using pharmacy- and diagnosis-based case-mix adjustment measures.

DATA SOURCES/STUDY SETTING

The study included veterans who used inpatient or outpatient services in Veterans Integrated Service Network (VISN) 20 during fiscal year 1998 (October 1997 to September 1998; N = 126,076). Utilization and pharmacy data were extracted from VHA national databases and the VISN 20 data warehouse.

STUDY DESIGN

We estimated concurrent regression models using pharmacy or diagnosis information in the base year (FY1998) to predict health service utilization in the same year. Utilization measures included bed days of care for inpatient care and provider visits for outpatient care.

PRINCIPAL FINDINGS

Rankings of predicted utilization measures across facilities vary by case-mix adjustment measure. There is greater consistency within the diagnosis-based models than between the diagnosis- and pharmacy-based models. The eight facilities were ranked differently by the diagnosis- and pharmacy-based models.

CONCLUSIONS

Choice of case-mix adjustment measure affects rankings of facilities on performance measures, raising concerns about the validity of profiling practices. Differences in rankings may reflect differences in comparability of data capture across facilities between pharmacy and diagnosis data sources, and unstable estimates due to small numbers of patients in a facility.

Spending For Specialized Mental Health Treatment In The VA: 1995–2001

The mid-1990s saw dramatic changes in mental health care in the Department of Veterans Affairs (VA), the largest provider of such care in the United States. Spending for specialized inpatient mental health care fell 21 percent from 1995 to 2001, while spending for specialized outpatient care rose 63 percent. The shift from inpatient to outpatient care was accompanied by rapid increases in outpatient medication costs. Overall, the VA reduced the average cost (per VA user) of specialized mental health care by 22 percent while it increased the number of users of these services by 35 percent.

Differences in length of stay in Veterans Health Administration and other United States hospitals: is the gap closing?

OBJECTIVES

Compare risk-adjusted length of stay (LOS) in VA and other United States (non-VA) hospitals and determine if relative differences in LOS have changed in recent years.

RESEARCH DESIGN

Retrospective cohort study.

PATIENTS

Patients with ten common medical diagnoses admitted to all VA hospitals and to non-VA hospitals included in the National Hospital Discharge Survey (NHDS) during 1996 through 1999.

DATA

Comparable data elements were obtained from VA administrative databases and the NHDS. LOS was adjusted for age, gender, marital status, and comorbidity. Comorbidity was assessed using a validated methodology that considers 30 conditions.

RESULTS

Unadjusted mean LOS was longer in VA than non-VA patient for all 4 years, in aggregate (7.1 vs. 4.9 days, respectively; P < 0.001), and for each year individually. However, the difference in mean LOS in VA and non-VA patients declined from 2.9 days in 1996 to 1.6 days in 1999. LOS in VA patients remained longer (P < 0.001) in linear regression analyses, adjusting for demographics and comorbidity. However, the difference in LOS declined from 28.5% (95% CI, 28.1%-29.0%) in 1996 to 17.0% (95% CI, 16.6%-17.4%) in 1999. These results were similar in analyses of individual geographic regions.

CONCLUSIONS

Risk-adjusted LOS was longer in VA hospitals than in other United States hospitals. However, differences in LOS narrowed between 1996 and 1999. These findings suggest that changes in the organization and delivery of VA health care in the mid-1990s may be closing the gap between the VA and other healthcare systems in hospital utilization.

Screening for post-traumatic stress disorder in female Veteran's Affairs patients: validation of the PTSD checklist

We evaluated the screening validity of a self-report measure for post traumatic stress disorder (PTSD), the PTSD Checklist (PCL), in female Veterans Affairs (VA) patients. All women seen for care at the VA Puget Sound Health Care system from October 1996-January 1999 (n=2,545) were invited to participate in a research interview. Participants (n=282) completed the 17-item PCL, followed by a gold standard diagnostic interview for PTSD, the Clinician Administered PTSD Scale (CAPS). Thirty-six percent of the participants (n=100) met CAPS diagnostic criteria for current PTSD. Receiver Operating Characteristic (ROC) analysis was used to evaluate the screening performance of the PCL. The area under the ROC curve was 0.86 (95% CI 0.82-0.90). A PCL score of 38 optimized the performance of the PCL as a screening test (sensitivity 0.79, specificity 0.79). The PCL performed well as a screening measure for the detection of PTSD in female VA patients.

Strengths and limitations of competitive versus non-competitive models of integrated capitated fundholding

Integrated budget-holding (fundholding) based on risk-adjusted capitation is commonly proposed as a central element of health system reform. Two contrasting models have been developed: the competitive model where fundholders or health plans compete for enrollees; and the non-competitive model, where plan membership is determined according to an objective attribute such as place of residence. Under the competitive model, efficiency is sought through consumer choice of plan. A range of regulatory elements may also be introduced to moderate undesirable elements of competition. Under the non-competitive model, efficiency is achieved through government regulation and the fact that the fundholder has continuing responsibility for the health of a defined population, supported by micro-management tools (such as quality assurance and selective payment arrangements). In theory, the non-competitive model encourages population-based health services planning. While both models assume risk-adjusted capitated funding, the requirements of any formula are more stringent under the competitive model. Economic theory, as well as documented health system experience, can help identify the relative strengths and limitations of each model. Concerns with the competitive model relate primarily to the capacity to develop robust risk adjusters for capitation sufficient to reduce the incentives for patient risk selection. Possible reductions in the quality of care are also a concern, compounded by difficulties for consumers in discriminating between plans. Efficiency under the non-competitive model requires a strong and appropriate regulatory/policy framework and effective use of micro-management tools. Funding equity objectives can be met through either model by the adoption of income-related contributions, but under the competitive model this may be compromised by incentives for the fundholders to select low-risk patients. Evidence drawn from regional fundholding in New South Wales (NSW, Australia), the US Veterans Health Agency and the literature on managed care in the USA illustrate these concerns. The problem of risk selection in the competitive model is a major theoretical concern, confirmed by the empirical evidence. This, together with concerns regarding other aspects of performance, suggests that the non-competitive model may be preferable, at least as an interim step in reform in public or mixed systems. Future research on this issue is clearly required.

Benchmarking Veterans Affairs Medical Centers in the delivery of preventive health services: comparison of methods

OBJECTIVE

To identify consistent provision of clinical preventive services, we sought to benchmark all acute care Veterans Affairs Medical Centers (VAMCs) against each other nationally on the basis of multiple evidence-based, performance measures to identify facilities performing consistently higher and lower than expected.

METHODS

The 1998 Veterans Health Survey assessed the self-reported delivery of evidence-based clinical preventive services in a stratified national sample of 450 ambulatory care patients seen at each VAMC. Proportions appropriately receiving each service within the recommended time interval were calculated for 138 VAMCs. Percentile ranks for each outcome were assigned. Two approaches were used for benchmarking performance. First, a scaled score for each facility was calculated across the set of 12 measures. Second, facilities were ranked based on the sum of the percentile ranks over a range of specific high cutoffs (eg, 70-80%) and above a range of lower cutoffs (eg, 40-50%). Ranking was validated by comparing with deciles of ranks on chart audit (External Peer Review Program, EPRP) data using Kendall's tau-b and chi2 quality-of-fit test. Differences between consistently high adherence (CHA) and low adherence (CLA) facilities were compared using the Wilcoxon rank sum test on 14 VHS and 11 EPRP outcomes.

RESULTS

Data from 39,939 patients (67% response rate) were examined. In combination, cutoffs of greater than 50th percentile and greater than 75th percentile rank yielded 12 of 14 VHS and 6 of 11 EPRP measures different between CHA and CLA facilities. The scaled-score approach resulted in 20 CHA and 14 CLA facilities. The sum of outcomes ranked above 50th percentile and over 75th percentile for CHA facilities (n = 17) was 15 or more. The sum of outcomes ranked above the same cutoffs for CLA facilities (n = 16) was 3 or less. EPRP and 1998 VHS data demonstrated that the survey measures and benchmarking approaches were both reliable and valid. Both approaches resulted in multiple differences between CHA and CLA facilities; differences were greater using the percentile rank approach.

CONCLUSIONS

The VA has successfully encouraged adoption of evidence-based clinical preventive services throughout its health care system. However, facilities show wide variation in their levels of delivery and can be distinguished on the basis of their consistently high or low levels of adherence. Examining service delivery across multiple performance indicators allows identification of opportunities to improve clinical practice guideline implementation and the delivery of preventive services. This approach identifies model institutions where focused investigation of factors associated with consistent performance may be particularly fruitful.

Risk-adjusted mortality rates as a potential outcome indicator for outpatient quality assessments

OBJECTIVE

The quality of outpatient medical care is increasingly recognized as having an important impact on mortality. We examined whether a clinically credible risk adjustment methodology can be developed for outpatient quality assessments.

RESEARCH DESIGN

This study used data from the 1998 National Survey of Ambulatory Care Patients, a prospective monitoring system of outcomes of patients receiving ambulatory care in the Veterans Affairs (VA) integrated service networks.

SUBJECTS

Thirty-one thousand eight hundred twenty-three patients were followed for 18 months.

MEASURES

The main study outcome measures were observed and risk-adjusted mortality rates.

RESULTS

Of the 31,823 patients, 1559 (5%) died during the 18-months of follow-up. Observed mortality rates across the 22 VA integrated service networks varied significantly from 3.3% to 6.7% (P <0.001). Age, gender, comorbidities (Charlson Index), physical health, and mental health were significant predictors of dying. The resulting risk-adjusted mortality model performed well in cross-validated tests of discrimination (c-statistic = 0.768; 95% CI, 0.749-0.788) and calibration. Analysis of variance confirmed that the 22 integrated service networks differed in their average level of expected risk (P <0.001). Risk-adjusted rates and ranks of the networks differed considerably from unadjusted ratings.

CONCLUSIONS

Risk-adjusted mortality rates may be a useful outcome measure for assessing quality of outpatient care. We have developed a clinically credible risk adjustment model with good performance properties using sociodemographics, diagnoses, and functional status data. The resulting risk adjustment model altered assessments of the performance of the integrated service networks when compared with the unadjusted mortality rates.

Quality of preventive medical care for patients with mental disorders

BACKGROUND/OBJECTIVES

This study compares quality of preventive services between persons with and without mental/substance use disorders for a national sample of medical outpatients.

RESEARCH DESIGN

Cross-sectional study.

SUBJECTS

A total of 113,505 veterans with chronic conditions and at least three general medical visits to Veterans Health Administration medical providers during 1998 to 1999.

MEASURES

Chart-derived rates of eight preventive services: two measures of immunization, four measures of cancer screening, and two of tobacco screening and counseling. Multivariable-generalized estimating equations compared rates of each preventive service among veterans with psychiatric disorders, substance use disorders, both, and neither, adjusting for demographic, health status, and facility-level characteristics.

RESULTS

On average, persons in the sample obtained 64% of the eight preventive procedures for which they were eligible. Overall rates of currency with preventive services were 58% for patients with combined psychiatric/substance use disorders, 60% and 65% for those with psychiatric and substance use disorders alone, and 66% for those with neither psychiatric nor substance use disorders. Each difference remained statistically significant in multivariable models.

CONCLUSIONS

In this sample of patients in active medical treatment, rates of preventive services were higher than rates reported for population-based, private-sector samples. Despite these high-baseline rates, persons with psychiatric disorders, particularly with comorbid substance use, were at risk for lower rate of receipt of preventive services.

Length of stay as a source of bias in comparing performance in VA and private sector facilities: lessons learned from a regional evaluation of intensive care outcomes

OBJECTIVES

Compare intensive care unit (ICU) mortality and length of stay (LOS) in a VA hospital and private sector hospitals and examine the impact of hospital utilization on mortality comparisons.

RESEARCH DESIGN

Retrospective cohort study.

SUBJECTS

Consecutive ICU admissions to a VA hospital (n = 1,142) and 27 private sector hospitals (n = 51,249) serving the same health care market in 1994 to 1995.

MEASURES

Mortality and ICU LOS were adjusted for severity of illness using a validated method that considers physiologic data from the first 24 hours of ICU admission. Mortality comparisons were made using two different multivariable techniques.

RESULTS

Unadjusted in-hospital mortality was higher in VA patients (14.5% vs. 12.0%; P = 0.01), as was hospital (28.3 vs. 11.3 days; P <0.001) and ICU (4.3 vs. 3.9 days; P <0.001) LOS. Using logistic regression to adjust for severity, the odds of death was similar in VA patients, relative to private sector patients (OR 1.16, 95% CI 0.93-1.44; P = 0.18). However, a higher proportion of VA deaths occurred after 21 hospital days (33% vs. 13%; P <0.001). Using proportional hazards regression and censoring patients at hospital discharge, the risk for death was lower in VA patients (hazard ratio 0.70; 95% CI 0.59-0.82; P <0.001). After adjusting for severity, differences in ICU LOS were no longer significant (P = 0.19).

CONCLUSIONS

Severity-adjusted mortality in ICU patients was lower in a VA hospital than in private sector hospitals in the same health care market, based on proportional hazards regression. This finding differed from logistic regression analysis, in which mortality was similar, suggesting that comparisons of hospital mortality between systems with different hospital utilization patterns may be biased if LOS is not considered. If generalizable to other markets, our findings further suggest that ICU outcomes are at least similar in VA hospitals.