Risk-Adjusting Mortality in the Nationwide Veterans Affairs Healthcare System

BACKGROUND

The US Veterans Affairs (VA) healthcare system began reporting risk-adjusted mortality for intensive care (ICU) admissions in 2005. However, while the VA's mortality model has been updated and adapted for risk-adjustment of all inpatient hospitalizations, recent model performance has not been published. We sought to assess the current performance of VA's 4 standardized mortality models: acute care 30-day mortality (acute care SMR-30); ICU 30-day mortality (ICU SMR-30); acute care in-hospital mortality (acute care SMR); and ICU in-hospital mortality (ICU SMR).

METHODS

Retrospective cohort study with split derivation and validation samples. Standardized mortality models were fit using derivation data, with coefficients applied to the validation sample. Nationwide VA hospitalizations that met model inclusion criteria during fiscal years 2017-2018(derivation) and 2019 (validation) were included. Model performance was evaluated using c-statistics to assess discrimination and comparison of observed versus predicted deaths to assess calibration.

RESULTS

Among 1,143,351 hospitalizations eligible for the acute care SMR-30 during 2017-2019, in-hospital mortality was 1.8%, and 30-day mortality was 4.3%. C-statistics for the SMR models in validation data were 0.870 (acute care SMR-30); 0.864 (ICU SMR-30); 0.914 (acute care SMR); and 0.887 (ICU SMR). There were 16,036 deaths (4.29% mortality) in the SMR-30 validation cohort versus 17,458 predicted deaths (4.67%), reflecting 0.38% over-prediction. Across deciles of predicted risk, the absolute difference in observed versus predicted percent mortality was a mean of 0.38%, with a maximum error of 1.81% seen in the highest-risk decile.

CONCLUSIONS AND RELEVANCE

The VA's SMR models, which incorporate patient physiology on presentation, are highly predictive and demonstrate good calibration both overall and across risk deciles. The current SMR models perform similarly to the initial ICU SMR model, indicating appropriate adaption and re-calibration.

Interpretability, credibility, and usability of hospital-specific template matching versus regression-based hospital performance assessments; a multiple methods study

BACKGROUND

Hospital-specific template matching (HS-TM) is a newer method of hospital performance assessment.

OBJECTIVE

To assess the interpretability, credibility, and usability of HS-TM-based vs. regression-based performance assessments.

RESEARCH DESIGN

We surveyed hospital leaders (January-May 2021) and completed follow-up semi-structured interviews. Surveys included four hypothetical performance assessment vignettes, with method (HS-TM, regression) and hospital mortality randomized.

SUBJECTS

Nationwide Veterans Affairs Chiefs of Staff, Medicine, and Hospital Medicine.

MEASURES

Correct interpretation; self-rated confidence in interpretation; and self-rated trust in assessment (via survey). Concerns about credibility and main uses (via thematic analysis of interview transcripts).

RESULTS

In total, 84 participants completed 295 survey vignettes. Respondents correctly interpreted 81.8% HS-TM vs. 56.5% regression assessments, p < 0.001. Respondents "trusted the results" for 70.9% HS-TM vs. 58.2% regression assessments, p = 0.03. Nine concerns about credibility were identified: inadequate capture of case-mix and/or illness severity; inability to account for specialized programs (e.g., transplant center); comparison to geographically disparate hospitals; equating mortality with quality; lack of criterion standards; low power; comparison to dissimilar hospitals; generation of rankings; and lack of transparency. Five concerns were equally relevant to both methods, one more pertinent to HS-TM, and three more pertinent to regression. Assessments were mainly used to trigger further quality evaluation (a "check oil light") and motivate behavior change.

CONCLUSIONS

HS-TM-based performance assessments were more interpretable and more credible to VA hospital leaders than regression-based assessments. However, leaders had a similar set of concerns related to credibility for both methods and felt both were best used as a screen for further evaluation.

Top priorities for the next decade of nursing health services research

BACKGROUND

The U.S. health care system faces increasing pressures for reform. The importance of nurses in addressing health care delivery challenges cannot be overstated.

PURPOSE

To present a Nursing Health Services Research (NHSR) agenda for the 2020s.

METHOD

A meeting of an interdisciplinary group of 38 health services researchers to discuss five key challenges facing health care delivery (behavioral health, primary care, maternal/neonatal outcomes, the aging population, health care spending) and identify the most pressing and feasible research questions for NHSR in the coming decade.

FINDINGS

Guided by a list of inputs affecting health care delivery (health information technology, workforce, delivery systems, payment, social determinants of health), meeting participants identified 5 to 6 research questions for each challenge. Also, eight cross-cutting themes illuminating the opportunities and barriers facing NHSR emerged.

DISCUSSION

The Agenda can act as a foundation for new NHSR - which is more important than ever - in the 2020s.

Next level of board accountability in health care quality

Purpose The purpose of this paper is to offer six principles that health system leaders can apply to establish a governance and management system for the quality of care and patient safety. Design/methodology/approach Leaders of a large academic health system set a goal of high reliability and formed a quality board committee in 2011 to oversee quality and patient safety everywhere care was delivered. Leaders of the health system and every entity, including inpatient hospitals, home care companies, and ambulatory services staff the committee. The committee works with the management for each entity to set and achieve quality goals. Through this work, the six principles emerged to address management structures and processes. Findings The principles are: ensure there is oversight for quality everywhere care is delivered under the health system; create a framework to organize and report the work; identify care areas where quality is ambiguous or underdeveloped (i.e. islands of quality) and work to ensure there is reporting and accountability for quality measures; create a consolidated quality statement similar to a financial statement; ensure the integrity of the data used to measure and report quality and safety performance; and transparently report performance and create an explicit accountability model. Originality/value This governance and management system for quality and safety functions similar to a finance system, with quality performance documented and reported, data integrity monitored, and accountability for performance from board to bedside. To the authors' knowledge, this is the first description of how a board has taken this type of systematic approach to oversee the quality of care.