Using a hospital quality improvement toolkit to improve performance on the AHRQ quality indicators

Composite neonatal morbidity indicators using hospital discharge data: A systematic review

BACKGROUND

Neonatal morbidity is associated with lifelong impairments, but the absence of a consensual definition and the need for large data sets limit research.

OBJECTIVES

To inform initiatives to define standard outcomes for research, we reviewed composite neonatal morbidity indicators derived from routine hospital discharge data.

DATA SOURCES

PubMed (updated on October 12, 2018). The search algorithm was based on three components: "morbidity," "neonatal," and "hospital discharge data."

STUDY SELECTION AND DATA EXTRACTION

Studies investigating neonatal morbidity using a composite indicator based on hospital discharge data were included. Indicators defined for specific conditions (eg congenital anomalies, maternal addictions) were excluded. The target population, objectives, component morbidities, diagnosis and procedure codes, validation methods, and prevalence of morbidity were extracted.

SYNTHESIS

For each study, we assessed construct validity by describing the methods used to select the indicator components and evaluated whether the authors assessed internal and external validity. We also calculated confidence intervals for the prevalence of the morbidity composite.

RESULTS

Seventeen studies fulfilled inclusion criteria. Indicators targeted all (n = 4), low-/moderate-risk (n = 9), and very preterm (VPT, n = 4) infants. Components were similar for VPT infants, but domains and diagnosis codes within domains varied widely for all and low-/moderate-risk infants. Component selection was described for 8/17 indicators and some form of validation reported for 12/17. Neonatal morbidity prevalence ranged from 4.6% to 9.0% of all infants, 0.4% to 8.0% of low-/moderate-risk infants, and 17.8% to 61.0% of VPT infants.

CONCLUSIONS

Multiple neonatal morbidity indicators based on hospital discharge data have been used for research, but their heterogeneity limits comparisons between studies. Standard neonatal outcome measures are needed for benchmarking and synthesis of research results.

Tailoring an educational program on the AHRQ Patient Safety Indicators to meet stakeholder needs: lessons learned in the VA

BACKGROUND

Given that patient safety measures are increasingly used for public reporting and pay-for performance, it is important for stakeholders to understand how to use these measures for improvement. The Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs) are one particularly visible set of measures that are now used primarily for public reporting and pay-for-performance among both private sector and Veterans Health Administration (VA) hospitals. This trend generates a strong need for stakeholders to understand how to interpret and use the PSIs for quality improvement (QI). The goal of this study was to develop an educational program and tailor it to stakeholders' needs. In this paper, we share what we learned from this program development process.

METHODS

Our study population included key VA stakeholders involved in reviewing performance reports and prioritizing and initiating quality/safety initiatives. A pre-program formative evaluation through telephone interviews and web-based surveys assessed stakeholders' educational needs/interests. Findings from the formative evaluation led to development and implementation of a cyberseminar-based program, which we tailored to stakeholders' needs/interests. A post-program survey evaluated program participants' perceptions about the PSI educational program.

RESULTS

Interview data confirmed that the concepts we had developed for the interviews could be used for the survey. Survey results informed us on what program delivery mode and content topics were of high interest. Six cyberseminars were developed-three of which focused on two content areas that were noted of greatest interest: learning how to use PSIs for monitoring trends and understanding how to interpret PSIs. We also used snapshots of VA PSI reports so that participants could directly apply learnings. Although initial interest in the program was high, actual attendance was low. However, post-program survey results indicated that perceptions about the program were positive.

CONCLUSIONS

Conducting a formative evaluation was a highly important process in program development. The useful information that we collected through the interviews and surveys allowed us to tailor the program to stakeholders' needs and interests. Our experiences, particularly with the formative evaluation process, yielded valuable lessons that can guide others when developing and implementing similar educational programs.

Simultaneous Prediction of New Morbidity, Mortality, and Survival Without New Morbidity From Pediatric Intensive Care: A New Paradigm for Outcomes Assessment

OBJECTIVES

Assessments of care including quality assessments adjusted for physiological status should include the development of new morbidities as well as mortalities. We hypothesized that morbidity, like mortality, is associated with physiological dysfunction and could be predicted simultaneously with mortality.

DESIGN

Prospective cohort study from December 4, 2011, to April 7, 2013.

SETTING

General and cardiac/cardiovascular PICUs at seven sites.

PATIENTS

Randomly selected PICU patients from their first PICU admission.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Among 10,078 admissions, the unadjusted morbidity rates (measured with the Functional Status Scale and defined as an increase of ≥ 3 from preillness to hospital discharge) were 4.6% (site range, 2.6-7.7%) and unadjusted mortality rates were 2.7% (site range, 1.3-5.0%). Morbidity and mortality were significantly (p < 0.001) associated with physiological instability (measured with the Pediatric Risk of Mortality III score) in dichotomous (survival and death) and trichotomous (survival without new morbidity, survival with new morbidity, and death) models without covariate adjustments. Morbidity risk increased with increasing Pediatric Risk of Mortality III scores and then decreased at the highest Pediatric Risk of Mortality III values as potential morbidities became mortalities. The trichotomous model with covariate adjustments included age, admission source, diagnostic factors, baseline Functional Status Scale, and the Pediatric Risk of Mortality III score. The three-level goodness-of-fit test indicated satisfactory performance for the derivation and validation sets (p > 0.20). Predictive ability assessed with the volume under the surface was 0.50 ± 0.019 (derivation) and 0.50 ± 0.034 (validation) (vs chance performance = 0.17). Site-level standardized morbidity ratios were more variable than standardized mortality ratios.

CONCLUSIONS

New morbidities were associated with physiological status and can be modeled simultaneously with mortality. Trichotomous outcome models including both morbidity and mortality based on physiological status are suitable for research studies and quality and other outcome assessments. This approach may be applicable to other assessments presently based only on mortality.

The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review

OBJECTIVES

The aim of this systematic review was to evaluate the effectiveness of toolkits as a knowledge translation (KT) strategy for facilitating the implementation of evidence into clinical care. Toolkits include multiple resources for educating and/or facilitating behaviour change.

DESIGN

Systematic review of the literature on toolkits.

METHODS

A search was conducted on MEDLINE, EMBASE, PsycINFO and CINAHL. Studies were included if they evaluated the effectiveness of a toolkit to support the integration of evidence into clinical care, and if the KT goal(s) of the study were to inform, share knowledge, build awareness, change practice, change behaviour, and/or clinical outcomes in healthcare settings, inform policy, or to commercialise an innovation. Screening of studies, assessment of methodological quality and data extraction for the included studies were conducted by at least two reviewers.

RESULTS

39 relevant studies were included for full review; 8 were rated as moderate to strong methodologically with clinical outcomes that could be somewhat attributed to the toolkit. Three of the eight studies evaluated the toolkit as a single KT intervention, while five embedded the toolkit into a multistrategy intervention. Six of the eight toolkits were partially or mostly effective in changing clinical outcomes and six studies reported on implementation outcomes. The types of resources embedded within toolkits varied but included predominantly educational materials.

CONCLUSIONS

Future toolkits should be informed by high-quality evidence and theory, and should be evaluated using rigorous study designs to explain the factors underlying their effectiveness and successful implementation.