Power calculation in stepped-wedge cluster randomized trial with reduced intervention sustainability effect

Stir-up Regimen After General Anesthesia in the Postanesthesia Care Unit: A Nurse Led Stepped Wedge Cluster Randomized Control Trial

PURPOSE

To implement a standardized Stir-up Regimen (deep breathing, coughing, repositioning, mobilization [moving arms/legs], assessing and managing pain and nausea) within the first 30 minutes of arrival in the postanesthesia care unit (PACU), with a goal of decreasing recovery time in the immediate postanesthesia period (Phase I).

DESIGN

A pragmatic stepped wedge cluster randomized control trial. Initially, data were collected on time in Phase I in three PACUs (control). Subsequently, the same three units were randomized to sequentially transition to the Stir-up Regimen (intervention).

METHODS

A stepped wedge cluster randomized control trial design was used to implement a standardized Stir-up Regimen in three PACUs in an academic hospital for adult patients who received at least 30 minutes of general anesthesia. The measured outcome was the PACU time in minutes from patient arrival to when the patient met Phase I discharge criteria. Differences between intervention and control groups were evaluated using a generalized mixed-effects model. Nurses were educated about the Stir-up Regimen in team huddles, in-services, video demonstrations, email notifications and reminders, and immediate feedback at the bedside. Implementation science principles were used to assess the adoption of the Stir-up Regimen through a presurvey, postsurvey and spot-check observations in all three PACUs.

FINDINGS

A total of 5,809 PACU adult patient admissions were included: control group (n = 2,860); intervention group (n = 2,949); males (n = 2,602), and females (n = 3,206). The intervention was associated with a reduction in overall mean Phase I recovery time of 4.9 minutes (95% CI: -8.4 to -1.4, P = .007). One PACU decreased time by 9.6 minutes (95% CI: -15.3 to -4.0, P < .001). The other units also reduced Phase I recovery time, but this did not reach statistical significance. The spot-check observations confirmed the intervention was adopted by the nurses, as most interventions were nurse-initiated versus patient-initiated during the first 30 minutes in PACU.

CONCLUSIONS

Standardization of a Stir-up Regimen within 30 minutes of patient PACU arrival resulted in decreased Phase I recovery time.

Researchers' experiences of the design and conduct challenges associated with parallel-group cluster-randomised trials and views on a novel open-cohort design

BACKGROUND

Two accepted designs exist for parallel-group cluster-randomised trials (CRTs). Closed-cohort designs follow the same individuals over time with a single recruitment period before randomisation, but face challenges in settings with high attrition. (Repeated) cross-sectional designs recruit at one or more timepoints before and/or after randomisation, collecting data from different individuals present in the cluster at these timepoints, but are unsuitable for assessment of individual change over time. An 'open-cohort' design allows individual follow-up with recruitment before and after cluster-randomisation, but little literature exists on acceptability to inform their use in CRTs.

AIM

To document the views and experiences of expert trialists to identify: a) Design and conduct challenges with established parallel-group CRT designs,b) Perceptions of potential benefits and barriers to implementation of open-cohort CRTs,c) Methods for minimising, and investigating the impact of, bias in open-cohort CRTs.

METHODS

Qualitative consultation via two expert workshops including triallists (n = 24) who had worked on CRTs over a range of settings. Workshop transcripts were analysed using Descriptive Thematic Analysis utilising inductive and deductive coding.

RESULTS

Two central organising concepts were developed. Design and conduct challenges with established CRT designs confirmed that current CRT designs are unable to deal with many of the complex research and intervention circumstances found in some trial settings (e.g. care homes). Perceptions of potential benefits and barriers of open cohort designs included themes on: approaches to recruitment; data collection; analysis; minimising/investigating the impact of bias; and how open-cohort designs might address or present CRT design challenges. Open-cohort designs were felt to provide a solution for some of the challenges current CRT designs present in some settings.

CONCLUSIONS

Open-cohort CRT designs hold promise for addressing the challenges associated with standard CRT designs. Research is needed to provide clarity around definition and guidance on application.

Identifying transient ischemic attack (TIA) patients at high-risk of adverse outcomes: development and validation of an approach using electronic health record data

Background

Risk-stratification tools that have been developed to identify transient ischemic attack (TIA) patients at risk of recurrent vascular events typically include factors which are not readily available in electronic health record systems. Our objective was to evaluate two TIA risk stratification approaches using electronic health record data.

Methods

Patients with TIA who were cared for in Department of Veterans Affairs hospitals (October 2015—September 2018) were included. The six outcomes were mortality, recurrent ischemic stroke, and the combined endpoint of stroke or death at 90-days and 1-year post-index TIA event. The cohort was split into development and validation samples. We examined the risk stratification of two scores constructed using electronic health record data. The Clinical Assessment Needs (CAN) score is a validated measure of risk of hospitalization or death. The PREVENT score was developed specifically for TIA risk stratification.

Results

A total of N = 5250 TIA patients were included in the derivation sample and N = 4248 in the validation sample. The PREVENT score had higher c-statistics than the CAN score across all outcomes in both samples. Within the validation sample the c-statistics for the PREVENT score were: 0.847 for 90-day mortality, 0.814 for 1-year mortality, 0.665 for 90-day stroke, and 0.653 for 1-year stroke, 0.699 for 90-day stroke or death, and 0.744 for 1-year stroke or death. The PREVENT score classified patients into categories with extreme nadir and zenith outcome rates. The observed 1-year mortality rate among validation patients was 7.1%; the PREVENT score lowest decile of patients had 0% mortality and the highest decile group had 30.4% mortality.

Conclusions

The PREVENT score had strong c-statistics for the mortality outcomes and classified patients into distinct risk categories. Learning healthcare systems could implement TIA risk stratification tools within electronic health records to support ongoing quality improvement.

Registration

ClinicalTrials.gov Identifier: NCT02769338.

The Perils of a "My Work Here is Done" perspective: a mixed methods evaluation of sustainment of an evidence-based intervention for transient ischemic attack

BACKGROUND

To evaluate quality improvement sustainment for Transient Ischemic Attack (TIA) and identify factors influencing sustainment, which is a challenge for Learning Healthcare Systems.

METHODS

Mixed methods were used to assess changes in care quality across periods (baseline, implementation, sustainment) and identify factors promoting or hindering sustainment of care quality. PREVENT was a stepped-wedge trial at six US Department of Veterans Affairs implementation sites and 36 control sites (August 2015-September 2019). Quality of care was measured by the without-fail rate: proportion of TIA patients who received all of the care for which they were eligible among brain imaging, carotid artery imaging, neurology consultation, hypertension control, anticoagulation for atrial fibrillation, antithrombotics, and high/moderate potency statins. Key informant interviews were used to identify factors associated with sustainment.

RESULTS

The without-fail rate at PREVENT sites improved from 36.7% (baseline, 58/158) to 54.0% (implementation, 95/176) and settled at 48.3% (sustainment, 56/116). At control sites, the without-fail rate improved from 38.6% (baseline, 345/893) to 41.8% (implementation, 363/869) and remained at 43.0% (sustainment, 293/681). After adjustment, no statistically significant difference in sustainment quality between intervention and control sites was identified. Among PREVENT facilities, the without-fail rate improved ≥2% at 3 sites, declined ≥2% at two sites, and remained unchanged at one site during sustainment. Factors promoting sustainment were planning, motivation to sustain, integration of processes into routine practice, leadership engagement, and establishing systems for reflecting and evaluating on performance data. The only factor that was sufficient for improving quality of care during sustainment was the presence of a champion with plans for sustainment. Challenges during sustainment included competing demands, low volume, and potential problems with medical coding impairing use of performance data. Four factors were sufficient for declining quality of care during sustainment: low motivation, champion inactivity, no reflecting and evaluating on performance data, and absence of leadership engagement.

CONCLUSIONS

Although the intervention improved care quality during implementation; performance during sustainment was heterogeneous across intervention sites and not different from control sites. Learning Healthcare Systems seeking to sustain evidence-based practices should embed processes within routine care and establish systems for reviewing and reflecting upon performance.

TRIAL REGISTRATION

Clinicaltrials.gov ( NCT02769338 ).

Pairing regression and configurational analysis in health services research: modelling outcomes in an observational cohort using a split-sample design

BACKGROUND

Configurational methods are increasingly being used in health services research.

OBJECTIVES

To use configurational analysis and logistic regression within a single data set to compare results from the two methods.

DESIGN

Secondary analysis of an observational cohort; a split-sample design involved randomly dividing patients into training and validation samples.

PARTICIPANTS AND SETTING

Patients who had a transient ischaemic attack (TIA) in US Department of Veterans Affairs hospitals.

MEASURES

The patient outcome was the combined endpoint of all-cause mortality or recurrent ischaemic stroke within 1 year post-TIA. The quality-of-care outcome was the without-fail rate (proportion of patients who received all processes for which they were eligible, among seven processes).

RESULTS

For the recurrent stroke or death outcome, configurational analysis yielded a three-pathway model identifying a set of (validation sample) patients where the prevalence was 15.0% (83/552), substantially higher than the overall sample prevalence of 11.0% (relative difference, 36%). The configurational model had a sensitivity (coverage) of 84.7% and specificity of 40.6%. The logistic regression model identified six factors associated with the combined endpoint (c-statistic, 0.632; sensitivity, 63.3%; specificity, 63.1%). None of these factors were elements of the configurational model. For the quality outcome, configurational analysis yielded a single-pathway model identifying a set of (validation sample) patients where the without-fail rate was 64.3% (231/359), nearly twice the overall sample prevalence (33.7%). The configurational model had a sensitivity (coverage) of 77.3% and specificity of 78.2%. The logistic regression model identified seven factors associated with the without-fail rate (c-statistic, 0.822; sensitivity, 80.3%; specificity, 84.2%). Two of these factors were also identified in the configurational analysis.

CONCLUSIONS

Configurational analysis and logistic regression represent different methods that can enhance our understanding of a data set when paired together. Configurational models optimise sensitivity with relatively few conditions. Logistic regression models discriminate cases from controls and provided inferential relationships between outcomes and independent variables.

Implementation Evaluation of a Complex Intervention to Improve Timeliness of Care for Veterans with Transient Ischemic Attack

BACKGROUND

The Protocol-guided Rapid Evaluation of Veterans Experiencing New Transient Neurologic Symptoms (PREVENT) program was designed to address systemic barriers to providing timely guideline-concordant care for patients with transient ischemic attack (TIA).

OBJECTIVE

We evaluated an implementation bundle used to promote local adaptation and adoption of a multi-component, complex quality improvement (QI) intervention to improve the quality of TIA care Bravata et al. (BMC Neurology 19:294, 2019).

DESIGN

A stepped-wedge implementation trial with six geographically diverse sites.

PARTICIPANTS

The six facility QI teams were multi-disciplinary, clinical staff.

INTERVENTIONS

PREVENT employed a bundle of key implementation strategies: team activation; external facilitation; and a community of practice. This strategy bundle had direct ties to four constructs from the Consolidated Framework for Implementation Research (CFIR): Champions, Reflecting & Evaluating, Planning, and Goals & Feedback.

MAIN MEASURES

Using a mixed-methods approach guided by the CFIR and data matrix analyses, we evaluated the degree to which implementation success and clinical improvement were associated with implementation strategies. The primary outcomes were the number of completed implementation activities, the level of team organization and > 15 points improvement in the Without Fail Rate (WFR) over 1 year.

KEY RESULTS

Facility QI teams actively engaged in the implementation strategies with high utilization. Facilities with the greatest implementation success were those with central champions whose teams engaged in planning and goal setting, and regularly reflected upon their quality data and evaluated their progress against their QI plan. The strong presence of effective champions acted as a pre-condition for the strong presence of Reflecting & Evaluating, Goals & Feedback, and Planning (rather than the other way around), helping to explain how champions at the +2 level influenced ongoing implementation.

CONCLUSIONS

The CFIR-guided bundle of implementation strategies facilitated the local implementation of the PREVENT QI program and was associated with clinical improvement in the national VA healthcare system.

TRIAL REGISTRATION

clinicaltrials.gov: NCT02769338.

Assessment of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) Program for Improving Quality of Care for Transient Ischemic Attack: A Nonrandomized Cluster Trial

IMPORTANCE

Patients with transient ischemic attack (TIA) are at high risk of recurrent vascular events. Timely management can reduce that risk by 70%; however, gaps in TIA quality of care exist.

OBJECTIVE

To assess the performance of the Protocol-Guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) intervention to improve TIA quality of care.

DESIGN, SETTING, AND PARTICIPANTS

This nonrandomized cluster trial with matched controls evaluated a multicomponent intervention to improve TIA quality of care at 6 diverse medical centers in 6 geographically diverse states in the US and assessed change over time in quality of care among 36 matched control sites (6 control sites matched to each PREVENT site on TIA patient volume, facility complexity, and quality of care). The study period (defined as the data period) started on August 21, 2015, and extended to May 12, 2019, including 1-year baseline and active implementation periods for each site. The intervention targeted clinical teams caring for patients with TIA.

INTERVENTION

The quality improvement (QI) intervention included the following 5 components: clinical programs, data feedback, professional education, electronic health record tools, and QI support.

MAIN OUTCOMES AND MEASURES

The primary outcome was the without-fail rate, which was calculated as the proportion of veterans with TIA at a specific facility who received all 7 guideline-recommended processes of care for which they were eligible (ie, anticoagulation for atrial fibrillation, antithrombotic use, brain imaging, carotid artery imaging, high- or moderate-potency statin therapy, hypertension control, and neurological consultation). Generalized mixed-effects models with multilevel hierarchical random effects were constructed to evaluate the intervention associations with the change in the mean without-fail rate from the 1-year baseline period to the 1-year intervention period.

RESULTS

Six facilities implemented the PREVENT QI intervention, and 36 facilities were identified as matched control sites. The mean (SD) age of patients at baseline was 69.85 (11.19) years at PREVENT sites and 71.66 (11.29) years at matched control sites. Most patients were male (95.1% [154 of 162] at PREVENT sites and 94.6% [920 of 973] at matched control sites at baseline). Among the PREVENT sites, the mean without-fail rate improved substantially from 36.7% (58 of 158 patients) at baseline to 54.0% (95 of 176 patients) during a 1-year implementation period (adjusted odds ratio, 2.10; 95% CI, 1.27-3.48; P = .004). Comparing the change in quality at the PREVENT sites with the matched control sites, the improvement in the mean without-fail rate was greater at the PREVENT sites than at the matched control sites (36.7% [58 of 158 patients] to 54.0% [95 of 176 patients] [17.3% absolute improvement] vs 38.6% [345 of 893 patients] to 41.8% [363 of 869 patients] [3.2% absolute improvement], respectively; absolute difference, 14%; P = .008).

CONCLUSIONS AND RELEVANCE

The implementation of this multifaceted program was associated with improved TIA quality of care across the participating sites. The PREVENT QI program is an example of a health care system using QI strategies to improve performance, and may serve as a model for other health systems seeking to provide better care.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02769338.

The protocol-guided rapid evaluation of veterans experiencing new transient neurological symptoms (PREVENT) quality improvement program: rationale and methods

BACKGROUND

Transient ischemic attack (TIA) patients are at high risk of recurrent vascular events; timely management can reduce that risk by 70%. The Protocol-guided Rapid Evaluation of Veterans Experiencing New Transient Neurological Symptoms (PREVENT) developed, implemented, and evaluated a TIA quality improvement (QI) intervention aligned with Learning Healthcare System principles.

METHODS

This stepped-wedge trial developed, implemented and evaluated a provider-facing, multi-component intervention to improve TIA care at six facilities. The unit of analysis was the medical center. The intervention was developed based on benchmarking data, staff interviews, literature, and electronic quality measures and included: performance data, clinical protocols, professional education, electronic health record tools, and QI support. The effectiveness outcome was the without-fail rate: the proportion of patients who receive all processes of care for which they are eligible among seven processes. The implementation outcomes were the number of implementation activities completed and final team organization level. The intervention effects on the without-fail rate were analyzed using generalized mixed-effects models with multilevel hierarchical random effects. Mixed methods were used to assess implementation, user satisfaction, and sustainability.

DISCUSSION

PREVENT advanced three aspects of a Learning Healthcare System. Learning from Data: teams examined and interacted with their performance data to explore hypotheses, plan QI activities, and evaluate change over time. Learning from Each Other: Teams participated in monthly virtual collaborative calls. Sharing Best Practices: Teams shared tools and best practices. The approach used to design and implement PREVENT may be generalizable to other clinical conditions where time-sensitive care spans clinical settings and medical disciplines.

TRIAL REGISTRATION

clinicaltrials.gov: NCT02769338 [May 11, 2016].