How is the sustainability of chronic disease health programmes empirically measured in hospital and related healthcare services?-a scoping review

Alternative metrics for characterizing longer-term clinical outcomes in difficult-to-treat depression: II. Sensitivity to treatment effects

OBJECTIVE

Characteristics of difficult-to-treat depression (DTD), including infrequent symptom remission and poor durability of benefit, compel reconsideration of the outcome metrics historically used to gauge the effectiveness of antidepressant interventions.

METHODS

Self-report and clinician assessments of depression symptom severity were obtained regularly over a 2-year period in a difficult-to-treat depression registry sample receiving treatment as usual (TAU), with or without vagus nerve stimulation (VNS). Alternative outcome metrics for characterizing symptom change were compared in effect size and discriminating power in distinguishing the vagus nerve stimulation + treatment as usual and treatment as usual treatment groups. We expected metrics based on remission status to produce weaker between-group separation than those based on the classifications of partial response or response and metrics that integrate information over time to produce greater separation than those based on single endpoint assessment.

RESULTS

Metrics based on remission status had smaller effect size and poorer discrimination in separating the treatment groups than metrics based on partial response or response classifications. Metrics that integrated information over the 2-year observation period had stronger performance characteristics than those based on symptom scores at single endpoint assessment. For both the clinician-rated and self-report depression ratings, the metrics with the strongest performance characteristics were the median percentage change in symptom scores over the observation period and the proportion of the observation period in partial response or better.

CONCLUSION

In difficult-to-treat depression, integrative symptom severity-based and time-based measures are sensitive and informative outcomes for assessing between-group treatment effects, while metrics based on remission status are not.

Alternative metrics for characterizing longer-term clinical outcomes in difficult-to-treat depression: I. Association with change in quality of life

BACKGROUND

In difficult-to-treat depression (DTD) the outcome metrics historically used to evaluate treatment effectiveness may be suboptimal. Metrics based on remission status and on single end-point (SEP) assessment may be problematic given infrequent symptom remission, temporal instability, and poor durability of benefit in DTD.

METHODS

Self-report and clinician assessment of depression symptom severity were regularly obtained over a 2-year period in a chronic and highly treatment-resistant registry sample (N = 406) receiving treatment as usual, with or without vagus nerve stimulation. Twenty alternative metrics for characterizing symptomatic improvement were evaluated, contrasting SEP metrics with integrative (INT) metrics that aggregated information over time. Metrics were compared in effect size and discriminating power when contrasting groups that did (N = 153) and did not (N = 253) achieve a threshold level of improvement in end-point quality-of-life (QoL) scores, and in their association with continuous QoL scores.

RESULTS

Metrics based on remission status had smaller effect size and poorer discrimination of the binary QoL outcome and weaker associations with the continuous end-point QoL scores than metrics based on partial response or response. The metrics with the strongest performance characteristics were the SEP measure of percentage change in symptom severity and the INT metric quantifying the proportion of the observation period in partial response or better. Both metrics contributed independent variance when predicting end-point QoL scores.

CONCLUSIONS

Revision is needed in the metrics used to quantify symptomatic change in DTD with consideration of INT time-based measures as primary or secondary outcomes. Metrics based on remission status may not be useful.

Evaluating the sustainability of patient navigation programs in oncology by length of existence, funding, and payment model participation

BACKGROUND

For this study, the authors examined whether specific programmatic factors were associated with the sustainability of patient navigation programs.

METHODS

This cross-sectional survey explored navigation programmatic factors associated with 3 measures of sustainability: 1) length of program existence, 2) reliance on sustainable funding, and 3) participation in alternative payment models. In total, 750 patient navigators or program administrators affiliated with oncology navigation programs in clinical-based and community-based settings completed the survey between April and July 2019.

RESULTS

Associations were observed between both accreditation and work setting and measures of program sustainability. Accredited programs and larger, more resourced clinical institutions were particularly likely to exhibit multiple measures of sustainability. The results also identified significant gaps at the programmatic level in data collection and reporting among navigation programs, but no association was observed between programmatic data collection/reporting and sustainability.

CONCLUSIONS

Navigation is not currently a reimbursable service and has historically been viewed as value-added in oncology settings. Therefore, factors associated with sustainability are critical to understand how to build a framework for successful navigation programs within the current system and also to develop the case for potential reimbursement in the future.

Implementability of healthcare interventions: an overview of reviews and development of a conceptual framework

Background

Implementation research may play an important role in reducing research waste by identifying strategies that support translation of evidence into practice. Implementation of healthcare interventions is influenced by multiple factors including the organisational context, implementation strategies and features of the intervention as perceived by people delivering and receiving the intervention. Recently, concepts relating to perceived features of interventions have been gaining traction in published literature, namely, acceptability, fidelity, feasibility, scalability and sustainability. These concepts may influence uptake of healthcare interventions, yet there seems to be little consensus about their nature and impact. The aim of this paper is to develop a testable conceptual framework of implementability of healthcare interventions that includes these five concepts.

Methods

A multifaceted approach was used to develop and refine a conceptual framework of implementability of healthcare interventions. An overview of reviews identified reviews published between January 2000 and March 2021 that focused on at least one of the five concepts in relation to a healthcare intervention. These findings informed the development of a preliminary framework of implementability of healthcare interventions which was presented to a panel of experts. A nominal group process was used to critique, refine and agree on a final framework.

Results

A total of 252 publications were included in the overview of reviews. Of these, 32% were found to be feasible, 4% reported sustainable changes in practice and 9% were scaled up to other populations and/or settings. The expert panel proposed that scalability and sustainability of a healthcare intervention are dependent on its acceptability, fidelity and feasibility. Furthermore, acceptability, fidelity and feasibility require re-evaluation over time and as the intervention is developed and then implemented in different settings or with different populations. The final agreed framework of implementability provides the basis for a chronological, iterative approach to planning for wide-scale, long-term implementation of healthcare interventions.

Conclusions

We recommend that researchers consider the factors acceptability, fidelity and feasibility (proposed to influence sustainability and scalability) during the preliminary phases of intervention development, evaluation and implementation, and iteratively check these factors in different settings and over time.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01171-7.

A study to determine if and how bereavement support programs provided by Irish and Canadian hospices are evaluated

A study of the bereavement programs provided by hospices in Ireland and the Canadian province of Alberta was done to determine if and how these are evaluated for efficacy. All but one hospice offered multiple different bereavement programs, with routine in-house evaluations of all programs performed. In all cases, staff and client unsolicited and solicited comments made during and at the end of each program provided data for continuing program refinement. Most hospices also routinely employed a self-devised questionnaire to gain specific sought information to retain or change programs. Other information, including news about program developments elsewhere, was also actively sought for program adoption or for the improvement of existing programs. With most bereavement programs having been in place for many years, participants were confident their programs are needed, safe, and effective. Their continuing quest for high-quality programming, however, meant they were active in monitoring these programs and in seeking developments in this field.

Defining sustainability in practice: views from implementing real-world innovations in health care

BACKGROUND

One of the key conceptual challenges in advancing our understanding of how to more effectively sustain innovations in health care is the lack of clarity and agreement on what sustainability actually means. Several reviews have helped synthesize and clarify how researchers conceptualize and operationalize sustainability. In this study, we sought to identify how individuals who implement and/or sustain evidence-informed innovations in health care define sustainability.

METHODS

We conducted in-depth, semi-structured interviews with implementation leaders and relevant staff involved in the implementation of evidence-based innovations relevant to cancer survivorship care (n = 27). An inductive approach, using constant comparative analysis, was used for analysis of interview transcripts and field notes.

RESULTS

Participants described sustainability as an ongoing and dynamic process that incorporates three key concepts and four important conditions. The key concepts were: (1) continued capacity to deliver the innovation, (2) continued delivery of the innovation, and (3) continued receipt of benefits. The key conditions related to (2) and (3), and included: (2a) innovations must continue in the absence of the champion or person/team who introduced it and (3a) adaptation is critical to ensuring relevancy and fit, and thus to delivering the intended benefits.

CONCLUSIONS

Participants provided a nuanced view of sustainability, with both continued delivery and continued benefits only relevant under certain conditions. The findings reveal the interconnected elements of what sustainability means in practice, providing a unique and important perspective to the academic literature.

Inclusion of a care bundle for fever, hyperglycaemia and swallow management in a National Audit for acute stroke: evidence of upscale and spread

BACKGROUND

In the Quality in Acute Stroke Care (QASC) trial undertaken in stroke units (SUs) located in New South Wales (NSW), Australia (2005-2010), facilitated implementation of a nurse-led care bundle to manage fever, hyperglycaemia and swallowing (FeSS protocols) reduced death and disability for patients with stroke. We aimed to determine subsequent adherence to the bundled FeSS processes (reflective of the protocols) between 2013 and 2017 in Australian hospitals, and examine whether changes in adherence to these processes varied based on previous participation in the QASC trial or subsequent statewide scale-up (QASCIP-Quality in Acute Stroke Care Implementation Project) and presence of an SU.

METHODS

Cross-sectional, observational study using self-reported organisational survey and retrospective clinical audit data from the National Acute Services Stroke Audit (2013, 2015, 2017). Mixed-effects logistic regression was performed with dependent variables: (1) composite outcome measure reflecting compliance with the FeSS protocols and (2) individual FeSS processes, including the year of audit as an independent variable, adjusted for correlation of outcomes within hospital. Separate models including interaction terms between the year of audit and previous participation in QASC/QASCIP and year of audit and SU were also generated.

RESULTS

Hospital participation included the following: 2013-124 hospitals, 3741 cases; 2015-112 hospitals, 4087 cases; and 2017-117 hospitals, 4192 cases. An 80% increase in the odds of receiving the composite outcome in 2017 compared to 2013 was found (2013, 30%; 2017, 41%; OR 1.8; 95% CI 1.6, 2.0; p < 0.001). The odds of FeSS adherence from 2013 to 2017 was greater for hospitals that had participated in QASC/QASCIP relative to those that had not (participated OR 2.1; 95% CI 1.7, 2.7; not participated OR 1.6; 95% CI 1.4, 1.8; p = 0.03). Similar uptake in adherence was evident in hospitals with and without an SU between 2013 and 2017.

CONCLUSION

The use of the FeSS protocols within Australia increased from 2013 to 2017 with the inclusion of these care processes in the National Audit. Greater uptake in hospitals previously involved in QASC/QASCIP was evident. Our implementation methods may be useful for other national initiatives for improving access to evidence-based practice.

Sustainable implementation of innovative, technology-based health care practices: A qualitative case study from stroke telemedicine

INTRODUCTION

Technology-based innovation requires long-term changes to workforce routines, otherwise practices will not be sustained. The aim of this study was to identify influential factors in the ongoing use of an acute stroke telemedicine programme.

METHODS

A new acute stroke telemedicine programme in a regional hospital receiving 375 patients with stroke or transient ischaemic attack per year was used as an exploratory case study. Semi-structured interviews with acute care and emergency department clinicians (n = 25) were conducted at two time-points: after a six-month pilot and then after a further 12-month implementation phase. Interviews (between 12-60 min) were recorded, transcribed and analysed inductively using descriptive thematic analysis. Reported barriers and facilitators were compared with those previously reported pre-implementation (deductive analysis) to identify changes over time. Using an implementation framework and a behaviour change taxonomy, strategies were developed to address influential factors on sustainability.

RESULTS

New facilitators were identified including hospital system changes, benefits to clinicians and telemedicine becoming standard practice. New and ongoing barriers included infrequent use, competing demands and the continued resistance to a specific treatment.

DISCUSSION

Understanding the factors supporting a health service in successfully implementing change can accelerate population benefits. The innovation itself may include barriers to be addressed, and barriers and facilitators can change over time. Individual attitudes remain critical to initial and ongoing success. Strategies proposed included promoting benefits across the organisation and allaying uncertainties with site-specific evidence. The effectiveness of these strategies, however, needs to be evaluated. Strategies sustaining change post-implementation should be considered.

Sustaining and spreading quality improvement: Decreasing intrapartum malpractice risk

BACKGROUND

Malpractice liability is an ongoing problem in obstetrics. However, developing, sustaining, and spreading effective interventions is challenging. The aim of this study is to examine the spread and sustainability of a multilevel integrated practice and coordinated communication model 66 months after its original implementation.

METHODS

Data on labor and delivery patients from 37 hospitals (5 beta sites and 32 expansion sites) were analyzed for the 81-month time period from January 2010 through September 2016.

RESULTS

High-risk occurrence rates per 1000 live births decreased by over 70% at both beta and expansion sites. The likelihood of a high-risk occurrence was statistically significantly lower during the final study period than in the preintervention period at both beta sites (odds ratio [OR] = 0.218; p < .0001) and expansion sites (OR = 0.288; p < .001).

CONCLUSION

The multilevel integrated practice and coordinated communication model was successfully spread and sustained. Key elements contributing to this success included developing and maintaining evidence-based guidelines, ensuring leadership buy-in and support, collecting and reporting performance measures, holding teams accountable, providing training, and ensuring transparent communication.

Transitioning from a single-site pilot project to a state-wide regional telehealth service: The experience from the Victorian Stroke Telemedicine programme

Scaling of projects from inception to establishment within the healthcare system is rarely formally reported. The Victorian Stroke Telemedicine (VST) programme provided a very useful opportunity to describe how rural hospitals in Victoria were able to access a network of Melbourne-based neurologists via telemedicine. The VST programme was initially piloted at one site in 2010 and has gradually expanded as a state-wide regional service operating with 16 hospitals in 2017. The aim of this paper is to summarise the factors that facilitated the state-wide transition of the VST programme. A naturalistic case-study was used and data were obtained from programme documents, e.g. minutes of governance committees, including the steering committee, the management committee and six working groups; operational and evaluation documentation, interviews and research field-notes taken by project staff. Thematic analysis was undertaken, with results presented in narrative form to provide a summary of the lived experience of developing and scaling the VST programme. The main success factors were attaining funding from various sources, identifying a clinical need and evidence-based solution, engaging stakeholders and facilitating co-design, including embedding the programme within policy, iterative evaluation including performing financial sustainability modelling, and conducting dissemination activities of the interim results, including promotion of early successes.

Press Ganey Outpatient Medical Practice Survey Scores Do Not Correlate With Patient-Reported Outcomes After Primary Joint Arthroplasty

BACKGROUND

Patient delivery of care satisfaction surveys have emerged as instruments to assess the quality of health care at both the hospital and provider levels. We evaluated the correlation between these care satisfaction surveys and patient-reported outcomes (PROs).

METHODS

We reviewed secondary data on 540 patients with 540 random visits who underwent primary total joint arthroplasty between January 2014 and February 2017. The Press Ganey Outpatient Medical Practice Survey was collected from outpatient clinical encounters to measure patient satisfaction with their experience and matched to PRO measures from the same encounter. The PROs evaluated included the Patient-Reported Outcomes Measurement Information System physical function computerized adaptive test, v1.2, and the Patient-Reported Outcomes Measurement Information System Global 10 health survey. In addition to the random selection, we reviewed separate cross-sections of the data including preoperative visits within 90 days of the index procedure, all postoperative visits at least 1 year from the index procedure, and the magnitude of change in PRO scores from preoperative to a minimum 1-year postoperative visit. Data were evaluated using the Spearman correlation coefficient (r_s).

RESULTS

There was little if any correlation between the Press Ganey scores and PROs at all time points evaluated (all, r_s: -0.13 to 0.14). When evaluating knee and hip arthroplasty cases separately, the data demonstrated similar results (all, r_s: -0.33 to 0.18).

CONCLUSION

We found little, if any, correlation between a patient's satisfaction with their care experience and their own perception of physical function and global health measures at all time points evaluated. These data question the utility of these scores as surrogate measures of health care quality, especially when reimbursements become tied to these metrics.

Navigating the sustainability landscape: a systematic review of sustainability approaches in healthcare

BACKGROUND

Improvement initiatives offer a valuable mechanism for delivering and testing innovations in healthcare settings. Many of these initiatives deliver meaningful and necessary changes to patient care and outcomes. However, many improvement initiatives fail to sustain to a point where their full benefits can be realised. This has led many researchers and healthcare practitioners to develop frameworks, models and tools to support and monitor sustainability. This work aimed to identify what approaches are available to assess and influence sustainability in healthcare and to describe the different perspectives, applications and constructs within these approaches to guide their future use.

METHODS

A systematic review was carried out following PRISMA guidelines to identify publications that reported approaches to support or influence sustainability in healthcare. Eligibility criteria were defined through an iterative process in which two reviewers independently assessed 20% of articles to test the objectivity of the selection criteria. Data were extracted from the identified articles, and a template analysis was undertaken to identify and assess the sustainability constructs within each reported approach.

RESULTS

The search strategy identified 1748 publications with 227 articles retrieved in full text for full documentary analysis. In total, 62 publications identifying a sustainability approach were included in this review (32 frameworks, 16 models, 8 tools, 4 strategies, 1 checklist and 1 process). Constructs across approaches were compared and 40 individual constructs for sustainability were found. Comparison across approaches demonstrated consistent constructs were seen regardless of proposed interventions, setting or level of application with 6 constructs included in 75% of the approaches. Although similarities were found, no approaches contained the same combination of the constructs nor did any single approach capture all identified constructs. From these results, a consolidated framework for sustainability constructs in healthcare was developed.

CONCLUSIONS

Choosing a sustainability method can pose a challenge because of the diverse approaches reported in the literature. This review provides a valuable resource to researchers, healthcare professionals and improvement practitioners by providing a summary of available sustainability approaches and their characteristics.

TRIAL REGISTRATION

This review was registered on the PROSPERO database: CRD42016040081 in June 2016.

The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care

There is strong interest in implementation science to address the gap between research and practice in public health. Research on the sustainability of evidence-based interventions has been growing rapidly. Sustainability has been defined as the continued use of program components at sufficient intensity for the sustained achievement of desirable program goals and population outcomes. This understudied area has been identified as one of the most significant translational research problems. Adding to this challenge is uncertainty regarding the extent to which intervention adaptation and evolution are necessary to address the needs of populations that differ from those in which interventions were originally tested or implemented. This review critically examines and discusses conceptual and methodological issues in studying sustainability, summarizes the multilevel factors that have been found to influence the sustainability of interventions in a range of public health and health care settings, and highlights key areas for future research.

Built to last? The sustainability of health system improvements, interventions and change strategies: a study protocol for a systematic review

INTRODUCTION

The sustainability of healthcare interventions and change programmes is of increasing importance to researchers and healthcare stakeholders interested in creating sustainable health systems to cope with mounting stressors. The aim of this protocol is to extend earlier work and describe a systematic review to identify, synthesise and draw meaning from studies published within the last 5 years that measure the sustainability of interventions, improvement efforts and change strategies in the health system.

METHODS AND ANALYSIS

The protocol outlines a method by which to execute a rigorous systematic review. The design includes applying primary and secondary data collection techniques, consisting of a comprehensive database search complemented by contact with experts, and searching secondary databases and reference lists, using snowballing techniques. The review and analysis process will occur via an abstract review followed by a full-text screening process. The inclusion criteria include English-language, peer-reviewed, primary, empirical research articles published after 2011 in scholarly journals, for which the full text is available. No restrictions on location will be applied. The review that results from this protocol will synthesise and compare characteristics of the included studies. Ultimately, it is intended that this will help make it easier to identify and design sustainable interventions, improvement efforts and change strategies.

ETHICS AND DISSEMINATION

As no primary data were collected, ethical approval was not required. Results will be disseminated in conference presentations, peer-reviewed publications and among policymaker bodies interested in creating sustainable health systems.

Developing a comprehensive definition of sustainability

Background

Understanding sustainability is one of the significant implementation science challenges. One of the big challenges in researching sustainability is the lack of consistent definitions in the literature. Most implementation studies do not present a definition of sustainability, even when assessing sustainability. The aim of the current study was to systematically develop a comprehensive definition of sustainability based on definitions already used in the literature.

Methods

We searched for knowledge syntheses of sustainability and abstracted sustainability definitions from the articles identified through any relevant systematic and scoping reviews. The constructs in the abstracted sustainability definitions were mapped to an existing definition. The comprehensive definition of sustainability was revised to include emerging constructs.

Results

We identified four knowledge syntheses of sustainability, which identified 209 original articles. Of the 209 articles, 24 (11.5%) included a definition of sustainability. These definitions were mapped to three constructs from an existing definition, and nine new constructs emerged. We reviewed all constructs and created a revised definition: (1) after a defined period of time, (2) a program, clinical intervention, and/or implementation strategies continue to be delivered and/or (3) individual behavior change (i.e., clinician, patient) is maintained; (4) the program and individual behavior change may evolve or adapt while (5) continuing to produce benefits for individuals/systems. All 24 definitions were remapped to the comprehensive definition (percent agreement among three coders was 94%). Of the 24 definitions, 17 described the continued delivery of a program (70.8%), 17 mentioned continued outcomes (70.8%), 13 mentioned time (54.2%), 8 addressed the individual maintenance of a behavior change (33.3%), and 6 described the evolution or adaptation (25.0%).

Conclusions

We drew from over 200 studies to identify 24 existing definitions of sustainability. Based on these definitions, we identified five key sustainability constructs, which can be used as the basis for future research on sustainability. Our next step is to identify sustainability frameworks and develop a meta-framework using a concept mapping approach to consolidate the factors and considerations across sustainability frameworks.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-017-0637-1) contains supplementary material, which is available to authorized users.

Improving discharge care: the potential of a new organisational intervention to improve discharge after hospitalisation for acute stroke, a controlled before-after pilot study

OBJECTIVE

Provision of a discharge care plan and prevention therapies is often suboptimal. Our objective was to design and pilot test an interdisciplinary, organisational intervention to improve discharge care using stroke as the case study using a mixed-methods, controlled before-after observational study design.

SETTING

Acute care public hospitals in Queensland, Australia (n=15). The 15 hospitals were ranked against a benchmark based on a composite outcome of three discharge care processes. Clinicians from a 'top-ranked' hospital participated in a focus group to elicit their success factors. Two pilot hospitals then participated in the organisational intervention that was designed with experts and consumers.

PARTICIPANTS

Hospital clinicians involved in discharge care for stroke and patients admitted with acute stroke or transient ischaemic attack.

INTERVENTION

A four-stage, multifaceted organisational intervention that included data reviews, education and facilitated action planning.

PRIMARY AND SECONDARY OUTCOME MEASURES

Three discharge processes collected in Queensland hospitals within the Australian Stroke Clinical Registry were used to select study hospitals: (1) discharge care plan; (2) antihypertensive medication prescription and (3) antiplatelet medication prescription (ischaemic events only). Primary measure: composite outcome. Secondary measures: individual adherence changes for each discharge process; sensitivity analyses. The performance outcomes were compared 3 months before the intervention (preintervention), 3 months postintervention and at 12 months (sustainability).

RESULTS

Data from 1289 episodes of care from the two pilot hospitals were analysed. Improvements from preintervention adherence were: antiplatelet therapy (88%vs96%, p=0.02); antihypertensive prescription (61%vs79%, p<0.001); discharge planning (72%vs94%, p<0.001); composite outcome (73%vs89%, p<0.001). There was an insignificant decay effect over the 12-month sustainability period (composite outcome: 89% postintervention vs 85% sustainability period, p=0.08).

CONCLUSION

Discharge care in hospitals may be effectively improved and sustained through a staged and peer-informed, organisational intervention. The intervention warrants further application and trialling on a larger scale.