Exploring variation in the use of feedback from national clinical audits: a realist investigation

Data-driven collaborative QUality improvement in Cardiac Rehabilitation (QUICR) to increase program completion: protocol for a cluster randomized controlled trial

BACKGROUND

Coronary heart disease (CHD) is the leading cause of deaths and disability worldwide. Cardiac rehabilitation (CR) effectively reduces the risk of future cardiac events and is strongly recommended in international clinical guidelines. However, CR program quality is highly variable with divergent data systems, which, when combined, potentially contribute to persistently low completion rates. The QUality Improvement in Cardiac Rehabilitation (QUICR) trial aims to determine whether a data-driven collaborative quality improvement intervention delivered at the program level over 12 months: (1) increases CR program completion in eligible patients with CHD (primary outcome), (2) reduces hospital admissions, emergency department presentations and deaths, and costs, (3) improves the proportion of patients receiving guideline-indicated CR according to national and international benchmarks, and (4) is feasible and sustainable for CR staff to implement routinely.

METHODS

QUICR is a multi-centre, type-2, hybrid effectiveness-implementation cluster-randomized controlled trial (cRCT) with 12-month follow-up. Eligible CR programs (n = 40) and the individual patient data within them (n ~ 2,000) recruited from two Australian states (New South Wales and Victoria) are randomized 1:1 to the intervention (collaborative quality improvement intervention that uses data to identify and manage gaps in care) or control (usual care with data collection only). This sample size is required to achieve 80% power to detect a difference in completion rate of 22%. Outcomes will be assessed using intention-to-treat principles. Mixed-effects linear and logistic regression models accounting for clusters within allocated groupings will be applied to analyse primary and secondary outcomes.

DISCUSSION

Addressing poor participation in CR by patients with CHD has been a longstanding challenge that needs innovative strategies to change the status-quo. This trial will harness the collaborative power of CR programs working simultaneously on common problem areas and using local data to drive performance. The use of data linkage for collection of outcomes offers an efficient way to evaluate this intervention and support the improvement of health service delivery.

ETHICS

Primary ethical approval was obtained from the Northern Sydney Local Health District Human Research Ethics Committee (2023/ETH01093), along with site-specific governance approvals.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry (ANZCTR) ACTRN12623001239651 (30/11/2023) ( https://anzctr.org.au/Trial/Registration/TrialReview.aspx?id=386540&isReview=true ).

Effects of a team Quality Improvement method in a national clinical audit programme of four clinical specialties in Ministry of Health hospitals in Saudi Arabia

In 2018, the Ministry of Health (MoH) in Saudi Arabia developed a clinical excellence strategy. An objective was to reduce variation in clinical practices in MoH hospitals, particularly for conditions with high mortality in Saudi Arabia, by applying best practice clinical standards and using the clinical audit process to measure clinical practice. The strategy included working with multiprofessional teams in hospitals to implement improvements needed in clinical practice. To test the feasibility of carrying out national clinical audits in MoH hospitals, audits were carried out in 16 MoH hospitals on four clinical subjects-acute myocardial infarction, major trauma, sepsis, and stroke. Clinical expert groups, including Saudi clinicians and an international clinical expert, developed clinical care standards for the four conditions from analyses of international and Saudi clinical guidelines. The audits were designed with the expert groups. Multiprofessional teams were appointed to carry out the audits in designated MoH hospitals. Data collectors in each hospital were trained to collect data. Workshops were held with the teams on the clinical care standards and how data would be collected for the audits, and later, on the findings of data collection and how to use the improvement process to implement changes to improve compliance with the standards. After 4 months, data collection was repeated to determine if compliance with the clinical care standards had improved. Data collected from each hospital for both cycles of data collection were independently reliably tested. All designated hospitals participated in the audits, collecting and submitting data for two rounds of data collection and implementing improvement plans after the first round of data collection. All hospitals made substantial improvements in clinical practices. Of a total of 84 measures used to assess compliance with a total of 52 clinical care standards for the four clinical conditions, improvements were made by hospital teams in 58 (69.1%) measures. Improvements were statistically significant for 34 (40.5%) measures. The project demonstrated that well-designed and executed audits using evidence-based clinical care standards can result in substantial improvements in clinical practices in MoH hospitals in Saudi Arabia. Keys to success were the improvement methodology built into the audit process and the requirement for hospitals to appoint multiprofessional teams to carry out the audits. The approach adds to evidence on the effectiveness of clinical audits in achieving improvements in clinical quality and can be replicated in national audit programmes.

How do healthcare providers use national audit data for improvement?

BACKGROUND

Substantial resources are invested by Health Departments worldwide in introducing National Clinical Audits (NCAs). Yet, there is variable evidence on the NCAs' effectiveness and little is known on factors underlying the successful use of NCAs to improve local practice. This study will focus on a single NCA (the National Audit of Inpatient Falls -NAIF 2017) to explore: (i) participants' perspectives on the NCA reports, local feedback characteristics and actions undertaken following the feedback underpinning the effective use of the NCA feedback to improve local practice; (ii) reported changes in local practice following the NCA feedback in England and Wales.

METHODS

Front-line staff perspectives were gathered through interviews. An inductive qualitative approach was used. Eighteen participants were purposefully sampled from 7 of the 85 participating hospitals in England and Wales. Analysis was guided by constant comparative techniques.

RESULTS

Regarding the NAIF annual report, interviewees valued performance benchmarking with other hospitals, the use of visual representations and the inclusion of case studies and recommendations. Participants stated that feedback should target front-line healthcare professionals, be straightforward and focused, and be delivered through an encouraging and honest discussion. Interviewees highlighted the value of using other relevant data sources alongside NAIF feedback and the importance of continuous data monitoring. Participants reported that engagement of front-line staff in the NAIF and following improvement activities was critical. Leadership, ownership, management support and communication at different organisational levels were perceived as enablers, while staffing level and turnover, and poor quality improvement (QI) skills, were perceived as barriers to improvement. Reported changes in practice included increased awareness and attention to patient safety issues and greater involvement of patients and staff in falls prevention activities.

CONCLUSIONS

There is scope to improve the use of NCAs by front-line staff. NCAs should not be seen as isolated interventions but should be fully embedded and integrated into the QI strategic and operational plans of NHS trusts. The use of NCAs could be optimised, but knowledge of them is poor and distributed unevenly across different disciplines. More research is needed to provide guidance on key elements to consider throughout the whole improvement process at different organisational levels.

A Systematic Review of Electronic Medical Record Driven Quality Measurement and Feedback Systems

Historically, quality measurement analyses utilize manual chart abstraction from data collected primarily for administrative purposes. These methods are resource-intensive, time-delayed, and often lack clinical relevance. Electronic Medical Records (EMRs) have increased data availability and opportunities for quality measurement. However, little is known about the effectiveness of Measurement Feedback Systems (MFSs) in utilizing EMR data. This study explores the effectiveness and characteristics of EMR-enabled MFSs in tertiary care. The search strategy guided by the PICO Framework was executed in four databases. Two reviewers screened abstracts and manuscripts. Data on effect and intervention characteristics were extracted using a tailored version of the Cochrane EPOC abstraction tool. Due to study heterogeneity, a narrative synthesis was conducted and reported according to PRISMA guidelines. A total of 14 unique MFS studies were extracted and synthesized, of which 12 had positive effects on outcomes. Findings indicate that quality measurement using EMR data is feasible in certain contexts and successful MFSs often incorporated electronic feedback methods, supported by clinical leadership and action planning. EMR-enabled MFSs have the potential to reduce the burden of data collection for quality measurement but further research is needed to evaluate EMR-enabled MFSs to translate and scale findings to broader implementation contexts.

Patient and healthcare provider perceptions on using patient-reported experience measures (PREMs) in routine clinical care: a systematic review of qualitative studies

BACKGROUND

Patient-reported experience measures (PREMs) assess quality-of-care from patients' perspectives. PREMs can be used to enhance patient-centered care and facilitate patient engagement in care. With increasing quality improvement studies in clinical practice, the use of PREMs has surged. As a result, knowledge about stakeholder experiences of using PREMs to assess quality of care across diverse clinical settings is needed to inform PREM implementation efforts. To address this, this review examines the qualitative literature on patient and healthcare provider experiences of using PREMs in clinical practice.

METHODS

Medline, Embase and PsycInfo were systematically searched from inception to May 2021. Additional searching of reference lists for all included articles and relevant review articles were performed. Retrieved articles were screened for eligibility by one reviewer and 25% cross-checked by a second reviewer across all stages of the review. Full texts meeting eligibility criteria were appraised against the COREQ checklist for quality assessment and thematic analysis was used to analyze textual data extracted from the results.

RESULTS

Electronic searches identified 2683 records, of which 20 studies met eligibility criteria. Extracted data were synthesized into six themes: facilitators to PREM implementation; barriers to PREM implementation; healthcare providers' perspectives towards using PREMs; patients' perspectives towards using PREMs; advantages of using PREMs in clinical practice; limitations and practical considerations to reduce resistance of PREM usage. The primary factors facilitating and impeding the use of PREMs include organizational-, staff- and patient-related factors.

CONCLUSION

Results can be used to guide the usage and implementation of PREMs in clinical settings by addressing the identified barriers and building on the perceived benefits to encourage adoption of PREMs. Results around facilitators to PREM implementation and practical considerations could also promote appropriate use of PREMs by healthcare providers, helping to improve practice and the quality of care based on patient feedback.

Analysis of a Web-Based Dashboard to Support the Use of National Audit Data in Quality Improvement: Realist Evaluation

Background

Dashboards can support data-driven quality improvements in health care. They visualize data in ways intended to ease cognitive load and support data comprehension, but how they are best integrated into working practices needs further investigation.

Objective

This paper reports the findings of a realist evaluation of a web-based quality dashboard (QualDash) developed to support the use of national audit data in quality improvement.

Methods

QualDash was co-designed with data users and installed in 8 clinical services (3 pediatric intensive care units and 5 cardiology services) across 5 health care organizations (sites A-E) in England between July and December 2019. Champions were identified to support adoption. Data to evaluate QualDash were collected between July 2019 and August 2021 and consisted of 148.5 hours of observations including hospital wards and clinical governance meetings, log files that captured the extent of use of QualDash over 12 months, and a questionnaire designed to assess the dashboard’s perceived usefulness and ease of use. Guided by the principles of realist evaluation, data were analyzed to understand how, why, and in what circumstances QualDash supported the use of national audit data in quality improvement.

Results

The observations revealed that variation across sites in the amount and type of resources available to support data use, alongside staff interactions with QualDash, shaped its use and impact. Sites resourced with skilled audit support staff and established reporting systems (sites A and C) continued to use existing processes to report data. A number of constraints influenced use of QualDash in these sites including that some dashboard metrics were not configured in line with user expectations and staff were not fully aware how QualDash could be used to facilitate their work. In less well-resourced services, QualDash automated parts of their reporting process, streamlining the work of audit support staff (site B), and, in some cases, highlighted issues with data completeness that the service worked to address (site E). Questionnaire responses received from 23 participants indicated that QualDash was perceived as useful and easy to use despite its variable use in practice.

Conclusions

Web-based dashboards have the potential to support data-driven improvement, providing access to visualizations that can help users address key questions about care quality. Findings from this study point to ways in which dashboard design might be improved to optimize use and impact in different contexts; this includes using data meaningful to stakeholders in the co-design process and actively engaging staff knowledgeable about current data use and routines in the scrutiny of the dashboard metrics and functions. In addition, consideration should be given to the processes of data collection and upload that underpin the quality of the data visualized and consequently its potential to stimulate quality improvement.

International Registered Report Identifier (IRRID)

RR2-10.1136/bmjopen-2019-033208