Developing 'high impact' guideline-based quality indicators for UK primary care: a multi-stage consensus process

Developing a quality and safety assessment framework for Iran's military hospitals

BACKGROUND

The first crucial step towards military hospitals performance improvement is to develop a local and scientific tool to assess quality and safety based on the context and aims of military hospitals. This study introduces a Quality and Safety Assessment Framework (Q&SAF) for Iran's military hospitals.

METHODS

This is a literature review which continued with a qualitative study. The Q&SAF for Iran's military hospitals was developed initially, through a review of the WHO's framework for hospital performance, literature review (other related framework), review of military hospital-related local documents, consultations with a national and sub-national expert. Finally, the Delphi technique used to finalize the framework.

RESULTS

Based on the literature review results; 13 hospital Q&SAF were identified. After reviewing literature review results and expert opinions; Iran's military hospitals Q&SAF was developed with 58 indictors in five dimensions including clinical effectiveness, safety, efficiency, patient-centeredness, and Responsive Management (Command and Control). The efficiency dimension had the highest number of indictors (19 indictors), whereas the patient-centered dimension had the lowest number of indices (4 indictors).

CONCLUSION

Regarding the comprehensiveness of the developed assessment framework due to its focus on the majority of quality dimensions and important components of the hospital's performance, it can be used as a useful tool for assessing and continuously improving the quality of hospitals, particularly military hospitals.

Priorities for implementation research on diagnosing cancer in primary care: a consensus process

BACKGROUND

The early detection and diagnosis of cancer to reduce avoidable mortality and morbidity is a challenging task in primary health care. There is a growing evidence base on how to enable earlier cancer diagnosis, but well-recognised gaps and delays exist around the translation of new research findings into routine clinical practice. Implementation research aims to accelerate the uptake of evidence by health care systems and professionals. We aimed to identify priorities for implementation research in early cancer diagnosis in primary care.

METHODS

We used a RAND/UCLA modified Delphi consensus process to identify and rank research priorities. We asked primary care physicians, patients and researchers to complete an online survey suggesting priorities for implementation research in cancer detection and diagnosis. We summarised and presented these suggestions to an 11-member consensus panel comprising nine primary care physicians and two patients. Panellists independently rated the importance of suggestions on a 1-9 scale (9 = very high priority; 1 = very low priority) before and after a structured group discussion. We ranked suggestions using median ratings.

RESULTS

We received a total of 115 suggested priorities for implementation research from 32 survey respondents (including 16 primary care professionals, 11 researchers, and 4 patient and public representatives; 88% of respondents were UK-based). After removing duplicates and ineligible suggestions, we presented 37 suggestions grouped within 17 categories to the consensus panel. Following two rounds of rating, 27 suggestions were highly supported (median rating 7-9). The most highly rated suggestions concerned diagnostic support (e.g., access to imaging) interventions (e.g., professional or patient education), organisation of the delivery of care (e.g., communication within and between teams) and understanding variations in care and outcomes.

CONCLUSIONS

We have identified a set of priorities for implementation research on the early diagnosis of cancer, ranked in importance by primary care physicians and patients. We suggest that researchers and research funders consider these in directing further efforts and resources to improve population outcomes.

What is the role of randomised trials in implementation science?

BACKGROUND

There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science.

MAIN BODY

There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies.

CONCLUSION

Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.

Common Features of Selection Processes of Health System Performance Indicators in Primary Healthcare: A Systematic Review

BACKGROUND

Health system performance indicators are widely used to assess primary healthcare (PHC) performance. Despite the numerous tools and some convergence on indicator criteria, there is not a clear understanding of the common features of indicator selection processes. We aimed to review the literature to identify papers that document indicator selection processes for health system performance indicators in PHC.

METHODS

We searched the online databases Scopus, Medline, and CINAHL, as well as the grey literature, without time restrictions, initially on July 31, 2019 followed by an update November 13, 2020. Empirical studies or reports were included if they described the selection of health system performance indicators or frameworks, that included PHC indicators. A combination of the process focussed research question and qualitative analysis meant a quality appraisal tool or assessment of bias could not meaningfully be applied to assess individual studies. We undertook an inductive analysis based on potential indicator selection processes criteria, drawn from health system performance indicator appraisal tools reported in the literature.

RESULTS

We identified 16 503 records of which 28 were included in the review. Most studies used a descriptive case study design. We found no consistent variations between indicator selection processes of health systems of high income and low- or lower-middle income countries. Identified common features of selection processes for indicators in PHC include literature review or adaption of an existing framework as an initial step; a consensus building process with stakeholders; structuring indicators into categories; and indicator criteria focusing on validity and feasibility. The evidence around field testing with utility and consideration of reporting burden was less clear.

CONCLUSION

Our findings highlight several characteristics of health system indicator selection processes. These features provide the groundwork to better understand how to value indicator selection processes in PHC.

Systematic development of quality indicators for skin cancer management in primary care: a mixed-methods study protocol

INTRODUCTION

Australia has the highest incidence of skin cancer in the world, with two out of three Australians expected to be diagnosed with skin cancer in their lifetime. Such incidence necessitates large-scale, effective skin cancer management practices. General practitioners (in mainstream practice and in skin cancer clinics) play an important role in skin cancer care provision, making decisions based on relevant evidence-based guidelines, protocols, experience and training. Diversity in these decision-making practices can result in unwarranted variation. Quality indicators are frequently implemented in healthcare contexts to measure performance quality at the level of the clinician and healthcare practice and mitigate unwarranted variation. Such measurements can facilitate performance comparisons between peers and a standard benchmark, often resulting in improved processes and outcomes. A standardised set of quality indicators is yet to be developed in the context of primary care skin cancer management.

AIMS

This research aims to identify, develop and generate expert consensus on a core set of quality indicators for skin cancer management in primary care.

METHODS

This mixed-methods study involves (1) a scoping review of the available evidence on quality indicators in skin cancer management in primary care, (2) identification and development of a core set of quality indicators through interviews/qualitative proforma surveys with participants, and (3) a focus group involving discussion of quality indicators according to Nominal Group Technique. Qualitative and quantitative data will be collected and analysed using thematic and descriptive statistical analytical methods.

ETHICS AND DISSEMINATION

Approval was granted by the university's Research Ethics Committee (HREC no. 520211051532420). Results from this study will be widely disseminated in publications, study presentations, educational events and reports.

Developing guideline-based key performance indicators for recurrent miscarriage care: lessons from a multi-stage consensus process with a diverse stakeholder group

BACKGROUND

Standardised care pathways tailored to women/couples who experience recurrent miscarriage are needed; however, clinical practice is inconsistent and poorly organised. In this paper, we outline our processes and experiences of developing guideline-based key performance indicators (KPIs) for recurrent miscarriage care with a diverse stakeholder group which will be used to evaluate national services. To date, such exercises have generally only involved clinicians, with the need for greater stakeholder involvement highlighted.

METHODS

Our study involved six stages: (i) identification and synthesis of recommendations for recurrent miscarriage care through a systematic review of clinical practice guidelines; (ii) a two-round modified e-Delphi survey with stakeholders to develop consensus on recommendations and outcomes; (iii) four virtual meetings to develop this consensus further; (iv) development of a list of candidate KPIs; (v) survey to achieve consensus on the final suite of KPIs and a (vi) virtual meeting to agree on the final set of KPIs. Through participatory methods, participants provided feedback on the process of KPI development.

RESULTS

From an initial list of 373 recommendations and 14 outcomes, 110 indicators were prioritised for inclusion in the final suite of KPIs: (i) structure of care (n = 20); (ii) counselling and supportive care (n = 7); (iii) investigations (n = 30); treatment (n = 34); outcomes (n = 19). Participants' feedback on the process comprised three main themes: accessibility, richness in diversity, streamlining the development process.

CONCLUSIONS

It is important and feasible to develop guideline-based KPIs with a diverse stakeholder group. One hundred and ten KPIs were prioritised for inclusion in a suite of guideline-based KPIs for recurrent miscarriage care. Insights into our experiences may help others undertaking similar projects, particularly those undertaken in the absence of a clinical guideline and/or involving a range of stakeholders.

Quality indicators and performance measures for prison healthcare: a scoping review

BACKGROUND

Internationally, people in prison should receive a standard of healthcare provision equivalent to people living in the community. Yet efforts to assess the quality of healthcare through the use of quality indicators or performance measures have been much more widely reported in the community than in the prison setting. This review aims to provide an overview of research undertaken to develop quality indicators suitable for prison healthcare.

METHODS

An international scoping review of articles published in English was conducted between 2004 and 2021. Searches of six electronic databases (MEDLINE, CINAHL, Scopus, Embase, PsycInfo and Criminal Justice Abstracts) were supplemented with journal searches, author searches and forwards and backwards citation tracking.

RESULTS

Twelve articles were included in the review, all of which were from the United States. Quality indicator selection processes varied in rigour, and there was no evidence of patient involvement in consultation activities. Selected indicators predominantly measured healthcare processes rather than health outcomes or healthcare structure. Difficulties identified in developing performance measures for the prison setting included resource constraints, data system functionality, and the comparability of the prison population to the non-incarcerated population.

CONCLUSIONS

Selecting performance measures for healthcare that are evidence-based, relevant to the population and feasible requires rigorous and transparent processes. Balanced sets of indicators for prison healthcare need to reflect prison population trends, be operable within data systems and be aligned with equivalence principles. More effort needs to be made to meaningfully engage people with lived experience in stakeholder consultations on prison healthcare quality. Monitoring healthcare structure, processes and outcomes in prison settings will provide evidence to improve care quality with the aim of reducing health inequalities experienced by people living in prison.

Explaining variable effects of an adaptable implementation package to promote evidence-based practice in primary care: a longitudinal process evaluation

Background

Implementing evidence-based recommendations is challenging in UK primary care, especially given system pressures and multiple guideline recommendations competing for attention. Implementation packages that can be adapted and hence applied to target multiple guideline recommendations could offer efficiencies for recommendations with common barriers to achievement. We developed and evaluated a package of evidence-based interventions (audit and feedback, educational outreach and reminders) incorporating behaviour change techniques to target common barriers, in two pragmatic trials for four “high impact” indicators: risky prescribing; diabetes control; blood pressure control; and anticoagulation in atrial fibrillation. We observed a significant, cost-effective reduction in risky prescribing but there was insufficient evidence of effect on the other outcomes. We explored the impact of the implementation package on both social processes (Normalisation Process Theory; NPT) and hypothesised determinants of behaviour (Theoretical Domains Framework; TDF).

Methods

We conducted a prospective multi-method process evaluation. Observational, administrative and interview data collection and analyses in eight primary care practices were guided by NPT and TDF. Survey data from trial and process evaluation practices explored fidelity.

Results

We observed three main patterns of variation in how practices responded to the implementation package. First, in integration and achievement, the package “worked” when it was considered distinctive and feasible. Timely feedback directed at specific behaviours enabled continuous goal setting, action and review, which reinforced motivation and collective action. Second, impacts on team-based determinants were limited, particularly when the complexity of clinical actions impeded progress. Third, there were delivery delays and unintended consequences. Delays in scheduling outreach further reduced ownership and time for improvement. Repeated stagnant or declining feedback that did not reflect effort undermined engagement.

Conclusions

Variable integration within practice routines and organisation of care, variable impacts on behavioural determinants, and delays in delivery and unintended consequences help explain the partial success of an adaptable package in primary care.

High adherence to recommended diabetes follow-up procedures by general practitioners is associated with lower estimated cardiovascular risk

AIMS

To explore whether the general practitioners' (GPs') performance of recommended processes of care was associated with estimated risk of cardiovascular disease (CVD) and poor glycaemic control in patients with type 2 diabetes.

METHODS

A cross-sectional study from Norwegian general practice including 6015 people with type 2 diabetes <75 years old, without CVD and their 275 GPs. The GPs were split into quintiles based on each GP's average performance of six recommended processes of care. The quintiles were the exposure variable in multilevel regression models with 10-year risk of cardiovascular events estimated by NORRISK 2 (total and modifiable fraction) and poor glycaemic control (HbA_1c >69 mmol/mol (>8.5%)) as outcome variables.

RESULTS

The mean total and modifiable estimated 10-year CVD risk was 12.3% and 3.3%, respectively. Compared with patients of GPs in the highest-performing quintile, patients treated by GPs in the lowest quintile had an adjusted total and modifiable CVD risk that was 1.88 (95% CI 1.17-2.60) and 1.78 (1.14-2.41) percent point higher. This represents a relative mean difference of 16.6% higher total and 74.8% higher modifiable risk among patients of GPs in the lowest compared with the highest quintile. For patients with GPs in the lowest-performing quintile, the adjusted odds of poor glycaemic control was 1.77 (1.27-2.46) times higher than that for patients with a GP in the highest quintile.

CONCLUSIONS

We found a pattern of lower CVD risk and better glycaemic control in patients of GPs performing more recommended diabetes processes of care.

Variation between general practitioners in type 2 diabetes processes of care

AIMS

To explore variation in general practitioners' (GPs') performance of six recommended procedures in type 2 diabetes patients <75 years without cardiovascular disease.

METHODS

Cross-sectional study of quality of diabetes care in Norway based on electronic health records from 2014. GPs (clustered in practices) were divided in quintiles based on a composite measure of performance of six processes of care. We fitted a multilevel partial ordinal regression model to identify GP factors associated with being in quintiles with better performance.

RESULTS

We identified 6015 type 2 diabetes patients from 275 GPs in 77 practices. The GPs performed on average 63.4% of the procedures; on average 46% in the poorest quintile to 81% in the best quintile with a larger range in individual GPs. After adjustments, use of a structured follow-up form was associated with GPs being in upper three quintiles (OR 12.4 (95% CI 2.37-65.1). Routines for reminders were associated with being in a better quintile (OR 2.6 (1.37-4.92). GPs' age >60 years and heavier workload were associated with poorer performance.

CONCLUSION

We found large variations in GPs' performance of processes of care. Factors reflecting structure and workload were strongly associated with performance.

Approaches of integrating the development of guidelines and quality indicators: a systematic review

Background

Guidelines and quality indicators (for example as part of a quality assurance scheme) aim to improve health care delivery and health outcomes. Ideally, the development of quality indicators should be grounded in evidence-based, trustworthy guideline recommendations. However, anecdotally, guidelines and quality assurance schemes are developed independently, by different groups of experts who employ different methodologies. We conducted an extension and update of a previous systematic review to identify, describe and evaluate approaches to the integrated development of guidelines and related quality indicators.

Methods

On May 24th, 2019 we searched in Medline, Embase and CINAHL and included studies if they reported a methodological approach to guideline-based quality indicator development and were published in English, French, or German.

Results: Out of 16,034 identified records, we included 17 articles that described a method to integrate guideline recommendations development and quality indicator development. Added to the 13 method articles from original systematic review we included a total 30 method articles. We did not find any evaluation studies. In most approaches, guidelines were a source of evidence to inform the quality indicator development. The criteria to select recommendations (e.g. level of evidence or strength of the recommendation) and to generate, select and assess quality indicators varied widely. We found methodological approaches that linked guidelines and quality indicator development explicitly, however none of the articles reported a conceptual framework that fully integrated quality indicator development into the guideline process or where quality indicator development was part of the question formulation for developing the guideline recommendations.

Conclusions

In our systematic review we found approaches which explicitly linked guidelines with quality indicator development, nevertheless none of the articles reported a comprehensive and well-defined conceptual framework which integrated quality indicator development fully into the guideline development process.

Developing and evaluating packages to support implementation of quality indicators in general practice: the ASPIRE research programme, including two cluster RCTs

Background

Dissemination of clinical guidelines is necessary but seldom sufficient by itself to ensure the reliable uptake of evidence-based practice. There are further challenges in implementing multiple clinical guidelines and clinical practice recommendations in the pressurised environment of general practice.

Objectives

We aimed to develop and evaluate an implementation package that could be adapted to support the uptake of a range of clinical guideline recommendations and be sustainably integrated within general practice systems and resources. Over five linked work packages, we developed ‘high-impact’ quality indicators to show where a measurable change in clinical practice can improve patient outcomes (work package 1), analysed adherence to selected indicators (work package 2), developed an adaptable implementation package (work package 3), evaluated the effects and cost-effectiveness of adapted implementation packages targeting four indicators (work package 4) and examined intervention fidelity and mechanisms of action (work package 5).

Setting and participants

Health-care professionals and patients from general practices in West Yorkshire, UK.

Design

We reviewed recommendations from existing National Institute for Health and Care Excellence clinical guidance and used a multistage consensus process, including 11 professionals and patients, to derive a set of ‘high-impact’ evidence-based indicators that could be measured using routinely collected data (work package 1). In 89 general practices that shared data, we found marked variations and scope for improvement in adherence to several indicators (work package 2). Interviews with 60 general practitioners, practice nurses and practice managers explored perceived determinants of adherence to selected indicators and suggested the feasibility of adapting an implementation package to target different indicators (work package 3). We worked with professional and patient panels to develop four adapted implementation packages. These targeted risky prescribing involving non-steroidal anti-inflammatory and antiplatelet drugs, type 2 diabetes control, blood pressure control and anticoagulation for atrial fibrillation. The implementation packages embedded behaviour change techniques within audit and feedback, educational outreach and (for risky prescribing) computerised prompts. We randomised 178 practices to implementation packages targeting either diabetes control or risky prescribing (trial 1), or blood pressure control or anticoagulation (trial 2), or to a further control (non-intervention) group, and undertook economic modelling (work package 4). In trials 1 and 2, practices randomised to the implementation package for one indicator acted as control practices for the other package, and vice versa. A parallel process evaluation included a further eight practices (work package 5).

Main outcome measures

Trial primary end points at 11 months comprised achievement of all recommended levels of glycated haemoglobin, blood pressure and cholesterol; risky prescribing levels; achievement of recommended blood pressure; and anticoagulation prescribing.

Results

We recruited 178 (73%) out of 243 eligible general practices. We randomised 80 practices to trial 1 (40 per arm) and 64 to trial 2 (32 per arm), with 34 non-intervention controls. The risky prescribing implementation package reduced risky prescribing (odds ratio 0.82, 97.5% confidence interval 0.67 to 0.99; p = 0.017) with an incremental cost-effectiveness ratio of £2337 per quality-adjusted life-year. The other three packages had no effect on primary end points. The process evaluation suggested that trial outcomes were influenced by losses in fidelity throughout intervention delivery and enactment, and by the nature of the targeted clinical and patient behaviours.

Limitations

Our programme was conducted in one geographical area; however, practice and patient population characteristics are otherwise likely to be sufficiently diverse and typical to enhance generalisability to the UK. We used an ‘opt-out’ approach to recruit general practices to the randomised trials. Subsequently, our trial practices may have engaged with the implementation package less than if they had actively volunteered. However, this approach increases confidence in the wider applicability of trial findings as it replicates guideline implementation activities under standard conditions.

Conclusions

This pragmatic, rigorous evaluation indicates the value of an implementation package targeting risky prescribing. In broad terms, an adapted ‘one-size-fits-all’ approach did not consistently work, with no improvement for other targeted indicators.

Future work

There are challenges in designing ‘one-size-fits-all’ implementation strategies that are sufficiently robust to bring about change in the face of difficult clinical contexts and fidelity losses. We recommend maximising feasibility and ‘stress testing’ prior to rolling out interventions within a definitive evaluation. Our programme has led on to other work, adapting audit and feedback for other priorities and evaluating different ways of delivering feedback to improve patient care.

Trial registration

Current Controlled Trials ISRCTN91989345.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 8, No. 4. See the NIHR Journals Library website for further project information.

Revitalising audit and feedback to improve patient care

Audit and feedback are widely used in quality improvement. Robbie Foy and colleagues argue that their full potential to improve patient care could be realised through a more evidence based and imaginative approach

An adaptable implementation package targeting evidence-based indicators in primary care: A pragmatic cluster-randomised evaluation

BACKGROUND

In primary care, multiple priorities and system pressures make closing the gap between evidence and practice challenging. Most implementation studies focus on single conditions, limiting generalisability. We compared an adaptable implementation package against an implementation control and assessed effects on adherence to four different evidence-based quality indicators.

METHODS AND FINDINGS

We undertook two parallel, pragmatic cluster-randomised trials using balanced incomplete block designs in general practices in West Yorkshire, England. We used 'opt-out' recruitment, and we randomly assigned practices that did not opt out to an implementation package targeting either diabetes control or risky prescribing (Trial 1); or blood pressure (BP) control or anticoagulation in atrial fibrillation (AF) (Trial 2). Within trials, each arm acted as the implementation control comparison for the other targeted indicator. For example, practices assigned to the diabetes control package acted as the comparison for practices assigned to the risky prescribing package. The implementation package embedded behaviour change techniques within audit and feedback, educational outreach, and computerised support, with content tailored to each indicator. Respective patient-level primary endpoints at 11 months comprised the following: achievement of all recommended levels of haemoglobin A1c (HbA1c), BP, and cholesterol; risky prescribing levels; achievement of recommended BP; and anticoagulation prescribing. Between February and March 2015, we recruited 144 general practices collectively serving over 1 million patients. We stratified computer-generated randomisation by area, list size, and pre-intervention outcome achievement. In April 2015, we randomised 80 practices to Trial 1 (40 per arm) and 64 to Trial 2 (32 per arm). Practices and trial personnel were not blind to allocation. Two practices were lost to follow-up but provided some outcome data. We analysed the intention-to-treat (ITT) population, adjusted for potential confounders at patient level (sex, age) and practice level (list size, locality, pre-intervention achievement against primary outcomes, total quality scores, and levels of patient co-morbidity), and analysed cost-effectiveness. The implementation package reduced risky prescribing (odds ratio [OR] 0.82; 97.5% confidence interval [CI] 0.67-0.99, p = 0.017) with an incremental cost-effectiveness ratio of £1,359 per quality-adjusted life year (QALY), but there was insufficient evidence of effect on other primary endpoints (diabetes control OR 1.03, 97.5% CI 0.89-1.18, p = 0.693; BP control OR 1.05, 97.5% CI 0.96-1.16, p = 0.215; anticoagulation prescribing OR 0.90, 97.5% CI 0.75-1.09, p = 0.214). No statistically significant effects were observed in any secondary outcome except for reduced co-prescription of aspirin and clopidogrel without gastro-protection in patients aged 65 and over (adjusted OR 0.62; 97.5% CI 0.39-0.99; p = 0.021). Main study limitations concern our inability to make any inferences about the relative effects of individual intervention components, given the multifaceted nature of the implementation package, and that the composite endpoint for diabetes control may have been too challenging to achieve.

CONCLUSIONS

In this study, we observed that a multifaceted implementation package was clinically and cost-effective for targeting prescribing behaviours within the control of clinicians but not for more complex behaviours that also required patient engagement.

TRIAL REGISTRATION

The study is registered with the ISRCTN registry (ISRCTN91989345).

Protocol for a mixed methods realist evaluation of regional District Health Board groupings in New Zealand

INTRODUCTION

Achieving effective integration of healthcare across primary, secondary and tertiary care is a key goal of the New Zealand (NZ) Health Strategy. NZ's regional District Health Board (DHB) groupings are fundamental to delivering integration, bringing the country's 20 DHBs together into four groups to collaboratively plan, fund and deliver health services within their defined geographical regions. This research aims to examine how, for whom and in what circumstances the regional DHB groupings work to improve health service integration, healthcare quality, health outcomes and health equity, particularly for Māori and Pacific peoples.

METHODS AND ANALYSIS

This research uses a mixed methods realist evaluation design. It comprises three linked studies: (1) formulating initial programme theory (IPT) through developing programme logic models to describe regional DHB working; (2) empirically testing IPT through both a qualitative process evaluation of regional DHB working using a case study design; and (3) a quantitative analysis of the impact that DHB regional groupings may have on service integration, health outcomes, health equity and costs. The findings of these three studies will allow refinement of the IPT and should lead to a programme theory which will explain how, for whom and in what circumstances regional DHB groupings improve service integration, health outcomes and health equity in NZ.

ETHICS AND DISSEMINATION

The University of Otago Human Ethics Committee has approved this study. The embedding of a clinician researcher within a participating regional DHB grouping has facilitated research coproduction, the research has been jointly conceived and designed and will be jointly evaluated and disseminated by researchers and practitioners. Uptake of the research findings by other key groups including policymakers, Māori providers and communities and Pacific providers and communities will be supported through key strategic relationships and dissemination activities. Academic dissemination will occur through publication and conference presentations.

Determining Best Practices for Management of Bacteriuria in Spinal Cord Injury: Protocol for a Mixed-Methods Study

Background

Bacteriuria, either asymptomatic (ASB) or symptomatic, urinary tract infection (UTI), is common in persons with spinal cord injury (SCI). Current Veterans Health Administration (VHA) guidelines recommend a screening urinalysis and urine culture for every veteran with SCI during annual evaluation, even when asymptomatic, which is contrary to other national guidelines. Our preliminary data suggest that a positive urine culture (even without signs or symptoms of infection) drives antibiotic use.

Objective

Through a series of innovative studies utilizing mixed methods, administrative databases, and focus groups, we will gain further knowledge about the attitudes driving current urine testing practices during the annual exam, as well as quantitative data on the clinical outcomes of these practices.

Methods

Aim 1 will identify patient, provider, and facility factors driving bacteriuria testing and subsequent antibiotic use after the SCI annual evaluation through qualitative interviews and quantitative surveys. Aim 2 will use national VHA databases to identify the predictors of urine testing and subsequent antibiotic use during the annual examination and compare the clinical outcomes of those who received antibiotics with those who did not. Aim 3 will use the information gathered from the previous 2 aims to develop the Test Smart, Treat Smart intervention, a combination of patient and provider education and resources that will help stakeholders have informed conversations about urine testing and antibiotic use; feasibility will be tested at a single site.

Results

This protocol received institutional review board and VHA Research and Development approval in July 2017, and Veterans Affairs Health Services Research and Development funding started on November 2017. As of submission of this manuscript, 10/15 (67%) of the target goal of provider interviews were complete, and 77/100 (77%) of the goal of surveys. With regard to patients, 5/15 (33%) of the target goal of interviews were complete, and 20/100 (20%) of the target goal of surveys had been completed. Preliminary analyses are ongoing; the study team plans to present these results in April 2019. Database analyses for aim 2 will begin in January 2019.

Conclusions

The negative consequences of antibiotic overuse and antibiotic resistance are well-documented and have national and even global implications. This study will develop an intervention aimed to educate stakeholders on evidence-based management of ASB and UTI and guide antibiotic stewardship in this high-risk population. The next step will be to refine the intervention and test its feasibility and effectiveness at multiple sites as well as reform policy for management of this common but burdensome condition.

International Registered Report Identifier (IRRID)

DERR1-10.2196/12272

Influence of patient characteristics on preventive service delivery and general practitioners' preventive performance indicators: A study in patients with hypertension or diabetes mellitus from Hungary

BACKGROUND

Regular primary healthcare (PHC) performance monitoring to produce a set of performance indicators for provider effectiveness is a fundamental method for improving guideline adherence but there are potential negative impacts of the inadequate application of this approach. Since performance indicators can reflect patient characteristics and working environments, as well as PHC team contributions, inadequate monitoring practices can reduce their effectiveness in the prevention of cardiometabolic disorders.

OBJECTIVES

To describe the influence of patients' characteristics on performance indicators of PHC preventive practices in patients with hypertension or diabetes mellitus.

METHODS

This cross-sectional analysis was based on a network of 165 collaborating GPs. A random sample of 4320 adults was selected from GP's patient lists. The response rate was 97.3% in this survey. Sociodemographic status, lifestyle, health attitudes and the use of recommended preventive PHC services were surveyed by questionnaire. The relationship between the use of preventive services and patient characteristics were analysed using hierarchical regression models in a subsample of 1659 survey participants with a known diagnosis of hypertension or diabetes mellitus.

RESULTS

Rates of PHC service utilization varied from 18.0% to 97.9%, and less than half (median: 44.4%; IQR: 30.8-62.5) of necessary services were used by patients. Patient attitude was as strong of an influencing factor as demographic properties but was remarkably weaker than patient socioeconomic status.

CONCLUSION

These findings emphasize that PHC performance indicators have to be evaluated concerning patient characteristics.

Identifying positive deviants in healthcare quality and safety: a mixed methods study

Objective Solutions to quality and safety problems exist within healthcare organisations, but to maximise the learning from these positive deviants, we first need to identify them. This study explores using routinely collected, publicly available data in England to identify positively deviant services in one region of the country. Design A mixed methods study undertaken July 2014 to February 2015, employing expert discussion, consensus and statistical modelling to identify indicators of quality and safety, establish a set of criteria to inform decisions about which indicators were robust and useful measures, and whether these could be used to identify positive deviants. Setting Yorkshire and Humber, England. Participants None - analysis based on routinely collected, administrative English hospital data. Main outcome measures We identified 49 indicators of quality and safety from acute care settings across eight data sources. Twenty-six indicators did not allow comparison of quality at the sub-hospital level. Of the 23 remaining indicators, 12 met all criteria and were possible candidates for identifying positive deviants. Results Four indicators (readmission and patient reported outcomes for hip and knee surgery) offered indicators of the same service. These were selected by an expert group as the basis for statistical modelling, which supported identification of one service in Yorkshire and Humber showing a 50% positive deviation from the national average. Conclusion Relatively few indicators of quality and safety relate to a service level, making meaningful comparisons and local improvement based on the measures difficult. It was possible, however, to identify a set of indicators that provided robust measurement of the quality and safety of services providing hip and knee surgery.

Role of patient and public involvement in implementation research: a consensus study

Background

Patient and public involvement (PPI) is often an essential requirement for research funding. Distinctions can be drawn between clinical research, which generally focuses on patients, and implementation research, which generally focuses on health professional behaviour. There is uncertainty about the role of PPI in this latter field. We explored and defined the roles of PPI in implementation research to inform relevant good practice guidance.

Methods

We used a structured consensus process using a convenience sample panel of nine experienced PPI and two researcher members. We drew on available literature to identify 21 PPI research roles. The panel rated their agreement with roles independently online in relation to both implementation and clinical research. Disagreements were discussed at a face-to-face meeting prior to a second online rating of all roles. Median scores were calculated and a final meeting held to review findings and consider recommendations.

Results

Ten panellists completed the consensus process. For clinical research, there was strong support and consensus for the role of PPI throughout most of the research process. For implementation research, there were eight roles with consensus and strong support, seven roles with consensus but weaker support and six roles with no consensus. There were more disagreements relating to PPI roles in implementation research compared with clinical research. PPI was rated as contributing less to the design and management of implementation research than for clinical research.

Conclusions

The roles of PPI need to be tailored according to the nature of research to ensure authentic and appropriate involvement. We provide a framework to guide the planning, conduct and reporting of PPI in implementation research, and encourage further research to evaluate its use.

To what extent can behaviour change techniques be identified within an adaptable implementation package for primary care? A prospective directed content analysis

BACKGROUND

Interpreting evaluations of complex interventions can be difficult without sufficient description of key intervention content. We aimed to develop an implementation package for primary care which could be delivered using typically available resources and could be adapted to target determinants of behaviour for each of four quality indicators: diabetes control, blood pressure control, anticoagulation for atrial fibrillation and risky prescribing. We describe the development and prospective verification of behaviour change techniques (BCTs) embedded within the adaptable implementation packages.

METHODS

We used an over-lapping multi-staged process. We identified evidence-based, candidate delivery mechanisms-mainly audit and feedback, educational outreach and computerised prompts and reminders. We drew upon interviews with primary care professionals using the Theoretical Domains Framework to explore likely determinants of adherence to quality indicators. We linked determinants to candidate BCTs. With input from stakeholder panels, we prioritised likely determinants and intervention content prior to piloting the implementation packages. Our content analysis assessed the extent to which embedded BCTs could be identified within the packages and compared them across the delivery mechanisms and four quality indicators.

RESULTS

Each implementation package included at least 27 out of 30 potentially applicable BCTs representing 15 of 16 BCT categories. Whilst 23 BCTs were shared across all four implementation packages (e.g. BCTs relating to feedback and comparing behaviour), some BCTs were unique to certain delivery mechanisms (e.g. 'graded tasks' and 'problem solving' for educational outreach). BCTs addressing the determinants 'environmental context' and 'social and professional roles' (e.g. 'restructuring the social and 'physical environment' and 'adding objects to the environment') were indicator specific. We found it challenging to operationalise BCTs targeting 'environmental context', 'social influences' and 'social and professional roles' within our chosen delivery mechanisms.

CONCLUSION

We have demonstrated a transparent process for selecting, operationalising and verifying the BCT content in implementation packages adapted to target four quality indicators in primary care. There was considerable overlap in BCTs identified across the four indicators suggesting core BCTs can be embedded and verified within delivery mechanisms commonly available to primary care. Whilst feedback reports can include a wide range of BCTs, computerised prompts can deliver BCTs at the time of decision making, and educational outreach can allow for flexibility and individual tailoring in delivery.

A protocol for the development and piloting of quality measures to support the Healthier You: The NHS Diabetes Prevention Programme

Background

The increasing prevalence of type 2 diabetes in the UK creates an additional, potentially preventable burden on health care and service providers. The Healthier You: NHS Diabetes Prevention Programme aims to reduce the incidence of type 2 diabetes through the identification of people at risk and the provision of intensive lifestyle change support. The provision of this care can be monitored through quality measurement at both the general practice and specialist service level.

Aim

To develop quality measures through piloting to assess the validity, credibility, acceptability, reliability, and feasibility of any proposed measures.

Design & setting

The non-experimental mixed design piloting study consists of consensus testing and exploratory research with GPs, commissioners, and patients from Herefordshire, England.

Method

A mixed-method approach will be used to develop and validate measures for diabetes prevention care and evaluate their performance over a 6-month pilot period consisting of consensus testing using a modified RAND approach with GPs and commissioners; four focus groups with 8-10 participants discussing experiences of non-diabetic hyperglycaemia (NDH), perceived ability to access care and prevent diabetes, and views on potential quality measures; and piloting final measures with at least five general practices for baseline and 6-month data.

Results

The findings will inform the implementation of the diabetes prevention quality measures on a national scale while addressing any issue with validity, credibility, feasibility, and cost-effectiveness.

Conclusion

Healthcare professionals and patients have the opportunity to evaluate the reliability, acceptability, and validity of measures.

Implementing Quality Indicators for Diabetes and Hypertension in Family Medicine in Slovenia

INTRODUCTION

A new form of family practices was introduced in 2011 through a pilot project introducing nurse practitioners as members of team and determining a set of quality indicators. The aim of this article was to assess the quality of diabetes and hypertension management.

METHODS

We included all family medicine practices that were participating in the project in December 2015 (N=584). The following data were extracted from automatic electronic reports on quality indicators: gender and specialisation of the family physician, status (public servant/self-contracted), duration of participation in the project, region of Slovenia, the number of inhabitants covered by a family medicine practice, the name of IT provider, and levels of selected quality indicators.

RESULTS

Out of 584 family medicine practices that were included in this project at the end of 2015, 568 (97.3%) had complete data and could be included in this analysis. The highest values were observed for structure quality indicator (list of diabetics) and the lowest for process and outcome quality indicators. The values of the selected quality indicators were independently associated with the duration of participation in the project, some regions of Slovenia where practices were located, and some IT providers of the practices.

CONCLUSION

First, the analysis of data on quality indicators for diabetes and hypertension in this primary care project pointed out the problems which are currently preventing higher quality of chronic patient management at the primary health care level.

Are characteristics of team members important for quality management of chronic patients at primary care level?

AIMS AND OBJECTIVES

To determine the possible associations between higher levels of selected quality indicators and the characteristics of providers.

BACKGROUND

In 2011, an ongoing project on a new model of family medicine practice was launched in Slovenia; the family physicians' working team (a family physician and a practice nurse) was extended by a nurse practitioner working 0.5 full-time equivalents. This was an example of a personalised team approach to managing chronic patients.

METHODS

We included all family medicine practices in the six units of the Community Health Centre Ljubljana which were participating in the project in December 2015 (N = 66). Data were gathered from automatic electronic reports on quality indicators provided monthly by each practice. We also collected demographic data.

RESULTS

There were 66 family medicine teams in the sample, with 165 members of their teams (66 family physicians, 33 nurse practitioners and 66 practice nurses). Fifty-six (84.4%) of the family physicians were women, as were 32 (97.0%) of the nurse practitioners, and 86 (95.5%) of the practice nurses. Multivariate analysis showed that a higher level of the quality indicator "Examination of diabetic foot once per year" was independently associated with nurse practitioners having attended additional education on diabetes, duration of participation in the project, age and years worked since graduation of nurse practitioners, working in the Center unit and not working in the Bezigrad unit.

CONCLUSIONS

Characteristics of team members are important in fostering quality management of chronic patients. Nurse practitioners working in new model family practices need obligatory, continuous professional education in the management of chronic patients.

RELEVANCE TO CLINICAL PRACTICE

The quality of care of chronic patients depends on the specific characteristics of the members of the team, which should be taken into account when planning quality improvements.

Variations in achievement of evidence-based, high-impact quality indicators in general practice: An observational study

BACKGROUND

There are widely recognised variations in the delivery and outcomes of healthcare but an incomplete understanding of their causes. There is a growing interest in using routinely collected 'big data' in the evaluation of healthcare. We developed a set of evidence-based 'high impact' quality indicators (QIs) for primary care and examined variations in achievement of these indicators using routinely collected data in the United Kingdom (UK).

METHODS

Cross-sectional analysis of routinely collected, electronic primary care data from a sample of general practices in West Yorkshire, UK (n = 89). The QIs covered aspects of care (including processes and intermediate clinical outcomes) in relation to diabetes, hypertension, atrial fibrillation, myocardial infarction, chronic kidney disease (CKD) and 'risky' prescribing combinations. Regression models explored the impact of practice and patient characteristics. Clustering within practice was accounted for by including a random intercept for practice.

RESULTS

Median practice achievement of the QIs ranged from 43.2% (diabetes control) to 72.2% (blood pressure control in CKD). Considerable between-practice variation existed for all indicators: the difference between the highest and lowest performing practices was 26.3 percentage points for risky prescribing and 100 percentage points for anticoagulation in atrial fibrillation. Odds ratios associated with the random effects for practices emphasised this; there was a greater than ten-fold difference in the likelihood of achieving the hypertension indicator between the lowest and highest performing practices. Patient characteristics, in particular age, gender and comorbidity, were consistently but modestly associated with indicator achievement. Statistically significant practice characteristics were identified less frequently in adjusted models.

CONCLUSIONS

Despite various policy and improvement initiatives, there are enduring inappropriate variations in the delivery of evidence-based care. Much of this variation is not explained by routinely collected patient or practice variables, and is likely to be attributable to differences in clinical and organisational behaviour.

Impact of Disease Prevalence Adjustment on Hospitalization Rates for Chronic Ambulatory Care-Sensitive Conditions in Germany

OBJECTIVES

To explore effects of disease prevalence adjustment on ambulatory care-sensitive hospitalization (ACSH) rates used for quality comparisons.

DATA SOURCES/STUDY SETTING

County-level hospital administrative data on adults discharged from German hospitals in 2011 and prevalence estimates based on administrative ambulatory diagnosis data were used.

STUDY DESIGN

A retrospective cross-sectional study using in- and outpatient secondary data was performed.

DATA COLLECTION

Hospitalization data for hypertension, diabetes, heart failure, chronic obstructive pulmonary disease, and asthma were obtained from the German Diagnosis Related Groups (DRG) database. Prevalence estimates were obtained from the German Central Research Institute of Ambulatory Health Care.

PRINCIPAL FINDINGS

Crude hospitalization rates varied substantially across counties (coefficients of variation [CV] 28-37 percent across conditions); this variation was reduced by prevalence adjustment (CV 21-28 percent). Prevalence explained 40-50 percent of the observed variation (r = 0.65-0.70) in ACSH rates for all conditions except asthma (r = 0.07). Between 30 percent and 38 percent of areas moved into or outside condition-specific control limits with prevalence adjustment.

CONCLUSIONS

Unadjusted ACSH rates should be used with caution for high-stakes public reporting as differences in prevalence may have a marked impact. Prevalence adjustment should be considered in models analyzing ACSH.

Process quality indicators for chronic kidney disease risk management: a systematic literature review

BACKGROUND AND OBJECTIVES

Quality indicators (QIs) can be used for measuring the quality of actions of healthcare providers. This systematic review gives an overview of such QIs measuring processes of care for chronic kidney disease (CKD), and identifies the QIs that have content, face, operational and/or predictive validity.

METHODS

PubMed and Embase were searched using a strategy combining the terms "quality of care," "quality indicators" and "chronic kidney disease". Papers were included if they focused on developing, testing or applying QIs for assessing the quality of care in adult patients with CKD not on renal replacement therapy.

RESULTS

Two hundred and seventy-three QIs from thirty-one papers were extracted, including QIs on adequate monitoring of kidney function and vascular risk factors, on indicated treatment, drug safety, adherence and referral to a specialist. The QIs that were considered content, face and operational valid focused on monitoring of glomerular filtration rate, albumin-creatinine ratio, lipid levels and blood pressure, the use of non-steroidal anti-inflammatory drugs, nitrofurantoin and biphosphonates in patients with CKD, and QIs on monitoring haemoglobin and treatment with angiotensin-converting-enzyme-inhibitors/angiotensin-receptor-II-blockers in patients with CKD and comorbidities. No QIs were tested for predictive validity. In addition, only two QIs focused on diet and no other QIs focused on lifestyle management.

CONCLUSIONS

Based on this review, sufficiently validated QIs can be selected for measuring the quality of CKD care. This review provides insight in QIs that need further validation, and in areas of care where QIs are still lacking.

Using the Theoretical Domains Framework (TDF) to understand adherence to multiple evidence-based indicators in primary care: a qualitative study

Background

There are recognised gaps between evidence and practice in general practice, a setting posing particular implementation challenges. We earlier screened clinical guideline recommendations to derive a set of ‘high-impact’ indicators based upon criteria including potential for significant patient benefit, scope for improved practice and amenability to measurement using routinely collected data. Here, we explore health professionals’ perceived determinants of adherence to these indicators, examining the degree to which determinants were indicator-specific or potentially generalisable across indicators.

Methods

We interviewed 60 general practitioners, practice nurses and practice managers in West Yorkshire, the UK, about adherence to four indicators: avoidance of risky prescribing; treatment targets in type 2 diabetes; blood pressure targets in treated hypertension; and anticoagulation in atrial fibrillation. Interview questions drew upon the Theoretical Domains Framework (TDF). Data were analysed using framework analysis.

Results

Professional role and identity and environmental context and resources featured prominently across all indicators whilst the importance of other domains, for example, beliefs about consequences, social influences and knowledge varied across indicators. We identified five meta-themes representing more general organisational and contextual factors common to all indicators.

Conclusions

The TDF helped elicit a wide range of reported determinants of adherence to ‘high-impact’ indicators in primary care. It was more difficult to pinpoint which determinants, if targeted by an implementation strategy, would maximise change. The meta-themes broadly underline the need to align the design of interventions targeting general practices with higher level supports and broader contextual considerations. However, our findings suggest that it is feasible to develop interventions to promote the uptake of different evidence-based indicators which share common features whilst also including content-specific adaptations.

Optimizing primary care research participation: a comparison of three recruitment methods in data-sharing studies

BACKGROUND

Recruitment of representative samples in primary care research is essential to ensure high-quality, generalizable results. This is particularly important for research using routinely recorded patient data to examine the delivery of care. Yet little is known about how different recruitment strategies influence the characteristics of the practices included in research.

OBJECTIVE

We describe three approaches for recruiting practices to data-sharing studies, examining differences in recruitment levels and practice representativeness.

METHODS

We examined three studies that included varying populations of practices from West Yorkshire, UK. All used anonymized patient data to explore aspects of clinical practice. Recruitment strategies were 'opt-in', 'mixed opt-in and opt-out' and 'opt-out'. We compared aggregated practice data between recruited and not-recruited practices for practice list size, deprivation, chronic disease management, patient experience and rates of unplanned hospital admission.

RESULTS

The opt-out strategy had the highest recruitment (80%), followed by mixed (70%) and opt-in (58%). Practices opting-in were larger (median 7153 versus 4722 patients, P = 0.03) than practices that declined to opt-in. Practices recruited by mixed approach were larger (median 7091 versus 5857 patients, P = 0.04) and had differences in the clinical quality measure (58.4% versus 53.9% of diabetic patients with HbA1c ≤ 59 mmol/mol, P < 0.01). We found no differences between practices recruited and not recruited using the opt-out strategy for any demographic or quality of care measures.

CONCLUSION

Opt-out recruitment appears to be a relatively efficient approach to ensuring participation of typical general practices. Researchers should, with appropriate ethical safeguards, consider opt-out recruitment of practices for studies involving anonymized patient data sharing.

Action to Support Practices Implement Research Evidence (ASPIRE): protocol for a cluster-randomised evaluation of adaptable implementation packages targeting 'high impact' clinical practice recommendations in general practice

BACKGROUND

There are recognised gaps between evidence and practice in general practice, a setting which provides particular challenges for implementation. We earlier screened clinical guideline recommendations to derive a set of 'high impact' indicators based upon criteria including potential for significant patient benefit, scope for improved practice and amenability to measurement using routinely collected data. We aim to evaluate the effectiveness and cost-effectiveness of a multifaceted, adaptable intervention package to implement four targeted, high impact recommendations in general practice.

METHODS/DESIGN

The research programme Action to Support Practice Implement Research Evidence (ASPIRE) includes a pair of pragmatic cluster-randomised trials which use a balanced incomplete block design. Clusters are general practices in West Yorkshire, United Kingdom (UK), recruited using an 'opt-out' recruitment process. The intervention package adapted to each recommendation includes combinations of audit and feedback, educational outreach visits and computerised prompts with embedded behaviour change techniques selected on the basis of identified needs and barriers to change. In trial 1, practices are randomised to adapted interventions targeting either diabetes control or risky prescribing and those in trial 2 to adapted interventions targeting either blood pressure control in patients at risk of cardiovascular events or anticoagulation in atrial fibrillation. The respective primary endpoints comprise achievement of all recommended target levels of haemoglobin A1c (HbA1c), blood pressure and cholesterol in patients with type 2 diabetes, a composite indicator of risky prescribing, achievement of recommended blood pressure targets for specific patient groups and anticoagulation prescribing in patients with atrial fibrillation. We are also randomising practices to a fifth, non-intervention control group to further assess Hawthorne effects. Outcomes will be assessed using routinely collected data extracted 1 year after randomisation. Economic modelling will estimate intervention cost-effectiveness. A process evaluation involving eight non-trial practices will examine intervention delivery, mechanisms of action and unintended consequences.

DISCUSSION

ASPIRE will provide 'real-world' evidence about the effects, cost-effectiveness and delivery of adapted intervention packages targeting high impact recommendations. By implementing our adaptable intervention package across four distinct clinical topics, and using 'opt-out' recruitment, our findings will provide evidence of wider generalisability.

TRIAL REGISTRATION

ISRCTN91989345.