Assessing fidelity of delivery of smoking cessation behavioural support in practice

Identifying the active content of interventions targeting the psychological well-being of carers of people with motor neuron disease: A systematic review

OBJECTIVES

The primary aim of this research was to use a taxonomy of behavior change techniques (BCTTv1) to identify, map, and describe the active components of intervention and comparator groups in studies evaluating the psychological well-being (PWB) of motor neuron disease (MND) carers. Secondary aims were to (a) identify absent active ingredients and (b) explore whether variability in the effectiveness of interventions targeting the PWB of MND carers could be better explained through improved characterization of the active content of these interventions.

METHODS

Mixed-methods systematic review based on Joanna Briggs Institute methodology for quantitative, qualitative, and mixed-methods reviews and Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Content-coding of interventions targeting the PWB of MND carers using BCTTv1 was conducted.

RESULTS

Sixteen manuscripts describing 14 studies were included. Forty-one of the possible 93 behavior change techniques (BCTs, 44%) were identified as active ingredients, while 52 BCTs (56%) were absent. BCTs were identified in all 14 intervention groups and 4 control groups. Four of the 16 overall BCTTv1 categories were absent. Eleven of the 14 studies demonstrated PWB benefits from their interventions.

SIGNIFICANCE OF RESULTS

Identified and absent BCTs and BCTTv1 categories were mapped for all study groups, enabling a transparent characterization of active intervention content associated with positive PWB outcomes. Directions to improve interventions in this nascent field of research included the investigation of relevant untested BCTs in this population and the management of reporting and methodological quality issues.

Care partners experience of an oral health intervention for individuals with mild cognitive impairment and mild dementia using behavior change technique: A qualitative study

Oral health declines in older adults with cognitive impairment. We aimed to improve oral hygiene outcomes for individuals with mild cognitive impairment (MCI) or mild dementia (MD) by fostering behavior changes among carepartners assisting them. We used qualitative data of verbatim transcripts of coaching sessions with carepartners (n = 17 dyads:10 dyads for MCI, 7 dyads for MD). Directed and emergent coding were used to understand behavior change techniques (BCTs). BCTs were compared with carepartners of participants with MCI and MD. Most frequently used BCTs in both groups: prompts and cues, instruction on how to perform the behavior, review behavioral goal, and problem solving. Different BCTs emerged in study: social support-unspecified of the MCI group and credible source for MD group. Findings clarified active intervention components, common BCTs used by carepartners, and different BCT approaches for both participants. Findings help to elucidate the mechanisms of changes in individuals' behaviors in these interventions.

Development of fidelity of delivery and enactment measures for interventions in communication disorders

OBJECTIVES

This study was part of a process evaluation for a single-blind, randomized controlled pilot study comparing Better Conversations with Primary Progressive Aphasia (BCPPA), an approach to communication partner training, with no speech and language therapy treatment. It was necessary to explore fidelity of delivery (delivery of intervention components) and intervention enactment (participants' use of intervention skills in the form of conversation behaviours comprising facilitators, that enhance the conversational flow, and barriers, that impeded the flow of conversation). This study aimed to: (1) Outline an adapted methodological process that uses video observation, to measure both fidelity of delivery and enactment. (2) Measure the extent to which the BCPPA pilot study was delivered as planned, and enacted.

DESIGN

Observational methods were used alongside statistical analysis to explore the fidelity of intervention and enactment using video recordings obtained from the BCPPA pilot study.

METHODS

A 5-step methodology, was developed to measure fidelity of delivery and enactment for the BCPPA study using video-recorded data. To identify delivery of intervention components, a random sample of eight video recorded and transcribed BCPPA intervention sessions was coded. To examine the enactment of conversation behaviours, 108 transcribed 10 -min-video recorded conversations were coded from 18 participants across the control and intervention group.

RESULTS

Checklists and guidelines for measurement of fidelity of treatment delivery and coding spreadsheets and guidelines for measurement of enactment are presented. Local collaborators demonstrated 87.2% fidelity to the BCPPA protocol. Participants in the BCPPA treatment group increased their use of facilitator behaviours enacted in conversation from a mean of 13.5 pre-intervention to 14.2 post-intervention, whilst control group facilitators decreased from a mean of 15.5 to 14.4, over the same timescale.

CONCLUSIONS

This study proposes a novel and robust methods, using video recorded intervention sessions and conversation samples, to measure both fidelity of intervention delivery and enactment. The learnings from this intervention are transferable to other communication interventions.

The Effects of Scheduled Smoking Reduction and Precessation Nicotine Replacement Therapy on Smoking Cessation: Randomized Controlled Trial With Compliance

BACKGROUND

Smoking remains a major public health problem, and it is important to provide a variety of efficacious and appealing options to encourage smokers to quit smoking. Scheduled smoking is a method of gradual reduction, preparing smokers to quit by systematically reducing cigarette consumption according to a predetermined schedule that increases the time between cigarette consumption. Gradual reduction may be preferred to abrupt quitting, but the efficacy of this cessation approach is unclear.

OBJECTIVE

This study aims, first, to evaluate the overall effectiveness of scheduled smoking alone, or in combination with precessation nicotine replacement therapy (NRT), versus standard NRT starting on the quit date with no prior smoking reduction and, second, to evaluate the impact of schedule compliance on the effectiveness of the intervention.

METHODS

A total of 916 participants recruited from the Houston metropolitan area were randomly assigned to 1 of the following 3 groups: scheduled smoking plus a precessation nicotine patch (n=306, 33.4%), scheduled smoking only with no precessation patch (n=309, 33.7%), and enhanced usual care (n=301, 32.9%) control. The primary abstinence outcomes were carbon monoxide-verified, self-reported, 7-day point prevalence abstinence at 2 and 4 weeks after the quit date. Unadjusted and adjusted logistic regression analyses were performed to evaluate the intervention effect. Scheduled smoking was implemented using a handheld device for 3 weeks before quitting. This trial was not registered because data collection began before July 1, 2005.

RESULTS

Results for the first aim showed no overall differences in abstinence among the 3 groups in both the unadjusted and adjusted models. However, the results for the second aim showed a clear effect on abstinence by schedule compliance at 2 and 4 weeks and 6 months after quitting (odds ratio [OR] 2.01, 95% CI 1.31-3.07), 4 weeks (OR 1.58, 95% CI 1.05-2.38), and 6 months (OR 1.68, 95% CI 1.04-2.64), with the differences at 2 and 4 weeks after quitting being the most robust. We also found that scheduled smoking was related to a reduction in nicotine withdrawal, negative affect, and craving when compared with the controls.

CONCLUSIONS

Scheduled smoking, when combined with precessation use of NRT, can result in significantly higher abstinence rates than usual care (abrupt quitting with NRT), particularly in the early postquit phase (2 and 4 weeks after cessation) when smokers are compliant with the procedure. Scheduled smoking also produced a better overall quitting experience by reducing symptoms of nicotine withdrawal and craving, in comparison with usual care, which could encourage future quit attempts. Studies in this area should focus on the use of counseling or other methods to improve adherence.

Examining public health practitioners' perceptions and use of behavioural sciences to design health promotion interventions

BACKGROUND

Behavioural sciences have been shown to support the development of more effective interventions aimed at promoting healthy lifestyles. However, the operationalization of this knowledge seems to be sub-optimal in public health. Effective knowledge transfer strategies are thus needed to optimize the use of knowledge from behavioural sciences in this field. To this end, the present study examined public health practitioners' perceptions and use of theories and frameworks from behavioural sciences to design health promotion interventions.

METHODS

This study adopted an exploratory qualitative design. Semi-structured interviews were conducted among 27 public health practitioners from across Canada to explore current intervention development processes, the extent to which they integrate theory and framework from behavioural sciences, and their perceptions regarding the use of this knowledge to inform intervention design. Practitioners from the public sector or non-profit/private organizations who were involved in the development of interventions aimed at promoting physical activity, healthy eating, or other healthy lifestyle habits (e.g., not smoking) were eligible to participate.

RESULTS

Public health practitioners generally agreed that behaviour change is an important goal of public health interventions. On the other hand, behavioural science theories and frameworks did not appear to be fully integrated in the design of public health interventions. The main reasons were (1) a perceived lack of fit with current professional roles and tasks; (2) a greater reliance on experiential-produced knowledge rather than academic knowledge (mainly for tailoring interventions to local setting characteristics); (3) the presence of a fragmented knowledge base; (4) the belief that theories and frameworks require too much time and resources to be operationalized; and 4) the belief that using behavioural sciences might undermine partnership building.

CONCLUSIONS

This study provided valuable insights that may inform knowledge transfer strategies that could be optimally designed to support the integration of behavioural sciences theories and frameworks into public health practices.

Introduction of the professional nurse advocate role using a quality implementation framework

BACKGROUND

Implementation of the professional nurse advocate (PNA) role and the Advocating and Educating for QUality ImProvement model (A-EQUIP) in nursing is relatively new. The model aims to build personal and professional resilience, enhance the quality of care and support preparedness for appraisal and professional revalidation.

AIM

To describe the implementation of the PNA role in a combined acute and community trust in England.

METHODS

A quality implementation framework was used to appraise and represent locally derived strategic activities for successful implementation of the role in an acute and community hospital in England. The content of this framework was derived from a synthesis of 25 implementation frameworks focusing on important elements understood to represent quality implementation.

FINDINGS

The article identifies strengths and weaknesses to implementation and ways to sustain early implementation success.

CONCLUSION

Using a quality implementation framework can provide a clear path for the successful implementation of the professional nurse advocate role. Professional nurse advocates should be supported to develop a culture of effective supervision within their organisation.

Does the design of the NHS Low-Calorie Diet Programme have fidelity to the programme specification? A documentary review of service parameters and behaviour change content in a type 2 diabetes intervention

BACKGROUND

NHS England commissioned four independent service providers to pilot low-calorie diet programmes to drive weight loss, improve glycaemia and potentially achieve remission of Type 2 Diabetes across 10 localities. Intervention fidelity might contribute to programme success. Previous research has illustrated a drift in fidelity in the design and delivery of other national diabetes programmes.

AIMS

(1) To describe and compare the programme designs across the four service providers; (2) To assess the fidelity of programme designs to the NHS England service specification.

METHODS

The NHS England service specification documents and each provider's programme design documents were double-coded for key intervention content using the Template for Intervention Description and Replication Framework and the Behaviour Change Technique (BCT) Taxonomy.

RESULTS

The four providers demonstrated fidelity to most but not all of the service parameters stipulated in the NHS England service specification. Providers included between 74% and 87% of the 23 BCTs identified in the NHS specification. Twelve of these BCTs were included by all four providers; two BCTs were consistently absent. An additional seven to 24 BCTs were included across providers.

CONCLUSIONS

A loss of fidelity for some service parameters and BCTs was identified across the provider's designs; this may have important consequences for programme delivery and thus programme outcomes. Furthermore, there was a large degree of variation between providers in the presence and dosage of additional BCTs. How these findings relate to the fidelity of programme delivery and variation in programme outcomes and experiences across providers will be examined.

Exploring the Association between Individual-Level Attributes and Fidelity to a Vocational Rehabilitation Intervention within a Randomised Controlled Trial

Understanding what attributes or characteristics of those delivering interventions affect intervention fidelity and patient outcomes is important for contextualising intervention effectiveness. It may also inform implementation of interventions in future research and clinical practice. This study aimed to explore the relationships between attributes of Occupational Therapists (OTs), their faithful delivery of an early stroke specialist vocational rehabilitation intervention (ESSVR), and stroke survivor return-to-work (RTW) outcomes. Thirty-nine OTs were surveyed about their experience and knowledge of stroke and vocational rehabilitation and were trained to deliver ESSVR. ESSVR was delivered across 16 sites in England and Wales between February 2018 and November 2021. OTs received monthly mentoring to support ESSVR delivery. The amount of mentoring each OT received was recorded in OT mentoring records. Fidelity was assessed using an intervention component checklist completed using retrospective case review of one randomly selected participant per OT. Linear and logistic regression analyses explored relationships between OT attributes, fidelity, and stroke survivor RTW outcome. Fidelity scores ranged from 30.8 to 100% (Mean: 78.8%, SD: 19.2%). Only OT engagement in mentoring was significantly associated with fidelity (b = 0.29, 95% CI = 0.05-0.53, p < 0.05). Increased fidelity (OR = 1.06, 95% CI = 1.01-1.1, p = 0.01) and increasing years of stroke rehabilitation experience (OR = 1.17, 95% CI = 1.02-1.35) was significantly associated with positive stroke survivor RTW outcomes. Findings of this study suggest that mentoring OTs may increase fidelity of delivery of ESSVR, which may also be associated with positive stroke survivor return-to-work outcomes. The results also suggest that OTs with more experience of stroke rehabilitation may be able to support stroke survivors to RTW more effectively. Upskilling OTs to deliver complex interventions, such as ESSVR, in clinical trials may require mentoring support in addition to training to ensure fidelity.

Developing an implementation fidelity checklist for a vocational rehabilitation intervention

BACKGROUND

Despite growing numbers of studies reporting the efficacy of complex interventions and their implementation, many studies fail to report information on implementation fidelity or describe how fidelity measures used within the study were developed. This study aimed to develop a fidelity checklist for measuring the implementation fidelity of an early, stroke-specialist vocational rehabilitation intervention (ESSVR) in the RETAKE trial.

METHODS

To develop the fidelity measure, previous checklists were reviewed to inform the assessment structure, and core intervention components were extracted from intervention descriptions into a checklist, which was ratified by eight experts in fidelity measurement and complex interventions. Guidance notes were generated to assist with checklist completion. To test the measure, two researchers independently applied the checklist to fifteen stroke survivor intervention case notes using retrospective observational case review. The scoring was assessed for interrater reliability.

RESULTS

A fidelity checklist containing 21 core components and 6 desirable components across 4 stages of intervention delivery was developed with corresponding guidance notes. Interrater reliability of each checklist item ranged from moderate to perfect (Cohen's kappa 0.69-1).

CONCLUSIONS

The resulting checklist to assess implementation fidelity is fit for assessing the delivery of vocational rehabilitation for stroke survivors using retrospective observational case review. The checklist proved its utility as a measure of fidelity and may be used to inform the design of future implementation strategies.

TRIAL REGISTRATION

ISRCTN, ISRCTN12464275. Registered on 13 March 2018.

Examining the complementarity between the ERIC compilation of implementation strategies and the behaviour change technique taxonomy: a qualitative analysis

BACKGROUND

Efforts to generate evidence for implementation strategies are frustrated by insufficient description. The Expert Recommendations for Implementing Change (ERIC) compilation names and defines implementation strategies; however, further work is needed to describe the actions involved. One potentially complementary taxonomy is the behaviour change techniques (BCT) taxonomy. We aimed to examine the extent and nature of the overlap between these taxonomies.

METHODS

Definitions and descriptions of 73 strategies in the ERIC compilation were analysed. First, each description was deductively coded using the BCT taxonomy. Second, a typology was developed to categorise the extent of overlap between ERIC strategies and BCTs. Third, three implementation scientists independently rated their level of agreement with the categorisation and BCT coding. Finally, discrepancies were settled through online consensus discussions. Additional patterns of complementarity between ERIC strategies and BCTs were labelled thematically. Descriptive statistics summarise the frequency of coded BCTs and the number of strategies mapped to each of the categories of the typology.

RESULTS

Across the 73 strategies, 41/93 BCTs (44%) were coded, with 'restructuring the social environment' as the most frequently coded (n=18 strategies, 25%). There was direct overlap between one strategy (change physical structure and equipment) and one BCT ('restructuring physical environment'). Most strategy descriptions (n=64) had BCTs that were clearly indicated (n=18), and others where BCTs were probable but not explicitly described (n=31) or indicated multiple types of overlap (n=15). For some strategies, the presence of additional BCTs was dependent on the form of delivery. Some strategies served as examples of broad BCTs operationalised for implementation. For eight strategies, there were no BCTs indicated, or they did not appear to focus on changing behaviour. These strategies reflected preparatory stages and targeted collective cognition at the system level rather than behaviour change at the service delivery level.

CONCLUSIONS

This study demonstrates how the ERIC compilation and BCT taxonomy can be integrated to specify active ingredients, providing an opportunity to better understand mechanisms of action. Our results highlight complementarity rather than redundancy. More efforts to integrate these or other taxonomies will aid strategy developers and build links between existing silos in implementation science.

Behaviour change interventions to promote health and well-being among older migrants: A systematic review

Background

Whether behaviour change interventions are effective for the maintenance of older migrants’ health and well-being is uncertain. A systematic review was conducted to assess evidence for the capacity of behaviour change techniques (BCTs) to promote the health and well-being of older migrants.

Methods

Electronic databases (Cochrane CENTRAL, Embase, Ovid MEDLINE and Web of Science) were searched systematically to identify relevant randomised controlled trials, pre–post studies and quasi-experimental studies published before March 2021. Additional articles were identified through citation tracking. Studies examining BCTs used to promote the health and/or well-being of older migrants were eligible. Two independent reviewers used the Behaviour Change Technique Taxonomy version 1 to extract data on BCTs. Data on intervention functions (IFs) and cultural adaption strategies were also extracted. Intervention contents (BCTs, IFs, culture adaption strategies) were compared across effective and ineffective interventions according to health and well-being outcome clusters (anthropometrics, health behaviour, physical functioning, mental health and cognitive functioning, social functioning and generic health and well-being).

Results

Forty-three studies (23 randomised controlled trials, 13 pre–post studies and 7 quasi-experimental studies) reporting on 39 interventions met the inclusion criteria. Thirteen BCTs were identified as promising for at least one outcome cluster: goal-setting (behaviour), problem-solving, behavioural contract, self-monitoring of behaviour, social support (unspecified), instruction on how to perform the behaviour, information about health consequences, information about social and environmental consequences, demonstration of the behaviour, social comparison, behavioural practice/rehearsal, generalisation of a target behaviour and addition of objects to the environment. Three BCTs (instruction on how to perform the behaviour, demonstration of the behaviour, and social comparison) and two IFs (modelling and training) were identified as promising for all outcome clusters.

Conclusions

Thirteen distinct BCTs are promising for use in future interventions to optimise health and well-being among older migrants. Future research should focus on the effectiveness of these BCTs (combinations) in various contexts and among different subgroups of older migrants, as well as the mechanisms through which they act. Given the scarcity of interventions in which cultural adaption has been taken into account, future behavioural change interventions should consider cultural appropriateness for various older migrant (sub)groups.

Trial registration

PROSPERO CRD42018112859.

Assessing and ensuring fidelity of the nationally implemented English NHS diabetes prevention programme: lessons learned for the implementation of large-scale behaviour change programmes

Background

Health services interventions are typically more effective in randomised controlled trials than in routine healthcare. One explanation for this ‘voltage drop', i.e. reduction in effectiveness, is a reduction in intervention fidelity, i.e. the extent to which a programme is implemented as intended. This article discusses how to optimise intervention fidelity in nationally implemented behaviour change programmes, using as an exemplar the National Health Service Diabetes Prevention Programme (NHS-DPP); a behaviour change intervention for adults in England at increased risk of developing Type 2 diabetes, delivered by four independent provider organisations. We summarise key findings from a thorough fidelity evaluation of the NHS-DPP assessing design (whether programme plans were in accordance with the evidence base), training (of staff to deliver key intervention components), delivery (of key intervention components), receipt (participant understanding of intervention content), and highlight lessons learned for the implementation of other large-scale programmes.

Results

NHS-DPP providers delivered the majority of behaviour change content specified in their programme designs. However, a drift in fidelity was apparent at multiple points: from the evidence base, during programme commissioning, and on to providers’ programme designs. A lack of clear theoretical rationale for the intervention contents was apparent in design, training, and delivery. Our evaluation suggests that many fidelity issues may have been less prevalent if there was a clear underpinning theory from the outset.

Conclusion

We provide recommendations to enhance fidelity of nationally implemented behaviour change programmes. The involvement of a behaviour change specialist in clarifying the theory of change would minimise drift of key intervention content. Further, as loss of fidelity appears notable at the design stage, this should be given particular attention. Based on these recommendations, we describe examples of how we have worked with commissioners of the NHS-DPP to enhance fidelity of the next roll-out of the programme.

Reporting Treatment Fidelity in Behavioral Tobacco Treatment Clinical Trials: Scoping Review and Measurement Recommendations

INTRODUCTION

Adoption of rigorous standards for reporting treatment fidelity is essential for advancing discovery, validation, and implementation of behavioral treatments. Whereas the NIH Behavior Change Consortium (BCC) developed an assessment tool to assess the quality of reporting and monitoring of treatment fidelity across health behavior change interventions, it has not yet been applied specifically to treatment fidelity in behavioral tobacco treatment trials.

AIMS AND METHODS

We conducted a scoping review of peer-reviewed, clinical trials of behavioral adult tobacco treatment interventions published in English between 2006 and 2018. Using the BCC treatment fidelity checklist, articles were coded for the presence or absence of various treatment fidelity strategies within each of 5 domains: Design, Training, Delivery, Receipt, and Enactment. Eligible articles (N = 755) were coded by two independent coders.

RESULTS

The proportion of reporting strategies varied within the fidelity domains, ranging from 5.2% to 96.3% in Design, 1.9% to 24.9% in Training, 2.6% to 32.3% in Delivery, 5.2% to 44.3% in Receipt, and 6.7% to 43.2% in Enactment. The mean proportion of adherence to treatment fidelity strategies within each domain was: Design (68%), Training (14%), Delivery (15%), Receipt (16%), and Enactment (25%). Only 11 studies achieved ≥80% reporting across >1 fidelity domain. There was no evidence for improvement in fidelity reporting across the 13-year time frame from the initial BCC publication to the present.

CONCLUSIONS

These findings illustrate the lack of consistency in fidelity reporting in tobacco treatment trials and underscore the challenges faced in evaluating rigor and reproducibility, as well as interpretation and dissemination of findings. Recommendations are made for improving fidelity reporting in tobacco treatment trials.

IMPLICATIONS

The SRNT Treatment Research Network sponsored a scoping review to summarize the current state of reporting treatment fidelity and make recommendations for best practices in reporting fidelity in tobacco treatment trials. The review identified a lack of consistency in fidelity reporting, illustrating the challenges faced in evaluating rigor, and reproducibility, as well as interpretation and dissemination of findings.

Implementation of pharmaceutical services in Brazilian primary health care: a cross-sectional study

BACKGROUND

In the Brazilian public health system, primary health care (PHC) is provided by the municipalities and is considered the entry level of the Unified Health System (SUS). Governmental pharmaceutical services (PharmSes) are part of the SUS, including PHC, and are the most significant way in which patients access medicine and services. Considering the diversity of the country, the municipalities have the autonomy to decide how PharmSes are implemented. Even though policies and procedures should be implemented as expected by policy makers and experts, municipality characteristics may interfere with implementation fidelity. Therefore, this study evaluated the degree to which the PharmSes in PHC were delivered as intended in Brazilian municipalities.

METHODS

We analysed data from a secondary database originating from a cross-sectional nationwide study carried out by the Ministry of Health and the World Bank from 2013 to 2015. Data on 465 municipalities and the Federal District were collected from 4939 governmental PharmSes. A rating system comprising 43 indicators was developed and applied to the dataset to obtain the implementation degree (ID) of each PharmSe. Additionally, the IDs of the two PharmSes dimensions and the nine components were measured.

RESULTS

Overall, the ID of the PharmSes in Brazilian PHC was evaluated as critical. The ID was critical in 81% of the municipalities (n = 369), incipient in 14% (n = 65) and unsatisfactory in 4.8% (n = 22). Regarding the PharmSes dimensions, the 'medicine management' (MM) ID was considered critical (Mean = 46%), while the 'care management' (CM) ID was incipient (Mean = 22%). In terms of the PharmSes components, the highest ID was achieved by 'forecasting' (58%). In contrast, 'continuing education and counselling' showed the lowest figure (ID = 11%) in the whole sample, followed by 'information and communication' and 'teamwork'.

CONCLUSIONS

The degree to which PharmSes were implemented was critical (ID< 50%). This analysis demonstrated that PharmSes were implemented with low fidelity, which may be related to the low availability of medicine in PHC. Although the care management component requires more attention, considering their incipient ID, all components must be reviewed. Municipalities must increase their investment in PharmSes implementation in order to maximize the benefits of these services and guarantee the essential right of access to medicine.

Implementation strategy fidelity evaluation for a multidisciplinary Chest Injury Protocol (ChIP)

Background

Blunt chest wall injuries can lead to complications, especially without early intervention. A blunt Chest Injury Protocol (ChIP) was developed to help improve the consistency of evidence-based care following admission to the emergency department. Implementation strategy fidelity is the extent to which the strategies of implementation are delivered in line with the intended plan. The aim of this study was to assess fidelity to the strategies of the implementation plan developed for ChIP.

Methods

A retrospective evaluation of strategies used for implementation was performed, specifically the behaviour change techniques (BCTs). BCTs were used as part of an implementation plan derived based on the Behaviour Change Wheel from results from a staff survey at two hospitals.

Levels of implementation or adaptation for BCTs were scored by implementers as follows: ‘Were the behaviour change interventions implemented?’ (0 = ‘not implemented’, 1 = partially implemented, and 2 = fully implemented); ‘Were adaptations made to the implementation plan?’, scored 1 (many changes from plan) to 4 (just as planned). Free text explanation to their responses was also collected with supporting evidence and documentation (such as emails, implementation checklists, audit reports, and incident reports).

Results

There was high overall fidelity of 97.6% for BCTs partially or fully implemented. More than three quarters (32/42, 76.2%) of the BCTs were fully implemented with an additional 9/42 (21.4%) partially implemented. BCTs that were not fully implemented were social support, feedback on behaviour, feedback on outcomes of behaviour, adding objects to the environment, and restructuring the environment. The modes of delivery with poorer implementation or increased adaptations were clinical champions and audit/feedback.

Conclusions

This study describes the evaluation of implementation strategy fidelity in the acute care context. The systematic use and application of the behaviour change wheel was used to develop an implementation plan and was associated with high implementation strategy fidelity. A fidelity checklist developed during the implementation process may help implementers assess fidelity.

Trial registration

Trial registered on ANZCTR. Registration number ACTRN12618001548224, date approved 17/09/2018

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-021-00189-8.

Fitness facility staff demonstrate high fidelity when implementing an evidence-based diabetes prevention program

Translating evidence-based diabetes prevention programs into the community is needed to make promising interventions accessible to individuals at-risk of type 2 diabetes. To increase the likelihood of successful translation, implementation evaluations should be conducted to understand program outcomes and provide feedback for future scale-up sites. The purpose of this research was to examine the delivery of, and engagement with, an evidence-based diet and exercise diabetes prevention program when delivered by fitness facility staff within a community organization. Ten staff from a community organization were trained to deliver the diabetes prevention program. Between August 2019-March 2020, 26 clients enrolled in the program and were assigned to one of the ten staff. Three fidelity components were accessed. First, staff completed session-specific fidelity checklists (n = 156). Second, two audio-recorded counseling sessions from all clients underwent an independent coder fidelity check (n = 49). Third, staff recorded client goals on session-specific fidelity checklists and all goals were independently assessed for (a) staff goal-setting fidelity, (b) client intervention receipt, and (c) client goal enactment by two coders (n = 285). Average self-reported fidelity was 90% for all six sessions. Independent coder scores for both counseling sessions were 83% and 81%. Overall staff helped clients create goals in line with program content and had a goal achievement of 78%. The program was implemented with high fidelity by staff at a community organization and clients engaged with the program. Findings increase confidence that program effects are due to the intervention itself and provide feedback to refine implementation strategies to support future scale-up efforts.

Agreement and Disparities between Women and Stop-Smoking Specialists about How to Promote Adherence to Nicotine Replacement Therapy in Pregnancy

Evidence for the effectiveness of nicotine replacement therapy (NRT) for smoking-cessation in pregnancy is weak. This has been attributed to insufficient dosing and low adherence. This study investigated the acceptability of key messages and delivery modes for a behavioral intervention to increase NRT adherence in pregnancy. Semi-structured telephone interviews were carried out with pregnant or postpartum women aged ≥16 from across England, who had been offered NRT during pregnancy as part of a quit attempt and who struggled to quit (n = 10), and a focus group with stop-smoking specialists from across England (n = 6). The two data sources were coded separately using a thematic approach and then integrated to compare perspectives. Women and specialists agreed on message tone and delivery modes. However, views diverged on the most influential sources for certain messages and whether some information should be given proactively or reactively. There was also disagreement over which messages were novel and which were routinely delivered. This study demonstrates the value of capturing and integrating different perspectives and informational requirements when developing behavior-change interventions. The findings provide useful insights for designing a pregnancy-specific NRT adherence intervention that is acceptable to both those who will deliver and receive it.

Measuring fidelity to delivery of a new smoking cessation intervention integrated into routine tuberculosis care in Pakistan and Bangladesh: Contextual differences and opportunities

INTRODUCTION

Tobacco smoking among tuberculosis (TB) patients leads to poorer treatment outcomes. Smoking cessation support should be integrated into routine TB care. We measured healthcare providers' fidelity to a smoking cessation intervention integrated into routine TB care, in Bangladesh and Pakistan. We aimed to understand the role of providers and settings in the implementation of behavior support (BS) messages for TB and smoking cessation.

METHODS

The integrated BS intervention was implemented in TB clinics (24 public and 1 private). Cross-sectional data were collected on the fidelity of delivery of the BS intervention using a predefined fidelity index based on an existing validated method of measuring intervention fidelity. Audio-recordings of patient-provider BS sessions were coded using the fidelity index. Intervention fidelity was presented as the proportion of sessions that implemented BS messages.

RESULTS

A total of 96 sessions were conducted, 37 in Bangladesh and 59 in Pakistan. In public settings, TB medication advice was offered in 91.9% (95% CI: 78.7- 97.2) of sessions in Bangladesh, and in 75.5% (95% CI: 62.4-85.1) of sessions in Pakistan; whilst it was offered in 83.3% (95% CI: 43.7-97.0) of sessions in the private setting in Pakistan. Patients' smoking status was assessed in 70.3% (95% CI: 54.2-82.5) of sessions in Bangladesh, and in 34.0% (95% CI: 22.7-47.4) of sessions in the public setting and in 66.7% (95% CI: 30.0-90.3) of sessions in the private setting in Pakistan. A quit date was set in 32.4% (95% CI: 19.6-48.5) of all sessions in Bangladesh, and in 33.3% (95% CI: 9.6-70.0) of all sessions in the public setting in Pakistan.

CONCLUSIONS

Fidelity to the intended delivery of the intervention was found to be high for TB-related messages but not for smoking cessation messages. Clinic contexts may play a mediating role in health workers' opportunities to deliver the intervention as planned.

TRIAL REGISTRATION

International Standard Randomized Clinical Trial Number (ISRCTN43811467). Registered 23 March 2016, https://doi.org/10.1186/ISRCTN43811467.

Fidelity of primary care nurses' delivery of a behavioural change intervention enhancing physical activity in patients at risk of cardiovascular disease: an observational study

OBJECTIVE

To evaluate the fidelity of delivery of a nurse-led intervention to enhance physical activity in patients at risk for cardiovascular diseases, the Activate intervention, by assessing: (1) self-reported fidelity of delivery; (2) observed fidelity of delivery; (3) quality of delivery of the Activate intervention and (4) nurses' beliefs about their capability, motivation, confidence and effectiveness towards delivering the Activate intervention, including behavioural change techniques.

DESIGN

An observational study.

SETTING

General practices in the Netherlands.

PARTICIPANTS

Primary care nurses (n=20) from 16 general practices.

PRIMARY AND SECONDARY OUTCOME MEASURES

Nurses' self-reported fidelity was evaluated using checklists (n=282), and the observed fidelity and quality of delivery were examined using audiorecordings of consultations of the delivery of the Activate intervention (n=42). Nurses' beliefs towards delivering the intervention were assessed using questionnaires (n=72).

RESULTS

The self-reported fidelity was 88.1% and observed fidelity was 85.4%, representing high fidelity. The observed fidelity of applied behavioural change techniques was moderate (75.0%). The observed quality of delivery was sufficient and varied among nurses (mean 2.9; SD 4.4; range 0-4). Nurses' beliefs about their capability, motivation, confidence and effectiveness towards delivering the intervention increased over time.

CONCLUSIONS

Nurses delivered most intervention components as intended with sufficient quality. Nurses believed they were capable, motivated and confident to deliver the intervention. They believed the intervention was effective to increase patients' physical activity level. Despite the high fidelity and moderate fidelity of applied behavioural change techniques, the varying quality of delivery within and across nurses might have diluted the effectiveness of the Activate intervention.

TRIAL REGISTRATION NUMBER

NCT02725203.

Smoking Cessation in Lower Socioeconomic Groups: Adaptation and Pilot Test of a Rolling Group Intervention

Background

Socioeconomic inequalities in smoking rates persist and tend to increase, as evidence-based smoking cessation programs are insufficiently accessible and appropriate for lower socioeconomic status (SES) smokers to achieve long-term abstinence. Our study is aimed at systematically adapting and pilot testing a smoking cessation intervention for this specific target group.

Methods

First, we conducted a needs assessment, including a literature review and interviews with lower SES smokers and professional stakeholders. Next, we selected candidate interventions for adaptation and decided which components needed to be adopted, adapted, or newly developed. We used Intervention Mapping to select effective methods and practical strategies and to build a coherent smoking cessation program. Finally, we pilot tested the adapted intervention to assess its potential effectiveness and its acceptability for lower SES smokers.

Results

The core of the adapted rolling group intervention was the evidence-based combination of behavioral support and pharmacotherapy. The intervention offered both group and individual support. It was open to smokers, smokers who had quit, and quitters who had relapsed. The professional-led group meetings had a fixed structure. Themes addressed included quitting-related coping skills and health-related and poverty-related issues. Methods applied were role modeling, practical learning, reinforcement, and positive feedback. In the pilot test, half of the 22 lower SES smokers successfully quit smoking. The intervention allowed them to “quit at their own pace” and to continue despite a possible relapse. Participants appraised the opportunities for social comparison and role modeling and the encouraging atmosphere. The trainers were appreciated for their competencies and personal feedback.

Conclusions

Our adapted rolling group intervention for lower SES smokers was potentially effective as well as feasible, suitable, and acceptable for the target group. Further research should determine the intervention's effectiveness. Our detailed report about the adaptation process and resulting intervention may help reveal the mechanisms through which such interventions might operate effectively.

Is the NHS Diabetes Prevention Programme Intervention Delivered as Planned? An Observational Study of Fidelity of Intervention Delivery

Background

The NHS Diabetes Prevention Programme (NHS-DPP) has been delivered by four commercial organizations across England, to prevent people with impaired glucose tolerance developing Type 2 diabetes. Evidence reviews underpinning the NHS-DPP design specification identified 19 Behavior Change Techniques (BCTs) that are the intervention “active ingredients.” It is important to understand the discrepancies between BCTs specified in design and BCTs actually delivered.

Purpose

To compare observed fidelity of delivery of BCTs that were delivered to (a) the NHS-DPP design specification, and (b) the programme manuals of four provider organizations.

Methods

Audio-recordings were made of complete delivery of NHS-DPP courses at eight diverse sites (two courses per provider organization). The eight courses consisted of 111 group sessions, with 409 patients and 35 facilitators. BCT Taxonomy v1 was used to reliably code the contents of NHS-DPP design specification documents, programme manuals for each provider organization, and observed NHS-DPP group sessions.

Results

The NHS-DPP design specification indicated 19 BCTs that should be delivered, whereas only seven (37%) were delivered during the programme in all eight courses. By contrast, between 70% and 89% of BCTs specified in programme manuals were delivered. There was substantial under-delivery of BCTs that were designed to improve self-regulation of behavior, for example, those involving problem solving and self-monitoring of behavior.

Conclusions

A lack of fidelity in delivery to the underlying evidence base was apparent, due to poor translation of design specification to programme manuals. By contrast, the fidelity of delivery to the programme manuals was relatively good. Future commissioning should focus on ensuring the evidence base is more accurately translated into the programme manual contents.

The Fidelity of Training in Behaviour Change Techniques to Intervention Design in a National Diabetes Prevention Programme

Background

The National Health Service Diabetes Prevention Programme (NHS-DPP) is a behavioural intervention for people identified as high risk for developing type 2 diabetes that has been rolled out across England. The present study evaluates whether the four commercial providers of the NHS-DPP train staff to deliver behaviour change technique (BCT) content with fidelity to intervention plans.

Method

One set of mandatory training courses across the four NHS-DPP providers (seven courses across 13 days) was audio-recorded, and all additional training materials used were collected. Recordings and training materials were coded for BCT content using the BCT Taxonomy v1. BCTs and depth of training (e.g. instruction, demonstration, practice) of BCT content was checked against providers’ intervention plans.

Results

Ten trainers and 78 trainees were observed, and 12 documents examined. The number of unique BCTs in audio recordings and associated training materials ranged from 19 to 44 across providers, and staff were trained in 53 unique BCTs across the whole NHS-DPP. Staff were trained in 66% of BCTs that were in intervention plans, though two providers trained staff in approximately half of BCTs to be delivered. The most common way that staff were trained in BCT delivery was through instruction. Training delivery style (e.g. experiential versus educational) varied between providers.

Conclusion

Observed training evidences dilution from providers’ intervention plans. NHS-DPP providers should review their training to ensure staff are trained in all key intervention components, ensuring thorough training of BCTs (e.g. demonstrating and practicing how to deliver) to enhance BCT delivery.

How Has Intervention Fidelity Been Assessed in Smoking Cessation Interventions? A Systematic Review

Introduction

Intervention fidelity concerns the degree to which interventions are implemented as intended. Fidelity frameworks propose fidelity is a multidimensional concept relevant at intervention designer, provider, and recipient levels; yet the extent to which it is assessed multidimensionally is unclear. Smoking cessation interventions are complex, including multiple components, often delivered over multiple sessions and/or at scale in clinical practice; this increases susceptibility variation in the fidelity with which they are delivered. This review examined the extent to which five dimensions from the Behaviour Change Consortium fidelity framework (design, training, delivery, receipt, and enactment) were assessed in fidelity assessments of smoking cessation interventions (randomised control trials (RCTs)).

Methods

Five electronic databases were searched using terms “smoking cessation,” “interventions,” “fidelity,” and “randomised control trials.” Eligible studies included RCTs of smoking cessation behavioural interventions, published post 2006 after publication of the framework, reporting assessment of fidelity. The data extraction form was structured around the framework, which specifies a number of items regarding assessment and reporting of each dimension. Data extraction included study characteristics, dimensions assessed, data collection, and analysis strategies. A score per dimension was calculated, indicating its presence.

Results

55 studies were reviewed. There was a wide variability in data collection approaches used to assess fidelity. Fidelity of delivery was the most commonly assessed and linked to the intervention outcomes (73% of the studies). Fidelity of enactment scored the highest according to the framework (average of 92.7%), and fidelity of training scored the lowest (average of 37.1%). Only a quarter of studies linked fidelity data to outcomes (27%).

Conclusion

There is wide variability in methodological and analytical approaches that precludes comparison and synthesis. In order to realise the potential of fidelity investigations to increase scientific confidence in the interpretation of observed trial outcomes, studies should include analyses of the association between fidelity data and outcomes. Findings have highlighted recommendations for improving fidelity evaluations and reporting practices.

Fidelity of the delivery of NHS Health Checks in general practice: an observational study

BACKGROUND

The NHS Health Check programme aims to reduce the risk of common preventable diseases by providing risk information and behaviour change advice. Failure to deliver the consultation appropriately could undermine its efficacy. To date, to the authors' knowledge, there are no published data on the fidelity of delivery of NHS Health Checks.

AIM

To assess the fidelity of delivery of NHS Health Checks in general practice.

DESIGN & SETTING

Fidelity assessment of video and audio recordings of NHS Health Check consultations conducted in four GP practices across the East of England.

METHOD

A secondary analysis of 38 NHS Health Check consultations, which were video or audio recorded as part of a pilot study of introducing discussions of cancer risk into NHS Health Checks. Using a checklist based on the NHS Health Check Best Practice Guidance, fidelity of delivery was assessed as the proportion of key elements completed during the consultations.

RESULTS

The mean number of elements of the NHS Health Check completed across all consultations was 14.5/18 (80.6%), with a range of 10 to 17 (55.6% to 94.4%). The mean fidelity for risk assessment, risk communication, and risk management sections was 8.7/10 (87.0%), 4.1/5 (82.0%), and 1.7/3 (56.7%), respectively. Clinically appropriate lifestyle advice was given in 34/38 consultations. Elements with the lowest fidelity were ethnicity assessment (n = 12/38; 31.6%), family history of cardiovascular disease (CVD) assessment (n = 25/38; 65.8%), AUDIT-C communication (n = 13/38; 34.2%), and dementia risk management (n = 6/38; 15.8%).

CONCLUSION

Although fidelity of delivery was high overall, important elements of the NHS Health Check were being regularly omitted. Opportunities for behaviour change, particularly relating to alcohol consumption and dementia risk management, may be being missed.

A Lay Health Worker Intervention to Increase Uptake and Completion of Pulmonary Rehabilitation in Chronic Obstructive Pulmonary Disease: Assessing Fidelity of Intervention Delivery

The benefits of pulmonary rehabilitation (PR) for chronic obstructive pulmonary disease (COPD) are restricted by poor uptake and completion. Lay health workers (LHWs) have been effective in improving access to treatment and services for other health conditions. We have successfully shown the feasibility of this approach in a PR setting and its acceptability to the LHWs and COPD patients. We present here the feasibility of assessment, and the fidelity of delivery of LHW support achieved for COPD patients referred for PR. LHWs, volunteer COPD patients experienced in PR, received training in the intervention including communication skills, confidentiality and behaviour change techniques (BCTs). Interactions between LHWs and patients were recorded, transcribed and coded for delivery style and BCTs. Inter-rater agreement on the coding of delivery style and BCTs was high at >84%. LHWs built rapport and communicated attentively in over 80% of interactions. LHWs most consistently delivered BCTs concerning information provision about the consequences of PR often making those consequences salient by referring to their own positive experience of PR. Social support BCTs were also used by the majority of LHWs. The use of BCTs varied between LHWs. The assessment of intervention delivery fidelity by LHWs was feasible. LHW training in the setting of PR should add emphasis to the acquisition of BCT skills relating to goal setting and action planning.

Does the design of the NHS Diabetes Prevention Programme intervention have fidelity to the programme specification? A document analysis

AIMS

To assess fidelity of the Healthier You: NHS Diabetes Prevention Programme (NHS-DPP), a behavioural intervention for people in England at high risk of developing type 2 diabetes, to the specified programme features.

METHODS

Document analysis of the NHS-DPP programme specification, including National Institute for Health and Care Excellence (NICE) PH38 diabetes prevention guidance. This was compared with the intervention design (framework response documents and programme manuals) from all four independent providers delivering the NHS-DPP. Documents were coded using the Template for Intervention Description and Replication framework (describing service parameters) and the Behaviour Change Technique Taxonomy v1.

RESULTS

Providers demonstrated good fidelity to service parameters of the NHS-DPP. The NHS-DPP specification indicated 19 unique behaviour change techniques. Framework responses for the four providers contained between 24 and 32 distinct behaviour change techniques, and programme manuals contained between 23 and 45 distinct behaviour change techniques, indicating variation in behaviour change content between providers' intervention documents. Thus, each provider planned to deliver 74% of the unique behaviour change techniques specified, and a large amount of behaviour change content not mandated.

CONCLUSIONS

There is good fidelity to the specified service parameters of the NHS-DPP; however, the four providers planned to deliver approximately three-quarters of behaviour change techniques specified by the NHS-DPP. Given that behaviour change techniques are the 'active ingredients' of interventions, and some of these techniques in the programme manuals may be missed in practice, this highlights possible limitations with fidelity to the NHS-DPP programme specification at the intervention design stage.

Evaluating process fidelity during the implementation of Group Antenatal Care in Mexico

BACKGROUND

CenteringPregnancy (CP) is a group antenatal care (G-ANC) model that has proven beneficial for mothers and their newborns. We conducted a feasibility study beginning in 2016 as part of the Mexican effort to implement G-ANC locally. This study reports on fidelity to the essential elements of CP during its implementation in Mexico.

METHODS

We collected prospective data using a standardized checklist at four primary-care centers that implemented our adapted G-ANC model. We performed a descriptive analysis of fidelity to 28 processes per G-ANC session (71 sessions made up of 10 groups and 129 women across 4 health centers). We calculated fidelity to each process as a proportion with 95% confidence intervals. We present overall results and stratified by health center and by facilitation team.

RESULTS

Overall fidelity to the G-ANC intervention was 82%, with variability by health center (78-88%). The elements with the highest fidelity were having space for activities such as checking vital signs, conversation in a circle, and medical check-ups (100% each) and the element with the lowest fidelity was using music to enhance privacy (27.3%). Fidelity was not significantly different by center.

CONCLUSIONS

Our study suggests good model fidelity during the implementation of G-ANC in Mexico. Our findings also contribute useful information about where to focus efforts in the future to maintain and improve G-ANC model fidelity.

Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study

BACKGROUND

It is important to evaluate fidelity of delivery and engagement during feasibility trials. However, there is little guidance on how to systematically develop strategies to improve implementation if problems arise. We aimed to: 1) Assess fidelity of delivery and engagement, 2) Identify factors influencing fidelity of delivery and engagement, and 3) Develop strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia, within a feasibility trial.

METHODS

A mixed methods evaluation of an intervention that aimed to improve independence in dementia. To assess fidelity of delivery and engagement, observation and self-report methods were used: 60% of audio-recorded intervention sessions were transcribed and reliably rated for fidelity. Providers (n = 12) and people with dementia/supporters (n = 34) were asked to complete checklists after each session. Descriptive statistics were used to analyse the data. To identify factors influencing fidelity and engagement, one-to-one semi-structured interviews were conducted with providers (n = 8), people with dementia (n = 7) and supporters (n = 7). Thematic analysis and content analysis were used to analyse data. To develop strategies, we followed four steps proposed by the authors of the Behaviour Change Wheel (1. Understanding the behaviour, 2. Identifying intervention functions, 3. Specifying intervention content, 4. Identifying mode of delivery).

RESULTS

Researcher ratings indicated moderate fidelity and provider/participant ratings indicated high fidelity of delivery. Knowledge, providers' attributes, ease of adaptation of the intervention in relation to participants' needs and logistical considerations influenced fidelity. We developed four strategies to improve fidelity of delivery of PRIDE: 1) showing a video, 2) giving an instruction sheet, 3) giving time to practice and 4) providing continued support. Participants reported high levels of engagement. Participants' attributes, capability and opportunity influenced engagement. We developed four strategies to improve engagement with PRIDE: 1) a session summary document, 2) clear instructions, 3) time to practice activity and 4) providing regular compulsory telephone support.

CONCLUSION

Fidelity of delivery and engagement are complex behaviours. This manuscript provides an example of how the Behaviour Change Wheel can be used during a feasibility trial to systematically develop strategies to improve implementation of complex interventions.

Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions

Background: Interventions to change behaviour have substantial potential to impact positively on individual and overall public health. Despite an increasing focus on health behaviour change intervention research, interventions do not always have the desired effect on outcomes, while others have diluted effects once implemented into real-life settings. There is little investment into understanding how or why such interventions work or do not work. Methodological inadequacies of trials of behavioural interventions have been previously suggested as a barrier to the quality and advancement of behavioural research, with intervention fidelity acknowledged as a key area for improvement. However, there is much ambiguity regarding the terminology and conceptualisation of intervention fidelity and a lack of practical guidance regarding how to address it sufficiently, particularly within trials of complex behavioural interventions. Objectives: This article outlines specific issues concerning intervention fidelity within trials of health behaviour change interventions and suggests practical considerations and specific recommendations for researchers, with examples from the literature presented. Conclusions: Recommendations pertain to (1) clarifying how fidelity is defined and conceptualised, (2) considering fidelity beyond intervention delivery, (3) considering strategies to both enhance and assess fidelity, (4) making use of existing frameworks and guidance, (5) considering the quality and comprehensiveness of fidelity assessment strategies, (6) considering the balance between fidelity and adaptation and (7) reporting the use of fidelity enhancement and assessment strategies and their results. Suggestions for future research to improve our understanding of, and ability to, address fidelity in behaviour change interventions are also provided.

Measuring fidelity to behavioural support delivery for smoking cessation and its association with outcomes

BACKGROUND AND AIMS

Behavioural support increases smoking cessation in clinical settings, but effect sizes differ among providers, due possibly to variations in delivery. This study evaluates a measure ('fidelity index') intended to capture fidelity to delivery of content- and interaction-based items of a behavioural support (BS) for smoking cessation and the association of fidelity with quit rates.

METHODS

A fidelity index for scoring the adherence and quality domains of a specific BS intervention, '5As for quit', was developed by classifying the intervention components using the taxonomy of behaviour change techniques. The index was applied to code 154 BS sessions audiotaped among 18 chest clinics in Pakistan to assess their fidelity and explore reliability of coding. The association between intervention fidelity and successful quit achieved by the same providers in a previous study was explored using regression analysis.

RESULTS

The index represented two domains: adherence to delivery of content-based activities of 5As (37 items) and quality of interaction-based activities (eight items). The intercoder reliability was good for content-based (average Krippendorff's α = 0.80) and moderate for interaction-based (average Krippendorff's α = 0.66) items. Approximately 70% (intraclass correlation coefficient: adherence scores = 0.72, quality scores = 0.71) of variation in BS delivery was contributed by providers, which increased to 97% (g-coefficient: adherence scores = 0.973, quality scores = 0.974) after accounting for other sources of variation. Higher quit rates were positively associated with average quality scores [risk ratio = 2.15; 95% confidence interval (CI) = 1.43-3.24], but negatively associated with average adherence scores (risk ratio = 0.55; 95% CI = 0.40-0.77) within services.

CONCLUSIONS

The fidelity index is a reliable measure for quantifying intervention fidelity of delivering smoking cessation behavioural support. Recommended revisions of the fidelity index include incorporation of additional interaction-based items, such as the relational techniques used in motivational interviewing.

Measuring fidelity of delivery of the Community Occupational Therapy in Dementia-UK intervention

Background

Interpreting data about intervention effectiveness requires an understanding of which intervention components were delivered and whether they were delivered as planned (fidelity of delivery). These studies aimed to develop a reliable measure for assessing fidelity of delivery of the Community Occupational Therapy in Dementia-UK intervention (COTiD-UK) (Study 1) and measure fidelity of delivery of COTiD-UK across sessions, sites and occupational therapists (Study 2).

Methods

The studies used a longitudinal observational design nested within a multi-site randomised controlled trial. Where practicable, all intervention sessions were audio-recorded. Fidelity checklists and coding guidelines were developed, piloted and refined until good agreement was achieved between two coders. Ten percent of sessions were purposively sampled from 12 sites and 31 occupational therapists. Transcripts were coded using checklists developed in Study 1; 10% of sets of intervention session transcripts were double coded to ensure that agreement was maintained. Percentages of components that were delivered were calculated for each session, site and occupational therapist.

Results

A reliable measure of fidelity of delivery for COTiD-UK was developed after several rounds of piloting and amendments. COTiD-UK was delivered with moderate fidelity across all six sessions (range: 52.4–75.5%). The mean range of fidelity varied across sites (26.7–91.2%) and occupational therapists (26.7–94.1%).

Conclusions

A reliable, systematic method for measuring fidelity of delivery of COTiD-UK was developed and applied, and can be adapted for use in similar interventions. As COTiD-UK was delivered with moderate fidelity, there is a reasonable degree of confidence that intervention effects were attributable to COTiD-UK.

Action, actor, context, target, time (AACTT): a framework for specifying behaviour

BACKGROUND

Designing implementation interventions to change the behaviour of healthcare providers and other professionals in the health system requires detailed specification of the behaviour(s) targeted for change to ensure alignment between intervention components and measured outcomes. Detailed behaviour specification can help to clarify evidence-practice gaps, clarify who needs to do what differently, identify modifiable barriers and enablers, design interventions to address these and ultimately provides an indicator of what to measure to evaluate an intervention's effect on behaviour change. An existing behaviour specification framework proposes four domains (Target, Action, Context, Time; TACT), but insufficiently clarifies who is performing the behaviour (i.e. the Actor). Specifying the Actor is especially important in healthcare settings characterised by multiple behaviours performed by multiple different people. We propose and describe an extension and re-ordering of TACT to enhance its utility to implementation intervention designers, practitioners and trialists: the Action, Actor, Context, Target, Time (AACTT) framework. We aim to demonstrate its application across key steps of implementation research and to provide tools for its use in practice to clarify the behaviours of stakeholders across multiple levels of the healthcare system.

METHODS AND RESULTS

We used French et al.'s four-step implementation process model to describe the potential applications of the AACTT framework for (a) clarifying who needs to do what differently, (b) identifying barriers and enablers, (c) selecting fit-for-purpose intervention strategies and components and (d) evaluating implementation interventions.

CONCLUSIONS

Describing and detailing behaviour using the AACTT framework may help to enhance measurement of theoretical constructs, inform development of topic guides and questionnaires, enhance the design of implementation interventions and clarify outcome measurement for evaluating implementation interventions.

Translating a walking intervention for health professional delivery within primary care: A mixed-methods treatment fidelity assessment

Objectives

Existing fidelity studies of physical activity interventions are limited in methodological quality and rigour, particularly those delivered by health care providers in clinical settings. The present study aimed to enhance and assess the fidelity of a walking intervention delivered by health care providers within general practice in line with the NIH Behavior Change Consortium treatment fidelity framework.

Design

Two practice nurses and six health care assistants delivered a theory‐based walking intervention to 63 patients in their own practices. A cross‐sectional mixed‐methods study assessed fidelity related to treatment delivery and treatment receipt, from the perspectives of health care providers and patients.

Methods

All providers received training and demonstrated delivery competence prior to the trial. Delivery of intervention content was coded from audio‐recordings using a standardized checklist. Qualitative interviews with 12 patients were conducted to assess patient perspectives of treatment receipt and analysed using framework analysis.

Results

Overall, 78% of intervention components were delivered as per the protocol (range 36–91%), with greater fidelity for components requiring active engagement from patients (e.g., completion of worksheets). The qualitative data highlighted differences in patients’ comprehension of specific intervention components. Understanding of, and engagement with, motivational components aimed at improving self‐efficacy was poorer than for volitional planning components.

Conclusions

High levels of fidelity of delivery were demonstrated. However, patient‐, provider‐, and component‐level factors impacted on treatment delivery and receipt. We recommend that methods for the enhancement and assessment of treatment fidelity are consistently implemented to enhance the rigour of physical activity intervention research.

Developing quality fidelity and engagement measures for complex health interventions

Objectives

To understand whether interventions are effective, we need to know whether the interventions are delivered as planned (with fidelity) and engaged with. To measure fidelity and engagement effectively, high‐quality measures are needed. We outline a five‐step method which can be used to develop quality measures of fidelity and engagement for complex health interventions. We provide examples from a fidelity study conducted within an evaluation of an intervention aimed to increase independence in dementia.

Methods

We propose five steps that can be systematically used to develop fidelity checklists for researchers, providers, and participants to measure fidelity and engagement. These steps include the following: (1) reviewing previous measures, (2) analysing intervention components and developing a framework outlining the content of the intervention, (3) developing fidelity checklists and coding guidelines, (4) obtaining feedback about the content and wording of checklists and guidelines, and (5) piloting and refining checklists and coding guidelines to assess and improve reliability.

Results

Three fidelity checklists that can be used reliably were developed to measure fidelity of and engagement with, the Promoting Independence in Dementia (PRIDE) intervention. As these measures were designed to be used by researchers, providers, and participants, we developed two versions of the checklists: one for participants and one for researchers and providers.

Conclusions

The five steps that we propose can be used to develop psychometrically robust and implementable measures of fidelity and engagement for complex health interventions that can be used by different target audiences. By considering quality when developing measures, we can be more confident in the interpretation of intervention outcomes drawn from fidelity and engagement studies.

Statement of contribution

What is already known on the subject?

Fidelity and engagement can be measured using a range of methods, such as observation and self‐report.

Studies seldom report psychometric and implementation qualities of fidelity measures.

What does this study add?

A method for developing fidelity and engagement measures for complex health interventions.

Guidance on how to consider quality when developing fidelity and engagement measures.

Evaluating NHS Stop Smoking Service engagement in community pharmacies using simulated smokers: fidelity assessment of a theory-based intervention

OBJECTIVES

Smokers are more likely to quit if they use the National Health Service (NHS) Stop Smoking Service (SSS). However, community pharmacies experience low service uptake. The Smoking Treatment Optimisation in Pharmacies (STOP) programme aims to address this problem by enhancing staff training using a theory-based intervention. In this study, we evaluated intervention fidelity using simulated smokers (actors) to assess smoker engagement and enactment of key intervention components by STOP trained staff.

DESIGN

An observational pilot study.

SETTINGS

Five community pharmacies in North East London with an NHS SSS.

METHODS

Six actors, representative of East London's population, were recruited and trained to complete intervention fidelity assessments. Consenting pharmacy staff from five participating pharmacies received STOP Intervention training. Four weeks after the staff training, the actors visited the participating pharmacies posing as smokers eligible for smoking cessation support. Engagement behaviour by pharmacy staff and enactment of intervention components was assessed using a scoring tool derived from the STOP logic model (scoring range of 0-36), and contemporaneous field notes taken by actors.

RESULTS

18 of 30 completed assessments were with STOP trained staff (10/18 were counter assistants). Mean score for smoker engagement was 24.4 (SD 9.0) points for trained and 16.9 (SD 7.8) for untrained staff, respectively. NHS SSS leaflets (27/30) were the most common smoking cessation materials seen on pharmacy visits. Most trained counter staff engaged with smokers using leaflets and a few proactively offered appointments with their cessation advisors. Appropriate use of body language was reported on 26/30 occasions alongside the use of key phrases from the STOP training session (n=8). Very few pharmacy staff wore STOP promotional badges (4/30).

CONCLUSIONS

STOP training may change client engagement behaviour in pharmacy staff and could improve the uptake of the NHS SSS. A cluster randomised controlled trial is currently in progress to evaluate its effectiveness and cost-effectiveness.

TRIAL REGISTRATION NUMBER

ISRCTN16351033.

Effects of a brief pain and smoking cessation intervention in adults with chronic pain: A randomized controlled trial

Tobacco cigarette smokers with comorbid chronic pain experience greater difficulty quitting smoking relative to those without chronic pain. A brief smoking cessation intervention was developed to address smoking in the context of chronic pain to increase the intention to engage in smoking cessation treatment. The primary aim of this randomized controlled trial was to investigate the effects of a brief pain and smoking (BPS) cessation intervention on the willingness to consider quitting smoking in adults with chronic pain seeking treatment in a pain specialty outpatient clinic. Subjects randomized to the BPS intervention were 7.5 times more likely to endorse willingness to consider quitting smoking. Subjects who received the BPS intervention were also greater than 2.5 times more likely to report an interest in learning about cessation programs, and nearly 5 times more likely to endorse willingness to consider participating in an intensive smoking cessation program. Moreover, subjects who received the BPS intervention evinced a trend-level reduction in perceived difficulty of quitting smoking. These results contribute to a growing multidisciplinary literature examining pain-smoking interrelations and suggest that smokers with chronic pain may become more willing to consider engaging a cessation attempt as awareness increases about how continued smoking may interfere with the clinical outcomes of pain treatment. These results are also consistent with clinical practice guidelines for promoting intention to quit among smokers currently unwilling to engage a quit attempt by incorporating strategies aimed at identifying ambivalence about the continued use of tobacco.

Cognitive-behavioral treatment with behavioral activation for smoking cessation: Randomized controlled trial

Introduction

Behavioral Activation is a behavioral-based treatment that has been proposed as suitable for smoking cessation, as it simultaneously addresses reinforcement-related variables and also mood management. The aim of this study was to compare the effects of a cognitive-behavioral smoking cessation treatment with components of behavioral activation (SCBSCT-BA) with a standard cognitive-behavioral treatment (SCBSCT), and a wait-list control group (WL).

Method

The sample was comprised of 275 adults smokers (61.4% females, mean age = 45.36, SD = 10.96). After baseline assessment sessions, participants were randomized (ratio: 2.2.1.) to SCBSCT-BA, SCBSCT, or WL. Active groups received 8 weekly 1-hour face-to-face group sessions. Biochemically verified smoking abstinence and depressive symptoms were assessed at the end of treatment, and at 3-, 6-, and 12-month follow-ups.

Results

Significant treatment effects in 7-dayspoint prevalence abstinence rates were found for both active groups at the end of treatment. Abstinence rates at 12-months follow-up were 30% for SCBSCT-BA, and 18% for SCBSCT. Using Multiple Imputation for missing data, regression analysis showed significantly greater ORs for the SCBSCT-BA condition (vs. SCBSCT) at the end of treatment and at 3-months follow-up. At 6-, and 12-months follow-ups, ORs for the SCBSCT-BA condition, although greater, did not reach statistical significance. Multilevel analysis showed that abstinence was related to reductions in depressive symptoms.

Conclusions

SCBSCT-BA obtained positive results at short and medium term. Participants who quit smoking experienced a significant reduction in depressive symptoms. Findings support the benefit of adding BA to a cognitive-behavioral smoking cessation treatment.

Trial registration

www.clinicaltrials.govNCT02844595.

Primary care management of cardiovascular risk for people with severe mental illnesses: the Primrose research programme including cluster RCT

Background

Effective interventions are needed to prevent cardiovascular disease (CVD) in people with severe mental illnesses (SMI) because their risk of CVD is higher than that of the general population.

Objectives

(1) Develop and validate risk models for predicting CVD events in people with SMI and evaluate their cost-effectiveness, (2) develop an intervention to reduce levels of cholesterol and CVD risk in SMI and (3) test the clinical effectiveness and cost-effectiveness of this new intervention in primary care.

Design

Mixed methods with patient and public involvement throughout. The mixed methods were (1) a prospective cohort and risk score validation study and cost-effectiveness modelling, (2) development work (focus groups, updated systematic review of interventions, primary care database studies investigating statin prescribing and effectiveness) and (3) cluster randomised controlled trial (RCT) assessing the clinical effectiveness and cost-effectiveness of a new practitioner-led intervention, and fidelity assessment of audio-recorded appointments.

Setting

General practices across England.

Participants

All studies included adults with SMI (schizophrenia, bipolar disorder or other non-organic psychosis). The RCT included adults with SMI and two or more CVD risk factors.

Interventions

The intervention consisted of 8–12 appointments with a practice nurse/health-care assistant over 6 months, involving collaborative behavioural approaches to CVD risk factors. The intervention was compared with routine practice with a general practitioner (GP).

Main outcome measures

The primary outcome for the risk score work was CVD events, in the cost-effectiveness modelling it was quality-adjusted life-years (QALYs) and in the RCT it was level of total cholesterol.

Data sources

Databases studies used The Health Improvement Network (THIN). Intervention development work included focus groups and systematic reviews. The RCT collected patient self-reported and routine NHS GP data. Intervention appointments were audio-recorded.

Results

Two CVD risk score models were developed and validated in 38,824 people with SMI in THIN: the Primrose lipid model requiring cholesterol levels, and the Primrose body mass index (BMI) model with no blood test. These models performed better than published Cox Framingham models. In health economic modelling, the Primrose BMI model was most cost-effective when used as an algorithm to drive statin prescriptions. Focus groups identified barriers to, and facilitators of, reducing CVD risk in SMI including patient engagement and motivation, staff confidence, involving supportive others, goal-setting and continuity of care. Findings were synthesised with evidence from updated systematic reviews to create the Primrose intervention and training programme. THIN cohort studies in 16,854 people with SMI demonstrated that statins effectively reduced levels of cholesterol, with similar effect sizes to those in general population studies over 12–24 months (mean decrease 1.2 mmol/l). Cluster RCT: 76 GP practices were randomised to the Primrose intervention (n = 38) or treatment as usual (TAU) (n = 38). The primary outcome (level of cholesterol) was analysed for 137 out of 155 participants in Primrose and 152 out of 172 in TAU. There was no difference in levels of cholesterol at 12 months [5.4 mmol/l Primrose vs. 5.5 mmol/l TAU; coefficient 0.03; 95% confidence interval (CI) –0.22 to 0.29], nor in secondary outcomes related to cardiometabolic parameters, well-being or medication adherence. Mean cholesterol levels decreased over 12 months in both arms (–0.22 mmol/l Primrose vs. –0.39 mmol/l TAU). There was a significant reduction in the cost of inpatient mental health attendances (–£799, 95% CI –£1480 to –£117) and total health-care costs (–£895, 95% CI –£1631 to –£160; p = 0.012) in the intervention group, but no significant difference in QALYs (–0.011, 95% CI –0.034 to 0.011). A total of 69% of patients attended two or more Primrose appointments. Audiotapes revealed moderate fidelity to intervention delivery (67.7%). Statin prescribing and adherence was rarely addressed.

Limitations

RCT participants and practices may not represent all UK practices. CVD care in the TAU arm may have been enhanced by trial procedures involving CVD risk screening and feedback.

Conclusions

SMI-specific CVD risk scores better predict new CVD if used to guide statin prescribing in SMI. Statins are effective in reducing levels of cholesterol in people with SMI in UK clinical practice. This primary care RCT evaluated an evidence-based practitioner-led intervention that was well attended by patients and intervention components were delivered. No superiority was shown for the new intervention over TAU for level of cholesterol, but cholesterol levels decreased over 12 months in both arms and the intervention showed fewer inpatient admissions. There was no difference in cholesterol levels between the intervention and TAU arms, which might reflect better than standard general practice care in TAU, heterogeneity in intervention delivery or suboptimal emphasis on statins.

Future work

The new risk score should be updated, deployed and tested in different settings and compared with the latest versions of CVD risk scores in different countries. Future research on CVD risk interventions should emphasise statin prescriptions more. The mechanism behind lower costs with the Primrose intervention needs exploring, including SMI-related training and offering frequent support to people with SMI in primary care.

Trial registration

Current Controlled Trials ISRCTN13762819.

Funding

This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full in Programme Grants for Applied Research; Vol. 7, No. 2. See the NIHR Journals Library website for further project information. Professor David Osborn is supported by the University College London Hospital NIHR Biomedical Research Centre and he was also in part supported by the NIHR Collaboration for Leadership in Applied Health Research and Care (CLAHRC) North Thames at Barts Health NHS Trust.

Use of dynamic systems methods to characterize dyadic interactions in smoking cessation behavioural support sessions: A feasibility study

BACKGROUND

Understanding how behaviour change techniques (BCTs) operate in practice requires a method for characterizing the reciprocal, dynamic, and real-time nature of behavioural support interactions between practitioners and clients. State space grids (SSGs) are an observational, dynamic systems methodology used to map the trajectory of dyadic interactions in real time. By mapping the flow of events in terms of practitioner and client actions, SSGs are potentially well suited to characterize behavioural support sessions.

PURPOSE

To develop reliable methods and examine the feasibility of using the SSG methodology for characterizing practitioners' delivery of and clients' response to BCTs in smoking cessation behavioural support sessions.

METHODS

Smoking cessation behavioural support sessions were video-recorded and transcribed verbatim (n = 6 recordings; 2,916 statements). All speech was coded independently by two researchers for content and duration using published frameworks for specifying practitioner-delivered and client-received BCTs in smoking cessation interactions. Inter-rater reliability was assessed. Indices of practitioner-client interaction dynamics were derived: (1) reciprocity (i.e., attractor states, content congruence, conditional pairing) and (2) temporal patterning (i.e., variability, inter-grid distance, combinatory micro-patterning, sessional macropatterning). The extent to which indices can describe differences between sessions involving different practitioners and clients was examined.

RESULTS

Inter-rater reliability was moderate at 72% agreement. Indices of reciprocity and temporal patterning characterized differences between sessions involving different practitioners and clients.

CONCLUSIONS

State space grids provide a method for characterizing the complexity and variability of practitioner-delivered and client-received BCTs in behavioural support sessions. This method has potential to add explanatory value to smoking cessation intervention outcomes. Statement of Contribution What is already known on this subject? Frameworks exist for characterizing practitioner-delivered and client-received behaviour change techniques (BCTs). Methods are still needed to investigate which BCTs are effective under what conditions. State space grids (SSGs) are a dynamic systems method that may better characterize behavioural support interactions. What does this study add? First reliable, dynamic systems, SSG coding procedures, methods, and measures to characterize behavioural support. A method for examining reciprocality and temporal patterning of BCT delivery and receipt. Establishes a dynamic systems method that adds explanatory value to the outcomes of interventions.

Contingency Management Versus Psychotherapy for Prenatal Smoking Cessation: A Meta-Analysis of Randomized Controlled Trials

OBJECTIVES

Prenatal smoking is the leading preventable cause of poor obstetric outcomes, yet treatment options are limited. Past reviews of prenatal smoking cessation have often grouped all counseling into a single category, which ignores the fact that psychotherapy is distinct from brief counseling. The objective of this study was to compare the effect sizes of two intensive interventions for prenatal smoking cessation: contingency management (i.e., financial incentives for abstinence) and psychotherapy.

METHODS

A systematic search for randomized controlled trials testing the efficacy of contingency management or psychotherapy was completed using PubMed, PsycINFO, Web of Science, the Cochrane Library, and EMBASE. Independent raters extracted data and assessed trials for risk of bias. Treatment effects were analyzed for three times points: late pregnancy, early postpartum, and late postpartum.

RESULTS

The search yielded 22 studies, and meta-analytic results indicated that interventions (compared with control groups) generally increased the odds of abstinence. Moderator analyses indicated that intervention type (contingency management vs. psychotherapy) accounted for variability in effect sizes. When comparing treatment type, effects of contingency management interventions were significantly greater than those of psychotherapeutic interventions. Although psychotherapy did not affect smoking abstinence, contingency management interventions had significant treatment effects at all three time points.

CONCLUSIONS

Contingency management seems to be a safe and efficacious prenatal smoking cessation treatment. Although psychotherapy alone did not show an effect on prenatal smoking abstinence, future research may seek to combine this approach with contingency management to promote prenatal smoking cessation.

Understanding Outcomes in Behavior Change Interventions to Prevent Pediatric Obesity: The Role of Dose and Behavior Change Techniques

BACKGROUND

Behavioral interventions to prevent pediatric obesity have shown inconsistent results across the field. Studying what happens within the "black box" of these interventions and how differences in implementation lead to different outcomes will help researchers develop more effective interventions.

AIM

To compare the implementation of three features of a phone-based intervention for parents (time spent discussing weight-related behaviors, behavior change techniques used in sessions, and intervention activities implemented by parents between sessions) with study outcomes.

METHODS

A random selection of 100 parent-child dyads in the intervention arm of a phone-based obesity prevention trial was included in this analysis. Sessions were coded for overall session length, length of time spent discussing specific weight-related behaviors, number of behavior change techniques used during the sessions, and number of intervention-recommended activities implemented by the parents between sessions (e.g., parent-reported implementation of behavioral practice/rehearsal between sessions). The primary study outcome, prevention of unhealthy increase in child body mass index (BMI) percentile, was measured at baseline and 12 months.

RESULTS

Overall session length was associated with decreases in child BMI percentile ( b = -0.02, p = .01). There was no association between the number of behavior change techniques used in the sessions and decreases in child BMI percentile ( b = -0.29, p = .27). The number of activities the parents reported implementing between sessions was associated with decreases in child BMI percentile ( b = -1.25, p = .02).

DISCUSSION

To improve future interventions, greater attention should be paid to the intended and delivered session length, and efforts should be made to facilitate parents' implementation of intervention-recommended activities between sessions (ClinicalTrials.gov, No. NCT01084590).

Thoroughness and Psychometrics of Fidelity Measures in Occupational and Physical Therapy: A Systematic Review

OBJECTIVE

This study evaluated the thoroughness and psychometric properties of fidelity measures used by or of relevance to occupational or physical therapy.

METHOD

A systematic review of the literature was completed. Assessments used to measure occupational or physical therapy intervention fidelity were evaluated for thoroughness, reliability, validity, and clinical utility.

RESULTS

Eight fidelity measures met inclusion criteria for this systematic review. Most of the measures had moderate levels of thoroughness in the coverage of key aspects of fidelity, reported adequate to excellent reliability and validity, and were highly variable in clinical utility.

CONCLUSION

Additional research is recommended to validate existing occupational or physical therapy fidelity measures and to develop novel measures for other occupational therapy and physical therapy interventions. Clinicians and researchers must place greater emphasis on the development and implementation of fidelity measures to ensure uniformity in intervention delivery and high-quality, evidence-based care.

An assessment of physiotherapist's delivery of behaviour change techniques within the SOLAS feasibility trial

OBJECTIVES

To investigate physiotherapist's (PTs) fidelity to 31 protocol-listed behaviour change techniques (BCTs) during a group-based self-management intervention. This study also explored the PTs delivery of these BCTs beyond the present or absent dichotomy, using a third variable, partial delivery (i.e., attempted).

DESIGN

Assessment of the intervention arm of the Self-management of Osteoarthritis and Low back pain through Activity and Skills (SOLAS) cluster, randomized controlled feasibility trial, using quantitative methods.

METHODS

Eight PTs delivered six SOLAS classes each, of which 50% were audio-recorded and transcribed. Transcripts were coded by two raters using the Behaviour Change Technique Taxonomy v1 and an intervention-specific manual and assessed for the delivery (i.e., full, partial, or absent) of the 31 BCTs and their target behaviours. Fidelity was calculated as fully delivered BCTs listed as a percentage of those due to take place within each class.

RESULTS

Physiotherapists delivered a mean 20.5 BCTs per class (68.3%; range = 64.9-72.4%). Of these, 17 BCTs were fully delivered in each class representing moderate fidelity to the protocol (56.8%; range = 53.5-59.3%). A further 3.5 BCTs per class (11.5%; range = 8.7-14.8%) were partially delivered. BCTs associated with 'goals and planning' were often poorly delivered.

CONCLUSIONS

Delivering the SOLAS intervention BCTs with high fidelity was not feasible. The assessment of partial delivery of BCTs provided greater insight into the techniques that should be removed from the protocol or that may require further training. Complex interventions should consider a list of 'core' or mandatory BCTs alongside 'optional' BCTs, depending on the target behaviour, and the needs of individual participants. Statement of contribution What is already known on this subject? BCTs are the smallest active components of behavioural interventions, yet typically their effectiveness is determined through meta-analyses. Attempted delivery of BCTs is often unaccounted for yet may provide valuable insight into difficulty with delivery. There is a need to investigate BCT implementation beyond simple presence/absence to identify protocol refinements or required BCT training. What does this study add? BCT delivery was assessed in greater depth than previous research, including partial delivery. Highlights the need for appropriate training in BCTs that are difficult to deliver, particularly those associated with 'goals and planning' Highlights the need for intervention-specific criteria as to what constitutes 'high', 'moderate', and 'low' fidelity.

Randomised Controlled Clinical Trial Investigating The Impact of Implementation Planning on Behaviour Related to The Diet

There is a perceived gap between dietary advice given by health practitioners and adherence to the advice by patients. We investigated whether a behaviour change technique (implementation-planning) was more effective than standard-of-care diet advice at reducing dietary acid intake using quantitative erosive tooth wear progression as an objective clinical outcome. This study was a randomised controlled, double-blind, single-centre clinical trial in the UK. Participants (n = 60) with high dietary acid intake (≥2 daily), were recruited and randomly assigned (1:1) to receive either implementation-planning or standard-of-care diet advice in a single clinical session. Questionnaires and impressions were taken at baseline and 6 months later. Dental casts were scanned using laser profilometry and superimposed using surface-matching software. Data were analysed per protocol and intention-to-treat using independent t-tests and Mann-Whitney tests. The intervention group reduced their dietary acid intake between meals to 1 intake per day compared to 2 intakes per day for the controls and demonstrated reduced dental hard tissue volume loss (-0.00 mm³ (SD = 0.01)) compared to controls (-0.07 mm³ (SD 0.17)), p = 0.049. This paper supports the use of implementation planning in clinical practice and presents a non-invasive method of intervention assessment in behaviour change. Larger trials are needed to confirm the generalisability of results.

The Effectiveness of a Computer-Tailored E-Learning Program for Practice Nurses to Improve Their Adherence to Smoking Cessation Counseling Guidelines: Randomized Controlled Trial

Background

Improving practice nurses’ (PN) adherence to smoking cessation counseling guidelines will benefit the quality of smoking cessation care and will potentially lead to higher smoking abstinence rates. However, support programs to aid PNs in improving their guideline uptake and adherence do not exist yet.

Objective

The aim of this study was to assess the effects of a novel computer-tailored electronic learning (e-learning) program on PNs’ smoking cessation guideline adherence.

Methods

A Web-based randomized controlled trial (RCT) was conducted in which an intervention group (N=147) with full access to the e-learning program for 6 months was compared with a control group (N=122) without access. Data collection was fully automated at baseline and 6-month follow-up via online questionnaires, assessing PNs’ demographics, work-related factors, potential behavioral predictors based on the I-Change model, and guideline adherence. PNs also completed counseling checklists to retrieve self-reported counseling activities for each consultation with a smoker (N=1175). To assess the program’s effectiveness in improving PNs’ guideline adherence (ie, overall adherence and adherence to individual counseling guideline steps), mixed linear and logistic regression analyses were conducted, thus accommodating for the smokers being nested within PNs. Potential effect moderation by work-related factors and behavioral predictors was also examined.

Results

After 6 months, 121 PNs in the intervention group (82.3%, 121/147) and 103 in the control group (84.4%, 103/122) completed the follow-up questionnaire. Mixed linear regression analysis revealed that counseling experience moderated the program’s effect on PNs’ overall guideline adherence (beta=.589; 95% CI 0.111-1.068; P_{Holm-Bonferroni} =.048), indicating a positive program effect on adherence for PNs with a more than average level of counseling experience. Mixed logistic regression analyses regarding adherence to individual guideline steps revealed a trend toward moderating effects of baseline levels of behavioral predictors and counseling experience. More specifically, for PNs with less favorable scores on behavioral predictors (eg, low baseline self-efficacy) and high levels of counseling experience, the program significantly increased adherence.

Conclusions

Results from our RCT showed that among PNs with more than average counseling experience, the e-learning program resulted in significantly better smoking cessation guideline adherence. Experienced PNs might have been better able to translate the content of our e-learning program into practically applicable counseling strategies compared with less experienced colleagues. Less favorable baseline levels of behavioral predictors among PNs possibly contributed to this effect, as there was more room for improvement by consulting the tailored content of the e-learning program. To further substantiate the effectiveness of e-learning programs on guideline adherence by health care professionals (HCPs), it is important to assess how to support a wider range of HCPs.

Trial Registration

Netherlands Trial Register NTR4436; http://www.trialregister.nl/trialreg/admin/rctview.asp?TC=4436 (Archived by WebCite at http://www.webcitation.org/6zJQuSRq0)

Protocol for the mixed-methods process and context evaluation of the TB & Tobacco randomised controlled trial in Bangladesh and Pakistan: a hybrid effectiveness-implementation study

INTRODUCTION

Tuberculosis (TB) remains a significant public health problem in South Asia. Tobacco use increases the risks of TB infection and TB progression. The TB& Tobacco placebo-controlled randomised trial aims to (1) assess the effectiveness of the tobacco cessation medication cytisine versus placebo when combined with behavioural support and (2) implement tobacco cessation medication and behavioural support as part of general TB care in Bangladesh and Pakistan. This paper summarises the process and context evaluation protocol embedded in the effectiveness-implementation hybrid design.

METHODS AND ANALYSIS

We are conducting a mixed-methods process and context evaluation informed by an intervention logic model that draws on the UK Medical Research Council's Process Evaluation Guidance. Our approach includes quantitative and qualitative data collection on context, recruitment, reach, dose delivered, dose received and fidelity. Quantitative data include patient characteristics, reach of recruitment among eligible patients, routine trial data on dose delivered and dose received, and a COM-B ('capability', 'opportunity', 'motivation' and 'behaviour') questionnaire filled in by participating health workers. Qualitative data include semistructured interviews with TB health workers and patients, and with policy-makers at district and central levels in each country. Interviews will be analysed using the framework approach. The behavioural intervention delivery is audio recorded and assessed using a predefined fidelity coding index based on behavioural change technique taxonomy.

ETHICS AND DISSEMINATION

The study complies with the guidelines of the Declaration of Helsinki. Ethics approval for the study and process evaluation was granted by the University of Leeds (qualitative components), University of York (trial data and fidelity assessment), Bangladesh Medical Research Council and Bangladesh Drug Administration (trial data and qualitative components) and Pakistan Medical Research Council (trial data and qualitative components). Results of this research will be disseminated through reports to stakeholders and peer-reviewed publications and conference presentations.

TRIAL REGISTRATION NUMBER

ISRCTN43811467; Pre-results.

Evaluating the effectiveness of behavior change techniques in health-related behavior: a scoping review of methods used

Behavior change interventions typically contain multiple potentially active components: behavior change techniques (BCTs). Identifying which specific BCTs or BCT combinations have the potential to be effective for a given behavior in a given context presents a major challenge. The aim of this study was to review the methods that have been used to identify effective BCTs for given behaviors in given contexts and evaluate their strengths and limitations. A scoping review was conducted of studies that had sought to identify effective BCTs. Articles referring to "behavio(u)r change technique(s)" in the abstract/text were located, and ones that involved identification of effective BCTs were selected. The methods reported were coded. The methods were analyzed in general terms using "PASS" criteria: Practicability (facility to apply the method appropriately), Applicability (facility to generalize from findings to contexts and populations of interest), Sensitivity (facility to identify effective BCTs), and Specificity (facility to rule out ineffective BCTs). A sample of 10% of the studies reviewed was then evaluated using these criteria to assess how far the strengths and limitations identified in principle were borne out in practice. One hundred and thirty-five studies were identified. The methods used in those studies were experimental manipulation of BCTs, observational studies comparing outcomes in the presence or absence of BCTs, meta-analyses of BCT comparisons, meta-regressions evaluating effect sizes with and without specific BCTs, reviews of BCTs found in effective interventions, and meta-classification and regression trees. The limitations of each method meant that only weak conclusions could be drawn regarding the effectiveness of specific BCTs or BCT combinations. Methods for identifying effective BCTs linked to target behavior and context all have important inherent limitations. A strategy needs to be developed that can systematically combine the strengths of the different methods and that can link these constructs in an ontology of behavior change interventions.

Characterising smoking cessation smartphone applications in terms of behaviour change techniques, engagement and ease-of-use features

The aim of this study was to assess whether or not behaviour change techniques (BCTs) as well as engagement and ease-of-use features used in smartphone applications (apps) to aid smoking cessation can be identified reliably. Apps were coded for presence of potentially effective BCTs, and engagement and ease-of-use features. Inter-rater reliability for this coding was assessed. Inter-rater agreement for identifying presence of potentially effective BCTs ranged from 66.8 to 95.1 % with ‘prevalence and bias adjusted kappas’ (PABAK) ranging from 0.35 to 0.90 (p < 0.001). The intra-class correlation coefficients between the two coders for scores denoting the proportions of (a) a set of engagement features and (b) a set of ease-of-use features, which were included, were 0.77 and 0.75, respectively (p < 0.001). Prevalence estimates for BCTs ranged from <10 % for medication advice to >50 % for rewarding abstinence. The average proportions of specified engagement and ease-of-use features included in the apps were 69 and 83 %, respectively. The study found that it is possible to identify potentially effective BCTs, and engagement and ease-of-use features in smoking cessation apps with fair to high inter-rater reliability.