The reporting of fidelity measures in primary prevention programmes for eating disorders in schools

Classifying Changes to Preventive Interventions: Applying Adaptation Taxonomies

High-quality implementation is important for preventive intervention effectiveness. Although this implies fidelity to a practice model, some adaptation may be inevitable or even advantageous in routine practice settings. In order to organize the study of adaptation and its effect on intervention outcomes, scholars have proposed various adaptation taxonomies. This paper examines how four published taxonomies retrospectively classify adaptations: the Ecological Validity Framework (EVF; Bernal et al. in J Abnorm Child Psychol 23(1):67-82, 1995), the Hybrid Prevention Program Model (HPPM; Castro et al. in Prev Sci 5(1):41-45, 2004. https://doi.org/10.1023/B:PREV.0000013980.12412.cd ), the Moore et al. (J Prim Prev 34(3):147-161, 2013. https://doi.org/10.1007/s10935-013-0303-6 ) taxonomy, and the Stirman et al. (Implement Sci 8:65, 2013. https://doi.org/10.1186/1748-5908-8-65 ) taxonomy. We used these taxonomies to classify teacher-reported adaptations made during the implementation of TOOLBOX™, a social emotional learning program implemented in 11 elementary schools during the 2014-2015 academic year. Post-implementation, 271 teachers and staff responded to an online survey that included questions about adaptation, yielding 98 adaptation descriptions provided by 42 respondents. Four raters used each taxonomy to try to classify these descriptions. We assessed the extent to which raters agreed they could classify the descriptions using each taxonomy (coverage), as well as the extent to which raters agreed on the subcategory they assigned (clarity). Results indicated variance among taxonomies, and tensions between the ideals of coverage and clarity emerged. Further studies of adaptation taxonomies as coding instruments may improve their performance, helping scholars more consistently assess adaptations and their effects on preventive intervention outcomes.

Intervention Fidelity Within Trials of Infant Feeding Behavioral Interventions to Prevent Childhood Obesity: A Systematic Review

Background

Intervention fidelity refers to whether an intervention has been implemented as intended. Trials of infant feeding behavioral interventions to prevent childhood obesity show inconsistent evidence of effectiveness. However, intervention fidelity has not been previously explored within these trials, limiting interpretation of findings.

Purpose

To review the use and/or reporting of strategies to enhance and assess intervention fidelity within trials of infant feeding interventions to prevent childhood obesity, and their association with study quality, effectiveness, and publication year.

Methods

Seven electronic databases were searched, with articles screened for inclusion by two reviewers. The National Institutes of Health Behaviour Change Consortium fidelity checklist was used to assess use and/or reporting of fidelity strategies across five domains (design, provider training, delivery, receipt, and enactment).

Results

Ten trials (16 papers) were identified. Average use/reporting of fidelity strategies was moderate (54%), ranging from 28.9% to 76.7%. Levels of use/reporting ranged from 15.9% in the domain of provider training to 95% for enactment. No association was found between these levels and study quality, effectiveness, or publication year.

Conclusions

The moderate use/reporting of fidelity strategies within trials of infant feeding interventions suggests that previous findings of inconsistent effectiveness may not fully reflect the intended interventions. The review highlights key considerations for improving future research, both in the area of behavioral infant feeding and wider behavior change literature. This includes improving reporting across all fidelity domains and ensuring an enhanced focus on provider training and control group content to optimize the translation of research into practice.

PROSPERO Registration number

CRD42016033492.

Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions

Background: Interventions to change behaviour have substantial potential to impact positively on individual and overall public health. Despite an increasing focus on health behaviour change intervention research, interventions do not always have the desired effect on outcomes, while others have diluted effects once implemented into real-life settings. There is little investment into understanding how or why such interventions work or do not work. Methodological inadequacies of trials of behavioural interventions have been previously suggested as a barrier to the quality and advancement of behavioural research, with intervention fidelity acknowledged as a key area for improvement. However, there is much ambiguity regarding the terminology and conceptualisation of intervention fidelity and a lack of practical guidance regarding how to address it sufficiently, particularly within trials of complex behavioural interventions. Objectives: This article outlines specific issues concerning intervention fidelity within trials of health behaviour change interventions and suggests practical considerations and specific recommendations for researchers, with examples from the literature presented. Conclusions: Recommendations pertain to (1) clarifying how fidelity is defined and conceptualised, (2) considering fidelity beyond intervention delivery, (3) considering strategies to both enhance and assess fidelity, (4) making use of existing frameworks and guidance, (5) considering the quality and comprehensiveness of fidelity assessment strategies, (6) considering the balance between fidelity and adaptation and (7) reporting the use of fidelity enhancement and assessment strategies and their results. Suggestions for future research to improve our understanding of, and ability to, address fidelity in behaviour change interventions are also provided.

A multidimensional approach to assessing intervention fidelity in a process evaluation of audit and feedback interventions to reduce unnecessary blood transfusions: a study protocol

Background

In England, NHS Blood and Transplant conducts national audits of transfusion and provides feedback to hospitals to promote evidence-based practice. Audits demonstrate 20% of transfusions fall outside guidelines. The AFFINITIE programme (Development & Evaluation of Audit and Feedback INterventions to Increase evidence-based Transfusion practIcE) involves two linked, 2×2 factorial, cluster-randomised trials, each evaluating two theoretically-enhanced audit and feedback interventions to reduce unnecessary blood transfusions in UK hospitals. The first intervention concerns the content/format of feedback reports. The second aims to support hospital transfusion staff to plan their response to feedback and includes a web-based toolkit and telephone support. Interpretation of trials is enhanced by comprehensively assessing intervention fidelity. However, reviews demonstrate fidelity evaluations are often limited, typically only assessing whether interventions were delivered as intended. This protocol presents methods for assessing fidelity across five dimensions proposed by the Behaviour Change Consortium fidelity framework, including intervention designer-, provider- and recipient-levels.

Methods

(1) Design: Intervention content will be specified in intervention manuals in terms of component behaviour change techniques (BCTs). Treatment differentiation will be examined by comparing BCTs across intervention/standard practice, noting the proportion of unique/convergent BCTs. (2) Training: draft feedback reports and audio-recorded role-play telephone support scenarios will be content analysed to assess intervention providers’ competence to deliver manual-specified BCTs. (3) Delivery: intervention materials (feedback reports, toolkit) and audio-recorded telephone support session transcripts will be content analysed to assess actual delivery of manual-specified BCTs during the intervention period. (4) Receipt and (5) enactment: questionnaires, semi-structured interviews based on the Theoretical Domains Framework, and objective web-analytics data (report downloads, toolkit usage patterns) will be analysed to assess hospital transfusion staff exposure to, understanding and enactment of the interventions, and to identify contextual barriers/enablers to implementation. Associations between observed fidelity and trial outcomes (% unnecessary transfusions) will be examined using mediation analyses.

Discussion

If the interventions have acceptable fidelity, then results of the AFFINITIE trials can be attributed to effectiveness, or lack of effectiveness, of the interventions. Hence, this comprehensive assessment of fidelity will be used to interpret trial findings. These methods may inform fidelity assessments in future trials.

Trial registration

ISRCTN 15490813. Registered 11/03/2015

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0528-x) contains supplementary material, which is available to authorized users.

Reporting of treatment fidelity in behavioural paediatric obesity intervention trials: a systematic review

Behavioural interventions for paediatric obesity are promising, but detailed information on treatment fidelity (i.e. design, training, delivery, receipt and enactment) is needed to optimize the implementation of more effective interventions. Little is known about current practices for reporting treatment fidelity in paediatric obesity studies. This systematic review, in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, describes the methods used to report treatment fidelity in randomized controlled trials. Treatment fidelity was double-coded using the National Institutes of Health Fidelity Framework checklist. Three hundred articles (N = 193 studies) were included. Mean inter-coder reliability across items was 0.83 (SD = 0.09). Reporting of treatment design elements within the field was high (e.g. 77% of studies reported designed length of treatment session), but reporting of other domains was low (e.g. only 7% of studies reported length of treatment sessions delivered). Few reported gold standard methods to evaluate treatment fidelity (e.g. coding treatment content delivered). General study quality was associated with reporting of treatment fidelity (p < 0.01) as was the number of articles published for a given study (p < 0.01). The frequency of reporting treatment fidelity components has not improved over time (p = 0.26). Specific recommendations are made to support paediatric obesity researchers in leading health behaviour disciplines towards more rigorous measurement and reporting of treatment fidelity.

Consideration of treatment fidelity to improve manual therapy research

OBJECTIVES

The purpose of this paper was to define treatment fidelity, review its use in health care research and suggest how it may be utilized in manual therapy research to improve the reliability and validity of the literature.

RESULTS

We offer an outline and a table of how manual therapy research may benefit from the concept of treatment fidelity.

DISCUSSION

While treatment fidelity is a newer concept, and has not been integrated into Physical Therapy or Manual Therapy research, when utilized, it can have positive effects on the reliability and validity of the techniques we evaluate.

Treatment integrity in school-wide programs: a review of the literature (1993-2012)

We assessed the reporting of treatment integrity in school-wide prevention programs in K-12 schools. This review was designed to determine (a) the extent to which treatment integrity was reported in school-wide prevention and intervention programs and how the reporting varied by research design, year, and journal; and (b) the procedures (e.g., method, frequency, informant) used to collect treatment integrity data. Results indicated that fewer than half of the studies in the review (n = 36, 45.6 %) measured and reported treatment integrity. Those studies reporting treatment integrity often used multiple methods and informants. Reporting treatment integrity in this body of literature has increased steadily over time.

Effectiveness of a brief school-based body image intervention 'Dove Confident Me: Single Session' when delivered by teachers and researchers: Results from a cluster randomised controlled trial

This study evaluated a 90-min single session school-based body image intervention (Dove Confident Me: Single Session), and investigated if delivery could be task-shifted to teachers. British adolescents (N = 1707; 11-13 years; 50.83% girls) participated in a cluster randomised controlled trial [lessons as usual control; intervention teacher-led (TL); intervention researcher-led (RL)]. Body image, risk factors, and psychosocial and disordered eating outcomes were assessed 1-week pre-intervention, immediate post-intervention, and 4-9.5 weeks follow-up. Multilevel mixed-models showed post-intervention improvements for intervention students relative to control in body esteem (TL; girls only), negative affect (TL), dietary restraint (TL; girls only), eating disorder symptoms (TL), and life engagement (TL; RL). Awareness of sociocultural pressures increased at post-intervention (TL). Effects were small-medium in size (ds 0.19-0.76) and were not maintained at follow-up. There were no significant differences between conditions at post or follow-up on body satisfaction, appearance comparisons, teasing, appearance conversations and self-esteem. The intervention had short-term benefits for girls' body image and dietary restraint, and for eating disorder symptoms and some psychosocial outcomes among girls and boys. A multi-session version of the intervention is likely to be necessary for sustained improvements. Teachers can deliver this intervention effectively with minimal training, indicating broader scale dissemination is feasible.

TRIAL REGISTRATION

ISRCTN16782819.

Effectiveness of pre-travel consultation in the prevention of travel-related diseases: a retrospective cohort study

BACKGROUND

This study aims to evaluate the effectiveness of pre-travel counselling carried out in Travel Clinics.

METHODS

This is a retrospective cohort. Three hundred international travellers were enrolled; 150 people were from users of Bari Travel Clinic, 150 were users of a travel agency. Enrolled subjects were interviewed using a questionnaire.

RESULTS

The average age of the enrolled subjects was 37.5 ± 13.9, without statistically significant differences between the two groups. 86% of cases and 19.3% of the controls reported the use of anti-malaria prophylaxis (p < 0.0001). Vaccination against cholera was given to 62% of cases and 7.3% of the controls (p < 0.001). Travel Clinic users, 6% reported diarrhoea and these figures were 27% in the control group (p < 0.0001). The proportion of those interviewed who reported fever (3.7) or insomnia (1.3) did not differ between the two groups. Mosquito bites were reported by 8% of cases and 20% of the controls (p = 0.003). Three cases of malaria were reported among the controls but no cases were detected among the cases (chi-square = 3.03; p = 0.08).

CONCLUSIONS

Our study demonstrated the effectiveness of pre-travel counselling; in the future, new studies must investigate the cost-effectiveness of pre-travel prevention measures.

What is the extent and quality of documentation and reporting of fidelity to implementation strategies: a scoping review

BACKGROUND

Implementation fidelity is critical to the internal and external validity of implementation research. Much of what is written about implementation fidelity addresses fidelity of evidence-informed interventions rather than fidelity of implementation strategies. The documentation and reporting of fidelity to implementation strategies requires attention. Therefore, in this scoping review, we identify the extent and quality of documentation and reporting of fidelity of implementation strategies that were used to implement evidence-informed interventions.

METHODS

A six-stage methodological framework for scoping studies guided our work. Studies were identified from the outputs of the Effective Practice and Organization of Care (EPOC) review group within the Cochrane Database of Systematic Reviews. EPOC's primary focus, implementation strategies influencing provider behavior change, optimized our ability to identify articles for inclusion. We organized the retrieved articles from the systematic reviews by journal and selected the three journals with the largest number of retrieved articles. Using a data extraction tool, we organized retrieved article data from these three journals. In addition, we summarized implementation strategies using the EPOC categories. Data extraction pertaining to the quality of reporting the fidelity of implementation strategies was facilitated with an "Implementation Strategy Fidelity Checklist" based on definitions adapted from Dusenbury et al. We conducted inter-rater reliability checks for all of the independently scored articles. Using linear regression, we assessed the fidelity scores in relation to the publication year.

RESULTS

Seventy-two implementation articles were included in the final analysis. Researchers reported neither fidelity definitions nor conceptual frameworks for fidelity in any articles. The most frequently employed implementation strategies included distribution of education materials (n = 35), audit and feedback (n = 32), and educational meetings (n = 25). Fidelity of implementation strategies was documented in 51 (71 %) articles. Inter-rater reliability coefficients of the independent reviews for each component of fidelity were as follows: adherence = 0.85, dose = 0.89, and participant responsiveness = 0.96. The mean fidelity score was 2.6 (SD = 2.25). We noted a statistically significant decline in fidelity scores over time.

CONCLUSIONS

In addition to identifying the under-reporting of fidelity of implementation strategies in the health literature, we developed and tested a simple checklist to assess the reporting of fidelity of implementation strategies. More research is indicated to assess the definitions and scoring schema of this checklist. Careful reporting of details about fidelity of implementation strategies will make an important contribution to implementation science.

Comparison of intervention fidelity between COPE TEEN and an attention-control program in a randomized controlled trial

Fidelity in implementing an intervention is critical to accurately determine and interpret the effects of an intervention. It is important to monitor the manner in which the behavioral intervention is implemented (e.g. adaptations, delivery as intended and dose). Few interventions are implemented with 100% fidelity. In this study, high school health teachers implemented the intervention. To attribute study findings to the intervention, it was vital to know to what degree the intervention was implemented. Therefore, the purposes of this study were to evaluate intervention fidelity and to compare implementation fidelity between the creating opportunities for personal empowerment (COPE) Healthy Lifestyles TEEN (thinking, emotions, exercise, and nutrition) program, the experimental intervention and Healthy Teens, an attention-control intervention, in a randomized controlled trial with 779 adolescents from 11 high schools in the southwest region of the United States. Thirty teachers participated in this study. Findings indicated that the attention-control teachers implemented their intervention with greater fidelity than COPE TEEN teachers. It is possible due to the novel intervention and the teachers' unfamiliarity with cognitive-behavioral skills building, COPE TEEN teachers had less fidelity. It is important to assess novel skill development prior to the commencement of experimental interventions and to provide corrective feedback during the course of implementation.

Differential methylation of the oxytocin receptor gene in patients with anorexia nervosa: a pilot study

Background and Aim

Recent studies in patients with anorexia nervosa suggest that oxytocin may be involved in the pathophysiology of anorexia nervosa. We examined whether there was evidence of variation in methylation status of the oxytocin receptor (OXTR) gene in patients with anorexia nervosa that might account for these findings.

Methods

We analyzed the methylation status of the CpG sites in a region from the exon 1 to the MT2 regions of the OXTR gene in buccal cells from 15 patients and 36 healthy women using bisulfite sequencing. We further examined whether methylation status was associated with markers of illness severity or form.

Results

We identified six CpG sites with significant differences in average methylation levels between the patient and control groups. Among the six differentially methylated CpG sites, five showed higher than average methylation levels in patients than those in the control group (64.9–88.8% vs. 6.6–45.0%). The methylation levels of these five CpG sites were negatively associated with body mass index (BMI). BMI, eating disorders psychopathology, and anxiety were identified in a regression analysis as factors affecting the methylation levels of these CpG sites with more variation accounted for by BMI.

Conclusions

Epigenetic misregulation of the OXTR gene may be implicated in anorexia nervosa, which may either be a mechanism linking environmental adversity to risk or may be a secondary consequence of the illness.