Evaluation of the fidelity of an interactive face-to-face educational intervention to improve general practitioner management of back pain

Insufficient structure and reporting of process evaluations of complex interventions for musculoskeletal conditions in randomized controlled trials: a systematic review

OBJECTIVES

To explore how process evaluation of complex interventions alongside randomized clinical trials (RCTs) in musculoskeletal conditions are conducted.

STUDY DESIGN AND SETTING

Systematic review.

METHODS

We searched the MEDLINE, SCOPUS, CINAHL, PsycINFO, Embase, Web of Science, and Cochrane databases. Studies were included if they reported process evaluation conducted alongside RCTs, within the main report or as separate reports, that assessed process evaluation of RCTs of complex nonsurgical and nonpharmacological interventions for musculoskeletal conditions. We performed a descriptive analysis of the included studies based on process evaluation parameters.

RESULTS

Data were extracted from 61 studies from 17 countries. Our findings showed studies used qualitative, quantitative, and mixed methods for process evaluations, typically reporting within the main RCT. Most studies were conducted in primary care settings. Only a few studies (16%) included a theoretical model to guide their process evaluation. Studies reported reach (8%), patients' and clinicians' perceptions of the interventions (44% and 8%, respectively), treatment fidelity and adherence (52% and 43%, respectively), training of patients and clinicians (eg, workshops, manuals and additional training) (54%), how the integration of process evaluation and outcome evaluation findings was performed (68%), barriers to perform the process evaluation (2%), and the strengths and weaknesses of the process evaluation (65%).

CONCLUSION

Reporting of process evaluations within RCTs is insufficiently reported. Researchers rarely adopted a theoretical model or framework to guide their process evaluation. Studies used a variety of methods to conduct process evaluations. We identified barriers, strengths, and weaknesses of methods used for assessing process evaluation as reported by authors from studies included in this review.

Optimising physiotherapist delivery fidelity of exercise and physical activity advice for achilles tendinopathy: A prospective repeated-measures observational study

OBJECTIVES

To assess physiotherapist delivery fidelity and identify factors optimising delivery fidelity of an intervention based on recommended guidelines for Achilles tendinopathy.

DESIGN

A prospective repeated-measures observational study of physiotherapist delivery fidelity with carefully defined exercise and physical activity advice.

SETTING

An inter-disciplinary clinic in Melbourne, Australia, embedded in a randomised controlled trial.

PARTICIPANTS

Two physiotherapists delivering the intervention to five participants each, at three timepoints.

INTERVENTION

All participants were expected to receive the same intervention. Feedback at timepoint one, guided boost-training to optimise delivery fidelity.

MAIN OUTCOME MEASURES

Proportion of exercise and physical activity advice components delivered as intended (high ≥80%; moderate 51-79%; low≤50%), with relationships between variables analysed using chi-square tests.

RESULTS

Physiotherapist delivery fidelity improved significantly between timepoint one and two (χ2 = 83.3, p < 0.001), then sustained at timepoint three. At timepoint one, seven (70%) of intervention components were delivered with high fidelity, one (10%) with moderate fidelity and two (20%) with low fidelity. At timepoint two, after boost-training, nine (90%) were delivered with high fidelity and one (10%) with moderate fidelity. At timepoint three, all intervention components (100%) were delivered with high fidelity by both physiotherapists.

CONCLUSION

Physiotherapist delivery fidelity can be optimised with feedback, collaboration and boost-training.

Increasing the delivery of upper limb constraint-induced movement therapy programs for stroke and brain injury survivors: evaluation of the ACTIveARM project

PURPOSE

To increase the number of constraint-induced movement therapy (CIMT) programs provided by rehabilitation services.

METHODS

A before-and-after implementation study involving nine rehabilitation services. The implementation package to help change practice included file audit-feedback cycles, 2-day workshops, poster reminders, a community-of-practice and drop-in support. File audits were conducted at baseline, every three months for 1.5 years, and once after support ceased to evaluate maintenance of change. CIMT participant outcomes were collected to evaluate CIMT effectiveness and maintenance (Action Research Arm Test and Motor Activity Log). Staff focus groups explored factors influencing CIMT delivery.

RESULTS

CIMT adoption improved from baseline where only 2% of eligible people were offered and/or received CIMT (n = 408 files) to more than 50% over 1.5 years post-implementation (n = 792 files, 52% to 73% offered CIMT, 27%-46% received CIMT). Changes were maintained at 6-month follow-up (n = 172 files, 56% offered CIMT, 40% received CIMT). CIMT participants (n = 74) demonstrated clinically significant improvements in arm function and occupational performance. Factors influencing adoption included interdisciplinary collaboration, patient support needs, intervention adaptations, a need for continued training, and clinician support.

CONCLUSIONS

The implementation package helped therapists overcome an evidence-practice gap and deliver CIMT more routinely.

Assessing the fidelity of a behavioural intervention involving academic detailing in general practice: a sub-study of the 'Implementing work-related Mental health guidelines in general PRacticE' (IMPRovE) trial

BACKGROUND

Assessing the fidelity of intervention components enables researchers to make informed judgements about the influence of those components on the observed outcome. The 'Implementing work-related Mental health guidelines in general PRacticE' (IMPRovE) trial is a hybrid III trial aiming to increase adherence to the 'Clinical Guidelines for the diagnosis and management of work-related mental health conditions in general practice'. IMPRovE is a multifaceted intervention, with one of the central components being academic detailing (AD). This study describes the fidelity to the protocol for the AD component of the IMPRovE intervention.

METHOD

All AD sessions for the trial were audio-recorded and a sample of 22% were randomly selected for fidelity assessment. Fidelity was assessed using a tailored proforma based on the Modified Conceptual Framework for fidelity assessment, measuring duration, coverage, frequency and content. A descriptive analysis was used to quantify fidelity to the protocol and a content analysis was used to elucidate qualitative aspects of fidelity.

RESULTS

A total of eight AD sessions were included in the fidelity assessment. The average fidelity score was 89.2%, ranging from 80 to 100% across the eight sessions. The sessions were on average 47 min long and addressed all of the ten chapters in the guideline. Of the guideline chapters, 9 were frequently discussed. The least frequently discussed chapter related to management of comorbid conditions. Most general practitioner (GP) participants used the AD sessions to discuss challenges with managing secondary mental conditions. In line with the protocol, opinion leaders who delivered the AD sessions largely offered evidence-based strategies aligning with the clinical guideline recommendations.

CONCLUSIONS/IMPLICATIONS

The IMPRovE AD intervention component was delivered to high fidelity. The sessions adhered to the intended duration, coverage, frequency, and content allowing participating GPs to comprehend the implementation of the guideline in their own practice. This study also demonstrates that the Modified Conceptual Fidelity Framework with a mixed methods approach can support the assessment of implementation fidelity of a behavioural intervention in general practice. The findings enhance the trustworthiness of reported outcomes from IMPRovE and show that assessing fidelity is amenable for AD and should be incorporated in other studies using AD.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ACTRN 12620001163998, November 2020.

Fidelity of Delivery in a Multisite Randomized Clinical Trial of Intervention Efficacy for Infants With Unilateral Cerebral Palsy

PURPOSE

To investigate the reliability of a measure of fidelity of therapist delivery, quantify fidelity of delivery, and determine factors impacting fidelity in the Rehabilitation EArly for Congenital Hemiplegia (REACH) clinical trial.

METHODS

Ninety-five infants (aged 3-9 months) with unilateral cerebral palsy participated in the REACH clinical trial. The Therapist Fidelity Checklist (TFC) evaluated key intervention components. Video-recorded intervention sessions were scored using the TFC.

RESULTS

Inter- and intrarater reliability was percentage agreement 77% to 100%. Fidelity of delivery was high for 88.9% of sessions and moderate for 11.1% of sessions. Sessions with moderate scores included infants receiving infant-friendly bimanual therapy and occurred at the intervention midpoint or later. No significant relationships were found for TFC scores and infant age, manual ability, or parent engagement.

CONCLUSIONS

Fidelity of delivery was high for the REACH trial in most intervention sessions. Standardized therapist training with intervention manuals and monthly peer-to-peer support likely contributed to these results.

Implementation Initiatives to Improve Low Back Pain Care in Australia: A Scoping Review

OBJECTIVE

This scoping review aimed to comprehensively review strategies for implementation of low back pain (LBP) guidelines, policies, and models of care in the Australian health care system.

METHODS

A literature search was conducted in MEDLINE, EMBASE, CINAHL, Allied and Complementary Medicine Database, and Web of Science to identify studies that aimed to implement or integrate evidence-based interventions or practices to improve LBP care within Australian settings.

RESULTS

Twenty-five studies met the inclusion criteria. Most studies targeted primary care settings (n = 13). Other settings included tertiary care (n = 4), community (n = 4), and pharmacies (n = 3). One study targeted both primary and tertiary care settings (n = 1). Only 40% of the included studies reported an underpinning framework, model, or theory. The implementation strategies most frequently used were evaluative and iterative strategies (n = 14, 56%) and train and educate stakeholders (n = 13, 52%), followed by engage consumers (n = 6, 24%), develop stakeholder relationships (n = 4, 16%), change in infrastructure (n = 4, 16%), and support clinicians (n = 3, 12%). The most common implementation outcomes considered were acceptability (n = 11, 44%) and adoption (n = 10, 40%), followed by appropriateness (n = 7, 28%), cost (n = 3, 12%), feasibility (n = 1, 4%), and fidelity (n = 1, 4%). Barriers included time constraints, funding, and teamwork availability. Facilitators included funding and collaboration between stakeholders.

CONCLUSIONS

Implementation research targeting LBP appears to be a young field, mostly focusing on training and educating stakeholders in primary care. Outcomes on sustainability and penetration of evidence-based interventions are lacking. There is a need for implementation research guided by established frameworks that consider interrelationships between organizational and system contexts beyond the clinician-patient dyad.

Fidelity assessment of nurse-led non-pharmacological package of care for knee pain in the package development phase of a feasibility randomised controlled trial based in secondary care: a mixed methods study

OBJECTIVES

To evaluate fidelity of delivery of a nurse-led non-pharmacological complex intervention for knee pain.

SETTING

Secondary care. Single-centre study.

STUDY DESIGN

Mixed methods study.

PARTICIPANTS

Eighteen adults with chronic knee pain.

INCLUSION CRITERIA

Age >40 years, knee pain present for longer than 3 months, knee pain for most days of the previous month, at least moderate pain in two of the five domains of Western Ontario and McMaster Universities Osteoarthritis Index pain scale.

INTERVENTIONS

Nurse-led non-pharmacological intervention comprising assessment, education, exercise, use of hot/cold treatments, footwear modification, walking aids and weight-loss advice (if required).

OUTCOMES

Primary: fidelity of delivery of intervention, secondary: nurses' experience of delivering intervention.

METHODS

Each intervention session with every participant was video recorded and formed part of fidelity assessment. Fidelity checklists were completed by the research nurse after each session and by an independent researcher, after viewing the video-recordings blinded to nurse ratings. Fidelity scores (%), percentage agreement and 95% Confidence Intervals (CI) were calculated. Two semi-structured interviews were conducted with the research nurse.

RESULTS

Fourteen participants completed all visits. 62 treatment sessions took place. Nurse self-report and assessor video rating scores for all 62 treatment sessions were included in fidelity assessment. Overall fidelity was higher on nurse self-report (97.7%) than on objective video-rating (84.2%). Percentage agreement between nurse self-report and video-rating was 73.3% (95% CI 71.3 to 75.3). Fidelity was lowest for advice on footwear and walking aids. The nurse reported difficulty advising on thermal treatments, footwear and walking aids, and did not feel confident negotiating achievable and realistic goals with participants.

CONCLUSIONS

A trained research nurse can deliver most components of a non-pharmacological intervention for knee pain to a high degree of fidelity. Future research should assess intervention fidelity in a routine clinical setting, and examine its clinical and cost-effectiveness.

TRIAL REGISTRATION NUMBER

NCT03670706.

Home environmental assessments and modification delivered by occupational therapists to reduce falls in people aged 65 years and over: the OTIS RCT

BACKGROUND

Falls and fall-related fractures are highly prevalent among older people and are a major contributor to morbidity and costs to individuals and society. Only one small pilot trial has evaluated the effectiveness of a home hazard assessment and environmental modification in the UK. This trial reported a reduction in falls as a secondary outcome, and no economic evaluation was undertaken. Therefore, the results need to be confirmed and a cost-effectiveness analysis needs to be undertaken.

OBJECTIVE

To determine the clinical effectiveness and cost-effectiveness of a home hazard assessment and environmental modification delivered by occupational therapists for preventing falls among community-dwelling people aged ≥ 65 years who are at risk of falling, relative to usual care.

DESIGN

This was a pragmatic, multicentre, modified cohort randomised controlled trial with an economic evaluation and a qualitative study.

SETTING

Eight NHS trusts in primary and secondary care in England.

PARTICIPANTS

In total, 1331 participants were randomised (intervention group, n = 430; usual-care group, n = 901) via a secure, remote service. Blinding was not possible.

INTERVENTIONS

All participants received a falls prevention leaflet and routine care from their general practitioner. The intervention group were additionally offered one home environmental assessment and modifications recommended or provided to identify and manage personal fall-related hazards, delivered by an occupational therapist.

MAIN OUTCOME MEASURES

The primary outcome was the number of falls per participant during the 12 months from randomisation. The secondary outcomes were the proportion of fallers and multiple fallers, time to fall, fear of falling, fracture rate, health-related quality of life and cost-effectiveness.

RESULTS

The primary analysis included all 1331 randomised participants and indicated weak evidence of a difference in fall rate between the two groups, with an increase in the intervention group relative to usual care (adjusted incidence rate ratio 1.17, 95% confidence interval 0.99 to 1.38; p = 0.07). A similar proportion of participants in the intervention group (57.0%) and the usual-care group (56.2%) reported at least one fall over 12 months. There were no differences in any of the secondary outcomes. The base-case cost-effectiveness analysis from an NHS and Personal Social Services perspective found that, on average per participant, the intervention was associated with additional costs (£18.78, 95% confidence interval £16.33 to £21.24), but was less effective (mean quality-adjusted life-year loss -0.0042, 95% confidence interval -0.0041 to -0.0043). Sensitivity analyses demonstrated uncertainty in these findings. No serious, related adverse events were reported. The intervention was largely delivered as intended, but recommendations were followed to a varying degree.

LIMITATIONS

Outcome data were self-reported by participants, which may have led to inaccuracies in the reported falls data.

CONCLUSIONS

We found no evidence that an occupational therapist-delivered home assessment and modification reduced falls in this population of community-dwelling participants aged ≥ 65 years deemed at risk of falling. The intervention was more expensive and less effective than usual care, and therefore it does not provide a cost-effective alternative to usual care.

FUTURE WORK

An evaluation of falls prevention advice in a higher-risk population, perhaps those previously hospitalised for a fall, or given by other professional staff could be justified.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN22202133.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 46. See the NIHR Journals Library website for further project information.

Fidelity of primary care nurses' delivery of a behavioural change intervention enhancing physical activity in patients at risk of cardiovascular disease: an observational study

OBJECTIVE

To evaluate the fidelity of delivery of a nurse-led intervention to enhance physical activity in patients at risk for cardiovascular diseases, the Activate intervention, by assessing: (1) self-reported fidelity of delivery; (2) observed fidelity of delivery; (3) quality of delivery of the Activate intervention and (4) nurses' beliefs about their capability, motivation, confidence and effectiveness towards delivering the Activate intervention, including behavioural change techniques.

DESIGN

An observational study.

SETTING

General practices in the Netherlands.

PARTICIPANTS

Primary care nurses (n=20) from 16 general practices.

PRIMARY AND SECONDARY OUTCOME MEASURES

Nurses' self-reported fidelity was evaluated using checklists (n=282), and the observed fidelity and quality of delivery were examined using audiorecordings of consultations of the delivery of the Activate intervention (n=42). Nurses' beliefs towards delivering the intervention were assessed using questionnaires (n=72).

RESULTS

The self-reported fidelity was 88.1% and observed fidelity was 85.4%, representing high fidelity. The observed fidelity of applied behavioural change techniques was moderate (75.0%). The observed quality of delivery was sufficient and varied among nurses (mean 2.9; SD 4.4; range 0-4). Nurses' beliefs about their capability, motivation, confidence and effectiveness towards delivering the intervention increased over time.

CONCLUSIONS

Nurses delivered most intervention components as intended with sufficient quality. Nurses believed they were capable, motivated and confident to deliver the intervention. They believed the intervention was effective to increase patients' physical activity level. Despite the high fidelity and moderate fidelity of applied behavioural change techniques, the varying quality of delivery within and across nurses might have diluted the effectiveness of the Activate intervention.

TRIAL REGISTRATION NUMBER

NCT02725203.

Enhancing the translation of health behaviour change research into practice: a selective conceptual review of the synergy between implementation science and health psychology

Health psychology is at the forefront of developing and disseminating evidence, theories, and methods that have improved the understanding of health behaviour change. However, current dissemination approaches may be insufficient for promoting broader application and impact of this evidence to benefit the health of patients and the public. Nevertheless, behaviour change theory/methods typically directed towards health behaviours are now used in implementation science to understand and support behaviour change in individuals at different health system levels whose own behaviour impacts delivering evidence-based health behaviour change interventions. Despite contributing to implementation science, health psychology is perhaps doing less to draw from it. A redoubled focus on implementation science in health psychology could provide novel prospects for enhancing the impact of health behaviour change evidence. We report a Health Psychology Review-specific review-of-reviews of trials of health behaviour change interventions published from inception to April 2020. We identified 34 reviews and assessed whether implementation readiness of behaviour change interventions was discussed. We then narratively review how implementation science has integrated theory/methods from health psychology and related discipline. Finally, we demonstrate how greater synergy between implementation science and health psychology could promote greater follow-through on advances made in the science of health behaviour change.

Developing strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia: a mixed methods study

BACKGROUND

It is important to evaluate fidelity of delivery and engagement during feasibility trials. However, there is little guidance on how to systematically develop strategies to improve implementation if problems arise. We aimed to: 1) Assess fidelity of delivery and engagement, 2) Identify factors influencing fidelity of delivery and engagement, and 3) Develop strategies to improve fidelity of delivery of, and engagement with, a complex intervention to improve independence in dementia, within a feasibility trial.

METHODS

A mixed methods evaluation of an intervention that aimed to improve independence in dementia. To assess fidelity of delivery and engagement, observation and self-report methods were used: 60% of audio-recorded intervention sessions were transcribed and reliably rated for fidelity. Providers (n = 12) and people with dementia/supporters (n = 34) were asked to complete checklists after each session. Descriptive statistics were used to analyse the data. To identify factors influencing fidelity and engagement, one-to-one semi-structured interviews were conducted with providers (n = 8), people with dementia (n = 7) and supporters (n = 7). Thematic analysis and content analysis were used to analyse data. To develop strategies, we followed four steps proposed by the authors of the Behaviour Change Wheel (1. Understanding the behaviour, 2. Identifying intervention functions, 3. Specifying intervention content, 4. Identifying mode of delivery).

RESULTS

Researcher ratings indicated moderate fidelity and provider/participant ratings indicated high fidelity of delivery. Knowledge, providers' attributes, ease of adaptation of the intervention in relation to participants' needs and logistical considerations influenced fidelity. We developed four strategies to improve fidelity of delivery of PRIDE: 1) showing a video, 2) giving an instruction sheet, 3) giving time to practice and 4) providing continued support. Participants reported high levels of engagement. Participants' attributes, capability and opportunity influenced engagement. We developed four strategies to improve engagement with PRIDE: 1) a session summary document, 2) clear instructions, 3) time to practice activity and 4) providing regular compulsory telephone support.

CONCLUSION

Fidelity of delivery and engagement are complex behaviours. This manuscript provides an example of how the Behaviour Change Wheel can be used during a feasibility trial to systematically develop strategies to improve implementation of complex interventions.

Intervention Fidelity Within Trials of Infant Feeding Behavioral Interventions to Prevent Childhood Obesity: A Systematic Review

Background

Intervention fidelity refers to whether an intervention has been implemented as intended. Trials of infant feeding behavioral interventions to prevent childhood obesity show inconsistent evidence of effectiveness. However, intervention fidelity has not been previously explored within these trials, limiting interpretation of findings.

Purpose

To review the use and/or reporting of strategies to enhance and assess intervention fidelity within trials of infant feeding interventions to prevent childhood obesity, and their association with study quality, effectiveness, and publication year.

Methods

Seven electronic databases were searched, with articles screened for inclusion by two reviewers. The National Institutes of Health Behaviour Change Consortium fidelity checklist was used to assess use and/or reporting of fidelity strategies across five domains (design, provider training, delivery, receipt, and enactment).

Results

Ten trials (16 papers) were identified. Average use/reporting of fidelity strategies was moderate (54%), ranging from 28.9% to 76.7%. Levels of use/reporting ranged from 15.9% in the domain of provider training to 95% for enactment. No association was found between these levels and study quality, effectiveness, or publication year.

Conclusions

The moderate use/reporting of fidelity strategies within trials of infant feeding interventions suggests that previous findings of inconsistent effectiveness may not fully reflect the intended interventions. The review highlights key considerations for improving future research, both in the area of behavioral infant feeding and wider behavior change literature. This includes improving reporting across all fidelity domains and ensuring an enhanced focus on provider training and control group content to optimize the translation of research into practice.

PROSPERO Registration number

CRD42016033492.

Focusing on fidelity: narrative review and recommendations for improving intervention fidelity within trials of health behaviour change interventions

Background: Interventions to change behaviour have substantial potential to impact positively on individual and overall public health. Despite an increasing focus on health behaviour change intervention research, interventions do not always have the desired effect on outcomes, while others have diluted effects once implemented into real-life settings. There is little investment into understanding how or why such interventions work or do not work. Methodological inadequacies of trials of behavioural interventions have been previously suggested as a barrier to the quality and advancement of behavioural research, with intervention fidelity acknowledged as a key area for improvement. However, there is much ambiguity regarding the terminology and conceptualisation of intervention fidelity and a lack of practical guidance regarding how to address it sufficiently, particularly within trials of complex behavioural interventions. Objectives: This article outlines specific issues concerning intervention fidelity within trials of health behaviour change interventions and suggests practical considerations and specific recommendations for researchers, with examples from the literature presented. Conclusions: Recommendations pertain to (1) clarifying how fidelity is defined and conceptualised, (2) considering fidelity beyond intervention delivery, (3) considering strategies to both enhance and assess fidelity, (4) making use of existing frameworks and guidance, (5) considering the quality and comprehensiveness of fidelity assessment strategies, (6) considering the balance between fidelity and adaptation and (7) reporting the use of fidelity enhancement and assessment strategies and their results. Suggestions for future research to improve our understanding of, and ability to, address fidelity in behaviour change interventions are also provided.

Developing quality fidelity and engagement measures for complex health interventions

OBJECTIVES

To understand whether interventions are effective, we need to know whether the interventions are delivered as planned (with fidelity) and engaged with. To measure fidelity and engagement effectively, high-quality measures are needed. We outline a five-step method which can be used to develop quality measures of fidelity and engagement for complex health interventions. We provide examples from a fidelity study conducted within an evaluation of an intervention aimed to increase independence in dementia.

METHODS

We propose five steps that can be systematically used to develop fidelity checklists for researchers, providers, and participants to measure fidelity and engagement. These steps include the following: (1) reviewing previous measures, (2) analysing intervention components and developing a framework outlining the content of the intervention, (3) developing fidelity checklists and coding guidelines, (4) obtaining feedback about the content and wording of checklists and guidelines, and (5) piloting and refining checklists and coding guidelines to assess and improve reliability.

RESULTS

Three fidelity checklists that can be used reliably were developed to measure fidelity of and engagement with, the Promoting Independence in Dementia (PRIDE) intervention. As these measures were designed to be used by researchers, providers, and participants, we developed two versions of the checklists: one for participants and one for researchers and providers.

CONCLUSIONS

The five steps that we propose can be used to develop psychometrically robust and implementable measures of fidelity and engagement for complex health interventions that can be used by different target audiences. By considering quality when developing measures, we can be more confident in the interpretation of intervention outcomes drawn from fidelity and engagement studies. Statement of contribution What is already known on the subject? Fidelity and engagement can be measured using a range of methods, such as observation and self-report. Studies seldom report psychometric and implementation qualities of fidelity measures. What does this study add? A method for developing fidelity and engagement measures for complex health interventions. Guidance on how to consider quality when developing fidelity and engagement measures.

Measuring fidelity of delivery of the Community Occupational Therapy in Dementia-UK intervention

Background

Interpreting data about intervention effectiveness requires an understanding of which intervention components were delivered and whether they were delivered as planned (fidelity of delivery). These studies aimed to develop a reliable measure for assessing fidelity of delivery of the Community Occupational Therapy in Dementia-UK intervention (COTiD-UK) (Study 1) and measure fidelity of delivery of COTiD-UK across sessions, sites and occupational therapists (Study 2).

Methods

The studies used a longitudinal observational design nested within a multi-site randomised controlled trial. Where practicable, all intervention sessions were audio-recorded. Fidelity checklists and coding guidelines were developed, piloted and refined until good agreement was achieved between two coders. Ten percent of sessions were purposively sampled from 12 sites and 31 occupational therapists. Transcripts were coded using checklists developed in Study 1; 10% of sets of intervention session transcripts were double coded to ensure that agreement was maintained. Percentages of components that were delivered were calculated for each session, site and occupational therapist.

Results

A reliable measure of fidelity of delivery for COTiD-UK was developed after several rounds of piloting and amendments. COTiD-UK was delivered with moderate fidelity across all six sessions (range: 52.4–75.5%). The mean range of fidelity varied across sites (26.7–91.2%) and occupational therapists (26.7–94.1%).

Conclusions

A reliable, systematic method for measuring fidelity of delivery of COTiD-UK was developed and applied, and can be adapted for use in similar interventions. As COTiD-UK was delivered with moderate fidelity, there is a reasonable degree of confidence that intervention effects were attributable to COTiD-UK.

Protocol for a pilot randomised controlled trial evaluating feasibility and acceptability of cognitive remediation group therapy compared with mutual aid group therapy for people ageing with HIV-associated neurocognitive disorder (HAND) in Toronto, Canada

INTRODUCTION

HIV-associated neurocognitive disorder (HAND) may affect 30%-50% of people ageing with HIV. HAND may increase stress and anxiety, and impede coping. Psychosocial group therapy may ameliorate HAND's symptoms, yet the ideal intervention is unclear. This protocol outlines a pilot randomised controlled trial (RCT)-designed using community-based participatory research-to pilot cognitive remediation group therapy (CRGT) against an active comparator.

METHODS AND ANALYSIS

This is a pilot, parallel design, two-arm RCT that will recruit participants diagnosed with the mild neurocognitive disorder form of HAND from a neurobehavioural research unit at a tertiary care hospital in Toronto, Canada. Eligibility criteria include age ≥40 years, known HIV status for 5+ years, English fluency, able to consent and able to attend 8 weeks of group therapy. Eligible participants will be randomised to one of two treatment arms, each consisting of eight-session group interventions delivered once weekly at 3 hours per session. Arm 1 (novel) is CRGT, combining mindfulness-based stress reduction with brain training activities. Arm 2 (active control) is mutual aid group therapy. The primary outcomes are feasibility, measured by proportions of recruitment and completion, and acceptability, determined by a satisfaction questionnaire. The secondary outcome is intervention fidelity, where content analysis will be used to assess facilitator session reports. A between-group analysis will be conducted on exploratory outcomes of stress, anxiety, coping and use of intervention activities that will be collected at three time points.

ETHICS AND DISSEMINATION

Ethical approval was obtained from the Research Ethics Boards of St. Michael's Hospital and the University of Toronto. Findings will be disseminated through peer-reviewed publications, conference presentations and community reporting. This study could provide insight into design (eg, recruitment, measures) and intervention considerations (eg, structure, content) for a larger trial to lessen the burden of cognitive decline among people ageing with HIV.

TRIAL REGISTRATION NUMBER

NCT03483740; Pre-results.

Links between the Hip and the Lumbar Spine (Hip Spine Syndrome) as they Relate to Clinical Decision Making for Patients with Lumbopelvic Pain

This narrative review focuses on the links between the hip and lumbar spine in the context of lumbopelvic pain. The literature regarding this topic is variable, found in many disciplines of care using different terminology. Collectively these inconsistencies add to the complexity of understanding the current evidence as it pertains to clinical practice. We have chosen to review studies that describe and assess overlapping pain distributions between the lumbar spine and hip, hip osteoarthritis and lumbopelvic pain, improvements in low back pain following hip arthroplasty, association of hip range of motion and lumbopelvic pain, and lumbopelvic motion and hip motion as they relate to low back pain and gender. The links between the hip and lumbar spine may provide the clinician important information to make decisions and recommendations for people presenting with the clinical symptom complex including lumbopelvic pain.

Comprehensive evaluation of interventions: eight vital parameters

BACKGROUND

It is critically important to determine the effectiveness of an intervention before it can be translated into clinical practice. However, the future implementation and sustainability of the intervention may be diminished if other intervention parameters are not assessed. This requires obtaining feedback from intervention recipients so interventions will be perceived as appealing, relevant, meaningful and beneficial to them; otherwise recipients may be unlikely to perform them over time, resulting in unsuccessful health outcomes.

AIM

To propose the addition of two intervention parameters to the existing six-parameter model and provide examples from recent research of how each parameter can be tested.

DISCUSSION

Definitions of the eight parameters are provided and methods for analysing each of them explained. While some studies show necessity, fidelity and cost have unique distinguishing characteristics, other studies indicate feasibility, acceptability and safety have common features, and efficacy and effectiveness are closely associated.

CONCLUSION

Researchers frequently examine one or two parameters, but few simultaneously apply the six-parameter model. This model is also missing two vital parameters - efficacy and cost.

IMPLICATIONS FOR PRACTICE

Comprehensive and systematic evaluation of all eight intervention parameters is recommended before researchers begin randomised controlled trials and translate them into practice.

IMP2ART systematic review of education for healthcare professionals implementing supported self-management for asthma

Despite a robust evidence base for its effectiveness, implementation of supported self-management for asthma is suboptimal. Professional education is an implementation strategy with proven effectiveness, though the specific features linked with effectiveness are often unclear. We performed a systematic review of randomised controlled trials and controlled clinical trials (published from 1990 and updated to May 2017 using forward citation searching) to determine the effectiveness of professional education on asthma self-management support and identify features of effective initiatives. Primary outcomes reflected professional behaviour change (provision of asthma action plans) and patient outcomes (asthma control; unscheduled care). Data were coded using the Effective Practice and Organisation of Care Taxonomy, the Theoretical Domains Framework (TDF), and Bloom's Taxonomy and synthesised narratively. Of 15,637 articles identified, 18 (reporting 15 studies including 21 educational initiatives) met inclusion criteria. Risk of bias was high for five studies, and unclear for 10. Three of 6 initiatives improved action plan provision; 1/2 improved asthma control; and 2/7 reduced unscheduled care. Compared to ineffective initiatives, effective initiatives were more often coded as being guideline-based; involving local opinion leaders; including inter-professional education; and addressing the TDF domains 'social influences'; 'environmental context and resources'; 'behavioural regulation'; 'beliefs about consequences'; and 'social/professional role and identity'. Findings should be interpreted cautiously as many strategies were specified infrequently. However, identified features warrant further investigation as part of implementation strategies aiming to improve the provision of supported self-management for asthma.

Measures of fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions: A systematic review of measure quality

PURPOSE

Understanding the effectiveness of complex, face-to-face health behaviour change interventions requires high-quality measures to assess fidelity of delivery and engagement. This systematic review aimed to (1) identify the types of measures used to monitor fidelity of delivery of, and engagement with, complex, face-to-face health behaviour change interventions and (2) describe the reporting of psychometric and implementation qualities.

METHODS

Electronic databases were searched, systematic reviews and reference lists were hand-searched, and 21 experts were contacted to identify articles. Studies that quantitatively measured fidelity of delivery of, and/or engagement with, a complex, face-to-face health behaviour change intervention for adults were included. Data on interventions, measures, and psychometric and implementation qualities were extracted and synthesized using narrative analysis.

RESULTS

Sixty-six studies were included: 24 measured both fidelity of delivery and engagement, 20 measured fidelity of delivery, and 22 measured engagement. Measures of fidelity of delivery included observation (n = 17; 38.6%), self-report (n = 15; 34%), quantitatively rated qualitative interviews (n = 1; 2.3%), or multiple measures (n = 11; 25%). Measures of engagement included self-report (n = 18; 39.1%), intervention records (n = 11; 24%), or multiple measures (n = 17; 37%). Fifty-one studies (77%) reported at least one psychometric or implementation quality; 49 studies (74.2%) reported at least one psychometric quality, and 17 studies (25.8%) reported at least one implementation quality.

CONCLUSION

Fewer than half of the reviewed studies measured both fidelity of delivery of, and engagement with complex, face-to-face health behaviour change interventions. More studies reported psychometric qualities than implementation qualities. Interpretation of intervention outcomes from fidelity of delivery and engagement measurements may be limited due to a lack of reporting of psychometric and implementation qualities. Statement of contribution What is already known on this subject? Evidence of fidelity and engagement is needed to understand effectiveness of complex interventions Evidence of fidelity and engagement are rarely reported High-quality measures are needed to measure fidelity and engagement What does this study add? Evidence that indicators of quality of measures are reported in some studies Evidence that psychometric qualities are reported more frequently than implementation qualities A recommendation for intervention evaluations to report indicators of quality of fidelity and engagement measures.

Implementation of a Multidisciplinary Guideline for Low Back Pain: Process-Evaluation Among Health Care Professionals

Background To reduce the burden of low back pain (LBP) in the Netherlands, a multidisciplinary guideline for LBP has been implemented in Dutch primary care using a multifaceted implementation strategy targeted at health care professionals (HCPs) and patients. The current paper describes the process evaluation of the implementation among HCPs. Methods The strategy aimed to improve multidisciplinary collaboration and communication, and consisted of 7 components. This process evaluation was performed using the Linnan and Steckler framework. Data were collected using a mixed methods approach of quantitative and qualitative data. Results 128 HCPs participated in the implementation study, of which 96 participated in quantitative and 21 participated in qualitative evaluation. Overall dose delivered for this study was 89 %, and the participants were satisfied with the strategy, mostly with the multidisciplinary approach, which contributed to the mutual understanding of each other's disciplines and perspectives. While the training sessions did not yield any new information, the strategy created awareness of the guideline and its recommendations, contributing to positively changing attitudes and aiding in improving guideline adherent behaviour. However, many barriers to implementation still exist, including personal and practical factors, confidence, dependence and distrust issues among the HCPs, as well as policy factors (e.g. reimbursement systems). Conclusions The data presented in this paper have shown that the strategy that was used to implement the guideline in a Dutch primary care setting was feasible, especially when using a multidisciplinary approach. However, identified barriers for implementation have been identified and should be addressed in future implementation.

Using mixed methods to assess fidelity of delivery and its influencing factors in a complex self-management intervention for people with osteoarthritis and low back pain

OBJECTIVES AND DESIGN

Despite an increasing awareness of the importance of fidelity of delivery within complex behaviour change interventions, it is often poorly assessed. This mixed methods study aimed to establish the fidelity of delivery of a complex self-management intervention and explore the reasons for these findings using a convergent/triangulation design.

SETTING

Feasibility trial of the Self-management of Osteoarthritis and Low back pain through Activity and Skills (SOLAS) intervention (ISRCTN49875385), delivered in primary care physiotherapy.

METHODS AND OUTCOMES

60 SOLAS sessions were delivered across seven sites by nine physiotherapists. Fidelity of delivery of prespecified intervention components was evaluated using (1) audio-recordings (n=60), direct observations (n=24) and self-report checklists (n=60) and (2) individual interviews with physiotherapists (n=9). Quantitatively, fidelity scores were calculated using percentage means and SD of components delivered. Associations between fidelity scores and physiotherapist variables were analysed using Spearman's correlations. Interviews were analysed using thematic analysis to explore potential reasons for fidelity scores. Integration of quantitative and qualitative data occurred at an interpretation level using triangulation.

RESULTS

Quantitatively, fidelity scores were high for all assessment methods; with self-report (92.7%) consistently higher than direct observations (82.7%) or audio-recordings (81.7%). There was significant variation between physiotherapists' individual scores (69.8% - 100%). Both qualitative and quantitative data (from physiotherapist variables) found that physiotherapists' knowledge (Spearman's association at p=0.003) and previous experience (p=0.008) were factors that influenced their fidelity. The qualitative data also postulated participant-level (eg, individual needs) and programme-level factors (eg, resources) as additional elements that influenced fidelity.

CONCLUSION

The intervention was delivered with high fidelity. This study contributes to the limited evidence regarding fidelity assessment methods within complex behaviour change interventions. The findings suggest a combination of quantitative methods is suitable for the assessment of fidelity of delivery. A mixed methods approach provided a more insightful understanding of fidelity and its influencing factors.

TRIAL REGISTRATION NUMBER

ISRCTN49875385; Pre-results.

Effectiveness of multifaceted implementation strategies for the implementation of back and neck pain guidelines in health care: a systematic review

BACKGROUND

For the optimal use of clinical guidelines in daily practice, mere distribution of guidelines and materials is not enough, and active implementation is needed. This review investigated the effectiveness of multifaceted implementation strategies compared to minimal, single, or no implementation strategy for the implementation of non-specific low back and/or neck pain guidelines in health care.

METHODS

The following electronic databases were searched from inception to June 1, 2015: MEDLINE, Embase, PsycInfo, the Cochrane Library, and CINAHL. The search strategy was restricted to low back pain, neck pain, and implementation research. Studies were included if their design was a randomized controlled trial, reporting on patients (age ≥18 years) with non-specific low back pain or neck pain (with or without radiating pain). Trials were eligible if they reported patient outcomes, measures of healthcare professional behaviour, and/or outcomes on healthcare level. The primary outcome was professional behaviour. Guidelines that were evaluated in the studies had to be implemented in a healthcare setting. No language restrictions were applied, and studies had to be published full-text in peer-reviewed journals, thus excluding abstract only publications, conference abstracts, and dissertation articles. Two researchers independently screened titles and abstract, extracted data from included studies, and performed risk of bias assessments.

RESULTS

After removal of duplicates, the search resulted in 4750 abstracts to be screened. Of 43 full-text articles assessed for eligibility, 12 were included in this review, reporting on 9 individual studies, and separate cost-effectiveness analyses of 3 included studies. Implementation strategies varied between studies. Meta-analyses did not reveal any differences in effect between multifaceted strategies and controls.

CONCLUSION

This review showed that multifaceted strategies for the implementation of neck and/or back pain guidelines in health care do not significantly improve professional behaviour outcomes. No effects on patient outcomes or cost of care could be found. More research is necessary to determine whether multifaceted implementation strategies are conducted as planned and whether these strategies are effective in changing professional behaviour and thereby clinical practice.

Development of a Feasible Implementation Fidelity Protocol Within a Complex Physical Therapy-Led Self-Management Intervention

BACKGROUND

Implementation fidelity is poorly addressed within physical therapy interventions, which may be due to limited research on how to develop and implement an implementation fidelity protocol.

OBJECTIVE

The purpose of this study was to develop a feasible implementation fidelity protocol within a pilot study of a physical therapy-led intervention to promote self-management for people with chronic low back pain or osteoarthritis.

DESIGN

A 2-phase mixed-methods design was used.

METHODS

Phase 1 involved the development of an initial implementation fidelity protocol using qualitative interviews with potential stakeholders to explore the acceptability of proposed strategies to enhance and assess implementation fidelity. Phase 2 involved testing and refining the initial implementation fidelity protocol to develop a finalized implementation fidelity protocol. Specifically, the feasibility of 3 different strategies (physical therapist self-report checklists, independently rated direct observations, and audio-recorded observations) for assessing implementation fidelity of intervention delivery was tested, followed by additional stakeholder interviews that explored the overall feasibility of the implementation fidelity protocol.

RESULTS

Phase 1 interviews determined the proposed implementation fidelity strategies to be acceptable to stakeholders. Phase 2 showed that independently rated audio recordings (n=6) and provider self-report checklists (n=12) were easier to implement than independently rated direct observations (n=12) for assessing implementation fidelity of intervention delivery. Good agreement (79.8%-92.8%) was found among all methods. Qualitative stakeholder interviews confirmed the acceptability, practicality, and implementation of the implementation fidelity protocol.

LIMITATIONS

The reliability and validity of assessment checklists used in this study have yet to be fully tested, and blinding of independent raters was not possible.

CONCLUSIONS

A feasible implementation fidelity protocol was developed based on a 2-phase development process involving intervention stakeholders. This study provides valuable information on the feasibility of rigorously addressing implementation fidelity within physical therapy interventions and provides recommendations for researchers wanting to address implementation fidelity in similar areas.

Sexual counselling for patients with cardiovascular disease: protocol for a pilot study of the CHARMS sexual counselling intervention

INTRODUCTION

Sexual problems are common with cardiovascular disease, and can negatively impact quality of life. To address sexual problems, guidelines have identified the importance of sexual counselling during cardiac rehabilitation, yet this is rarely provided. The Cardiac Health and Relationship Management and Sexuality (CHARMS) intervention aims to improve the provision of sexual counselling in cardiac rehabilitation in Ireland.

METHODS AND ANALYSIS

This is a multicentre pilot study for the CHARMS intervention, a complex, multilevel intervention delivered within hospital-based cardiac rehabilitation programmes. The intervention includes (1) training in sexual counselling for staff, (2) a staff-led patient education and support intervention embedded within the cardiac rehabilitation programme, (3) a patient information booklet and (4) an awareness raising poster. The intervention will be delivered in two randomly selected cardiac rehabilitation centres. In each centre 30 patients will be recruited, and partners will also be invited to participate. Data will be collected from staff and patients/partners at T1 (study entry), T2 (3-month follow-up) and T3 (6-month follow-up). The primary outcome for patients/partners will be scores on the Sexual Self-Perception and Adjustment Questionnaire. Secondary outcomes for patients/partners will include relationship satisfaction; satisfaction with and barriers to sexual counselling in services; sexual activity, functioning and knowledge; physical and psychological well-being. Secondary outcomes for staff will include sexuality-related practice; barriers to sexual counselling; self-ratings of capability, opportunity and motivation; sexual attitudes and beliefs; knowledge of cardiovascular disease and sex. Fidelity of intervention delivery will be assessed using trainer self-reports, researcher-coded audio recordings and exit interviews. Longitudinal feasibility data will be gathered from patients/partners and staff via questionnaires and interviews.

ETHICS AND DISSEMINATION

This study is approved by the Research Ethics Committee (REC) of the National University of Ireland, Galway. Findings will be disseminated to cardiac rehabilitation staff, patients/partners and relevant policymakers via appropriate publications and presentations.