Qualitative research to inform hypothesis testing for fidelity-based sub-group analysis in clinical trials: lessons learnt from the process evaluation of a multifaceted podiatry intervention for falls prevention

Impact of Social Stories on social and emotional health of autism spectrum primary school children: the ASSSIST2 RCT with economic evaluation

Background

Differences in the way autistic children experience the world can contribute to anxiety and stress. Carol Gray's Social Stories™ are a highly personalised intervention to support children by providing social information about specific situations in an individual story.

Objectives

This randomised controlled trial aimed to establish whether Social Stories are clinically effective and cost-effective in improving social responsiveness and social and emotional health in children on the autism spectrum in schools.

Design

A multisite pragmatic cluster randomised controlled trial comparing Social Stories with care as usual.

Setting

Eighty-seven schools (clusters) across Yorkshire and the Humber.

Participants

Two hundred and forty-nine children were randomised via a bespoke system hosted at York Trials Unit (129 Social Stories and 120 care as usual). Recruitment was completed in May 2021. Participants were children aged 4-11 years with a diagnosis of autism, alongside teachers, interventionists and caregivers. Recruitment was via schools, NHS trusts, support groups and local publicity.

Intervention

The intervention included training for educational professionals and caregivers covering psychoeducation and implementation of Social Stories. Stories were written around contextualised goals around the child's need for social information. Interventionists read the Social Story™ with the child at least six times over 4 weeks during school.

Main outcome measure

The primary outcome was the Social Responsiveness Scale-2 completed by teachers at 6 months (the primary end point), which measures social awareness, cognition, communication and behaviour. Data were collected from caregivers and educational professionals at 6 weeks and 6 months through questionnaires. Blinding of participants was not possible.

Results

At 6 months, the estimated difference in expected teacher-reported Social Responsiveness Scale-2 T-score (the primary end point) was -1.61 (95% confidence interval -4.18 to 0.96, p = 0.220), slightly favouring the intervention group. The estimated differences for the parent-reported secondary outcomes at 6 months were small and generally favoured the control group except the measure of children's quality-adjusted life-year (+ 0.001, 95% confidence interval -0.032 to 0.035) and parental stress (-1.49, 95% confidence interval -5.43 to 2.46, p = 0.460), which favoured the intervention group. Children in the intervention group met their individual goals more frequently than children who received usual care alone (0.97 confidence interval 0.21 to 1.73, p = 0.012). The intervention is likely to save small costs (-£191 per child, 95% confidence interval -767.7 to 337.7) and maintain a similar quality of life compared to usual care. The probability of Social Stories being a preferred option is 75% if the society is willing to pay £20,000 per quality-adjusted life-year gained. Limitations include considerable disruptions during the coronavirus disease 2019 pandemic.

Conclusion

Social Stories are used in schools and represent a low-cost intervention. There is no clinically evident impact on social responsiveness, anxiety and/or depression, parental stress or general health. Benefits were observed for specific behavioural goals as assessed by the teacher, and Social Stories may serve as a useful tool for facilitating dialogue between children and school staff to address specific behavioural challenges. Usage should be at the school's discretion.

Future work

Given the uncertainty of the results in light of coronavirus disease 2019, further work to establish the impact of Social Stories is merited.

Trial registration

This trial is registered as ISRCTN11634810.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 16/111/91) and is published in full in Health Technology Assessment; Vol. 28, No. 39. See the NIHR Funding and Awards website for further award information.

Implementation fidelity in a multifaceted program to foster rational antibiotics use in primary care: an observational study

BACKGROUND

The ARena study (Sustainable Reduction of Antimicrobial Resistance in German Ambulatory Care) is a three-arm, cluster randomized trial to evaluate a multifaceted implementation program in a German primary care setting. In the context of a prospective process evaluation conducted alongside ARena, this study aimed to document and explore fidelity of the implementation program.

METHODS

This observational study is based on data generated in a three-wave survey of 312 participating physicians in the ARena program and attendance documentation. Measures concerned persistence of participation in the ARena program and adherence to intervention components (thematic quality circles, e-learning, basic expenditure reimbursements, additional bonus payments and a computerized decision support system). Participants' views on five domains of the implementation were also measured. Binary logistic and multiple linear regression analyses were used to explore which views on the implementation were associated with participants' adherence to quality circles and use of additional bonus compensation.

RESULTS

The analysis of fidelity showed overall high persistence of participation in the intervention components across the three intervention arms (90,1%; 97,9%; 92,9%). 96.4% of planned quality circles were delivered to study participants and, across waves, 30.4% to 93% of practices participated; 56.1% of physicians attended the maximum of four quality circles. 84% of the practices (n = 158) with a minimum of one index patient received a performance-based additional bonus payment at least once. In total, bonus compensation was triggered for 51.8% of affected patients. Participation rate for e-learning (a prerequisite for reimbursement of project-related expenditure) covered 90.8% of practices across all intervention arms, with the highest rate in arm II (96.5%). Uptake of expenditure reimbursement was heterogeneous across study arms, with a mean rate of 86.5% (89.1% in arm I, 96.4% in arm II and 74.1% in arm III). Participants' views regarding participant responsiveness (OR = 2.298) 95% CI [1.598, 3.305] and Context (OR = 2.146) 95% CI [1.135, 4.055] affected additional bonus payment. Participants' views on participant responsiveness (Beta = 0.718) 95% CI [0.479, 0.957], Context (Beta = 0.323) 95% CI [0.055, 0.590] and Culture of shared decision-making (Beta = -0.334) 95% CI [-0.614, -0.053] affected quality circle attendance.

CONCLUSION

This study showed an overall high fidelity to the implementation program. Participants' views on the implementation were associated with degree of intervention fidelity.

TRIAL REGISTRATION

ISRCTN, ISRCTN58150046.

Home environmental assessments and modification delivered by occupational therapists to reduce falls in people aged 65 years and over: the OTIS RCT

BACKGROUND

Falls and fall-related fractures are highly prevalent among older people and are a major contributor to morbidity and costs to individuals and society. Only one small pilot trial has evaluated the effectiveness of a home hazard assessment and environmental modification in the UK. This trial reported a reduction in falls as a secondary outcome, and no economic evaluation was undertaken. Therefore, the results need to be confirmed and a cost-effectiveness analysis needs to be undertaken.

OBJECTIVE

To determine the clinical effectiveness and cost-effectiveness of a home hazard assessment and environmental modification delivered by occupational therapists for preventing falls among community-dwelling people aged ≥ 65 years who are at risk of falling, relative to usual care.

DESIGN

This was a pragmatic, multicentre, modified cohort randomised controlled trial with an economic evaluation and a qualitative study.

SETTING

Eight NHS trusts in primary and secondary care in England.

PARTICIPANTS

In total, 1331 participants were randomised (intervention group, n = 430; usual-care group, n = 901) via a secure, remote service. Blinding was not possible.

INTERVENTIONS

All participants received a falls prevention leaflet and routine care from their general practitioner. The intervention group were additionally offered one home environmental assessment and modifications recommended or provided to identify and manage personal fall-related hazards, delivered by an occupational therapist.

MAIN OUTCOME MEASURES

The primary outcome was the number of falls per participant during the 12 months from randomisation. The secondary outcomes were the proportion of fallers and multiple fallers, time to fall, fear of falling, fracture rate, health-related quality of life and cost-effectiveness.

RESULTS

The primary analysis included all 1331 randomised participants and indicated weak evidence of a difference in fall rate between the two groups, with an increase in the intervention group relative to usual care (adjusted incidence rate ratio 1.17, 95% confidence interval 0.99 to 1.38; p = 0.07). A similar proportion of participants in the intervention group (57.0%) and the usual-care group (56.2%) reported at least one fall over 12 months. There were no differences in any of the secondary outcomes. The base-case cost-effectiveness analysis from an NHS and Personal Social Services perspective found that, on average per participant, the intervention was associated with additional costs (£18.78, 95% confidence interval £16.33 to £21.24), but was less effective (mean quality-adjusted life-year loss -0.0042, 95% confidence interval -0.0041 to -0.0043). Sensitivity analyses demonstrated uncertainty in these findings. No serious, related adverse events were reported. The intervention was largely delivered as intended, but recommendations were followed to a varying degree.

LIMITATIONS

Outcome data were self-reported by participants, which may have led to inaccuracies in the reported falls data.

CONCLUSIONS

We found no evidence that an occupational therapist-delivered home assessment and modification reduced falls in this population of community-dwelling participants aged ≥ 65 years deemed at risk of falling. The intervention was more expensive and less effective than usual care, and therefore it does not provide a cost-effective alternative to usual care.

FUTURE WORK

An evaluation of falls prevention advice in a higher-risk population, perhaps those previously hospitalised for a fall, or given by other professional staff could be justified.

TRIAL REGISTRATION

Current Controlled Trials ISRCTN22202133.

FUNDING

This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 46. See the NIHR Journals Library website for further project information.