We need to move the debate on open clinical trial data forward

Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science

INTRODUCTION

To improve the rigor of science, experimental evidence for scientific claims ideally needs to be replicated repeatedly with comparable analyses and new data to increase the collective confidence in the veracity of those claims. Large replication projects in psychology and cancer biology have evaluated the replicability of their fields but no collaborative effort has been undertaken in sports and exercise science. We propose to undertake such an effort here. As this is the first large replication project in this field, there is no agreed-upon protocol for selecting studies to replicate. Criticism of previous selection protocols include claims they were non-randomised and non-representative. Any selection protocol in sports and exercise science must be representative to provide an accurate estimate of replicability of the field. Our aim is to produce a protocol for selecting studies to replicate for inclusion in a large replication project in sports and exercise science.

METHODS

The proposed selection protocol uses multiple inclusion and exclusion criteria for replication study selection, including: the year of publication and citation rankings, research disciplines, study types, the research question and key dependent variable, study methods and feasibility. Studies selected for replication will be stratified into pools based on instrumentation and expertise required, and will then be allocated to volunteer laboratories for replication. Replication outcomes will be assessed using a multiple inferential strategy and descriptive information will be reported regarding the final number of included and excluded studies, and original author responses to requests for raw data.

ReseArch with Patient and Public invOlvement: a RealisT evaluation – the RAPPORT study

Background

Patient and public involvement (PPI) is a prerequisite for many funding bodies and NHS research ethics approval. PPI in research is defined as research carried out with or by the public rather than to, about or for them. While the benefits of PPI have been widely discussed, there is a lack of evidence on the impact and outcomes of PPI in research.

Objectives

To determine the types of PPI in funded research, describe key processes, analyse the contextual and temporal dynamics of PPI and explore the experience of PPI in research for all those involved. Mechanisms contributing to the routine incorporation of PPI in the research process were assessed, the impact of PPI on research processes and outcomes evaluated, and barriers and enablers to effective PPI identified.

Design

A three-staged realist evaluation drawing on Normalisation Process Theory to understand how far PPI was embedded within health-care research in six areas: diabetes mellitus, arthritis, cystic fibrosis, dementia, public health and learning disabilities. The first two stages comprised a scoping exercise and online survey to chief investigators to assess current PPI activity. The third stage consisted of case studies tracked over 18 months through interviews and document analysis. The research was conducted in four regions of England.

Participants

Non-commercial studies currently running or completed within the previous 2 years eligible for adoption on the UK Clinical Research Network portfolio. A total of 129 case study participants included researchers and PPI representatives from 22 research studies, and representatives from funding bodies and PPI networks.

Results

In the scoping 51% (n = 92) of studies had evidence of PPI and in the survey 79% (n = 80), with funder requirements and study design the strongest influence on the extent of PPI. There was little transparency about PPI in publicly accessible information. In case studies, context–mechanism–outcome configurations suggested that six salient actions were required for effective PPI. These were a clear purpose, role and structure for PPI; ensuring diversity; whole research team engagement with PPI; mutual understanding and trust between the researchers and lay representatives; ensuring opportunities for PPI throughout the research process; and reflecting on, appraising and evaluating PPI within a research study. PPI models included a ‘one-off’ model with limited PPI, a fully intertwined model in which PPI was fully embedded and an outreach model with lay representatives linking to broader communities. Enabling contexts included funder, topic/design, resources, research host, organisation of PPI and, most importantly, relationships. In some case studies, lack of coherence in defining PPI persisted, with evidence of a dual role of PPI representative/study participant. Evidence of PPI outcomes included changes to study design, improvements to recruitment materials and rates, and dissemination.

Conclusions

Six salient actions were required for effective PPI and were characterised by a shared understanding of moral and methodological purposes of PPI, a key individual co-ordinating PPI, ensuring diversity, a research team positive about PPI input and fully engaged with it, based on relationships that were established and maintained over time, and PPI being evaluated in a proactive and systematic approach. Future work recommendations include exploring the impact of virtual PPI, cost analysis and economic evaluation of the different models of PPI, and a longer-term follow-up study of the outcomes of PPI on research findings and impact on services and clinical practice.

Funding

The National Institute for Health Research Health Services and Delivery Research programme.