Patient and public involvement in numerical aspects of trials (PoINT): exploring patient and public partners experiences and identifying stakeholder priorities

Patient and public involvement and engagement in the development of innovative patient-centric early phase dose-finding trial designs

BACKGROUND

In light of the FDA's Project Optimus initiative, there is fresh interest in leveraging Patient-reported Outcome (PRO) data to enhance the assessment of tolerability for investigational therapies within early phase dose-finding oncology trials. Typically, dose escalation in most trial designs is solely reliant on clinician assessed adverse events. Research has shown a disparity between patients and clinicians when assessing whether an investigational therapy is tolerable, leading to the recommendation of potentially intolerable doses for further investigation in subsequent trials. It is also increasingly recognized that patient and public involvement and engagement (PPIE) plays a pivotal role in enriching trial design and conduct. However, to our knowledge, no PPIE has explored the optimal integration of PROs in the development of advanced statistical trial designs within early phase dose-finding oncology trials.

METHODS

A virtual PPIE session was held with nine participants on 18th October 2023 to discuss the incorporation of PROs within a dose-finding trial design. This cross disciplinary session was developed and led by a team of statisticians, clinical specialists, qualitative experts, and trial methodologists. Following the session, in-depth perspectives were provided by two patient advocates who actively engaged in the PPIE session. We discuss the importance of PPIE in shaping advanced dose-finding trial designs, share insights from patients on integrating PROs to inform treatment tolerability, and present a template for meaningful patient involvement in trial design development.

RESULTS

Participants generally supported the introduction of PROs within dose-finding trials but showed some apprehensiveness as to how PROs may reduce the size of the recommended dose (and potentially efficacious effect). Some participants shared that they may be reluctant to record the real severity of their symptoms via PROs if it would mean that they would have to discontinue treatment. They discussed that PROs could be used to assess tolerability rather than toxicity of a dose.

CONCLUSIONS

Amplifying patient voice in the development of patient-centric dose-finding trial designs is now essential. This paper offers an exemplary illustration of how trialists and methodologists can effectively incorporate patient voice in the future development of advanced dose-finding trial designs.

Public involvement in chronic respiratory diseases research: A qualitative study of patients', carers' and citizens' perspectives

INTRODUCTION

Patient and public involvement (PPI) initiatives involving patients with chronic respiratory disease (CRD) are rare. Therefore, this study aimed to explore the perspectives of patients with CRD, carers and interested citizens regarding the relevance and need for a PPI network and suggestions for its implementation.

METHODS

A qualitative study based on focus groups was conducted. Recruitment occurred through invitations on social media platforms and to patients who have participated in previous asthma studies of the team. Three focus groups were conducted, via video conference, using a semi-structured guide. Thematic analysis was performed by two independent researchers and discussed with the extended team.

RESULTS

Fifteen patients with CRD, one carer and one interested citizen (13 females, median 36 (range: 18-72) years) participated. All participants acknowledged the importance of implementing a collaborative network and demonstrated interest in being integrated. Participants acknowledged the importance of their involvement in several phases of the research cycle. The main aim identified for this network was to facilitate communication between patients and researchers. Participants regarded the integration of patients, carers, researchers and healthcare professionals from different scientific areas as relevant. The use of digital platforms to attract members and support the work, together with group dynamics and regular meetings, were some of the most relevant practical considerations for implementing the network. The identified facilitators for their engagement were sharing experiences, researchers' and healthcare professionals' support and feedback and schedule flexibility. The identified barriers included the amount of time dedicated, low health/digital literacy and the potential detachment of nondiagnosed patients or those with low symptom impact in daily life.

CONCLUSION

Patients, carers and citizens acknowledged the relevance of implementing a collaborative network and demonstrated interest in active participation in every stage of the health research cycle. A deeper knowledge of the barriers and facilitators identified in this study could support implementing these initiatives in Portugal.

PATIENT OR PUBLIC CONTRIBUTION

This study was designed by a research team that included one patient with asthma and one carer. They were specifically involved in building the study protocol and the interview guide. They also gave feedback regarding the electronic consent form and the short sociodemographic questionnaire created, namely by removing noncontributing words or phrases and rewording expressions. The lay summary was written by another patient with asthma. All participants of this study were invited to implement and integrate the ConectAR network-a collaborative network of research in respiratory health.

PUBLIC SUMMARY

In Portugal, chronic respiratory patients do not have an active role as 'coinvestigators'. This study aimed to acknowledge if patients and citizens considered a patient and public involvement network useful, whose main purpose would be to facilitate communication between patients and researchers. A study based on online group interviews was carried out with patients with chronic respiratory diseases and interested citizens, both recruited on social media platforms. Participants considered that bringing together patients, carers, researchers and healthcare professionals is valuable because sharing different experiences and perspectives may help patients to improve their daily lives and increase research quality. In conclusion, patients agree that implementing a collaborative network with researchers and healthcare professionals and participating in the health research cycle is quite preponderant. Acknowledging what can help and deter this network may be beneficial to implementing this type of initiative in Portugal.

Evaluating participant experiences of Community Panels to scrutinise policy modelling for health inequalities: the SIPHER Consortium

Data-intensive research, including policy modelling, poses some distinctive challenges for efforts to mainstream public involvement into health research. There is a need for learning about how to design and deliver involvement for these types of research which are highly technical, and where researchers are at a distance from the people whose lives data depicts. This article describes our experiences involving members of the public in the SIPHER Consortium, a data-intensive policy modelling programme with researchers and policymakers working together over five years to try to address health inequalities. We focus on evaluating people's experiences as part of Community Panels for SIPHER. Key issues familiar from general public involvement efforts include practical details, careful facilitation of meetings, and payment for participants. We also describe some of the more particular learning around how to communicate technical research to non-academic audiences, in order to enable public scrutiny of research decisions. We conclude that public involvement in policy modelling can be meaningful and enjoyable, but that it needs to be carefully organised, and properly resourced.

Involving patients and the public In sTatistIcal Analysis pLans (INITIAL): A delphi survey

BACKGROUND

Patient and public involvement (PPI) in trials aims to enhance research by improving its relevance and transparency. Planning for statistical analysis begins at the design stage of a trial within the protocol and is refined and detailed in a Statistical Analysis Plan (SAP). While PPI is common in design and protocol development it is less common within SAPs. This study aimed to reach consensus on the most important and relevant statistical analysis items within an SAP to involve patients and the public.

METHODS

We developed a UK-based, two-round Delphi survey through an iterative consultation with public partners, statisticians, and trialists. The consultation process started with 55 items from international guidance for statistical analysis plans. We aimed to recruit at least 20 participants per key stakeholder group for inclusion in the final analysis of the Delphi survey. Participants were asked to vote on each item using a Likert scale from 1 to 9, where a rating of 1 to 3 was labelled as having 'limited importance'; 4 to 6 as 'important but not critical' and 7 to 9 as 'critical' to involve patients and the public. Results from the second round determined consensus on critical items for PPI.

RESULTS

The consultation exercise led to the inclusion of 15 statistical items in the Delphi survey. We recruited 179 participants, of whom 72% (129: 36 statisticians, 29 patients or public partners, 25 clinical researchers or methodologists, 27 trial managers, and 12 PPI coordinators) completed both rounds. Participants were on average 48 years old, 60% were female, 84% were White, 64% were based in England and 84% had at least five years' experience in trials. Four items reached consensus regarding critical importance for patient and public involvement: presentation of results to trial participants; summary and presentation of harms; interpretation and presentation of findings in an academic setting; factors impacting how well a treatment works. No consensus was reached for the remaining 11 items. In general, the results were consistent across stakeholder groups.

DISCUSSION

We identified four critical items to involve patients and the public in statistical analysis plans. The remaining 11 items did not reach consensus and need to be considered in a case-by-case basis with most responders considering patient and public involvement important (but not critical). Our research provides a platform to enable focused future efforts to improve patient and public involvement in trials and enhance the relevance of statistical analyses to patients and the public.

Barriers and facilitators of meaningful patient participation at the collective level in healthcare organizations: A systematic review

INTRODUCTION

Collective patient participation, such as patient participation in policy making, has become increasingly important to achieve high-quality care. However, there is little knowledge on how to let patients participate in a meaningful manner at this level. The aim of this systematic literature review was to provide an overview of barriers, facilitators, and associated impact of collective patient participation.

METHODS

PubMed and EMBASE were searched until May 2023 for studies that evaluated collective patient participation. Study characteristics, methods for patient participation, barriers and facilitators, and impact (if measured) of patient participation were extracted from the articles.

RESULTS

We included 59 articles. Identified barriers and facilitators of collective patient participation were grouped into five categories: (1) preconditions for patient participation, (2) strategy for patient participation, (3) preparation of patients and staff for patient participation, (4) support for patients and staff during patient participation, and (5) evaluation of patient participation. Impact of patient participation was reported in 34 included studies at three levels: quality of care and research, the team and organization, and the participants themselves. Only three studies reported quantitative outcomes.

CONCLUSION

Interestingly, similar challenges were experienced during a period of twenty years, indicating that little progress has been made in structuring patient participation. Our overview of barriers and facilitators will therefore help to improve and structure collective patient participation.

Incorporation of patient and public involvement in statistical methodology research: a survey assessing current practices and attitudes of researchers

BACKGROUND

Patient and public involvement (PPI) ensures that research is designed and conducted in a manner that is most beneficial to the individuals whom it will impact. It has an undisputed place in applied research and is required by many funding bodies. However, PPI in statistical methodology research is more challenging and work is needed to identify where and how patients and the public can meaningfully input in this area.

METHODS

A descriptive cross-sectional research study was conducted using an online questionnaire, which asked statistical methodologists about themselves and their experience conducting PPI, either to inform a grant application or during a funded statistical methodology project. The survey included both closed-text responses, which were reported using summary statistics, and open-ended questions for which common themes were identified.

RESULTS

119 complete responses were recorded. Individuals who completed the survey displayed an even range of ages, career lengths and positions, with the majority working in academia. 40.3% of participants reported undertaking PPI to inform a grant application and the majority reported that the inclusion of PPI was received positively by the funder. Only 21.0% of participants reported undertaking PPI during a methodological project. 31.0% of individuals thought that PPI was "very" or "extremely" relevant to statistical methodology research, with 45.5% responding "somewhat" and 24.4% answering "not at all" or "not very". Arguments for including PPI were that it can provide the motivation for research and shape the research question. Negative opinions included that it is too technical for the public to understand, so they cannot have a meaningful impact.

CONCLUSIONS

This survey found that the views of statistical methodologists on the inclusion of PPI in their research are varied, with some individuals having particularly strong opinions, both positive and negative. Whilst this is clearly a divisive topic, one commonly identified theme was that many researchers are willing to try and incorporate meaningful PPI into their research but would feel more confident if they had access to resources such as specialised training, guidelines, and case studies.

Patient and public involvement in pragmatic trials: online survey of corresponding authors of published trials

BACKGROUND

There are few data on patient and public involvement (PPI) in pragmatic trials. We aimed to describe the prevalence and nature of PPI within pragmatic trials, describe variation in prevalence of PPI by trial characteristics and compare prevalence of PPI reported by trial authors to that reported in trial publications.

METHODS

We applied a search filter to identify pragmatic trials published from 2014 to 2019 in MEDLINE. We invited the corresponding authors of pragmatic trials to participate in an online survey about their specific trial.

RESULTS

Of 3163 authors invited, 2585 invitations were delivered, 710 (27.5%) reported on 710 unique trials and completed the survey; 334 (47.0%) conducted PPI. Among those who conducted PPI, for many the aim was to increase the research relevance (86.3%) or quality (76.5%). Most PPI partners were engaged at protocol development stages (79.1%) and contributed to the co-design of interventions (70.9%) or recruitment or retention strategies (60.5%). Patient and public involvement was more common among trials involving children, trials conducted in the United Kingdom, cluster randomized trials, those explicitly labelled as "pragmatic" in the study manuscript, and more recent trials. Less than one-quarter of trials (22.8%) that reported PPI in the survey also reported PPI in the trial manuscript.

INTERPRETATION

Nearly half of trialists in this survey reported conducting PPI and listed several benefits of doing so, but researchers who did not conduct PPI often cited a lack of requirement for it. Patient and public involvement appears to be significantly underreported in trial publications. Consistent and standardized reporting is needed to promote transparency about PPI methods, outcomes, challenges and benefits.

Starting a conversation about estimands with public partners involved in clinical trials: a co-developed tool

BACKGROUND

Clinical trials aim to draw conclusions about the effects of treatments, but a trial can address many different potential questions. For example, does the treatment work well for patients who take it as prescribed? Or does it work regardless of whether patients take it exactly as prescribed? Since different questions can lead to different conclusions on treatment benefit, it is important to clearly understand what treatment effect a trial aims to investigate-this is called the 'estimand'. Using estimands helps to ensure trials are designed and analysed to answer the questions of interest to different stakeholders, including patients and public. However, there is uncertainty about whether patients and public would like to be involved in defining estimands and how to do so. Public partners are patients and/or members of the public who are part of, or advise, the research team. We aimed to (i) co-develop a tool with public partners that helps explain what an estimand is and (ii) explore public partner's perspectives on the importance of discussing estimands during trial design.

METHODS

An online consultation meeting was held with 5 public partners of mixed age, gender and ethnicities, from various regions of the UK. Public partner opinions were collected and a practical tool describing estimands, drafted before the meeting by the research team, was developed. Afterwards, the tool was refined, and additional feedback sought via email.

RESULTS

Public partners want to be involved in estimand discussions. They found an introductory tool, to be presented and described to them by a researcher, helpful for starting a discussion about estimands in a trial design context. They recommended storytelling, analogies and visual aids within the tool. Four topics related to public partners' involvement in defining estimands were identified: (i) the importance of addressing questions that are relevant to patients and public in trials, (ii) involving public partners early on, (iii) a need for education and communication for all stakeholders and (iv) public partners and researchers working together.

CONCLUSIONS

We co-developed a tool for researchers and public partners to use to facilitate the involvement of public partners in estimand discussions.

How do we know a treatment is good enough? A survey of non-inferiority trials

BACKGROUND

Non-inferiority and equivalence trials aim to determine whether a new treatment is good enough (non-inferior) or as good as (equivalent to) another treatment. To inform the decision about non-inferiority or equivalence, a margin is used. We aimed to identify the current methods used to determine non-inferiority or equivalence margins, as well as the main challenges and suggestions from trialists.

METHODS

We developed an online questionnaire that included both closed and open-ended questions about methods to elicit non-inferiority or equivalence margins, underlying principles, and challenges and suggestions for improvement. We recruited trialists with experience of determining a margin by contacting corresponding authors for non-inferiority or equivalence trials. We used descriptive statistics and content analysis to identify categories in qualitative data.

RESULTS

We had forty-one responses, all from non-inferiority trials. More than half of the trials were non-pharmacological (n = 21, 51%), and the most common primary outcome was clinical (n = 29, 71%). The two most used methods to determine the margin were as follows: a review of the evidence base (n = 27, 66%) and opinion seeking methods (n = 24, 59%). From those using reviews, the majority used systematic reviews or reviews of multiple RCTs to determine the margin (n = 17, 63%). From those using opinion seeking methods, the majority involved clinicians with or without other professionals (n = 19, 79%). Respondents reported that patients' opinions on the margin were sought in four trials (16%). Median confidence in overall quality of the margin was 5 out of 7 (maximum confidence); however, around a quarter of the respondents were "completely unconfident" that the margin reflected patient's views. We identified "stakeholder involvement" as the most common category to determine respondent's confidence in the quality of the margins and whether it reflected stakeholder's views. The most common suggestion to improve the definition of margins was "development of methods to involve stakeholders," and the most common challenge identified was "communication of margins."

CONCLUSIONS

Responders highlighted the need for clearer guidelines on defining a margin, more and better stakeholder involvement in its selection, and better communication tools that enable discussions about non-inferiority trials with stakeholders. Future research should focus on developing best practice recommendations.

Reporting of and explanations for under-recruitment and over-recruitment in pragmatic trials: a secondary analysis of a database of primary trial reports published from 2014 to 2019

OBJECTIVES

To describe the extent to which pragmatic trials underachieved or overachieved their target sample sizes, examine explanations and identify characteristics associated with under-recruitment and over-recruitment.

STUDY DESIGN AND SETTING

Secondary analysis of an existing database of primary trial reports published during 2014-2019, registered in ClinicalTrials.gov, self-labelled as pragmatic and with target and achieved sample sizes available.

RESULTS

Of 372 eligible trials, the prevalence of under-recruitment (achieving <90% of target sample size) was 71 (19.1%) and of over-recruitment (>110% of target) was 87 (23.4%). Under-recruiting trials commonly acknowledged that they did not achieve their targets (51, 71.8%), with the majority providing an explanation, but only 11 (12.6%) over-recruiting trials acknowledged recruitment excess. The prevalence of under-recruitment in individually randomised versus cluster randomised trials was 41 (17.0%) and 30 (22.9%), respectively; prevalence of over-recruitment was 39 (16.2%) vs 48 (36.7%), respectively. Overall, 101 025 participants were recruited to trials that did not achieve at least 90% of their target sample size. When considering trials with over-recruitment, the total number of participants recruited in excess of the target was a median (Q1-Q3) 319 (75-1478) per trial for an overall total of 555 309 more participants than targeted. In multinomial logistic regression, cluster randomisation and lower journal impact factor were significantly associated with both under-recruitment and over-recruitment, while using exclusively routinely collected data and educational/behavioural interventions were significantly associated with over-recruitment; we were unable to detect significant associations with obtaining consent, publication year, country of recruitment or public engagement.

CONCLUSIONS

A clear explanation for under-recruitment or over-recruitment in pragmatic trials should be provided to encourage transparency in research, and to inform recruitment to future trials with comparable designs. The issues and ethical implications of over-recruitment should be more widely recognised by trialists, particularly when designing cluster randomised trials.

Patient-reported outcomes and target effect sizes in pragmatic randomized trials in ClinicalTrials.gov: A cross-sectional analysis

BACKGROUND

Use of patient-reported outcomes (PROs) and patient and public engagement are critical ingredients of pragmatic trials, which are intended to be patient centered. Engagement of patients and members of the public in selecting the primary trial outcome and determining the target difference can better ensure that the trial is designed to inform the decisions of those who ultimately stand to benefit. However, to the best of our knowledge, the use and reporting of PROs and patient and public engagement in pragmatic trials have not been described. The objectives of this study were to review a sample of pragmatic trials to describe (1) the prevalence of reporting patient and public engagement; (2) the prevalence and types of PROs used; (3) how its use varies across trial characteristics; and (4) how sample sizes and target differences are determined for trials with primary PROs.

METHODS AND FINDINGS

This was a methodological review of primary reports of pragmatic trials. We used a published electronic search filter in MEDLINE to identify pragmatic trials, published in English between January 1, 2014 and April 3, 2019; we identified the subset that were registered in ClinicalTrials.gov and explicitly labeled as pragmatic. Trial descriptors were downloaded from ClinicalTrials.gov; information about PROs and sample size calculations were extracted from the manuscript. Chi-squared, Cochran-Armitage, and Wilcoxon rank sum tests were used to examine associations between trial characteristics and use of PROs. Among 4,337 identified primary trial reports, 1,988 were registered in CT.gov, of which 415 were explicitly labeled as pragmatic. Use of patient and public engagement was identified in 39 (9.4%). PROs were measured in 235 (56.6%): 144 (34.7%) used PROs as primary outcomes and 91 (21.9%) as only secondary outcomes. Primary PROs were symptoms (64; 44%), health behaviors (36; 25.0%), quality of life (17; 11.8%), functional status (16; 11.1%), and patient experience (10; 6.9%). Trial characteristics with lower prevalence of use of PROs included being conducted exclusively in children or adults over age 65 years, cluster randomization, recruitment in low- and middle-income countries, and primary purpose of prevention; trials conducted in Europe had the highest prevalence of PROs. For the 144 trials with a primary PRO, 117 (81.3%) reported a sample size calculation for that outcome; of these, 71 (60.7%) justified the choice of target difference, most commonly, using estimates from pilot studies (31; 26.5%), standardized effect sizes (20; 17.1%), or evidence reviews (16; 13.7%); patient or stakeholder opinions were used to justify the target difference in 8 (6.8%). Limitations of this study are the need for trials to be registered in ClinicalTrials.gov, which may have reduced generalizability, and extracting information only from the primary trial report.

CONCLUSIONS

In this study, we observed that pragmatic trials rarely report patient and public engagement and do not commonly use PROs as primary outcomes. When provided, target differences are often not justified and rarely informed by patients and stakeholders. Research funders, scientific journals, and institutions should support trialists to incorporate patient engagement to fulfill the mandate of pragmatic trials to be patient centered.