Evaluation of patient engagement in medicine development: A multi-stakeholder framework with metrics

Patient engagement in designing, conducting, and disseminating clinical pain research: IMMPACT recommended considerations

ABSTRACT

In the traditional clinical research model, patients are typically involved only as participants. However, there has been a shift in recent years highlighting the value and contributions that patients bring as members of the research team, across the clinical research lifecycle. It is becoming increasingly evident that to develop research that is both meaningful to people who have the targeted condition and is feasible, there are important benefits of involving patients in the planning, conduct, and dissemination of research from its earliest stages. In fact, research funders and regulatory agencies are now explicitly encouraging, and sometimes requiring, that patients are engaged as partners in research. Although this approach has become commonplace in some fields of clinical research, it remains the exception in clinical pain research. As such, the Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials convened a meeting with patient partners and international representatives from academia, patient advocacy groups, government regulatory agencies, research funding organizations, academic journals, and the biopharmaceutical industry to develop consensus recommendations for advancing patient engagement in all stages of clinical pain research in an effective and purposeful manner. This article summarizes the results of this meeting and offers considerations for meaningful and authentic engagement of patient partners in clinical pain research, including recommendations for representation, timing, continuous engagement, measurement, reporting, and research dissemination.

Overview of Chatbots with special emphasis on artificial intelligence-enabled ChatGPT in medical science

The release of ChatGPT has initiated new thinking about AI-based Chatbot and its application and has drawn huge public attention worldwide. Researchers and doctors have started thinking about the promise and application of AI-related large language models in medicine during the past few months. Here, the comprehensive review highlighted the overview of Chatbot and ChatGPT and their current role in medicine. Firstly, the general idea of Chatbots, their evolution, architecture, and medical use are discussed. Secondly, ChatGPT is discussed with special emphasis of its application in medicine, architecture and training methods, medical diagnosis and treatment, research ethical issues, and a comparison of ChatGPT with other NLP models are illustrated. The article also discussed the limitations and prospects of ChatGPT. In the future, these large language models and ChatGPT will have immense promise in healthcare. However, more research is needed in this direction.

Systematic scoping review protocol of Stroke Patient and Stakeholder Engagement (SPSE)

This protocol describes a systematic scoping review of Stroke Patient and Stakeholder Engagement (SPSE), concepts, definitions, models, implementation strategies, indicators, or frameworks. The active engagement of patients and other stakeholders is increasingly acknowledged as essential to patient-centered research to answer questions of importance to patients and their caregivers. Stroke is a debilitating, long-lasting burden for individuals, their families, and healthcare professionals. They require rehabilitation services, health care system assistance, and social support. Their difficulties are unique and require the continued involvement of all parties involved. Understanding SPSE in research is fundamental to healthcare planning and extends the role of patients and stakeholders beyond that of the study subject. We will conduct a systematic literature search to identify the types of existing evidence related to SPSE, implementation strategies, indicators, or frameworks related to Patient and Stakeholder Engagement (PSE); clarify key concepts, definitions, and components of SPSE; compile experiences and prerequisites; and identify stroke research internationally. Two independent reviewers will extract data from selected studies onto a customized extraction form that has already been piloted. We integrate existing knowledge to address gaps in the literature on SPSE research by presenting the model, implementation strategies, indicators, and frameworks for stroke patients. We hope that these findings will offer future researchers a clear picture and conceptual model of SPSE.

Researchers' experiences with patient engagement in health research: a scoping review and thematic synthesis

CONTEXT

Implicating patients in research is gaining popularity around the world and is now the reference of many funding agencies. Understanding these partnerships is necessary to grasp this new reality. The experiences of researchers who have involved patient-partners (PPs) in health research are important for a better understanding of these practices.

OBJECTIVE

This study aimed to identify and analyze the existing qualitative scientific literature on the experiences of academic researchers involved in health research with patient engagement (PE).

DESIGN

A scoping review of the available literature with an inductive thematic synthesis, guided by the methodological framework of Arksey and O'Malley.

DATA COLLECTION

A search strategy was developed to include keywords relating to researchers, patient-partners, experiences, and the qualitative methodologies of the targeted studies. Five databases were searched using the EBSCO-host engine. The search results were screened by four reviewers to only include articles written in English on the topic of the experience of academic researchers having worked with PPs in health research based on qualitative studies or mixed-methods studies with a distinct qualitative section.

ANALYSIS

Articles included were charted for general information. All "results" sections were coded line by line. These codes were organized inductively to form descriptive and analytical themes. This led to the synthesis of the ideas found in the selected articles.

RESULTS

The search strategy yielded 7616 results, of which 2468 duplicates were removed. The remaining 5148 articles were screened, resulting in the exclusion of 5114 off-topic studies. The remaining 29 full-text articles were evaluated for inclusion from which 5 additional studies were identified. The final selection consisted of 11 articles that met all the criteria. These articles were published between 2009 and 2019. Five general themes inductively emerged from the analysis: the understanding of PE, motivations, contexts, attitudes, and practical aspects of PE that are central to researchers.

CONCLUSION

This scoping review provides a better understanding of the experiences of researchers who have implemented patient partnerships in health research projects. Our findings reveal many positive elements central to health researchers' discourses about PE, but they provide insights into the challenges and postures of resistance. This knowledge can support the development of empirically sound improvements in PE practices.

Using Patient Perspectives to Inform Better Clinical Trial Design and Conduct: Current Trends and Future Directions

The approach to patient engagement (PE) in drug development has changed rapidly due to many factors, including the complexity of innovative drugs and the need to demonstrate outcomes of relevance to patients, the desire to show 'value add' of PE, and the pandemic-related changes to how clinical trials are run, e.g., decentralised studies. In parallel, there have been changes in technology-assisted ways of running clinical trials, capturing patient health outcomes and preferences, an increasing societal demand for diversity and inclusion, and efforts to improve clinical trial efficiency, transparency, and accountability. Organisations are beginning to monitor PE activities and outcomes more effectively to learn and inform future PE strategies. As a result, these factors are facilitating the incorporation of patients' lived experience, preferences and needs into the design and running of clinical trials more than ever before. In this paper, the authors reflect upon these last few years, the emerging trends and their drivers, and where we may expect PE in clinical research to progress in the near future.

Measurable outcomes of consumer engagement in health research: A scoping review

Background

Consumer engagement is increasingly recognized as an instrumental component of health research, with many institutions and international bodies mandating it as part of the research and funding process. Given an increasing utilization of consumer engagement in health research, it is critical to identify the literature which support its value and tools that capture successful outcomes. To develop an overview of the literature, we conducted an umbrella scoping review exploring important outcomes of consumer engagement in health research combined with a scoping review of relevant frameworks. Specifically, we aimed to capture outcomes which reflect authentic and meaningful consumer engagement.

Methods

Four databases (PubMed, Embase, CINAHL and Cochrane Library) were searched using key search terms. Records were included if they were review articles or frameworks that addressed outcomes of consumer engagement in health research. Data was analyzed using descriptive statistics and an inductive qualitative content analysis. Identified outcomes were sorted based on the three most relevant stakeholder groups (consumer, researcher, institution).

Results

A total of twenty articles that explored a variety of health disciplines were included. We identified fifteen measurable outcomes of consumer engagement in health research. Eight core outcomes were relevant to all stakeholder groups, and were considered fundamental to authentic consumer engagement including (1) trust, (2) empowerment, (3) respect, (4) confidence in the outcomes of the research, (5) transparency of the research process, (6) satisfaction with the consumer engagement program, (7) knowledge and experiences of consumers, and (8) degree of consumer engagement. Outcomes pertaining to specific stakeholder groups included representativeness and diversity of the consumer groups, research relevance to consumers, funding opportunities, quality/validity of the research, recruitment/retention rates, translation and dissemination of research, and interpretation of results.

Conclusion

This review identified key measurable outcomes that could be captured when evaluating the impacts of consumer engagement on health research and the success of consumer engagement programs. All outcomes identified were relatively underexplored within the literature, and inadequately and/or inconsistently evaluated amongst studies. Future research should consult all stakeholder groups to identify outcomes perceived to be reflective of optimal consumer engagement.

Practices of patient engagement in drug development: a systematic scoping review

BACKGROUND

During the past decade, patient engagement (PE) has attracted significant attention in the field of drug development. Readiness to accept the central importance of patients' knowledge and contributions has become evident. This study aimed to synthesize evidence on the current state of PE in drug development: what is actually being done and how.

METHODS

A systematic scoping review was conducted based on a PRISMA-informed protocol. Search was performed in PubMed, EMBASE and Web of Science, covering the period between 2011 and 2021. For analysis of extracted data, we developed a framework for analyzing PE in Drug Development. The Framework distinguishes a number of different PE types that take place at different stages of drug development and are characterized by the different degrees of power patients have in the process. It allowed us to assess depth and intensity of PE initiatives included in this review.

RESULTS

Most included PE initiatives took place at the stage of designing studies (40 in total). At this stage drug development goals are already set, but the mode of reaching them has not yet been fully determined. PE initiatives on the finetuning details stage followed (16 in total). The finetuning details stage covers the last parts of the drug development trajectory, when only relatively minor issues are still open for patients' contributions. The least numerous were PE initiatives on the stage of setting up R&D program (13 in total). This stage refers to the early steps in drug development where PE has the potential to make the most impact on shaping the subsequent process. In terms of intensity of engagement, most PE initiatives included in this review align with consultation and involvement types, 26 and 30 initiatives, respectively. Partnership was less frequent in the published accounts of PE (13 initiatives).

CONCLUSIONS

This review delineated a contemporary landscape of PE in drug development. Although attention to PE in drug development is relatively recent, a wide range of PE practices has already been initiated. The results indicate the necessity of distinguishing between different types of PE in order to understand consequences of choices regarding depth and intensity of PE.

Evaluating the Impacts of Patient Engagement on Health Services Research Teams: Lessons from the Veteran Consulting Network

BACKGROUND

Despite increasing commitment to patient engagement in research, evaluation of the impact of these efforts on research processes, products, and teams is limited.

OBJECTIVE

To explore the impacts of engaging patients as consultants to research studies by examining the experiences, impacts, and lessons learned from a program facilitating patient engagement at a Veterans Health Administration research center.

DESIGN

We developed a logic model to articulate the activities being implemented to support patient engagement and their anticipated outcomes. Then, we conducted qualitative, semi-structured interviews with participants in the local Veteran Consulting Network to qualitatively explore these outcomes.

PARTICIPANTS

Twelve researchers and eleven Veteran patients with experience working on at least one grant or funded study.

APPROACH

Interview transcripts were inductively coded using a consensus-based approach. Findings were synthesized using framework analysis and mapped back onto our logic model of expected patient engagement impacts.

KEY RESULTS

Patient engagement improved the perceived quality and relevance of research studies as patient consultants challenged researchers' assumptions about patient populations and clinical contexts and gave feedback that helped improve the feasibility of proposed grants, readability of study materials, comprehensiveness of study assessments, and cultural sensitivity and relevance of interventions. Patient engagement also had personal benefits to researchers and patients. Researchers reported improved communication skills and higher job satisfaction. Patients reported a sense of purpose and satisfaction from their work with greater awareness of and appreciation for research.

CONCLUSIONS

Engaging patients in research can have multiple benefits to the people and work involved. Our evaluation process can serve as a template for other organizations to plan for and assess the impact of their own patient engagement programs. Creating logic models and updating them based on feedback from program users make engagement goals explicit, help verify expected mechanisms to achieve impact, and facilitate organizational learning.

Monitoring and Evaluation of Patient Engagement in Health Product Research and Development: Co-Creating a Framework for Community Advisory Boards

PURPOSE

While patient engagement is becoming more customary in developing health products, its monitoring and evaluation to understand processes and enhance impact are challenging. This article describes a patient engagement monitoring and evaluation (PEME) framework, co-created and tailored to the context of community advisory boards (CABs) for rare diseases in Europe. It can be used to stimulate learning and evaluate impacts of engagement activities.

METHODS

A participatory approach was used in which data collection and analysis were iterative. The process was based on the principles of interactive learning and action and guided by the PEME framework. Data were collected via document analysis, reflection sessions, a questionnaire, and a workshop.

RESULTS

The tailored framework consists of a theory of change model with metrics explaining how CABs can reach their objectives. Of 61 identified metrics, 17 metrics for monitoring the patient engagement process and short-term outcomes were selected, and a "menu" for evaluating long-term impacts was created. Example metrics include "Industry representatives' understanding of patients' unmet needs;" "Feeling of trust between stakeholders;" and "Feeling of preparedness." "Alignment of research programs with patients' needs" was the highest-ranked metric for long-term impact.

CONCLUSIONS

Findings suggest that process and short-term outcome metrics could be standardized across CABs, whereas long-term impact metrics may need to be tailored to the collaboration from a proposed menu. Accordingly, we recommend that others adapt and refine the PEME framework as appropriate. The next steps include implementing and testing the evaluation framework to stimulate learning and share impacts.

Institutionalizing Reflexivity for Sustainability: Two Cases in Health Care

Unsustainability in health care comprises diminishing returns and misalignment between the health care regime and the needs of the population. To deal with complex sustainability problems, niche solutions can be collaboratively designed and implemented through reflexive methods. For second-order sustainability, however, the institutionalization of the reflexive element itself is also needed. This paper aims to provide insight into the possibilities of embedding reflexivity into institutions to support second-order sustainability by reporting on two consecutive participatory research programs that sought to address unsustainability in terms of misalignment and diminishing returns. The first case study reflexively monitored the system’s innovation toward an integrated perinatal care system. Reflection within the project and implementation was supported successfully, but for stronger embedding and institutionalization, greater alignment of the reflexive practices with regime standards was needed. Building on these lessons, the second case study, which was part of the IMI-PARADIGM consortium, collaboratively built a structured tool to monitor and evaluate “the return on engagement” in medicine development. To institutionalize reflexivity, the creation of “reflexive standards” together with regime actors appears to be most promising. Broader and deeper institutionalization of reflexive standards can be attained by building enforcement structures for reflexive standards in the collaborative process as part of the reflexive methodologies for addressing complex sustainability problems.

Sustaining Meaningful Patient Engagement Across the Lifecycle of Medicines: A Roadmap for Action

BACKGROUND

There is increased recognition that incorporating patients' perspectives and insights into the medicines development process results in better health outcomes and benefits for all involved stakeholders. Despite the increased interest and the existence of frameworks and practical recommendations, patient engagement (PE) is not yet considered standard practice. The objective of this work was to provide a roadmap to support systematic change in all stakeholder organisations involved in medicines development across Europe, patients and patient organisations, medicines developers, academia, regulatory authorities, Health Technology Assessment bodies, payers, policy-makers and public research funders, to sustain PE practices.

METHODS

A mixed-methods approach was used by the EU-funded Innovative Medicines Initiative PARADIGM Consortium to co-develop the sustainability roadmap including background work to identify success factors and scenarios for sustainable PE. The roadmap development was based on the Theory of Change concept and populated with findings from (1) interviews with national/ and international institutions with the potential to increase PE uptake by other stakeholders; (2) multi-stakeholder workshops and webinars; and (3) consultations with specific stakeholder groups, Consortium members and a consultative body formed by international PE initiatives.

RESULTS

This roadmap sets strategic goals for the PE community to achieve meaningful and systematic PE through changes in the culture, processes and resources of stakeholder organisations. It brings in key PARADIGM outputs to work in a coordinated fashion with existing frameworks and mechanisms to achieve system-wide sustained PE.

CONCLUSIONS

The roadmap provides a framework for all stakeholders to take collective action within their organisations and across Europe to implement PE in a sustainable manner.

Evaluation of patient engagement in medicine development: A multi-stakeholder framework with metrics

BACKGROUND

Patient engagement is becoming more customary in medicine development. However, embedding it in organizational decision-making remains challenging, partly due to lack of agreement on its value and the means to evaluate it. The objective of this project was to develop a monitoring and evaluation framework, with metrics, to demonstrate impact and enhance learning.

METHODS

A consortium of five patient groups, 15 biopharmaceutical companies and two academic groups iteratively created a framework in a multi-phase participatory process, including analysis of its application in 24 cases.

RESULTS

The framework includes six components, with 87 metrics and 15 context factors distributed among (sub)components: (a) Input: expectations, preparations, resources, representativeness of stakeholders; (b) Activities/process: structure, management, interactions, satisfaction; (c) Learnings and changes; (d) Impacts: research relevance, study ethics and inclusiveness, study quality and efficiency, quality of evidence and uptake of products, empowerment, reputation and trust, embedding of patient engagement; (e) Context: policy, institutional, community, decision-making contextual factors. Case study findings show a wide variation in use of metrics. There is no 'one size fits all' set of metrics appropriate for every initiative or organization. Presented sample sets of metrics can be tailored to individual situations.

CONCLUSION

Introducing change into any process is best done when the value of that change is clear. This framework allows participants to select what metrics they value and assess to what extent patient engagement has contributed.

PATIENT CONTRIBUTION

Five patient groups were involved in all phases of the study (design, conduct, interpretation of data) and in writing the manuscript.