"We measure what we can measure": Struggles in defining and evaluating institutional review board quality

Resources to aid ethical review of clinical studies: an exploratory scoping review identifying gaps and opportunities

BACKGROUND

Research Ethics Committees (RECs) review the ethical, legal, and methodological standards of clinical research. Complying with all requirements and professional expectations while maintaining the necessary scientific and ethical standards can be challenging for applicants and members of the REC alike. There is a need for accessible guidelines and resources to help medical researchers and REC members navigate the legal and ethical requirements and the process of their review.

METHODS

We employed an explorative search for resources on the websites of a purposively selected sample of relevant stakeholders, including 12 national umbrella organizations (six German-language and six English-language), three English-language international umbrella organizations, and 16 national RECs of major university hospitals (eight German- and eight English-language). We qualitatively mapped the identified resources onto the guiding principles of ethical clinical research and 35 related checkpoints. To describe the content of the resources, we conducted a thematic analysis.

RESULTS

We extracted a total of 233 resources, including templates (n = 134, 58.5%), guidelines/recommendations (n = 62, 26.6%), checklists (n = 23, 9.9%), tools (n = 5, 2.2%), flowcharts (n = 5, 2.2%), glossaries (n = 3, 1.3%), and one (0.4%) software program. We extracted 101 German and 132 English resources created between 2004 and 2023. The majority (n = 204; 87.6%) could be assigned to one checkpoint. The remaining 29 (12.5%) resources were considered unspecific (e.g., a checklist which documents to be submitted for a German drug trial). The specific resources are discussed per checkpoint.

CONCLUSION

While much support is available for some aspects, such as participant information and informed consent forms, it is lacking in other areas, such as study design, analysis, and biometrics. More support should be provided in these areas to ensure that research projects are methodologically sound. A more detailed analysis of the quality of available resources could help identify other areas of need.

A conceptual model and practical guidance for the development, administration, and evaluation of individualized therapies

Bespoke therapies represent a promising tool to address a diverse range of genetic and acquired conditions, offering new hope where conventional treatments have fallen short. With the rapid rise of bespoke therapies, profound ethical and regulatory challenges emerge, making it crucial to establish a comprehensive framework that ensures these treatments reach clinical settings and meet patients' needs as quickly as possible while protecting all parties involved. Although current guidelines are continually evolving to address the range of ethical tensions raised by these therapies, several gaps remain. A significant unresolved question is determining where personalized interventions fall on the research-care continuum and understanding the institutional, regulatory, and ethical implications when custom therapies are classified as research, care, or a mix of both. To address these questions, we introduce a conceptual model alongside practical guidance for the development, administration, and evaluation of individualized therapies, using CRISPR/Cas9-based interventions for Duchenne Muscular Dystrophy as a case study. We argue that the goals of an intervention should be as individualized as the bespoke product itself, tailored to the specifics of each case. Rather than attempting to pinpoint the exact location of an intervention on the continuum, which may be hard to operationalize and have limited utility, our approach focuses on the practical details of how such interventions are administered and the individual component parts of an intervention. It advocates for transparent discussions among all partners to anticipate and adjust various components/parameters along the process of administering individualized interventions. Our paper highlights the most critical of these parameters in (1) the planning and development of individualized therapies in laboratory settings, (2) their regulatory oversight, and (3) evaluation. By discussing these stages and parameters in detail, we aim to provide guidance on how to navigate the ethical complexities inherent to individualized interventions and offer a preliminary framework for balancing the interplay between research objectives and patient care needs. Acknowledging that the scientific rigor and adequacy of any new model must be evaluated, we also identify the types of evidence that are required to validate that our model effectively meets individual and societal needs.

Inclusive, engaged, and accountable institutional review boards

In February 2023, the U.S. Government Accountability Office (GAO) released another report acknowledging that we still lack meaningful, validated, widely-accepted measures for evaluating institutional review board (IRB) quality and effectiveness. This challenge is well known to the Consortium to Advance Effective Research Ethics Oversight (www.AEREO.org), a collaborative group of human research protection (HRP) professionals, researchers, and research ethicists founded in 2018 to do precisely what GAO recommends: examine approaches for measuring IRB effectiveness in protecting human subjects, and implement the approaches as appropriate. Two underlying tenets have been central to AEREO's as approach to thinking about IRB quality and effectiveness: (1) IRBs exist to protect participants and thus the participant perspective should be central to all IRBs do; and (2) because IRBs are tasked with applying subjective ethical and regulatory standards about which people may disagree, their approach and decisions should at least meet the basic standard of reasonableness in terms of accounting for relevant perspectives, considering key factors, and providing defensible justifications. Critical to each of these tenets, IRBs should include diverse perspectives in their deliberations, find ways to meaningfully engage with relevant communities about their views regarding ethical research and appropriate participant protections, and be accountable to the public.

Research ethics and collaborative research in health and social care: Analysis of UK research ethics policies, scoping review of the literature, and focus group study

Current research ethics frameworks were developed on the footprint of biomedical, experimental research and present several pitfalls when applied to non-experimental social sciences. This work explores how the normative principles underpinning policy and regulatory frameworks of research ethics and the related operational processes work in practice in the context of collaborative health and social care research. The work was organised in three phases. First, UK research ethics policy documents were analysed thematically, with themes further organised under the categories of 'Principles' and 'Processes'. Next, we conducted a scoping review of articles about research ethics in the context of collaborative health and social care research, published in English between 2010 and 2022. We then held an exploratory focus group with ten academic researchers with relevant experience to gather their views on how the research ethics system works in practice in England (UK). The thematic framework developed in the first phase supported the analysis of the articles included in the scoping review and of focus group data. The analysis of policy documents identified twelve themes. All were associated to both a principle and a related operational process. The scoping review identified 31 articles. Across these, some themes were barely acknowledged (e.g., Compliance with legislation). Other themes were extensively covered (e.g., The working of Research Ethics Committees), often to discuss issues and limitations in how, in practice, the research ethics system and its processes deal with collaborative research and to suggest options for improvement. Focus group data were largely consistent with the findings of the scoping review. This work provides evidence of the poor alignment between how the research ethics system is normatively expected to work and how it works in practice and offers options that could make research ethics more fit for purpose when addressing collaborative research in health and social care.

How Do Accredited Organizations Evaluate the Quality and Effectiveness of Their Human Research Protection Programs?

BACKGROUND

Meaningfully evaluating the quality of institutional review boards (IRBs) and human research protection programs (HRPPs) is a long-recognized challenge. To be accredited by the Association for the Accreditation of Human Research Protection Programs (AAHRPP), organizations must demonstrate that they measure and improve HRPP "quality, effectiveness, and efficiency" (QEE). We sought to learn how AAHRPP-accredited organizations interpret and satisfy this standard, in order to assess strengths, weaknesses, and gaps in current approaches and to inform recommendations for improvement.

METHODS

We conducted 3 small-group interviews with a total of 19 participant representatives of accredited organizations at the 2019 AAHRPP annual meeting. Participants were eligible if they had familiarity with their organization's approach to satisfying the relevant QEE standard.

RESULTS

Participants reported lacking clear definitions for HRPP quality or effectiveness but described various approaches to assessing QEE, typically focused on turnaround time, compliance, and researcher satisfaction. Evaluation of IRB members was described as relatively superficial and information regarding research subject experience was not reported as central to QEE assessment, although participants described several efforts to improve consideration of patient, subject, and community perspectives in IRB review. Participants also described efforts to educate and build relationships with key stakeholders as important features of a high-quality HRPP. While generally satisfied with their approaches, participants expressed concern about resource and time constraints that pushed them to be reactive and automatic about QEE, rather than proactive and critical.

CONCLUSIONS

The relevant AAHRPP accreditation standard may obscure critical gaps in defining and measuring QEE elements. We recommend that AAHRPP: (1) offer a definition of QEE or require accredited organizations to provide their own, to help clarify the rationale and goals behind assessment and improvement efforts, and (2) require accredited organizations to establish QEE objectives and measures focused on participant outcomes and deliberative quality during protocol review.

Research ethics in a changing social sciences landscape

The role of research ethics committees, and research ethics issues more broadly are often not viewed in the context of the development of scientific methods and the academic community. This topic piece seeks to redress this gap. I begin with a brief outline of the changes we experience within the social sciences before exploring in more detail their impact on research ethics and the practices of research ethics committees. I conclude with recommendations for how the existing research ethics processes may be made more future-proof.

Engaging key stakeholders to overcome barriers to studying the quality of research ethics oversight

The primary purpose of Institutional Review Boards (IRBs) is to protect the rights and welfare of human research participants. Evaluation and measurement of how IRBs satisfy this purpose and other important goals are open questions that demand empirical research. Research on IRBs, and the Human Research Protection Programs (HRPPs) of which they are often a part, is necessary to inform evidence-based practices, policies, and approaches to quality improvement in human research protections. However, to date, HRPP and IRB engagement in empirical research about their own activities and performance has been limited. To promote engagement of HRPPs and IRBs in self-reflective research on HRPP and IRB quality and effectiveness, barriers to their participation need to be addressed. These include: extensive workloads, limited information technology systems, and few universally accepted or consistently measured metrics for HRPP/IRB quality and effectiveness. Additionally, institutional leaders may have concerns about confidentiality. Professional norms around the value of participating in this type of research are lacking. Lastly, obtaining external funding for research on IRBs and HRPPs is challenging. As a group of HRPP professionals and researchers actively involved in a research consortium focused on IRB quality and effectiveness, we identify potential levers for supporting and encouraging HRPP and IRB engagement in research on quality and effectiveness. We maintain that this research should be informed by the core principles of patient- and community-engaged research, in which members and key stakeholders of the community to be studied are included as key informants and members of the research team. This ensures that relevant questions are asked and that data are interpreted to produce meaningful recommendations. As such, we offer several ways to increase the participation of HRPP professionals in research as participants, as data sharers, and as investigators.