A Framework for Assessing Scientific Merit in Ethical Review of Clinical Research

Scientific review of protocols to enhance informativeness of global health clinical trials

BACKGROUND

Trial informativeness describes the likelihood of a clinical trial to have a meaningful impact on clinical practice, research, or policy decisions. A dedicated scientific review process for protocols at the post-funding stage is not common, yet is an opportunity to enhance trial informativeness. The Bill and Melinda Gates Foundation (BMGF), one of the largest funders of clinical trials, created a group called Design, Analyze, Communicate (DAC). DAC's first completion of an expert scientific review of a grantee's trial protocol was in 2020. We categorized and quantified areas of scientific review feedback provided for 52 clinical trial protocols submitted to DAC over a 3-year period. Most trials planned to study treatment interventions and included at least one trial site in a low- and middle-income country. Feedback themes offer insight into areas of trial design weakness.

METHODS

We conducted a retrospective analysis of protocol review feedback provided by DAC to grantees. Reviews were completed by BMGF between 2020 and 2022. A qualitative content analysis was conducted by developing a codebook of clinical trial methodology topics and subtopics and systematically coding free-text review feedback. Manual text classification of individual feedback statements enabled quantification and frequency analysis of review feedback.

RESULTS

A total of 1537 individual recommendations were made across all 52 protocols. The median number of recommendations per protocol was 28 (range: 13 to 52), covering a wide range of issues related to clinical trial design, implementation, analysis, and impact. Nearly half of all recommendations (47%) were characterized by the review team as high priority. The areas with the highest frequency of recommendations were statistics and data analysis, trial procedures, and intervention/dose.

CONCLUSIONS

This study provides a taxonomy of scientific review feedback topic areas that can be used to categorize clinical trial design topics. The high number of recommendations per protocol review across several distinct topic areas highlights the need for a scientific review to enhance trial informativeness. This review must take place prior to trial initiation and review teams should include statistical and trial design expertise with additional expertise tailored to the trial/intervention type and phase.

Flying Blind: How Thorough are IRBs when Assessing Scientific Value?

BACKGROUND

Institutional Review Boards (IRBs) in the United States play a crucial role in ensuring the ethical conduct of clinical trials, including assessing the scientific merit of studies to justify the risks to participants. However, prior research suggests that many IRBs do not systematically evaluate scientific merit, raising concerns about the approval of low-quality trials.

OBJECTIVE

To investigate whether IRBs provide adequate guidance on assessing scientific merit in their Standard Operating Procedures (SOPs) and other relevant materials.

DESIGN

A systematic pilot investigation of IRB SOPs and related guidance documents from a sample of U.S.-based non-profit institutions.

PARTICIPANTS

IRB materials from 35 U.S.-based non-profit institutions selected from the FDA's Bioresearch Monitoring Information System database, representing 39.9% of submissions between 2018 and 2021. Additionally, materials from one U.S.-based for-profit IRB were included.

INTERVENTIONS

Not applicable.

MAIN MEASURES

The presence of guidance on 15 dimensions of scientific merit, clustered into four PICO (Population, Intervention, Comparator, Outcome) categories, was assessed by reviewing IRB SOPs and related documents. Evidence of guidance was determined by mention of keywords related to each dimension.

KEY RESULTS

Most IRB materials mentioned basic study elements such as study design (99%), subject recruitment (90%), and intervention justification (97%). However, critical aspects related to study quality, such as bias reduction (53%) and outcome measurement tools (57%), were less frequently mentioned. The least represented dimension was confounder control (10%).

CONCLUSIONS

IRB guidance materials vary in their coverage of scientific merit dimensions, with significant gaps in areas critical for assessing study quality. Strengthening guidance materials by including comprehensive instructions for all 15 dimensions could improve IRB assessments of scientific merit, thereby enhancing the ethical oversight of clinical trials.

Assessing the acceptability of individual studies that use deception: A systematic review of normative guidance documents

Research participants are often deceived for methodological reasons. However, assessing the ethical acceptability of an individual study that uses deception is not straightforward. The academic literature is scattered on the subject and several aspects of the acceptability assessment are only scarcely addressed, which parallels reports of inconsistent ethics review. Therefore, we aimed to investigate where normative guidance documents agree and disagree about this assessment. A PRISMA-Ethics-guided systematic review of normative guidance documents that discuss deception of research participants was conducted. Our search strategy resulted in 55 documents that were subsequently analyzed through abductive thematic analysis. While guidance documents mention little about specific risks and opportunities of deception, our analysis describes a rich picture of the thresholds for acceptability of the risks and benefits of deception and their integration, the comparison with the risk-benefit analysis of alternative non-deceptive methods, and the bodies of people who are positioned to do the review. Our review reveals an agreement on the general process of assessing the acceptability of studies that use deception, although significant variability remains in the details and several topics are largely or completely unaddressed in guidance documents.

The Social Value Misconception in Clinical Research

Clinical researchers should help respect the autonomy and promote the well-being of prospective study participants by helping them make voluntary, informed decisions about enrollment. However, participants often exhibit poor understanding of important information about clinical research. Bioethicists have given special attention to "misconceptions" about clinical research that can compromise participants' decision-making, most notably the "therapeutic misconception." These misconceptions typically involve false beliefs about a study's purpose, or risks or potential benefits for participants. In this article, we describe a misconception involving false beliefs about a study's potential benefits for non-participants, or its expected social value. This social value misconception can compromise altruistically motivated participants' decision-making, potentially threatening their autonomy and well-being. We show how the social value misconception raises ethical concerns for inherently low-value research, hyped research, and even ordinary research, and advocate for empirical and normative work to help understand and counteract this misconception's potential negative impacts on participants.

Should Children Be Included in Human Challenge Studies?

Human challenge studies, in which human research subjects are intentionally exposed to pathogens to contribute to scientific knowledge, raise many ethical complexities. One controversial question is whether it is ethically permissible to include children as participants. Commentary of the past decades endorses the exclusion of children, while new guidance suggests that pediatric human challenge studies can be ethically permissible. This paper argues that neither children's exclusion nor their inclusion are well justified. I examine and reject three arguments for exclusion, but suggest that these arguments establish pediatric human challenge studies as a complex ethical category of research that requires caution. I then argue for a strong presumption against children's inclusion, by drawing on an analogy to children's inclusion in phase I trials, emphasizing a requirement of necessity, and suggesting that accommodating children's vulnerability promotes an age de-escalation approach for pediatric human challenge studies research. In the final section, I suggest a procedure for ethics review.

A maturity model for the scientific review of clinical trial designs and their informativeness

BACKGROUND

Informativeness, in the context of clinical trials, defines whether a study's results definitively answer its research questions with meaningful next steps. Many clinical trials end uninformatively. Clinical trial protocols are required to go through reviews in regulatory and ethical domains: areas that focus on specifics outside of trial design, biostatistics, and research methods. Private foundations and government funders rarely require focused scientific design reviews for these areas. There are no documented standards and processes, or even best practices, toward a capability for funders to perform scientific design reviews after their peer review process prior to a funding commitment.

MAIN BODY

Considering the investment in and standardization of ethical and regulatory reviews, and the prevalence of studies never finishing or failing to provide definitive results, it may be that scientific reviews of trial designs with a focus on informativeness offer the best chance for improved outcomes and return-on-investment in clinical trials. A maturity model is a helpful tool for knowledge transfer to help grow capabilities in a new area or for those looking to perform a self-assessment in an existing area. Such a model is offered for scientific design reviews of clinical trial protocols. This maturity model includes 11 process areas and 5 maturity levels. Each of the 55 process area levels is populated with descriptions on a continuum toward an optimal state to improve trial protocols in the areas of risk of failure or uninformativeness.

CONCLUSION

This tool allows for prescriptive guidance on next investments to improve attributes of post-funding reviews of trials, with a focus on informativeness. Traditional pre-funding peer review has limited capacity for trial design review, especially for detailed biostatistical and methodological review. Select non-industry funders have begun to explore or invest in post-funding review programs of grantee protocols, based on exemplars of such programs. Funders with a desire to meet fiduciary responsibilities and mission goals can use the described model to enhance efforts supporting trial participant commitment and faster cures.

"Over-reviewing" of research? An analysis of orthodontic reviews

INTRODUCTION

Research overviews may be undertaken to identify gaps in the literature, evaluate existing systematic reviews (SRs), and summarize evidence. This paper aims to profile overviews that have been conducted in orthodontics and related interventions since 2012 and to evaluate the degree of overlap among these overviews.

METHODS

Overviews published between January 1, 2012 and June 20, 2023 were identified using an electronic search involving Google Scholar and PubMed. A descriptive summary was produced, and citation matrices were used to evaluate the percentage of overlap between overviews using corrected covered area and covered area. This was classified as slight, moderate, high, or very high.

RESULTS

A total of 35 overviews were identified across a wide range of topics. Eight overviews included <10 SRs; 21 had 10-20 SRs; and 6 included >20 SRs (median no. of SRs per overview, 15; range, 3-62). Meta-analysis was conducted in only 5 overviews. Overlap between overviews on the same topic ranged from slight (2.7%) to very high (53.8%).

CONCLUSIONS

Almost all overview topics address treatments and their effects, with a wide variation in the number and quality of SRs included. There is considerable overlap in some orthodontic overviews, suggesting unnecessary duplication and research waste. Researchers should be encouraged to focus on primary data collection to add more high-quality data to SRs, which will ultimately enhance the yield from secondary and tertiary orthodontic research.

BrainSwarming, blockchain, and bioethics: applying Innovation Enhancing Techniques to healthcare and research

Innovation in healthcare and biomedicine is in decline, yet there exist no widely-known alternatives to traditional brainstorming that can be employed for innovative idea generation. McCaffrey's Innovation Enhancing Techniques (IETs) were developed to enhance creative problem-solving by helping the solver to overcome common psychological obstacles to generating innovative ideas. These techniques were devised for engineering and design problems, which involve solving practical goals using physical materials. Healthcare and science problems however often involve solving abstract goals using intangible resources. Here we adapt two of McCaffrey's IETs, BrainSwarming and the Generic Parts Technique, to effectively enhance idea generation for such problems. To demonstrate their potential, we apply these techniques to a case study involving the use of blockchain technologies to facilitate ethical goals in biomedicine, and successfully identify 100 potential solutions to this problem. Being simple to understand and easy to implement, these and other IETs have significant potential to improve innovation and idea generation in healthcare, scientific, and technological contexts. By catalysing idea generation in problem-solving, these techniques may be used to target the innovative stagnation currently facing the scientific world.

Ensuring the Scientific Value and Feasibility of Clinical Trials: A Qualitative Interview Study

BACKGROUND

Ethical and scientific principles require that clinical trials address an important question and have the resources needed to complete the study. However, there are no clear standards for review that would ensure that these principles are upheld.

METHODS

We conducted semi-structured interviews with a convenience sample of nineteen experts in clinical trial design, conduct, and/or oversight to elucidate current practice and identify areas of need with respect to ensuring the scientific value and feasibility of clinical trials prior to initiation and while ongoing. We used a priori and grounded theory to analyze the data and constant comparative method to induce higher order themes.

RESULTS

Interviewees perceived determination of scientific value as the responsibility of the investigator and, secondarily, other parties who review or oversee research. Interviewees reported that ongoing trials are rarely reevaluated due to emerging evidence from external sources, evaluation is complex, and there would be value in the development of standards for monitoring and evaluating evidence systematically. Investigators, IRBs, and/or data monitoring committees (DMCs) could undertake these responsibilities. Feasibility assessments are performed but are typically inadequate; potential solutions are unclear.

CONCLUSIONS

There are three domains where current approaches are suboptimal and in which further guidance is needed. First, who has the responsibility for conducting scientific review, whether it be the investigator, IRB, and/or DMC is often unclear. Second, the standards for scientific review (e.g., appropriate search terms, data sources, and analytic plan) should be defined. Third, guidance is needed on the evaluation of ongoing studies in light of potentially new and evolving evidence, with particular reference to evidence from outside the trial itself.

Ethical Considerations in Clinical Trials for Disorders of Consciousness

As the clinical trial landscape for patients with disorders of consciousness (DoC) expands, consideration of associated ethical challenges and opportunities is of ever-increasing importance. Responsible conduct of research in the vulnerable population of persons with DoC, including those with coma, vegetative state/unresponsive wakefulness syndrome (VS/UWS), minimally conscious state (MCS), covert cortical processing (CCP), and cognitive motor dissociation (CMD), demands proactive deliberation of unique ethical issues that may arise and the adoption of robust protections to safeguard patients, surrogates, and other key stakeholders. Here we identify and critically evaluate four central categories of ethical considerations in clinical trials involving participants with DoC: (1) autonomy, respect for persons and informed consent of individuals with liminal consciousness; (2) balancing unknown benefits and risks, especially considering the epistemological gap between behavior and consciousness that complicates ordinary ascription of subjective states; (3) disclosure to surrogates and clinical teams of investigational results pertaining to consciousness; and (4) justice considerations, including equitable access to clinical trial enrollment across communities and geographies. We outline guiding principles and research opportunities for clinicians, neuroethicists, and researchers engaged in DoC clinical trials to advance ethical study design and deployment in this complex yet crucial area of investigation.

Demystifying the Institutional Review Board

Perioperative nurses are responsible for generating and using evidence to improve patient care. To protect human participants during research activities, government regulations enacted after widely publicized research misconduct specify that institutions receiving federal funds must have an institutional review board (IRB) comprising at least five members. Board members have many responsibilities, including completing a thorough review of each section of the application and attachments (eg, consent documents). To expedite the IRB approval processes, applicants should create and submit a well-written application. Applicants should understand that the application addresses the important ethical concepts of respect for persons, beneficence, and justice. Quality improvement activities (ie, local activities that seek to improve patient care or clinical outcomes) differ from research activities that focus on creating new knowledge. Depending on the purpose, design, and generalizability of a quality improvement project, the applicant may need to submit the project to the IRB for approval.

Do we really know how many clinical trials are conducted ethically? Why research ethics committee review practices need to be strengthened and initial steps we could take to strengthen them

Research Ethics Committees (RECs) play a critical gatekeeping role in clinical trials. This role is meant to ensure that only those trials that meet certain ethical thresholds proceed through their gate. Two of these thresholds are that the potential benefits of trials are reasonable in relation to risks and that trials are capable of producing a requisite amount of social value. While one ought not expect perfect execution by RECs of their gatekeeping role, one should expect routine success in it. This article reviews a range of evidence showing that substantial numbers of ethically tainted trials are receiving REC approvals. Many of the trials are early phase trials that evidence shows have benefits that may not be reasonable compared with their risks and many others are later trials that evidence shows may lack sufficient social value. The evidence pertains to such matters as methodologically inadequate preclinical studies incapable of supporting the inferences that REC members must make about the prospects for potential benefit needed to offset the risks in early phase trials and sponsorship bias that can cause improperly designed, conducted, analysed and reported later phase trials. The analysis of the evidence makes clear that REC practices need to be strengthened if they are to adequately fulfil their gatekeeping role. The article also explores options that RECs could use in order to improve their gatekeeping function.

Ethics review of big data research: What should stay and what should be reformed?

BACKGROUND

Ethics review is the process of assessing the ethics of research involving humans. The Ethics Review Committee (ERC) is the key oversight mechanism designated to ensure ethics review. Whether or not this governance mechanism is still fit for purpose in the data-driven research context remains a debated issue among research ethics experts.

MAIN TEXT

In this article, we seek to address this issue in a twofold manner. First, we review the strengths and weaknesses of ERCs in ensuring ethical oversight. Second, we map these strengths and weaknesses onto specific challenges raised by big data research. We distinguish two categories of potential weakness. The first category concerns persistent weaknesses, i.e., those which are not specific to big data research, but may be exacerbated by it. The second category concerns novel weaknesses, i.e., those which are created by and inherent to big data projects. Within this second category, we further distinguish between purview weaknesses related to the ERC's scope (e.g., how big data projects may evade ERC review) and functional weaknesses, related to the ERC's way of operating. Based on this analysis, we propose reforms aimed at improving the oversight capacity of ERCs in the era of big data science.

CONCLUSIONS

We believe the oversight mechanism could benefit from these reforms because they will help to overcome data-intensive research challenges and consequently benefit research at large.

A CTSA One Health Alliance guidance on institutional review of veterinary clinical studies

Harmonized institutional processes and reviewer training are vital to maintain integrity and ethical rigor of the veterinary clinical research pipeline and are a prerequisite to future work that might establish centralized or single-site ethical and regulatory review to ease initiation of multi-center studies. Funded by a CTSA One Health Alliance (COHA) pilot award, a diverse working group of veterinary clinicians and institutional representatives was convened in February 2020 to develop a guidance document detailing broadly agreed upon practices for ethical review and approval of veterinary clinical studies conducted in the United States.The working group defined key areas of need for consensus, developed a set of associated guidelines, and circulated these for review by COHA's fifteen member institutions. Six focus areas were identified by the working group and included vital items of protocol review, composition of the review committee, post-approval monitoring and adverse event reporting, consideration of special circumstances such as satellite sites and the use of healthy veterinary subjects in research, and the informed consent process.This document outlines a broadly agreed-upon framework through which to approach vital items associated with veterinary clinical study protocol review and approval. These approaches represent current best practice in the review and approval of veterinary clinical studies, and can serve as a guidance for veterinary clinician-scientists and regulatory experts, to ensure robust and ethically conducted studies that can contribute to the advancement of both animal and human health.

Reporting of prior clinical studies in Investigator's Brochures did not adhere to the basic principles of evidence synthesis: a cross-sectional study

OBJECTIVES

International regulations require Investigator's Brochures (IBs) to compile all available evidence that inform the risk-benefit assessment for the newly planned clinical trial. This study examined the adherence of IBs to the basic principles of evidence synthesis when compiling prior clinical studies.

STUDY DESIGN AND SETTING

For 97 IBs for phase I/II trials reviewed at one German research ethics committee we assessed the reporting on search, appraisal, and synthesis procedures for prior clinical studies. For a random subsample of 30 IBs, we evaluated the quality of reporting of the compiled 247 prior clinical studies.

RESULTS

Only 2% of all 97 IBs reported a comprehensive search strategy, provided a critical appraisal of the compiled prior clinical studies or presented respective study results in a structured manner. For the 247 prior clinical studies compiled in 30 IBs, the information required to appraise their risk of bias (eg, sample size calculation or baseline characteristics) was rarely reported.

CONCLUSION

When compiling all available evidence supporting the rationale for the proposed clinical study IBs do not acknowledge the broadly established principles for reviewing and reporting evidence. This may impact negatively on the trustworthiness and efficiency of risk-benefit assessment.

Evaluating the Quality of Research Ethics Review and Oversight: A Systematic Analysis of Quality Assessment Instruments

BACKGROUND

Research ethics review committees (RERCs) and Human Research Protection Programs (HRPPs) are responsible for protecting the rights and welfare of research participants while avoiding unnecessary inhibition of valuable research. Evaluating RERC/HRPP quality is vital to determining whether they are achieving these goals effectively and efficiently, as well as what adjustments might be necessary. Various tools, standards, and accreditation mechanisms have been developed in the United States and internationally to measure and promote RERC/HRPP quality.

METHODS

We systematically reviewed 10 quality assessment instruments, examining their overall approaches, factors considered relevant to quality, how they compare to each other, and what they leave out. For each tool, we counted the number of times each of 34 topics (divided into structure, process, and outcome categories) was mentioned. We generated lists of which topics are most and least mentioned for each tool, which are most prevalent across tools, and which are left unmentioned. We also conducted content analysis for the 10 most common topics.

RESULTS

We found wide variability between instruments, common emphasis on process and structure with little attention to participant outcomes, and failure to identify clear priorities for assessment. The most frequently mentioned topics are Review Type, IRB Member Expertise, Training and Educational Resources, Protocol Maintenance, Record Keeping, and Mission, Approach, and Culture. Participant Outcomes is unmentioned in 8 tools; the remaining 2 tools include assessments based on adverse events, failures of informed consent, and consideration of participant experiences.

CONCLUSIONS

Our analysis confirms that RERC/HRPP quality assessment instruments largely rely on surrogate measures of participant protection. To prioritize between these measures and preserve limited resources for evaluating the most important criteria, we recommend that instruments focus on elements relevant to participant outcomes, robust board deliberation, and procedures most likely to address participant risks. Validation of these approaches remains an essential next step.

Considerations and methods for placebo controls in surgical trials (ASPIRE guidelines)

Placebo comparisons are increasingly being considered for randomised trials assessing the efficacy of surgical interventions. The aim of this Review is to provide a summary of knowledge on placebo controls in surgical trials. A placebo control is a complex type of comparison group in the surgical setting and, although powerful, presents many challenges. This Review outlines what a placebo control entails and present understanding of this tool in the context of surgery. We consider when placebo controls in surgery are acceptable (and when they are desirable) in terms of ethical arguments and regulatory requirements, how a placebo control should be designed, how to identify and mitigate risk for participants in these trials, and how such trials should be done and interpreted. Use of placebo controls is justified in randomised controlled trials of surgical interventions provided there is a strong scientific and ethical rationale. Surgical placebos might be most appropriate when there is poor evidence for the efficacy of the procedure and a justified concern that results of a trial would be associated with high risk of bias, particularly because of the placebo effect. Feasibility work is recommended to optimise the design and implementation of randomised controlled trials. This Review forms an outline for best practice and provides guidance, in the form of the Applying Surgical Placebo in Randomised Evaluations (known as ASPIRE) checklist, for those considering the use of a placebo control in a surgical randomised controlled trial.

The ethics of psychedelic research in disorders of consciousness

This article provides an ethical analysis of psychedelic research involving disorders of consciousness patients. We apply two internationally accepted approaches for analyzing the ethics of human research, the Value-Validity Framework and Component Analysis, to a research program recently proposed by Scott and Carhart-Harris. We focus on Scott and Carhart-Harris's proposal, but the ethical frameworks outlined are applicable to other novel research protocols in the science of consciousness.

Challenges in the ethics review process of clinical scientific research projects in China

Purpose

Ethics review processes have become increasingly complex. The objective of this study was to explore the challenges currently faced in ethics reviews of clinical scientific research projects in China, with the goal of standardizing the structure of medical ethics committees and better protecting the rights and interests of research participants.

Methods

We reviewed and comprehensively analyzed the available literature discussing standardized ethics reviews of clinical scientific research projects.

Results

We identified the following problems: incomplete legislation, absence of supervision, vague review criteria, limitations of ethics committee competence, inadequate ethics consciousness, and poor tracking of reviews. In this paper, we suggest strategies for the development of future ethical reviews of clinical scientific research projects.

Conclusion

To standardize the ethics review process of clinical scientific research projects in China, it is necessary to establish relevant laws and regulations and implement supervisory responsibilities. Professional training of medical ethics committees is suggested as an effective way to improve the quality of ethics reviews.