Independence of the statistician who analyses unblinded data

Factors influencing the statistical planning, design, conduct, analysis and reporting of trials in health care: A systematic review

Background

Trials in health care are prospective human research studies designed to test the effectiveness and safety of health care interventions, such as medications, surgeries, medical devices and other treatment or prevention interventions. Statistics is an important and powerful tool in trials. Inappropriately designed trials and/or inappropriate statistical analysis produce unreliable results and a lack of transparency when reported, with limited clinical use.

Aim

This systematic literature review aimed to identify, describe and synthesise factors contributing to or influencing the statistical planning, design, conduct, analysis and reporting of trials.

Methods

Information sources were retrieved from the following electronic citation databases: PubMed, Web of Science, PsycINFO, and CINAHL and the grey literature repository: OpenGrey. 90 articles and guidelines were included in this review. A narrative, thematic synthesis identified the key factors influencing the statistical planning, design, conduct, analysis and reporting of trials in health care.

Findings and conclusion

We identified three analytical themes within which factors are grouped. These are: “what makes a statistician?“, “the need for dynamic statistical involvement and collaboration throughout a trial – it's not just about the numbers”, “and the “accountability of statisticians in ensuring the safety of trial participants and the integrity of trial data”. While important insights emerged about the qualifications, training, roles, and responsibilities of statisticians and their collaboration with other team members in a trial, further empirical research is warranted to elicit the perceptions of trial team members at the centre of statistics in trials.

How to construct an optimal interim report: What the data monitoring committee does and doesn’t need to know

Background:

Data monitoring committees for randomized clinical trials have the responsibility of safeguarding interests of trial participants. To do so, the data monitoring committee must receive reports on safety and efficacy to assess risk/benefit and on trial conduct to ensure that the study can achieve its goals. This article outlines the key components of reports to the data monitoring committee and the important role of the unblinded statistician in preparing those reports.

Methods:

Most data monitoring committee meetings include open and closed sessions. For each session, there is a report of interim results. The open session is attended by the sponsor and lead investigators, including the statistician(s) responsible for the trial design. These investigators are blinded to the interim treatment comparisons. The closed session is attended by the data monitoring committee members and by the statistician(s) who prepared the closed report. These individuals are unblinded to interim treatment comparisons and therefore are not involved in study design changes. The optimal content of data monitoring committee reports and qualifications of the unblinded statistician(s) are discussed.

Reports:

Open reports should include responses to data monitoring committee recommendations, a synopsis of the protocol, a review of the protocol history and amendments, and information on enrollment, baseline characteristics, completeness of follow-up, and data quality. The open report is also a vehicle through which the sponsor and investigators should inform the data monitoring committee of relevant external information. Data in the open report are pooled over the treatment groups. The open report should not include data summaries by treatment group. The closed report should include a written summary with references to key tables and figures and methods used to prepare them. Tables and figures should summarize baseline characteristics, follow-up completeness, treatment adherence, and major safety and efficacy outcomes by treatment group. Text summaries should accompany the tables and figures. The data monitoring committee monitoring history (e.g. treatment differences at previous meetings) should be summarized. The unblinded statistician preparing the closed report should be familiar with the protocol and data collection plan and be capable of customizing the report to the current stage of the trial. This includes anticipating questions that may arise during the data monitoring committee review and pro-actively including data summaries to address these questions.

Conclusions:

There is considerable variation in the quality of open and closed data monitoring committee reports. Open and closed data monitoring committee reports should be concise, up to date, and informative. To achieve this, unblinded statisticians responsible for preparing closed data monitoring committee reports should be familiar with the statistical methods, the trial protocol, and the data collection plan. They should be capable of anticipating questions from the data monitoring committee and responding to requests for additional analyses.

Reports to Independent Data Monitoring Committees: An Appeal for Clarity, Completeness, and Comprehensibility

BACKGROUND

Organizations presenting reports to independent data monitoring committees (IDMCs) should present data in a way that facilitates the ability of the IDMC to make informed judgments about the trial.

METHODS

This paper reviews reports to IDMCs and suggests approaches an independent statistical reporting group (ISRG) might take to prepare clear, complete, and comprehensible reports.

RESULTS

Sensible reporting by an ISRG and informed decision making by an IDMC require a productive partnership between the quantitative and clinical disciplines involved in a clinical trial. IDMC reports differ in structure and purpose from clinical study reports that summarize data at the end of a trial. The ISRG must have intellectual independence, recognizing that although the sponsor may be paying the bills, the ISRG is responsible to the IDMC. Ideally, it should have access to all data from the trial and should be capable of responding to requests from the IDMC without the sponsor's specific permission. The ISRG and sponsor must understand the differences between clean data at the end of the trial and data collected during the trial. To perform its role most effectively, the ISRG must collaborate with sponsor and IDMC clinicians to become conversant with the disease area, the product's mechanism of action, and the clinical relevance of important outcome measures.

CONCLUSIONS

An IDMC is best served by an independent ISRG that will prepare clear, complete, and comprehensible reports. Given the complexities of interim data and IDMC requirements, the ISRG must be an active and informed participant in the monitoring process.

Sharing interim trial results by the Data Safety Monitoring Board with those responsible for the trial's conduct and progress: a narrative review

BACKGROUND

Sharing interim data, results or result extrapolations is an important issue that can affect trial integrity. The different ways in which Data Safety Monitoring Boards (DSMBs) share interim results with non-DSMB members and the acceptability of such practices are poorly understood. Our objective was to undertake a narrative review specifically on what kind of interim results, if any, should be shared by the DSMB with non-DSMB members and why.

METHODS

We conducted a narrative review using a systematic search strategy of several databases and major health research stakeholders. Literature was included if there was some discussion within the full text about sharing interim trial results with non-DSMB members.

RESULTS

About 79.6% (129/162) of included citations were based on author's views, 16.7% (27/162) on research guidelines and 3.7% (6/162) on surveys. The largest group of citations, 73/162 (45%), expresses the opinion or argument against sharing interim results with exceptions. Trailing closely, 71/162 (43.8%) of the included citations support the opinion or argument that interim results should not be shared and should remain confidential with the DSMB. Half of the six surveys support sharing in some capacity, while the other three do not. Eleven circumstances were found that potentially warrant interim result sharing by the DSMB; they relate to (1) usual practices by DSMBs, (2) trial completion threatened, (3) patient safety, (4) regulatory approval and (5) other circumstances. Dominant risks for sharing under these conditions are associated with introducing trial bias.

DISCUSSION/CONCLUSION

There was no majority view in the literature. However, the largest group of citations included express the idea that interim results should remain confidential with the DSMB but also acknowledge circumstances when they could be shared with non-DSMB members. Limitations of this review are that (1) the included literature predominately provides personal perspectives, not evidence, and (2) surveys found globally focus on trial monitoring practices lacking detailed information on what specifically to share, with whom and why. More research is needed with the use of a detailed survey of the clinical trial community focused on DSMB sharing interim results, to better understand and guide DSMB interim result sharing practices.

Data monitoring in randomized controlled trials: surveys of recent practice and policies

Background Data Monitoring Committees (DMCs) are increasingly involved in the conduct of randomized controlled trials, but there is little documented evidence of what they do. Three interlinked surveys were carried out as part of the DAMOCLES project to explore recent and current DMC practice and policy.

Methods 1) A questionnaire about DMC practice was sent to sample of 45 authors of trials published in selected journals in 2000. The sample was stratified by centre (single/multiple), disease area, and presence of DMC. 2) A sample of investigators in trials ongoing in the United Kingdom in 2001–02 was also sent a questionnaire about DMC practice. The sample was drawn from trials funded by the Medical Research Council, the United Kingdom Department of Health's Health Technology Assessment Programme, and a local and a multicentre research ethics committee. The sample was additionally stratified by funder (public/industry), centre (single/ multiple), and disease area. 3) A sample of major organisations involved in randomised controlled trials was sent a questionnaire about DMC policies.

Results Information about DMC practice from the first survey was obtained from 31 trials (69%), of which four had a DMC. Information about DMC practice from the second survey was obtained about 36 trials (90%), of which 20 had a DMC. Information about DMC policy from the third survey was obtained from 25 out of 25 organisations. There was general agreement about the sorts of trials particularly needing independent DMCs, but there were few uniform approaches to their modes of functioning, and few of the organisations surveyed had developed formal policies. Conclusions The roles of existing DMCs and policies governing DMC functioning vary widely across trials and organisations that sponsor or oversee trials, both within the UK and internationally. These findings reinforce previous calls for development of such policies across a wider range of organisations, better means to monitor their implementation within trials, and wider use of structured “charters”, which set out DMC modus operandi in advance.

Protecting clinical trial participants and protecting data integrity: are we meeting the challenges?

Susan Ellenberg discusses alternative approaches towards evaluating data as it accumulates in clinical trials, and to protecting the integrity and preventing undue risks to participants, as the trial continues.

Continuous safety monitoring for randomized controlled clinical trials with blinded treatment information. Part 1: Ethical considerations

The protection of patient safety is the principal responsibility of clinical trial investigators, and must be assured even if that were to prevent successful completion of a trial. Yet, the decision to prematurely stop a blinded, randomized controlled clinical trial can be extremely complicated, involving a tangle of ethical, statistical, and practical issues. Questions are quickly answered when conclusive evidence of harm has been established for trial participants, or when the potential for harm exceeds an acceptable limit of comfort for an oversight body. Less readily addressed are those situations in which early alarms warn of possible harm, but the data are too preliminary or incomplete to reach a satisfactory decision as to whether or not to stop the study. Early study termination without sufficient evidence disallows the study question from being answered and may allow an inferior treatment to remain in use, or prevent a superior one from being discovered. Even without early stopping, as a study proceeds, worrisome trends may lead to overzealous (or overly cautious) looks at study data which could jeopardize the integrity of the findings. Trial investigators and safety monitoring groups, aided by objective statistical rules and thoughtful deliberations, share responsibility for patient welfare. Statistical guidelines must not frustrate ethical concerns, but, rather, should be designed to promote the highest ethical and scientific outcomes possible, safeguarding both trial participants and the public - the ultimate beneficiaries of clinical trials.

Experiences and challenges in data monitoring for clinical trials within an international tropical disease research network

BACKGROUND

Models for the structure and procedures of data and safety monitoring boards (DSMBs) continue to evolve in response to issues of new and of old concern. Some authors have called for an open dialogue on these questions through publication of the experiences of DSMBs in addressing them.

PURPOSE

The goal of this paper is to add to the current discussion about acceptable models for establishing, serving on, and reporting to monitoring committees, particularly those that oversee multiple studies in less developed countries. The paper seeks to do so by describing the establishment and subsequent operation of one such multi-trial DSMB over a five-year period. This DSMB was formed to monitor trials conducted by members of the International Centers for Tropical Disease Research (ICTDR) network of the National Institute of Allergy and Infectious Diseases (NIAID).

METHODS

The operational model and experiences are summarized by the authors, who had immediate responsibilities for directing the DSMB's activities.

RESULTS

The board played an active, traditional role in assuring that patient safety was maintained and that current standards for clinical research were met. In addition, both NIAID and the board members viewed education of investigators to be an important role for the board to play in this particular setting. This affected the threshold for identifying which trials would be monitored, and it impacted several procedures adopted by the board.

LIMITATIONS

This report reflects the observations of those involved in managing the DSMB, including comments offered by the DSMB and by investigators, but not data gathered in a systematic way.

CONCLUSIONS

The operational model described here has allowed the DSMB to fulfill its role in the oversight of the trials. We hope that the ideas we present may help others facing similar situations and may stimulate further critical thinking about DSMB structure and function.