Principles and procedures for data and safety monitoring in pragmatic clinical trials

Evaluating the Safety of an Educational Deprescribing Intervention: Lessons from the Optimize Trial

BACKGROUND

Patients, family members, and clinicians express concerns about potential adverse drug withdrawal events (ADWEs) following medication discontinuation or fears of upsetting a stable medical equilibrium as key barriers to deprescribing. Currently, there are limited methods to pragmatically assess the safety of deprescribing and ascertain ADWEs. We report the methods and results of safety monitoring for the OPTIMIZE trial of deprescribing education for patients, family members, and clinicians.

METHODS

This was a pragmatic cluster randomized trial with multivariable Poisson regression comparing outcome rates between study arms. We conducted clinical record review and adjudication of sampled records to assess potential causal relationships between medication discontinuation and outcomes. This study included adults aged 65+ with dementia or mild cognitive impairment, one or more additional chronic conditions, and prescribed 5+ chronic medications. The intervention included an educational brochure on deprescribing that was mailed to patients prior to primary care visits, a clinician notification about individual brochure mailings, and an educational tip sheets was provided monthly to primary care clinicians. The outcomes of the safety monitoring were rates of hospitalizations and mortality during the 4 months following brochure mailings and results of record review and adjudication. The adjudication process was conducted throughout the trial and included classifications: likely, possibly, and unlikely.

RESULTS

There was a total of 3012 (1433 intervention and 1579 control) participants. There were 420 total hospitalizations involving 269 (18.8%) people in the intervention versus 517 total hospitalizations involving 317 (20.1%) people in the control groups. Adjusted risk ratios comparing intervention to control groups were 0.92 [95% confidence interval (CI) 0.72, 1.16] for hospitalization and 1.19 (95% CI 0.67, 2.11) for mortality. Both groups had zero deaths "likely" attributed to a medication change prior to the event. A total of 3 out of 30 (10%) intervention group hospitalizations and 7 out of 35 (20%) control group hospitalizations were considered "likely" due to a medication change.

CONCLUSIONS

Population-based deprescribing education is safe in the older adult population with cognitive impairment in our study. Pragmatic methods for safety monitoring are needed to further inform deprescribing interventions.

TRIAL REGISTRATION

NCT03984396. Registered on 13 June 2019.

Guidelines for Data and Safety Monitoring in Pragmatic Randomized Clinical Trials Using Case Studies

Pragmatic randomized clinical trials (pRCTs) have a unique set of considerations for data and safety monitoring. Because of their unconventional trial designs coupled with collection of multilevel data and implementation outcomes in real-world settings, thoughtful consideration is needed on the presentation of the trial design and accruing data to facilitate review and decision-making by the trial's data and safety monitoring board (DSMB). To our knowledge, there is limited information available in practical guidelines for generalists and medical general practitioners on what to monitor and to report to the DSMB during the conduct of pRCTs and what the DSMB should focus on in its review of reports. This article discusses these matters in the context of 3 case studies focusing on a set of critical data and safety monitoring questions that would be of interest to the generalist conducting pRCTs. In considering these questions, we provide tabular and graphical illustrations of how data can be presented to the DSMB while drawing attention to those areas that the DSMB should focus on in its review of the trial. The strategies and viewpoints discussed herein provide practical guidelines and can serve as a resource for the generalist conducting pRCTs.

Comparing the Safety Action Feedback and Engagement (SAFE) Loop with an established incident reporting system: Study protocol for a pragmatic cluster randomized controlled trial

Background

Incident reporting is widely used in hospitals to improve patient safety, but current reporting systems do not function optimally. The utility of incident reports is limited because hospital staff may not know what to report, may fear retaliation, and may doubt whether administrators will review reports and respond effectively.

Methods

This is a clustered randomized controlled trial of the Safety Action Feedback and Engagement (SAFE) Loop, an intervention designed to transform hospital incident reporting systems into effective tools for improving patient safety. The SAFE Loop has six key attributes: obtaining nurses' input about which safety problems to prioritize on their unit; focusing on learning about selected high-priority events; training nurses to write more informative event reports; prompting nurses to report high-priority events; integrating information about events from multiple sources; and providing feedback to nurses on findings and mitigation plans. The study will focus on medication errors and randomize 20 nursing units at a large academic/community hospital in Los Angeles. Outcomes include: (1) incident reporting practices (rates of high-priority reports, contributing factors described in reports), (2) nurses' attitudes toward incident reporting, and (3) rates of high-priority events. Quantitative analyses will compare changes in outcomes pre- and post-implementation between the intervention and control nursing units, and qualitative analyses will explore nurses' experiences with implementation.

Conclusion

If effective, SAFE Loop will have several benefits: increasing nurses' engagement with reporting, producing more informative reports, enabling safety leaders to understand problems, designing system-based solutions more effectively, and lowering rates of high-priority patient safety events.

Reporting Ethical Procedures in Suicide Prevention Research: Current Status and Recommendations

OBJECTIVE

Ethical concerns frequently arise in suicide prevention research regarding participant safety and confidentiality. Despite a substantial literature on managing and navigating ethical concerns in suicide research, little attention has been paid to the reporting of ethical procedures. Furthermore, standard procedures for reporting ethical risk management procedures have not been developed.

METHOD

A review of the current literature was performed to examine the current state of reporting of ethical procedures within suicide research. Articles published in 2020 (N = 263) from three suicide-focused publications were screened and then coded (n = 131) to identify reporting of procedures for the ethical conduct of research and suicide risk management steps taken by the research teams.

RESULTS

The majority of articles reported ethical review or approval (84.7%) and reported the use of an informed consent process (77.9%). Only 28.2% included risk mitigation procedures. Of those 29.7% of those articles reported conducting risk evaluation, 66.7% reported resource dissemination, and 51.4% reported an intervention.

CONCLUSION

As empirical support for brief interventions accrues, suicide prevention researchers should consider establishing standards for the reporting of procedures to ensure the safety of participants with suicidal risk.HighlightsReporting suicide safety protocols helps ensure high ethical standards in research.Fewer than 1/3 of articles reviewed reported risk mitigation procedures in 2020.Standard procedures for reporting safety protocols in suicide research are needed.

The evolving role of data & safety monitoring boards for real-world clinical trials

Introduction

Clinical trials provide the "gold standard" evidence for advancing the practice of medicine, even as they evolve to integrate real-world data sources. Modern clinical trials are increasingly incorporating real-world data sources - data not intended for research and often collected in free-living contexts. We refer to trials that incorporate real-world data sources as real-world trials. Such trials may have the potential to enhance the generalizability of findings, facilitate pragmatic study designs, and evaluate real-world effectiveness. However, key differences in the design, conduct, and implementation of real-world vs traditional trials have ramifications in data management that can threaten their desired rigor.

Methods

Three examples of real-world trials that leverage different types of data sources - wearables, medical devices, and electronic health records are described. Key insights applicable to all three trials in their relationship to Data and Safety Monitoring Boards (DSMBs) are derived.

Results

Insight and recommendations are given on four topic areas: A. Charge of the DSMB; B. Composition of the DSMB; C. Pre-launch Activities; and D. Post-launch Activities. We recommend stronger and additional focus on data integrity.

Conclusions

Clinical trials can benefit from incorporating real-world data sources, potentially increasing the generalizability of findings and overall trial scale and efficiency. The data, however, present a level of informatic complexity that relies heavily on a robust data science infrastructure. The nature of monitoring the data and safety must evolve to adapt to new trial scenarios to protect the rigor of clinical trials.

Characterizing modifications to a comparative effectiveness research study: the OPTIMIZE trial-using the Framework for Reporting Adaptations and Modifications to Evidence-based Interventions (FRAME)

BACKGROUND

The OPTIMIZE trial is a multi-site, comparative effectiveness research (CER) study that uses a Sequential Multiple Assessment Randomized Trial (SMART) designed to examine the effectiveness of complex health interventions (cognitive behavioral therapy, physical therapy, and mindfulness) for adults with chronic low back pain. Modifications are anticipated when implementing complex interventions in CER. Disruptions due to COVID have created unanticipated challenges also requiring modifications. Recent methodologic standards for CER studies emphasize that fully characterizing modifications made is necessary to interpret and implement trial results. The purpose of this paper is to outline the modifications made to the OPTIMIZE trial using the Framework for Reporting Adaptations and Modifications to Evidence-Based Interventions (FRAME) to characterize modifications to the OPTIMIZE trial in response to the COVID pandemic and other challenges encountered.

METHODS

The FRAME outlines a strategy to identify and report modifications to evidence-based interventions or implementation strategies, whether planned or unplanned. We use the FRAME to characterize the process used to modify the aspects of the OPTIMIZE trial. Modifications were made to improve lower-than-anticipated rates of treatment initiation and COVID-related restrictions. Contextual modifications were made to permit telehealth delivery of treatments originally designed for in-person delivery. Training modifications were made with study personnel to provide more detailed information to potential participants, use motivational interviewing communication techniques to clarify potential participants' motivation and possible barriers to initiating treatment, and provide greater assistance with scheduling of assigned treatments.

RESULTS

Modifications were developed with input from the trial's patient and stakeholder advisory panels. The goals of the modifications were to improve trial feasibility without compromising the interventions' core functions. Modifications were approved by the study funder and the trial steering committee.

CONCLUSIONS

Full and transparent reporting of modifications to clinical trials, whether planned or unplanned, is critical for interpreting the trial's eventual results and considering future implementation efforts.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03859713. Registered on March 1, 2019.

A Pragmatic Clinical Trial Approach to Assessing and Monitoring Suicidal Ideation: Results from A National US Trauma Care System Study

OBJECTIVE

Few investigations have comprehensively described methods for assessing and monitoring suicidal ideation in pragmatic clinical trials of mental health services interventions. This investigation's goal was to assess a collaborative care intervention's effectiveness in reducing suicidal ideation and describe suicide monitoring implementation in a nationwide protocol.

METHOD

The investigation was a secondary analysis of a stepped wedge cluster randomized trial at 25-Level I trauma centers. Injury survivors (N = 635) were randomized to control (n = 370) and intervention (n = 265) conditions and assessed at baseline hospitalization and follow-up at 3-, 6- and 12-months post-injury. The Patient Health Questionnaire (PHQ-9) item-9 was used to evaluate patients for suicidal ideation. Mixed model regression was used to assess intervention versus control group changes in PHQ-9 item-9 scores over time and associations between baseline characteristics and development of suicidal ideation longitudinally. As part of the study implementation process assessment, suicide outreach call logs were also reviewed.

RESULTS

Over 50% of patients endorsed suicidal ideation at ≥1 assessment. Intervention patients relative to control patients demonstrated reductions in endorsements of suicidal ideation that did not achieve statistical significance (F[3,1461] = 0.74, P = .53). The study team completed outreach phone calls, texts or voice messages to 268 patients with PHQ-9 item-9 scores ≥1 (n = 161 control, n = 107 intervention).

CONCLUSIONS

Suicide assessment and monitoring can be feasibly implemented in large-scale pragmatic clinical trials. Intervention patients demonstrated less suicidal ideation over time; however, these comparisons did not achieve statistical significance. Intensive pragmatic trial monitoring may mask treatment effects by providing control patients a supportive intervention.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02655354.

Stakeholder perspectives regarding pragmatic clinical trial collateral findings

CONTEXT

Pragmatic clinical trials (PCTs), which are becoming widespread since they are relatively inexpensive and offer important benefits for healthcare decision-making, can also present practical, ethical, and legal challenges. One such challenge involves managing "pragmatic clinical trial collateral findings" (PCT-CFs), or information emerging in a PCT that is unrelated to the primary research question(s), yet may have implications for individual patients, clinicians, or health care systems from whom or within which data were collected. The expansion of PCTs makes it likely healthcare systems will increasingly encounter PCT-CFs, yet little guidance exists regarding their appropriate management.

METHODS

We conducted semi-structured interviews with key stakeholders experienced in the conduct or oversight of PCTs and those in health system leadership. Interviews explored respondents' experience with PCTs and PCT-CFs, and actual or hypothetical reactions to PCT-CF management. We used standard methods of qualitative analysis to identify key themes.

FINDINGS

Forty-one stakeholders participated. Four key themes emerged. First, discussions of PCT-CFs are complicated by layers of ambiguity related to both the nature of PCTs themselves, and unanticipated results that emanate from them. Second, management of PCT-CFs is context-specific, and not amenable to a "one-size-fits-all" approach. Third, there was a wide diversity of attitudes regarding the scope of researcher responsibilities in PCTs. Fourth, PCT-CFs had generally not been previously considered by respondents, but there was widespread belief in the importance of prospective planning to anticipate such issues in future PCTs.

CONCLUSIONS

PCT-CFs are likely to increase, yet those charged with PCT-CF decision-making and their disclosure are unlikely to have experience with these issues. Further deliberation about the ethical obligations and implementation processes regarding PCT-CFs is needed. To enhance the likelihood of developing sound policies and practices, such deliberations should include the input and perspectives of key stakeholders in PCTs, including professionals, policy makers, and patients.

Central data monitoring in the multicentre randomised SafeBoosC-III trial - a pragmatic approach

BACKGROUND

Data monitoring of clinical trials is a tool aimed at reducing the risks of random errors (e.g. clerical errors) and systematic errors, which include misinterpretation, misunderstandings, and fabrication. Traditional 'good clinical practice data monitoring' with on-site monitors increases trial costs and is time consuming for the local investigators. This paper aims to outline our approach of time-effective central data monitoring for the SafeBoosC-III multicentre randomised clinical trial and present the results from the first three central data monitoring meetings.

METHODS

The present approach to central data monitoring was implemented for the SafeBoosC-III trial, a large, pragmatic, multicentre, randomised clinical trial evaluating the benefits and harms of treatment based on cerebral oxygenation monitoring in preterm infants during the first days of life versus monitoring and treatment as usual. We aimed to optimise completeness and quality and to minimise deviations, thereby limiting random and systematic errors. We designed an automated report which was blinded to group allocation, to ease the work of data monitoring. The central data monitoring group first reviewed the data using summary plots only, and thereafter included the results of the multivariate Mahalanobis distance of each centre from the common mean. The decisions of the group were manually added to the reports for dissemination, information, correcting errors, preventing furture errors and documentation.

RESULTS

The first three central monitoring meetings identified 156 entries of interest, decided upon contacting the local investigators for 146 of these, which resulted in correction of 53 entries. Multiple systematic errors and protocol violations were identified, one of these included 103/818 randomised participants. Accordingly, the electronic participant record form (ePRF) was improved to reduce ambiguity.

DISCUSSION

We present a methodology for central data monitoring to optimise quality control and quality development. The initial results included identification of random errors in data entries leading to correction of the ePRF, systematic protocol violations, and potential protocol adherence issues. Central data monitoring may optimise concurrent data completeness and may help timely detection of data deviations due to misunderstandings or fabricated data.

Designing and undertaking randomised implementation trials: guide for researchers

Implementation science is the study of methods to promote the systematic uptake of evidence based interventions into practice and policy to improve health. Despite the need for high quality evidence from implementation research, randomised trials of implementation strategies often have serious limitations. These limitations include high risks of bias, limited use of theory, a lack of standard terminology to describe implementation strategies, narrowly focused implementation outcomes, and poor reporting. This paper aims to improve the evidence base in implementation science by providing guidance on the development, conduct, and reporting of randomised trials of implementation strategies. Established randomised trial methods from seminal texts and recent developments in implementation science were consolidated by an international group of researchers, health policy makers, and practitioners. This article provides guidance on the key components of randomised trials of implementation strategies, including articulation of trial aims, trial recruitment and retention strategies, randomised design selection, use of implementation science theory and frameworks, measures, sample size calculations, ethical review, and trial reporting. It also focuses on topics requiring special consideration or adaptation for implementation trials. We propose this guide as a resource for researchers, healthcare and public health policy makers or practitioners, research funders, and journal editors with the goal of advancing rigorous conduct and reporting of randomised trials of implementation strategies.

The role of data and safety monitoring boards in implementation trials: When are they justified?

The National Institutes of Health requires data and safety monitoring boards (DSMBs) for all phase III clinical trials. The National Heart, Lung and Blood Institute requires DSMBs for all clinical trials involving more than one site and those involving cooperative agreements and contracts. These policies have resulted in the establishment of DSMBs for many implementation trials, with little consideration regarding the appropriateness of DSMBs and/or key adaptations needed by DSMBs to monitor data quality and participant safety. In this perspective, we review the unique features of implementation trials and reflect on key questions regarding the justification for DSMBs and their potential role and monitoring targets within implementation trials.