Using routinely recorded data in the UK to assess outcomes in a randomised controlled trial: The Trials of Access

The use of healthcare systems data for RCTs

BACKGROUND

Healthcare systems data (HSD) has the potential to optimise the efficiency of randomised controlled trials (RCTs), by decreasing trial-specific data demands. Therefore, the use of HSD in trials is expected to increase. In 2019, it was estimated that 47% of NIHR-funded trials were planning to use HSD. We aim to understand the extent and nature of its current use and its evolution over time.

METHODS

We identified a cohort of RCTs within the NIHR Journals Library that commenced after 2019 and were described as being in progress on 6 June 2022. Details on the source and use of HSD were extracted from eligible RCTs. The use of HSD was categorised according to whether it was used as the sole data source for outcomes and whether the outcomes were primary or secondary. HSD is often insufficient for patient-reported outcomes (PROs). We aimed to determine methods used by trialists for collecting PRO data alongside HSD.

RESULTS

Of the 84 eligible studies, 52 (62%) planned to use HSD and 79 (94%) planned to collect PROs. The number of RCTs planning to use HSD for at least one outcome was 28 (54%) with 24 of these planning to use HSD as the sole data source for at least one outcome. The number of studies planning to use HSD for primary and secondary outcomes was 10 (20%) and 21 (40%) respectively. The sources of HSD were National Health Service (NHS) Digital (n = 37, 79%), patient registries (n = 7, 29%), primary care (n = 5, 21%), The Office for National Statistics (ONS) (n = 3, 13%) and other (n = 2, 8%). PROs were collected for 92% of the trials planning to use HSD. Methods for collection of PROs included in-person (n = 26, 54%), online (n = 22, 46%), postal (n = 18, 38%), phone (n = 14, 29%) and app (n = 2, 4%).

CONCLUSIONS

HSD is being used in around two thirds of the studies but cannot yet be used to support PRO data collection within the cohort we examined. Comparison with an earlier cohort demonstrates an increase in the number of RCTs planning to use HSD.

Augmenting clinical trial economic analysis by linking cancer trial data to administrative data: current landscape and future opportunities

BACKGROUND

Economic analyses based on clinical trial data are costly and time consuming, and alternative methods for performing economic analyses should be explored.

OBJECTIVE AND METHODS

In this perspective, we examine the emerging role of administrative data for economic analyses in cancer.

RESULTS

Compared with routinely collected clinical trial data, routinely collected administrative data have several strengths including high capture rates for healthcare encounters, less resource utilisation, low rates of misclassification, long follow-up periods and the opportunity to collect data points not traditionally captured in clinical trials. However, there are also limitations including the need for accurate data linkage across multiple databases and systems, the costs and time associated with data linkage, the potential time lag between trial data collection and the availability of administrative data, and limited data on quality of life, toxicity and indirect costs. In this perspective, we identify important barriers and potential solutions to performing economic analyses for oncology using administrative data, and outline strategies to increase research in this field.

CONCLUSION

The use of routinely collected administrative data sets for economic analyses of clinical trials presents a unique opportunity that could complement and validate economic analyses based on trial-level data.

Using routine databases to evaluate Electronic Palliative Care Co-ordination Systems (EPaCCS)

In response to the government's drive to expand Electronic Palliative Care Co-ordination Systems (EPaCCS) across England by 2020, further evidence for this intervention needs to be established quickly. With palliative and end-of-life care research being an underfunded area, the availability and lower costs of routine databases make it an attractive resource to integrate into studies evaluating EPaCCS without jeopardising research quality. This article describes how routine databases can be used to address the current paucity of high-quality evidence; they can be used in a range of study designs, including randomised controlled trials and quasi-experimental designs, and may also be able to contribute quality of life or patient-reported outcome measures.

E-Infrastructures and the divergent assetization of public health data: Expectations, uncertainties, and asymmetries

In this article, we examine some of the expectations, frictions and uncertainties involved with the assetization of de-identified NHS patient data by (primary care) research services in UK. Pledges to Electronic Health Record (EHR) data-driven research attempt to reconfigure public health data as an asset for realizing multiple values across healthcare, research and finance. We introduce the concept of 'asymmetrical divergence' in public health data assetization to study the various practices of configuring and using this data, both as a continuously generated resource to be extracted and as an asset to be circulated in the knowledge economy. As data assetization and exploitations grow bigger and more diverse, the capitalization of these datasets may constitute EHR data-driven research in healthcare as an attractive technoscientific activity, but one limited to those actors with specific sociotechnical resources in place to fully exploit them at the required scale.

A study protocol for the development of a SPIRIT extension for trials conducted using cohorts and routinely collected data (SPIRIT-ROUTINE)

Background: Protocols are an essential document for conducting randomised controlled trials (RCTs). However, the completeness of the information provided is often inadequate. To help improve the content of trial protocols, an international group of stakeholders published the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) Initiative in 2013. Presently, there is increasing use of cohorts and routinely collected data (RCD) for RCTs because these data have the potential to improve efficiencies by facilitating recruitment, simplifying, and reducing the cost of data collection. Reporting guidelines have been shown to improve the quality of reporting, but there is currently no specific SPIRIT guidance on protocols for trials conducted using cohorts and RCD. This protocol outlines steps for developing SPIRIT-ROUTINE, which aims to address this gap by extending the SPIRIT guidance to protocols for trials conducted using cohorts and RCD.  Methods: The development of the SPIRIT-ROUTINE extension comprises five stages. Stage 1 consists of a project launch and a meeting to finalise the membership of the steering group and scope of the extension. In Stage 2, a rapid review will be performed to identify possible modifications to the original SPIRIT 2013 checklist. Other key reporting guidelines will be reviewed to identify areas where additional items may be needed, such as the Consolidated Standards of Reporting Trials (CONSORT) extension for trials conducted using cohorts and RCD (CONSORT-ROUTINE). Stage 3 will involve an online Delphi exercise, consisting of two rounds and involving key international stakeholders to gather feedback on the preliminary checklist items. In Stage 4, a consensus meeting of the SPIRIT-ROUTINE steering group will finalise the items to include in the extension. Stage 5 will involve the publication preparation and dissemination of the final checklist.  Conclusion: The SPIRIT-ROUTINE extension will contribute to improving design of trials using cohorts and RCD and transparency of reporting.

Using routinely recorded data in a UK RCT: a comparison to standard prospective data collection methods

BACKGROUND

Routinely recorded data held in electronic health records can be used to inform the conduct of randomised controlled trials (RCTs). However, limitations with access and accuracy have been identified.

OBJECTIVE

Using epilepsy as an exemplar condition, we assessed the attributes and agreement of routinely recorded data compared to data collected using case report forms in a UK RCT assessing antiepileptic drug treatments for individuals newly diagnosed with epilepsy.

METHODS

The case study RCT is the Standard and New Antiepileptic Drugs II (SANAD II) trial, a pragmatic, UK multicentre RCT assessing the clinical and cost-effectiveness of antiepileptic drugs as treatments for epilepsy. Ninety-eight of 470 eligible participants provided consent for access to routinely recorded secondary care data that were retrieved from NHS Digital Hospital Episode Statistics (N=71) and primary and secondary care data from The Secure Anonymised Information Linkage Databank (N=27). We assessed data items relevant to the identification of individuals eligible for inclusion in SANAD II, baseline and follow-up visits. The attributes of routinely recorded data were assessed including the degree of missing data. The agreement between routinely recorded data and data collected on case report forms in SANAD II was assessed using calculation of Cohen's kappa for categorical data and construction of Bland-Altman plots for continuous data.

RESULTS

There was a significant degree of missing data in the routine record for 15 of the 20 variables assessed, including all clinical variables. Agreement was poor for the majority of comparisons, including the assessments of seizure occurrence and adverse events. For example, only 23/62 (37%) participants had a date of first-ever seizure identified in routine datasets. Agreement was satisfactory for the date of prescription of antiepileptic drugs and episodes of healthcare resource use.

CONCLUSIONS

There are currently significant limitations preventing the use of routinely recorded data for participant identification and assessment of clinical outcomes in epilepsy, and potentially other chronic conditions. Further research is urgently required to assess the attributes, agreement, additional benefits, cost-effectiveness and 'optimal mix' of routinely recorded data compared to data collected using standard methods such as case report forms at clinic visits for people with epilepsy.

TRIAL REGISTRATION

Standard and New Antiepileptic Drugs II (SANAD II (EudraCT No: 2012-001884-64, registered 05/09/2012; ISRCTN Number: ISRCTN30294119 , registered 03/07/2012)).

Realising the full potential of data-enabled trials in the UK: a call for action

RATIONALE

Clinical trials are the gold standard for testing interventions. COVID-19 has further raised their public profile and emphasised the need to deliver better, faster, more efficient trials for patient benefit. Considerable overlap exists between data required for trials and data already collected routinely in electronic healthcare records (EHRs). Opportunities exist to use these in innovative ways to decrease duplication of effort and speed trial recruitment, conduct and follow-up.

APPROACH

The National Institute of Health Research (NIHR), Health Data Research UK and Clinical Practice Research Datalink co-organised a national workshop to accelerate the agenda for 'data-enabled clinical trials'. Showcasing successful examples and imagining future possibilities, the plenary talks, panel discussions, group discussions and case studies covered: design/feasibility; recruitment; conduct/follow-up; collecting benefits/harms; and analysis/interpretation.

REFLECTION

Some notable studies have successfully accessed and used EHR to identify potential recruits, support randomised trials, deliver interventions and supplement/replace trial-specific follow-up. Some outcome measures are already reliably collected; others, like safety, need detailed work to meet regulatory reporting requirements. There is a clear need for system interoperability and a 'route map' to identify and access the necessary datasets. Researchers running regulatory-facing trials must carefully consider how data quality and integrity would be assessed. An experience-sharing forum could stimulate wider adoption of EHR-based methods in trial design and execution.

DISCUSSION

EHR offer opportunities to better plan clinical trials, assess patients and capture data more efficiently, reducing research waste and increasing focus on each trial's specific challenges. The short-term emphasis should be on facilitating patient recruitment and for postmarketing authorisation trials where research-relevant outcome measures are readily collectable. Sharing of case studies is encouraged. The workshop directly informed NIHR's funding call for ambitious data-enabled trials at scale. There is the opportunity for the UK to build upon existing data science capabilities to identify, recruit and monitor patients in trials at scale.

Seizure first aid training for people with epilepsy attending emergency departments and their significant others: the SAFE intervention and feasibility RCT

Background

No seizure first aid training intervention exists for people with epilepsy who regularly attend emergency departments and their significant others, despite such an intervention’s potential to reduce clinically unnecessary and costly visits.

Objectives

The objectives were to (1) develop Seizure first Aid training For Epilepsy (SAFE) by adapting a broader intervention and (2) determine the feasibility and optimal design of a definitive randomised controlled trial to test SAFE’s efficacy.

Design

The study involved (1) the development of an intervention informed by a co-design approach with qualitative feedback and (2) a pilot randomised controlled trial with follow-ups at 3, 6 and 12 months and assessments of treatment fidelity and the cost of SAFE’s delivery.

Setting

The setting was (1) third-sector patient support groups and professional health-care organisations and (2) three NHS emergency departments in England.

Participants

Participants were (1) people with epilepsy who had visited emergency departments in the prior 2 years, their significant others and emergency department, paramedic, general practice, commissioning, neurology and nursing representatives and (2) people with epilepsy aged ≥ 16 years who had been diagnosed for ≥ 1 year and who had made two or more emergency department visits in the prior 12 months, and one of their significant others. Emergency departments identified ostensibly eligible people with epilepsy from attendance records and patients confirmed their eligibility.

Interventions

Participants in the pilot randomised controlled trial were randomly allocated 1 : 1 to SAFE plus treatment as usual or to treatment as usual only.

Main outcome measures

Consent rate and availability of routine data on emergency department use at 12 months were the main outcome measures. Other measures of interest included eligibility rate, ease with which people with epilepsy could be identified and routine data secured, availability of self-reported emergency department data, self-reported emergency department data’s comparability with routine data, SAFE’s effect on emergency department use, and emergency department use in the treatment as usual arm, which could be used in sample size calculations.

Results

(1) Nine health-care professionals and 23 service users provided feedback that generated an intervention considered to be NHS feasible and well positioned to achieve its purpose. (2) The consent rate was 12.5%, with 53 people with epilepsy and 38 significant others recruited. The eligibility rate was 10.6%. Identifying people with epilepsy from attendance records was resource intensive for emergency department staff. Those recruited felt more stigmatised because of epilepsy than the wider epilepsy population. Routine data on emergency department use at 12 months were secured for 94.1% of people with epilepsy, but the application process took 8.5 months. Self-reported emergency department data were available for 66.7% of people with epilepsy, and people with epilepsy self-reported more emergency department visits than were captured in routine data. Most participants (76.9%) randomised to SAFE received the intervention. The intervention was delivered with high fidelity. No related serious adverse events occurred. Emergency department use at 12 months was lower in the SAFE plus treatment as usual arm than in the treatment as usual only arm, but not significantly so. Calculations indicated that a definitive trial would need ≈ 674 people with epilepsy and ≈ 39 emergency department sites.

Limitations

Contrary to patient statements on recruitment, routine data secured at the pilot trial’s end indicated that ≈ 40% may not have satisfied the inclusion criterion of two or more emergency department visits.

Conclusions

An intervention was successfully developed, a pilot randomised controlled trial conducted and outcome data secured for most participants. The consent rate did not satisfy a predetermined ‘stop/go’ level of ≥ 20%. The time that emergency department staff needed to identify eligible people with epilepsy is unlikely to be replicable. A definitive trial is currently not feasible.

Future work

Research to more easily identify and recruit people from the target population is required.

Trial registration

Current Controlled Trials ISRCTN13871327.

Funding

This project was funded by the National Institute for Health Research (NIHR) Health Services and Delivery Research programme and will be published in full in Health Services and Delivery Research; Vol. 8, No. 39. See the NIHR Journals Library website for further project information.

Linkage of the CHHiP randomised controlled trial with primary care data: a study investigating ways of supplementing cancer trials and improving evidence-based practice

BACKGROUND

Randomised controlled trials (RCTs) are the gold standard for evidence-based practice. However, RCTs can have limitations. For example, translation of findings into practice can be limited by design features, such as inclusion criteria, not accurately reflecting clinical populations. In addition, it is expensive to recruit and follow-up participants in RCTs. Linkage with routinely collected data could offer a cost-effective way to enhance the conduct and generalisability of RCTs. The aim of this study is to investigate how primary care data can support RCTs.

METHODS

Secondary analysis following linkage of two datasets: 1) multicentre CHHiP radiotherapy trial (ISRCTN97182923) and 2) primary care database from the Royal College of General Practitioners Research and Surveillance Centre. Comorbidities and medications recorded in CHHiP at baseline, and radiotherapy-related toxicity recorded in CHHiP over time were compared with primary care records. The association of comorbidities and medications with toxicity was analysed with mixed-effects logistic regression.

RESULTS

Primary care records were extracted for 106 out of 2811 CHHiP participants recruited from sites in England (median age 70, range 44 to 82). Complementary information included longitudinal body mass index, blood pressure and cholesterol, as well as baseline smoking and alcohol usage but was limited by the considerable missing data. In the linked sample, 9 (8%) participants were recorded in CHHiP as having a history of diabetes and 38 (36%) hypertension, whereas primary care records indicated incidence prior to trial entry of 11 (10%) and 40 (38%) respectively. Concomitant medications were not collected in CHHiP but available in primary care records. This indicated that 44 (41.5%) men took aspirin, 65 (61.3%) statins, 14 (13.2%) metformin and 46 (43.4%) phosphodiesterase-5-inhibitors at some point before or after trial entry.

CONCLUSIONS

We provide a set of recommendations on linkage and supplementation of trials. Data recorded in primary care are a rich resource and linkage could provide near real-time information to supplement trials and an efficient and cost-effective mechanism for long-term follow-up. In addition, standardised primary care data extracts could form part of RCT recruitment and conduct. However, this is at present limited by the variable quality and fragmentation of primary care data.

Role of health plan administrative claims data in participant recruitment for pragmatic clinical trials: An Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness (ADAPTABLE) example

Aim:

The purpose of this study is to evaluate HealthCore/Anthem Research Network recruitment strategies, compare response and enrollment rates for different recruitment strategies, and describe demographic and clinical characteristics of responders and enrollees.

Methods:

HealthCore/Anthem Research Network, a part of the Health Plan Research Network of the Patient-Centered Clinical Data Research Network, used administrative claims data to identify eligible health plan members for potential participation in the Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness study. We approached health plan members, identified with a validated Patient-Centered Clinical Data Research Network common data model computable phenotype, and their clinical providers during November 2017 to August 2018. Providers were offered the option to exclude their patients’ participation in Aspirin Dosing: A Patient-centric Trial Assessing Benefits and Long-term Effectiveness prior to our direct patient (member) outreach. Member identification was in two phases: Phase 1: 1 January 2006 to 1 April 2017, and Phase 2: 1 January 2006 to 2 February 2018. Phase 1 consisted of two batches of mail and one phone call per patient. In Phase 2, which included two similar batches of patients, outreach was via either mail or brochure and one phone call.

Results:

Phase 1 and Phase 2 included 133,373 and 51,777 members, respectively. We engaged 28,593 providers in Phase 1, and 5077 in Phase 2. In Phase 1, 264,158 mixed email/mail messages were delivered to 133,373 members, followed by 90,481 phone calls from November 2017 to February 2018. In Phase 2, after simple randomization to letter or brochure, 51,777 members were sent email/mail or mailed brochure in three waves from May 2018 to July 2018. In this 9-week period, 51,623 communications were sent to 25,914 members in the email/mail group, and 50,160 brochures to 25,863 in the brochure group. Following email/mail or mailed brochure outreach, 16,624 and 16,580 calls were made to the groups, respectively. Overall, 1549 health plan members visited the study portal by 1 September 2018; 355 electronically signed the Informed Consent Form and enrolled. Mailed brochures drove more portal visits in Phase 2, but a lower percentage of responders enrolled. Recruitment was better in Phase 2—2.3 enrollees per 1000 outreach members versus 1.8 in Phase 1.

Conclusion:

This study showed the ability of a health plan within Patient-Centered Clinical Data Research Network to identify potential study participants with administrative claims, and use different outreach methods to facilitate recruitment and enrollment for pragmatic clinical trials.

Electronic health record nested pragmatic randomized controlled trial of a reminder system for serum lithium level monitoring in patients with mood disorder: KONOTORI study protocol

Background

The weaknesses of classical explanatory randomized controlled trials (RCTs) include limited generalizability, high cost, and time burden. Pragmatic RCTs nested within electronic health records (EHRs) can be useful to overcome such limitations. Serum lithium monitoring has often been underutilized in real-world practice in Japan. This trial aims to evaluate the effectiveness of the EHR-nested reminder system for serum lithium level monitoring in the maintenance of therapeutic lithium concentration and in the improvement of the quality of care for patients on lithium maintenance therapy.

Methods

The Kyoto Toyooka nested controlled trial of reminders (KONOTORI trial) is an EHR-nested, parallel-group, superiority, stratified, permuted block-randomized controlled trial. Screening, random allocation, reminder output, and outcome collection will be conducted automatically by the EHR-nested trial program. Patients with a mood disorder taking lithium carbonate for maintenance therapy will be randomly allocated to the two-step reminder system for serum lithium monitoring or to usual care. The primary outcome is the achievement of therapeutic serum lithium concentration between 0.4 and 1.0 mEq/L at 18 months after informed consent.

Discussion

The KONOTORI trial uses EHRs to enable the efficient conduct of a pragmatic trial of the reminder system for lithium monitoring. This may contribute to improved quality of care for patients on lithium maintenance therapy.

Trial registration

University Hospital Medical Information Network (UMIN) Clinical Trials Registry, UMIN000033633. Registered on 3 July 2018.

Does Preexisting Practice Modify How Video Games Recalibrate Physician Heuristics in Trauma Triage?

Background:

A majority of severely injured patients fail to receive care at trauma centers (undertriage), in part, because of physician judgment. We previously developed two educational video games that reduced physicians’ undertriage compared with control in two clinical trials. In this secondary analysis, we investigated heterogeneity of treatment effect of the interventions by assessing physicians’ preexisting practice patterns in claims data. We hypothesized that physicians with high preexisting undertriage would benefit most from game-based training.

Methods:

Using Medicare claims records from 2010 to 2015, we measured physicians’ preexisting triage practices before their participation in one of two trials conducted in 2016 and 2017. We categorized physicians as having received game-based training versus control and noted their postintervention simulation triage performance in the trials. We used multivariable linear regression models to assess the heterogeneity of game-based training effect among physicians with high and low preexisting undertriage.

Results:

Of the 394 eligible physicians from our trials, we identified 275 (70%) with claims for Medicare fee-for-service beneficiaries suffering severe injury between 2010 and 2015. On average, the physicians were 44 y old (SD 8.4) with 12 y (SD 8.2) of experience. We found significant interaction between preexisting practice and intervention efficacy (P = 0.04). Physicians with high undertriage before enrollment improved significantly with game-based training compared with the control (46% versus 63%, P < 0.001). Those with low preexisting undertriage did not (58% versus 56%, P = 0.76).

Conclusions:

Using claims-based data, we found heterogeneity of treatment effect of interventions designed to recalibrate physician heuristics. Physicians with high preexisting undertriage benefited most from game-based training.

Challenges in accessing routinely collected data from multiple providers in the UK for primary studies: Managing the morass

INTRODUCTION

Researchers are increasingly using routinely collected data in addition to, or instead of, other data collection methods. The UK government continues to invest in research centres to encourage use of these data, and trials and cohort studies utilise data linkage methods in the follow-up of participants. This does not come without its limitations and challenges, such as data access delays.

OBJECTIVE

This paper outlines the challenges faced by three projects utilising individual-level routinely-collected linked data for the longer-term follow-up of participants.

METHODS

These studies are varied in design, study population and data providers. One researcher was common to the three studies and collated relevant study correspondence, formal documentary evidence such as data sharing agreements and, where relevant, meeting records to review. Key themes were identified and reviewed by other members of the research teams. Mitigating strategies were identified and discussed with a data provider representative and a broader group of researchers to finalise the recommendations presented.

RESULTS

The challenges discussed are grouped into five themes: Data application process; Project timelines; Dependencies and considerations related to consent; Information Governance; Contractual. In presenting our results descriptively we summarise each case study, identify the main cross-cutting themes and consider the potential for mitigation of challenges.

CONCLUSIONS

We make recommendations that identify responsibilities for both researchers and data providers for mitigating and managing data access challenges. A continued conversation within the research community and with data providers is needed to continue to enable researchers to access and utilise the wealth of routinely-collected data available. The suggestions made in this paper will help researchers be better prepared to deal with the challenges of applying for data from multiple data providers.

Challenges in accessing routinely collected data from multiple providers in the UK for primary studies: Managing the morass

Introduction

Researchers are increasingly using routinely collected data in addition to, or instead of, other data collection methods. The UK government continues to invest in research centres to encourage use of these data, and trials and cohort studies utilise data linkage methods in the follow-up of participants. This does not come without its limitations and challenges, such as data access delays.

Objective

This paper outlines the challenges faced by three projects utilising individual-level routinely-collected linked data for the longer-term follow-up of participants.

Methods

These studies are varied in design, study population and data providers. One researcher was common to the three studies and collated relevant study correspondence, formal documentary evidence such as data sharing agreements and, where relevant, meeting records to review. Key themes were identified and reviewed by other members of the research teams. Mitigating strategies were identified and discussed with a data provider representative and a broader group of researchers to finalise the recommendations presented.

Results

The challenges discussed are grouped into five themes: Data application process; Project timelines; Dependencies and considerations related to consent; Information Governance; Contractual. In presenting our results descriptively we summarise each case study, identify the main cross-cutting themes and consider the potential for mitigation of challenges.

Conclusions

We make recommendations that identify responsibilities for both researchers and data providers for mitigating and managing data access challenges. A continued conversation within the research community and with data providers is needed to continue to enable researchers to access and utilise the wealth of routinely-collected data available. The suggestions made in this paper will help researchers be better prepared to deal with the challenges of applying for data from multiple data providers.