Accessing routinely collected health data to improve clinical trials: recent experience of access

A pilot study examining the impact of a pragmatic process for improving the cultural responsiveness of non-Aboriginal alcohol and other drug treatment services using routinely collected data in Australia

Objective: Routine health data has the potential to identify changes in patient-related outcomes, in close to real time. This pilot project used routine data to explore and compare the impact of changes to cultural responsiveness on service use by Aboriginal and Torres Strait Islander (hereafter, Aboriginal) clients in Australia.Methods: The New South Wales Minimum Data Set (MDS) for alcohol and other drug use treatment services was provided for 11 services for a period of 30 months from March 2019 to September 2021 (four months prior to two years after the intervention; data were analysed between March 2022 to February 2023). Change in cultural responsiveness was assessed via practice audits of services at baseline and two years. The average change in audit rating was analysed using a linear mixed regression model. Generalised Linear Mixed Models were used to identify changes in service use by Aboriginal clients. Results: All 11 services showed increased audit scores at two years, with a statistically significant mean increase of 18.6 (out of 63 points; b = 18.32, 95% CI 12.42-24.22). No statistically significant pre-to post-changes were identified in: (1) the proportion of episodes delivered to Aboriginal versus non-Aboriginal clients (OR = 1.15, 95% CI = 0.94-1.40), (2) the number of episodes of care provided to Aboriginal clients per month (IRR = 1.01, 95% CI = 0.84-1.23), or (3) the proportion of episodes completed by Aboriginal clients (OR = 0.96, 95% CI = 0.82-1.13). Conclusions: The lack of statistically significant impact on service use outcomes using MDS contrasts to the improvements in cultural responsiveness, suggesting further work is needed to identify appropriate outcome measures. This may include patient-reported experience measures. This project showed that routine data has potential as an efficient method for measuring changes in patient-related outcomes in response to health services improvements.

Getting our ducks in a row: The need for data utility comparisons of healthcare systems data for clinical trials

BACKGROUND

Better use of healthcare systems data, collected as part of interactions between patients and the healthcare system, could transform planning and conduct of randomised controlled trials. Multiple challenges to widespread use include whether healthcare systems data captures sufficiently well the data traditionally captured on case report forms. "Data Utility Comparison Studies" (DUCkS) assess the utility of healthcare systems data for RCTs by comparison to data collected by the trial. Despite their importance, there are few published UK examples of DUCkS.

METHODS-AND-RESULTS

Building from ongoing and selected recent examples of UK-led DUCkS in the literature, we set out experience-based considerations for the conduct of future DUCkS. Developed through informal iterative discussions in many forums, considerations are offered for planning, protocol development, data, analysis and reporting, with comparisons at "patient-level" or "trial-level", depending on the item of interest and trial status.

DISCUSSION

DUCkS could be a valuable tool in assessing where healthcare systems data can be used for trials and in which trial teams can play a leading role. There is a pressing need for trials to be more efficient in their delivery and research waste must be reduced. Trials have been making inconsistent use of healthcare systems data, not least because of an absence of evidence of utility. DUCkS can also help to identify challenges in using healthcare systems data, such as linkage (access and timing) and data quality. We encourage trial teams to incorporate and report DUCkS in trials and funders and data providers to support them.

The use of routinely collected healthcare records for outcome assessment in clinical trials: a UK perspective

The use of routinely collected electronic healthcare records (EHR) for outcome assessment in clinical trials has been described as a 'disruptive' new technique more than a decade ago. Despite this potential, significant methodological issues and regulatory barriers have hampered the progress in this area. This article discusses the key considerations that trialists should take into account when incorporating EHR into their trials. These include considerations of the clinical relevance of the outcome, data timeliness and quality, ethical and regulatory issues, and some practical considerations for clinical trials units. In addition, this article describes the benefits of using EHR which include cost, reduced trial burden for participants and staff, follow up efficiencies, and improved health economic evaluation procedures. We also describe the major regulatory and start up costs of using EHR in clinical trials. This article focuses on the UK specific EHR landscape in clinical trials and would help researchers and trials units considering the use of this method of outcome data collection in their next trial. If the issues described are mitigated, this method will be a formidable tool for conducting pragmatic clinical trials.

Evaluation of venous thromboembolism risk assessment models for hospital inpatients: the VTEAM evidence synthesis

Background

Pharmacological prophylaxis during hospital admission can reduce the risk of acquired blood clots (venous thromboembolism) but may cause complications, such as bleeding. Using a risk assessment model to predict the risk of blood clots could facilitate selection of patients for prophylaxis and optimise the balance of benefits, risks and costs.

Objectives

We aimed to identify validated risk assessment models and estimate their prognostic accuracy, evaluate the cost-effectiveness of different strategies for selecting hospitalised patients for prophylaxis, assess the feasibility of using efficient research methods and estimate key parameters for future research.

Design

We undertook a systematic review, decision-analytic modelling and observational cohort study conducted in accordance with Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines.

Setting

NHS hospitals, with primary data collection at four sites.

Participants

Medical and surgical hospital inpatients, excluding paediatric, critical care and pregnancy-related admissions.

Interventions

Prophylaxis for all patients, none and according to selected risk assessment models.

Main outcome measures

Model accuracy for predicting blood clots, lifetime costs and quality-adjusted life-years associated with alternative strategies, accuracy of efficient methods for identifying key outcomes and proportion of inpatients recommended prophylaxis using different models.

Results

We identified 24 validated risk assessment models, but low-quality heterogeneous data suggested weak accuracy for prediction of blood clots and generally high risk of bias in all studies. Decision-analytic modelling showed that pharmacological prophylaxis for all eligible is generally more cost-effective than model-based strategies for both medical and surgical inpatients, when valuing a quality-adjusted life-year at £20,000. The findings were more sensitive to uncertainties in the surgical population; strategies using risk assessment models were more cost-effective if the model was assumed to have a very high sensitivity, or the long-term risks of post-thrombotic complications were lower. Efficient methods using routine data did not accurately identify blood clots or bleeding events and several pre-specified feasibility criteria were not met. Theoretical prophylaxis rates across an inpatient cohort based on existing risk assessment models ranged from 13% to 91%.

Limitations

Existing studies may underestimate the accuracy of risk assessment models, leading to underestimation of their cost-effectiveness. The cost-effectiveness findings do not apply to patients with an increased risk of bleeding. Mechanical thromboprophylaxis options were excluded from the modelling. Primary data collection was predominately retrospective, risking case ascertainment bias.

Conclusions

Thromboprophylaxis for all patients appears to be generally more cost-effective than using a risk assessment model, in hospitalised patients at low risk of bleeding. To be cost-effective, any risk assessment model would need to be highly sensitive. Current evidence on risk assessment models is at high risk of bias and our findings should be interpreted in this context. We were unable to demonstrate the feasibility of using efficient methods to accurately detect relevant outcomes for future research.

Future work

Further research should evaluate routine prophylaxis strategies for all eligible hospitalised patients. Models that could accurately identify individuals at very low risk of blood clots (who could discontinue prophylaxis) warrant further evaluation.

Study registration

This study is registered as PROSPERO CRD42020165778 and Researchregistry5216.

Funding

This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: NIHR127454) and will be published in full in Health Technology Assessment; Vol. 28, No. 20. See the NIHR Funding and Awards website for further award information.

Leveraging Virtual Technology to Conduct Clinical Trials in Inflammatory Bowel Disease

Clinical trials have led to major advances in inflammatory bowel disease (IBD) care over the last few decades, yet in that time most clinical trial protocols in IBD have remained markedly the same. Many IBD protocols often still require face-to-face visits and monitoring, hospital-based medication administration, paper-based forms and questionnaires, and short follow-up periods resulting in limited long-term data. These factors have recently been recognized as likely contributors to the low recruitment and lack of diversity of participants across clinical trials in IBD. However, with increasing technological advances, there is now an opportunity for improvement. This article assesses a range of virtual innovations for how they may offer digital solutions to challenges currently encountered in IBD clinical trials. Such solutions include consideration for increasing patient diversity, digital invitation, remote consent and recruitment, virtual visits, remote patient monitoring and data collection, remote medication delivery and administration, remote clinical trial monitoring, and routinely collected health data for long-term follow-up. Adoption of virtual technology may drive the field toward patient centricity and more efficient trial protocols to allow for a new era in IBD clinical trials.

A DELPHI study priority setting the remaining challenges for the use of routinely collected data in trials: COMORANT-UK

BACKGROUND

Researchers are increasingly seeking to use routinely collected data to support clinical trials. This approach has the potential to transform the way clinical trials are conducted in the future. The availability of routinely collected data for research, whether healthcare or administrative, has increased, and infrastructure funding has enabled much of this. However, challenges remain at all stages of a trial life cycle. This study, COMORANT-UK, aimed to systematically identify, with key stakeholders across the UK, the ongoing challenges related to trials that seek to use routinely collected data.

METHODS

This three-step Delphi method consisted of two rounds of anonymous web-based surveys and a virtual consensus meeting. Stakeholders included trialists, data infrastructures, funders of trials, regulators, data providers and the public. Stakeholders identified research questions or challenges that they considered were of particular importance and then selected their top 10 in the second survey. The ranked questions were taken forward to the consensus meeting for discussion with representatives invited from the stakeholder groups.

RESULTS

In the first survey, 66 respondents yielded over 260 questions or challenges. These were thematically grouped and merged into a list of 40 unique questions. Eighty-eight stakeholders then ranked their top ten from the 40 questions in the second survey. The most common 14 questions were brought to the virtual consensus meeting in which stakeholders agreed a top list of seven questions. We report these seven questions which are within the following domains: trial design, Patient and Public Involvement, trial set-up, trial open and trial data. These questions address both evidence gaps (requiring further methodological research) and implementation gaps (requiring training and/or service re-organisation).

CONCLUSION

This prioritised list of seven questions should inform the direction of future research in this area and should direct efforts to ensure that the benefits in major infrastructure for routinely collected data are achieved and translated. Without this and future work to address these questions, the potential societal benefits of using routinely collected data to help answer important clinical questions will not be realised.

Healthcare systems data in the context of clinical trials - A comparison of cardiovascular data from a clinical trial dataset with routinely collected data

BACKGROUND

Routinely-collected healthcare systems data (HSD) are proposed to improve the efficiency of clinical trials. A comparison was undertaken between cardiovascular (CVS) data from a clinical trial database with two HSD resources.

METHODS

Protocol-defined and clinically reviewed CVS events (heart failure (HF), acute coronary syndrome (ACS), thromboembolic stroke, venous and arterial thromboembolism) were identified within the trial data. Data (using pre-specified codes) was obtained from NHS Hospital Episode Statistics (HES) and National Institute for Cardiovascular Outcomes Research (NICOR) HF and myocardial ischaemia audits for trial participants recruited in England between 2010 and 2018 who had provided consent. The primary comparison was trial data versus HES inpatient (APC) main diagnosis (Box-1). Correlations are presented with descriptive statistics and Venn diagrams. Reasons for non-correlation were explored.

RESULTS

From 1200 eligible participants, 71 protocol-defined clinically reviewed CVS events were recorded in the trial database. 45 resulted in a hospital admission and therefore could have been recorded by either HES APC/ NICOR. Of these, 27/45 (60%) were recorded by HES inpatient (Box-1) with an additional 30 potential events also identified. HF and ACS were potentially recorded in all 3 datasets; trial data recorded 18, HES APC 29 and NICOR 24 events respectively. 12/18 (67%) of the HF/ACS events in the trial dataset were recorded by NICOR.

CONCLUSION

Concordance between datasets was lower than anticipated and the HSD used could not straightforwardly replace current trial practices, nor directly identify protocol-defined CVS events. Further work is required to improve the quality of HSD and consider event definitions when designing clinical trials incorporating HSD.

Accuracy of efficient data methods to determine the incidence of hospital-acquired thrombosis and major bleeding in medical and surgical inpatients: a multicentre observational cohort study in four UK hospitals

OBJECTIVES

We evaluated the accuracy of using routine health service data to identify hospital-acquired thrombosis (HAT) and major bleeding events (MBE) compared with a reference standard of case note review.

DESIGN

A multicentre observational cohort study.

SETTING

Four acute hospitals in the UK.

PARTICIPANTS

A consecutive unselective cohort of general medical and surgical patients requiring hospitalisation for a period of >24 hours during the calendar year 2021. We excluded paediatric, obstetric and critical care patients due to differential risk profiles.

INTERVENTIONS

We compared preidentified sources of routinely collected information (using hospital coding data and local contractually mandated thrombosis datasets) to data extracted from case notes using a predesigned workflow methodology.

PRIMARY AND SECONDARY OUTCOME MEASURES

We defined HAT as objectively confirmed venous thromboembolism occurring during hospital stay or within 90 days of discharge and MBE as per international consensus.

RESULTS

We were able to source all necessary routinely collected outcome data for 87% of 2008 case episodes reviewed. The sensitivity of hospital coding data (International Classification of Diseases 10th Revision, ICD-10) for the diagnosis of HAT and MBE was 62% (95% CI, 54 to 69) and 38% (95% CI, 27 to 50), respectively. Sensitivity improved to 81% (95% CI, 75 to 87) when using local thrombosis data sets.

CONCLUSIONS

Using routinely collected data appeared to miss a substantial proportion of outcome events, when compared with case note review. Our study suggests that currently available routine data collection methods in the UK are inadequate to support efficient study designs in venous thromboembolism research.

TRIAL REGISTRATION NUMBER

NIHR127454.

A Hybrid Architecture (CO-CONNECT) to Facilitate Rapid Discovery and Access to Data Across the United Kingdom in Response to the COVID-19 Pandemic: Development Study

BACKGROUND

COVID-19 data have been generated across the United Kingdom as a by-product of clinical care and public health provision, as well as numerous bespoke and repurposed research endeavors. Analysis of these data has underpinned the United Kingdom's response to the pandemic, and informed public health policies and clinical guidelines. However, these data are held by different organizations, and this fragmented landscape has presented challenges for public health agencies and researchers as they struggle to find relevant data to access and interrogate the data they need to inform the pandemic response at pace.

OBJECTIVE

We aimed to transform UK COVID-19 diagnostic data sets to be findable, accessible, interoperable, and reusable (FAIR).

METHODS

A federated infrastructure model (COVID - Curated and Open Analysis and Research Platform [CO-CONNECT]) was rapidly built to enable the automated and reproducible mapping of health data partners' pseudonymized data to the Observational Medical Outcomes Partnership Common Data Model without the need for any data to leave the data controllers' secure environments, and to support federated cohort discovery queries and meta-analysis.

RESULTS

A total of 56 data sets from 19 organizations are being connected to the federated network. The data include research cohorts and COVID-19 data collected through routine health care provision linked to longitudinal health care records and demographics. The infrastructure is live, supporting aggregate-level querying of data across the United Kingdom.

CONCLUSIONS

CO-CONNECT was developed by a multidisciplinary team. It enables rapid COVID-19 data discovery and instantaneous meta-analysis across data sources, and it is researching streamlined data extraction for use in a Trusted Research Environment for research and public health analysis. CO-CONNECT has the potential to make UK health data more interconnected and better able to answer national-level research questions while maintaining patient confidentiality and local governance procedures.

A good use of time? Providing evidence for how effort is invested in primary and secondary outcome data collection in trials

BACKGROUND

Data collection is a substantial part of trial workload for participants and staff alike. How these hours of work are spent is important because stakeholders are more interested in some outcomes than others. The ORINOCO study compared the time spent collecting primary outcome data to the time spent collecting secondary outcome data in a cohort of trials.

METHODS

We searched PubMed for phase III trials indexed between 2015 and 2019. From these, we randomly selected 120 trials evaluating a therapeutic intervention plus an additional random selection of 20 trials evaluating a public health intervention. We also added eligible trials from a cohort of 189 trials in rheumatology that had used the same core outcome set. We then obtained the time taken to collect primary and secondary outcomes in each trial. We used a hierarchy of methods that included data in trial reports, contacting the trial team and approaching individuals with experience of using the identified outcome measures. We calculated the primary to secondary data collection time ratio and notional data collection cost for each included trial.

RESULTS

We included 161 trials (120 phase III; 21 core outcome set; 20 public health), which together collected 230 primary and 688 secondary outcomes. Full primary and secondary timing data were obtained for 134 trials (100 phase III; 17 core outcome set; 17 public health). The median time spent on primaries was 56.1 h (range: 0.0-10,746.7, IQR: 226.89) and the median time spent on secondaries was 190.7 hours (range: 0.0-1,356,832.9, IQR: 617.6). The median primary to secondary data collection time ratio was 1.0:3.0 (i.e. for every minute spent on primary outcomes, 3.0 were spent on secondaries). The ratio varied by trial type: phase III trials were 1.0:3.1, core outcome set 1.0:3.4 and public health trials 1.0:2.2. The median notional overall data collection cost was £8015.73 (range: £52.90-£31,899,140.70, IQR: £20,096.64).

CONCLUSIONS

Depending on trial type, between two and three times as much time is spent collecting secondary outcome data than collecting primary outcome data. Trial teams should explicitly consider how long it will take to collect the data for an outcome and decide whether that time is worth it given importance of the outcome to the trial.

Generalizability and reach of a randomized controlled trial to improve oral health among home care recipients: comparing participants and nonparticipants at baseline and during follow-up

Background

The generalizability of randomized controlled trials (RCTs) with a low response can be limited by systematic differences between participants and nonparticipants. This participation bias, however, is rarely investigated because data on nonparticipants is usually not available. The purpose of this article is to compare all participants and nonparticipants of a RCT to improve oral health among home care recipients at baseline and during follow-up using claims data.

Methods

Seven German statutory health and long-term care insurance funds invited 9656 home care recipients to participate in the RCT MundPflege. Claims data for all participants (n = 527, 5.5% response) and nonparticipants (n = 9129) were analyzed. Associations between trial participation and sex, age, care dependency, number of Elixhauser diseases, and dementia, as well as nursing, medical, and dental care utilization at baseline, were investigated using multivariable logistic regression. Associations between trial participation and the probability of (a) moving into a nursing home, (b) being hospitalized, and (c) death during 1 year of follow-up were examined via Cox proportional hazards regressions, controlling for baseline variables.

Results

At baseline, trial participation was positively associated with male sex (odds ratio 1.29 [95% confidence interval 1.08–1.54]), high (vs. low 1.46 [1.15–1.86]) care dependency, receiving occasional in-kind benefits to relieve caring relatives (1.45 [1.15–1.84]), having a referral by a general practitioner to a medical specialist (1.62 [1.21–2.18]), and dental care utilization (2.02 [1.67–2.45]). It was negatively associated with being 75–84 (vs. < 60 0.67 [0.50–0.90]) and 85 + (0.50 [0.37–0.69]) years old. For morbidity, hospitalizations, and formal, respite, short-term, and day or night care, no associations were found. During follow-up, participants were less likely to move into a nursing home than nonparticipants (hazard ratio 0.50 [0.32–0.79]). For hospitalizations and mortality, no associations were found.

Conclusions

For half of the comparisons, differences between participants and nonparticipants were observed. The RCT’s generalizability is limited, but to a smaller extent than one would expect because of the low response. Routine data provide a valuable source for investigating potential differences between trial participants and nonparticipants, which might be used by future RCTs to evaluate the generalizability of their findings.

Trial registration

German Clinical Trials Register DRKS00013517. Retrospectively registered on June 11, 2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-022-06470-y.

Development and implementation of a national online application system for cross-jurisdictional linked data

The Population Health Research Network (PHRN) is an Australian national data linkage infrastructure that links a wide range of health and human services data in privacy-preserving ways. The data linkage infrastructure enables researchers to apply for access to routinely collected, linked, administrative data from the six states and two territories which make up the Commonwealth of Australia, as well as data collected by the Australian Government. The PHRN is a distributed network where data is collected and managed at the respective jurisdictional and/or cross-jurisdictional levels. As a result, access to linked data from multiple jurisdictions requires complex approval processes. This paper describes Australia's approach to enabling access to linked data from multiple jurisdictions. It covers the identification of, and agreement to, a minimum set of data items to be included in a unified national application form, the development and implementation of a national online application system and the harmonisation of business processes for cross-jurisdictional research projects. Utilisation of the online application system and the ongoing challenges of data linkage across jurisdictions are discussed. Changes to the data custodian and ethics committee approval criteria were out of scope for this project.

Identifying researcher learning needs to develop online training for UK researchers working with administrative data: CENTRIC training

BACKGROUND

The use of administrative data in health and social science research continues to expand, with increased availability of data and interest from funders. Researchers, however, continue to experience delays in access, storage and sharing of administrative data. Training opportunities are limited and typically specific to individual data providers or focussed on the analytical aspects of working with administrative data. The CENTRIC study was funded by the Information Commissioners Office, with the aim of developing a broader training curriculum for researchers working with administrative data in the UK.

METHODS

A mixed-methods design informed curriculum content, including surveys with researchers, focus group discussions with data providers and workshops with members of the public. Researchers were identified from relevant administrative data networks and invited to participate in an online survey identifying training needs. Data providers were approached with a request to input to a face-to-face or online meeting with two members of the research team about their experiences of working with researchers. Data were analysed within the broad framework of the interview schedule, free text responses in the survey were analysed thematically.

RESULTS

107 researchers responded to the online survey and four data providers participated in the focus groups. We identified five main themes, relating to research training needs for UK researchers working with administrative data: communication; timelines; changes & amendments; future-proofing applications; and, the availability of training and support. Data providers either provided additional evidence on these learning needs or ways to address identified challenges. Six modules were developed addressing these training needs. Quotes from the survey and focus groups are used anonymously in the online training modules.

CONCLUSION

The CENTRIC online training curriculum was launched in September 2020 and is available, free of charge for UK researchers. CENTRIC specifically addresses commonly identified training needs of researchers working with administrative data.

The road to hell is paved with good intentions: the experience of applying for national data for linkage and suggestions for improvement

BACKGROUND

We can improve healthcare services by better understanding current provision. One way to understand this is by linking data sets from clinical and national audits, national registries and other National Health Service (NHS) encounter data. However, getting to the point of having linked national data sets is challenging.

OBJECTIVE

We describe our experience of the data application and linkage process for our study 'LAUNCHES QI', and the time, processes and resource requirements involved. To help others planning similar projects, we highlight challenges encountered and advice for applications in the current system as well as suggestions for system improvements.

FINDINGS

The study set up for LAUNCHES QI began in March 2018, and the process through to data acquisition took 2.5 years. Several challenges were encountered, including the amount of information required (often duplicate information in different formats across applications), lack of clarity on processes, resource constraints that limit an audit's capacity to fulfil requests and the unexpected amount of time required from the study team. It is incredibly difficult to estimate the resources needed ahead of time, and yet necessary to do so as early on as funding applications. Early decisions can have a significant impact during latter stages and be hard to change, yet it is difficult to get specific information at the beginning of the process.

CONCLUSIONS

The current system is incredibly complex, arduous and slow, stifling innovation and delaying scientific progress. NHS data can inform and improve health services and we believe there is an ethical responsibility to use it to do so. Streamlining the number of applications required for accessing data for health services research and providing clarity to data controllers could facilitate the maintenance of stringent governance, while accelerating scientific studies and progress, leading to swifter application of findings and improvements in healthcare.