Design considerations, architecture, and use of the Mini-Sentinel distributed data system

Developing a PRogram to Educate and Sensitize Caregivers to Reduce the Inappropriate Prescription Burden in the Elderly with Alzheimer's Disease (D-PRESCRIBE-AD): Trial protocol and rationale of an open-label pragmatic, prospective randomized controlled trial

CONTEXT

Potentially inappropriate prescribing of medications in older adults, particular those with dementia, can lead to adverse drug events including falls and fractures, worsening cognitive impairment, emergency department visits, and hospitalizations. Educational mailings from health plans to patients and their providers to encourage deprescribing conversations may represent an effective, low-cost, "light touch", approach to reducing the burden of potentially inappropriate prescription use in older adults with dementia.

OBJECTIVES

The objective of the Developing a PRogram to Educate and Sensitize Caregivers to Reduce the Inappropriate Prescription Burden in Elderly with Alzheimer's Disease (D-PRESCRIBE-AD) trial is to evaluate the effect of a health plan based multi-faceted educational outreach intervention to community dwelling patients with dementia who are currently prescribed sedative/hypnotics, antipsychotics, or strong anticholinergics.

METHODS

The D-PRESCRIBE-AD is an open-label pragmatic, prospective randomized controlled trial (RCT) comparing three arms: 1) educational mailing to both the health plan patient and their prescribing physician (patient plus physician arm, n = 4814); 2) educational mailing to prescribing physician only (physician only arm, n = 4814); and 3) usual care (n = 4814) among patients with dementia enrolled in two large United States based health plans. The primary outcome is the absence of any dispensing of the targeted potentially inappropriate prescription during the 6-month study observation period after a 3-month black out period following the mailing. Secondary outcomes include dose-reduction, polypharmacy, healthcare utilization, mortality and therapeutic switching within targeted drug classes.

CONCLUSION

This large pragmatic RCT will contribute to the evidence base on promoting deprescribing of potentially inappropriate medications among older adults with dementia. If successful, such light touch, inexpensive and highly scalable interventions have the potential to reduce the burden of potentially inappropriate prescribing for patients with dementia. ClinicalTrials.gov Identifier: NCT05147428.

Risk of Arterial and Venous Thrombotic Events Among Patients with COVID-19: A Multi-National Collaboration of Regulatory Agencies from Canada, Europe, and United States

Purpose

Few studies have examined how the absolute risk of thromboembolism with COVID-19 has evolved over time across different countries. Researchers from the European Medicines Agency, Health Canada, and the United States (US) Food and Drug Administration established a collaboration to evaluate the absolute risk of arterial (ATE) and venous thromboembolism (VTE) in the 90 days after diagnosis of COVID-19 in the ambulatory (eg, outpatient, emergency department, nursing facility) setting from seven countries across North America (Canada, US) and Europe (England, Germany, Italy, Netherlands, and Spain) within periods before and during COVID-19 vaccine availability.

Patients and Methods

We conducted cohort studies of patients initially diagnosed with COVID-19 in the ambulatory setting from the seven specified countries. Patients were followed for 90 days after COVID-19 diagnosis. The primary outcomes were ATE and VTE over 90 days from diagnosis date. We measured country-level estimates of 90-day absolute risk (with 95% confidence intervals) of ATE and VTE.

Results

The seven cohorts included 1,061,565 patients initially diagnosed with COVID-19 in the ambulatory setting before COVID-19 vaccines were available (through November 2020). The 90-day absolute risk of ATE during this period ranged from 0.11% (0.09-0.13%) in Canada to 1.01% (0.97-1.05%) in the US, and the 90-day absolute risk of VTE ranged from 0.23% (0.21-0.26%) in Canada to 0.84% (0.80-0.89%) in England. The seven cohorts included 3,544,062 patients with COVID-19 during vaccine availability (beginning December 2020). The 90-day absolute risk of ATE during this period ranged from 0.06% (0.06-0.07%) in England to 1.04% (1.01-1.06%) in the US, and the 90-day absolute risk of VTE ranged from 0.25% (0.24-0.26%) in England to 1.02% (0.99-1.04%) in the US.

Conclusion

There was heterogeneity by country in 90-day absolute risk of ATE and VTE after ambulatory COVID-19 diagnosis both before and during COVID-19 vaccine availability.

Factors Influencing the Acceptance of Distributed Research Networks in Korea: Data Accessibility and Data Security Risk

OBJECTIVES

Distributed research networks (DRNs) facilitate multicenter research by enabling the use of multicenter data; therefore, they are increasingly utilized in healthcare fields. Despite the numerous advantages of DRNs, it is crucial to understand researchers' acceptance of these networks to ensure their effective application in multicenter research. In this study, we sought to identify the factors influencing the adoption of DRNs among researchers in Korea.

METHODS

We used snowball sampling to collect data from 149 researchers between July 7 and August 28, 2020. Five factors were used to formulate the hypotheses and research model: data accessibility, usefulness, ease of use, data security risk, and intention to use DRNs. We applied a structural equation model to identify relationships within the research model.

RESULTS

Data accessibility and data security were critical to the acceptance and use of DRNs. The usefulness of DRNs partially mediated the relationship between data accessibility and the intention to use DRNs. Interestingly, ease of use did not influence the intention to use DRNs, but it was affected by data accessibility. Furthermore, ease of use impacted the perceived usefulness of DRNs.

CONCLUSIONS

This study highlighted major factors that can promote the broader adoption and utilization of DRNs. Consequently, these findings can contribute to the expansion of active multicenter research using DRNs in the field of healthcare research.

Six Years of the US Food and Drug Administration's Postmarket Active Risk Identification and Analysis System in the Sentinel Initiative: Implications for Real World Evidence Generation

Congress mandated the creation of a postmarket Active Risk Identification and Analysis (ARIA) system containing data on 100 million individuals for monitoring risks associated with drug and biologic products using data from disparate sources to complement the US Food and Drug Administration's (FDA's) existing postmarket capabilities. We report on the first 6 years of ARIA utilization in the Sentinel System (2016-2021). The FDA has used the ARIA system to evaluate 133 safety concerns; 54 of these evaluations have closed with regulatory determinations, whereas the rest remain in progress. If the ARIA system and the FDA's Adverse Event Reporting System are deemed insufficient to address a safety concern, then the FDA may issue a postmarket requirement to a product's manufacturer. One hundred ninety-seven ARIA insufficiency determinations have been made. The most common situation for which ARIA was found to be insufficient is the evaluation of adverse pregnancy and fetal outcomes following in utero drug exposure, followed by neoplasms and death. ARIA was most likely to be sufficient for thromboembolic events, which have high positive predictive value in claims data alone and do not require supplemental clinical data. The lessons learned from this experience illustrate the continued challenges using administrative claims data, especially to define novel clinical outcomes. This analysis can help to identify where more granular clinical data are needed to fill gaps to improve the use of real-world data for drug safety analyses and provide insights into what is needed to efficiently generate high-quality real-world evidence for efficacy.

Risk of admission to hospital with arterial or venous thromboembolism among patients diagnosed in the ambulatory setting with covid-19 compared with influenza: retrospective cohort study

Objective

To measure the 90 day risk of arterial thromboembolism and venous thromboembolism among patients diagnosed with covid-19 in the ambulatory (ie, outpatient, emergency department, or institutional) setting during periods before and during covid-19 vaccine availability and compare results to patients with ambulatory diagnosed influenza.

Design

Retrospective cohort study.

Setting

Four integrated health systems and two national health insurers in the US Food and Drug Administration's Sentinel System.

Participants

Patients with ambulatory diagnosed covid-19 when vaccines were unavailable in the US (period 1, 1 April-30 November 2020; n=272 065) and when vaccines were available in the US (period 2, 1 December 2020-31 May 2021; n=342 103), and patients with ambulatory diagnosed influenza (1 October 2018-30 April 2019; n=118 618).

Main outcome measures

Arterial thromboembolism (hospital diagnosis of acute myocardial infarction or ischemic stroke) and venous thromboembolism (hospital diagnosis of acute deep venous thrombosis or pulmonary embolism) within 90 days after ambulatory covid-19 or influenza diagnosis. We developed propensity scores to account for differences between the cohorts and used weighted Cox regression to estimate adjusted hazard ratios of outcomes with 95% confidence intervals for covid-19 during periods 1 and 2 versus influenza.

Results

90 day absolute risk of arterial thromboembolism with covid-19 was 1.01% (95% confidence interval 0.97% to 1.05%) during period 1, 1.06% (1.03% to 1.10%) during period 2, and with influenza was 0.45% (0.41% to 0.49%). The risk of arterial thromboembolism was higher for patients with covid-19 during period 1 (adjusted hazard ratio 1.53 (95% confidence interval 1.38 to 1.69)) and period 2 (1.69 (1.53 to 1.86)) than for patients with influenza. 90 day absolute risk of venous thromboembolism with covid-19 was 0.73% (0.70% to 0.77%) during period 1, 0.88% (0.84 to 0.91%) during period 2, and with influenza was 0.18% (0.16% to 0.21%). Risk of venous thromboembolism was higher with covid-19 during period 1 (adjusted hazard ratio 2.86 (2.46 to 3.32)) and period 2 (3.56 (3.08 to 4.12)) than with influenza.

Conclusions

Patients diagnosed with covid-19 in the ambulatory setting had a higher 90 day risk of admission to hospital with arterial thromboembolism and venous thromboembolism both before and after covid-19 vaccine availability compared with patients with influenza.

Development and implementation of a distributed data network between an academic institution and state health departments to investigate variation in time to HIV viral suppression in the Deep South

BACKGROUND

Achieving early and sustained viral suppression (VS) following diagnosis of HIV infection is critical to improving outcomes for persons with HIV (PWH). The Deep South of the United States (US) is a region that is disproportionately impacted by the domestic HIV epidemic. Time to VS, defined as time from diagnosis to initial VS, is substantially longer in the South than other regions of the US. We describe the development and implementation of a distributed data network between an academic institution and state health departments to investigate variation in time to VS in the Deep South.

METHODS

Representatives of state health departments, the Centers for Disease Control and Prevention (CDC), and the academic partner met to establish core objectives and procedures at the beginning of the project. Importantly, this project used the CDC-developed Enhanced HIV/AIDS Reporting System (eHARS) through a distributed data network model that maintained the confidentiality and integrity of the data. Software programs to build datasets and calculate time to VS were written by the academic partner and shared with each public health partner. To develop spatial elements of the eHARS data, health departments geocoded residential addresses of each newly diagnosed individual in eHARS between 2012-2019, supported by the academic partner. Health departments conducted all analyses within their own systems. Aggregate results were combined across states using meta-analysis techniques. Additionally, we created a synthetic eHARS data set for code development and testing.

RESULTS

The collaborative structure and distributed data network have allowed us to refine the study questions and analytic plans to conduct investigations into variation in time to VS for both research and public health practice. Additionally, a synthetic eHARS data set has been created and is publicly available for researchers and public health practitioners.

CONCLUSIONS

These efforts have leveraged the practice expertise and surveillance data within state health departments and the analytic and methodologic expertise of the academic partner. This study could serve as an illustrative example of effective collaboration between academic institutions and public health agencies and provides resources to facilitate future use of the US HIV surveillance system for research and public health practice.

The need for multimodal health data modeling: a practical approach for a federated-learning healthcare platform

Federated learning initiatives in healthcare are being developed to collaboratively train predictive models without the need to centralize sensitive personal data. GenoMed4All is one such project, with the goal of connecting European clinical and -omics data repositories on rare diseases through a federated learning platform. Currently, the consortium faces the challenge of a lack of well-established international datasets and interoperability standards for federated learning applications on rare diseases. This paper presents our practical approach to select and implement a Common Data Model (CDM) suitable for the federated training of predictive models applied to the medical domain, during the initial design phase of our federated learning platform. We describe our selection process, composed of identifying the consortium's needs, reviewing our functional and technical architecture specifications, and extracting a list of business requirements. We review the state of the art and evaluate three widely-used approaches (FHIR, OMOP and Phenopackets) based on a checklist of requirements and specifications. We discuss the pros and cons of each approach considering the use cases specific to our consortium as well as the generic issues of implementing a European federated learning healthcare platform. A list of lessons learned from the experience in our consortium is discussed, from the importance of establishing the proper communication channels for all stakeholders to technical aspects related to -omics data. For federated learning projects focused on secondary use of health data for predictive modeling, encompassing multiple data modalities, a phase of data model convergence is sorely needed to gather different data representations developed in the context of medical research, interoperability of clinical care software, imaging, and -omics analysis into a coherent, unified data model. Our work identifies this need and presents our experience and a list of actionable lessons learned for future work in this direction.

A new analytic tool for assessing the impact of the US Food and Drug Administration regulatory actions

PURPOSE

Develop and test a flexible, scalable tool using interrupted time series (ITS) analysis to assess the impact of Food and Drug Administration (FDA) regulatory actions on drug use.

METHODS

We applied the tool in the Sentinel Distributed Database to assess the impact of FDA's 2010 drug safety communications (DSC) concerning the safety of long-acting beta2-agonists (LABA) in adult asthma patients. We evaluated changes in LABA use by measuring the initiation of LABA alone and concomitant use of LABA and asthma controller medications (ACM) after the DSCs. The tool generated ITS graphs and used segmented regression to estimate baseline slope, level change, slope change, and absolute and relative changes at up to two user-specified time point (s) after the intervention. We tested the tool and compared our results against prior analyses that used similar measures.

RESULTS

Initiation of LABA alone declined among asthma patients aged 18-45 years before FDA DSCs (-0.10% per quarter; 95%CI: -0.11% to -0.09%) and the downward trend continued after. Concomitant use of LABA and ACM was stable before FDA DSCs. After FDA DSCs, there was a small trend decrease of 0.006% per quarter (95% CI, -0.008% to -0.003%). We found similar results among those aged 46-64 years and patients with poorly-controlled asthma. Our results were consistent with previous studies, confirming the performance of the new tool.

CONCLUSIONS

We developed and tested a reusable ITS tool in real-world databases formatted to the Sentinel Common Data Model that can assess the impact of regulatory actions on drug use.

Comparative Risk of Angioedema With Sacubitril-Valsartan vs Renin-Angiotensin-Aldosterone Inhibitors

BACKGROUND

Data on angioedema risk among sacubitril-valsartan (SV) users in real-world settings are limited.

OBJECTIVES

We sought to evaluate the risk of angioedema among SV new users compared with angiotensin-converting enzyme (ACE) inhibitor and angiotensin-receptor-blocker (ARB) new users separately.

METHODS

We conducted a propensity score-matched cohort study, comparing SV new users (no use of SV, ACE inhibitor, ARB 6 months before) and SV new users with prior use (within 183 or 14 days) of ACE inhibitor or ARB (ACE inhibitor-SV and ARB-SV users; recent ACE inhibitor-SV and recent ARB-SV users, respectively) vs ACE inhibitor and ARB new users separately.

RESULTS

Compared with ACE inhibitor, SV new (HR: 0.18; 95% CI: 0.11-0.29) and ACE inhibitor-SV users (HR: 0.31; 95% CI: 0.23-0.43) showed lower risk of angioedema. On the other hand, there was no difference in angioedema risk when SV new users (HR: 0.59; 95% CI: 0.35-1.01) or ARB-SV users (HR: 0.85; 95% CI: 0.58-1.26) were compared with ARB new users. Compared with SV new users, ACE inhibitor-SV users (HR: 1.62; 95% CI: 0.91-2.89) trended toward higher angioedema risk, which intensified when the ACE inhibitor to SV switch occurred within 14 days (recent ACE inhibitor-SV) (HR: 1.98; 95% CI: 1.11-3.53). Similarly, ARB-SV users (HR: 2.03; 95% CI: 1.16-3.54) experienced an increased risk compared with SV new users, which intensified for the more recent switchers (recent ARB-SV) (HR: 2.45; 95% CI: 1.36-4.43).

CONCLUSIONS

We did not observe an increased risk of angioedema among SV new users compared with ACE inhibitor or ARB users. However, there was an increased risk of angioedema among SV users who recently switched from ACE inhibitor or ARB compared with SV new users.

Ovarian Real-World International Consortium (ORWIC): A multicentre, real-world analysis of epithelial ovarian cancer treatment and outcomes

Introduction

Much drug development and published analysis for epithelial ovarian cancer (EOC) focuses on early-line treatment. Full sequences of treatment from diagnosis to death and the impact of later lines of therapy are rarely studied. We describe the establishment of an international network of cancer centers configured to compare real-world treatment pathways in UK, Portugal, Germany, South Korea, France and Romania (the Ovarian Real-World International Consortium; ORWIC).

Methods

3344 patients diagnosed with EOC (2012-2018) were analysed using a common data model and hub and spoke programming approach applied to existing electronic medical records. Consistent definition of line of therapy between sites and an efficient approach to analysis within the limitations of local information governance was achieved.

Results

Median age of participants was 53-67 years old and 5-29% were ECOG >1. Between 62% and 84% of patients were diagnosed with late-stage disease (FIGO III-IV). Sites treating younger and fitter patients had higher rates of debulking surgery for those diagnosed at late stage than sites with older, more frail patients. At least 21% of patients treated with systemic anti-cancer therapy (SACT) had recurrent disease following second-line therapy (2L); up to 11 lines of SACT treatment were recorded for some patients. Platinum-based SACT was consistently used across sites at 1L, but choices at 2L varied, with hormone therapies commonly used in the UK and Portugal. The use (and type) of maintenance therapy following 1L also varied. Beyond 2L, there was little consensus between sites on treatment choice: trial compounds and unspecified combinations of other agents were common.

Discussion

Specific treatment sequences are reported up to 4L and the establishment of this network facilitates future analysis of comparative outcomes per line of treatment with the aim of optimizing available options for patients with recurrent EOC. In particular, this real-world network can be used to assess the growing use of PARP inhibitors. The real-world optimization of advanced line treatment will be especially important for patients not usually eligible for involvement with clinical trials. The resources to enable this analysis to be implemented elsewhere are supplied and the network will seek to grow in coverage of further sites.

Assessing the impact of privacy-preserving record linkage on record overlap and patient demographic and clinical characteristics in PCORnet®, the National Patient-Centered Clinical Research Network

OBJECTIVE

This article describes the implementation of a privacy-preserving record linkage (PPRL) solution across PCORnet®, the National Patient-Centered Clinical Research Network.

MATERIAL AND METHODS

Using a PPRL solution from Datavant, we quantified the degree of patient overlap across the network and report a de-duplicated analysis of the demographic and clinical characteristics of the PCORnet population.

RESULTS

There were ∼170M patient records across the responding Network Partners, with ∼138M (81%) of those corresponding to a unique patient. 82.1% of patients were found in a single partner and 14.7% were in 2. The percentage overlap between Partners ranged between 0% and 80% with a median of 0%. Linking patients' electronic health records with claims increased disease prevalence in every clinical characteristic, ranging between 63% and 173%.

DISCUSSION

The overlap between Partners was variable and depended on timeframe. However, patient data linkage changed the prevalence profile of the PCORnet patient population.

CONCLUSIONS

This project was one of the largest linkage efforts of its kind and demonstrates the potential value of record linkage. Linkage between Partners may be most useful in cases where there is geographic proximity between Partners, an expectation that potential linkage Partners will be able to fill gaps in data, or a longer study timeframe.

The US Food and Drug Administration Sentinel System: a national resource for a learning health system

The US Food and Drug Administration (FDA) created the Sentinel System in response to a requirement in the FDA Amendments Act of 2007 that the agency establish a system for monitoring risks associated with drug and biologic products using data from disparate sources. The Sentinel System has completed hundreds of analyses, including many that have directly informed regulatory decisions. The Sentinel System also was designed to support a national infrastructure for a learning health system. Sentinel governance and guiding principles were designed to facilitate Sentinel's role as a national resource. The Sentinel System infrastructure now supports multiple non-FDA projects for stakeholders ranging from regulated industry to other federal agencies, international regulators, and academics. The Sentinel System is a working example of a learning health system that is expanding with the potential to create a global learning health system that can support medical product safety assessments and other research.

Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning

We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015-2019 in 2 integrated health-care institutions in the Northwest United States. We used one site's manually reviewed gold-standard outcomes data for model development and the other's for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events.

Developing a common data model approach for DISCOVER CKD: A retrospective, global cohort of real-world patients with chronic kidney disease

Objectives

To describe a flexible common data model (CDM) approach that can be efficiently tailored to study-specific needs to facilitate pooled patient-level analysis and aggregated/meta-analysis of routinely collected retrospective patient data from disparate data sources; and to detail the application of this CDM approach to the DISCOVER CKD retrospective cohort, a longitudinal database of routinely collected (secondary) patient data of individuals with chronic kidney disease (CKD).

Methods

The flexible CDM approach incorporated three independent, exchangeable components that preceded data mapping and data model implementation: (1) standardized code lists (unifying medical events from different coding systems); (2) laboratory unit harmonization tables; and (3) base cohort definitions. Events between different coding vocabularies were not mapped code-to-code; for each data source, code lists of labels were curated at the entity/event level. A study team of epidemiologists, clinicians, informaticists, and data scientists were included within the validation of each component.

Results

Applying the CDM to the DISCOVER CKD retrospective cohort, secondary data from 1,857,593 patients with CKD were harmonized from five data sources, across three countries, into a discrete database for rapid real-world evidence generation.

Conclusions

This flexible CDM approach facilitates evidence generation from real-world data within the DISCOVER CKD retrospective cohort, providing novel insights into the epidemiology of CKD that may expedite improvements in diagnosis, prognosis, early intervention, and disease management. The adaptable architecture of this CDM approach ensures scalable, fast, and efficient application within other therapy areas to facilitate the combined analysis of different types of secondary data from multiple, heterogeneous sources.

A concentric circles view of health data relations facilitates understanding of sociotechnical challenges for learning health systems and the role of federated data networks

The ability to use clinical and research data at scale is central to hopes for data-driven medicine. However, in using such data researchers often encounter hurdles–both technical, such as differing data security requirements, and social, such as the terms of informed consent, legal requirements and patient and public trust. Federated or distributed data networks have been proposed and adopted in response to these hurdles. However, to date there has been little consideration of how FDNs respond to both technical and social constraints on data use. In this Perspective we propose an approach to thinking about data in terms that make it easier to navigate the health data space and understand the value of differing approaches to data collection, storage and sharing. We set out a socio-technical model of data systems that we call the “Concentric Circles View” (CCV) of data-relationships. The aim is to enable a consistent understanding of the fit between the local relationships within which data are produced and the extended socio-technical systems that enable their use. The paper suggests this model can help understand and tackle challenges associated with the use of real-world data in the health setting. We use the model to understand not only how but why federated networks may be well placed to address emerging issues and adapt to the evolving needs of health research for patient benefit. We conclude that the CCV provides a useful model with broader application in mapping, understanding, and tackling the major challenges associated with using real world data in the health setting.

Design, Implementation, and Evolution of the Medicaid Outcomes Distributed Research Network (MODRN)

BACKGROUND

In the US, Medicaid covers over 80 million Americans. Comparing access, quality, and costs across Medicaid programs can provide policymakers with much-needed information. As each Medicaid agency collects its member data, multiple barriers prevent sharing Medicaid data between states. To address this gap, the Medicaid Outcomes Distributed Research Network (MODRN) developed a research network of states to conduct rapid multi-state analyses without sharing individual-level data across states.

OBJECTIVE

To describe goals, design, implementation, and evolution of MODRN to inform other research networks.

METHODS

MODRN implemented a distributed research network using a common data model, with each state analyzing its own data; developed standardized measure specifications and statistical software code to conduct analyses; and disseminated findings to state and federal Medicaid policymakers. Based on feedback on Medicaid agency priorities, MODRN first sought to inform Medicaid policy to improve opioid use disorder treatment, particularly medication treatment.

RESULTS

Since its 2017 inception, MODRN created 21 opioid use disorder quality measures in 13 states. MODRN modified its common data model over time to include additional elements. Initial barriers included harmonizing utilization data from Medicaid billing codes across states and adapting statistical methods to combine state-level results. The network demonstrated its utility and addressed barriers to conducting multi-state analyses of Medicaid administrative data.

CONCLUSIONS

MODRN created a new, scalable, successful model for conducting policy research while complying with federal and state regulations to protect beneficiary health information. Platforms like MODRN may prove useful for emerging health challenges to facilitate evidence-based policymaking in Medicaid programs.

Association of COVID-19 vs Influenza With Risk of Arterial and Venous Thrombotic Events Among Hospitalized Patients

IMPORTANCE

The incidence of arterial thromboembolism and venous thromboembolism in persons with COVID-19 remains unclear.

OBJECTIVE

To measure the 90-day risk of arterial thromboembolism and venous thromboembolism in patients hospitalized with COVID-19 before or during COVID-19 vaccine availability vs patients hospitalized with influenza.

DESIGN, SETTING, AND PARTICIPANTS

Retrospective cohort study of 41 443 patients hospitalized with COVID-19 before vaccine availability (April-November 2020), 44 194 patients hospitalized with COVID-19 during vaccine availability (December 2020-May 2021), and 8269 patients hospitalized with influenza (October 2018-April 2019) in the US Food and Drug Administration Sentinel System (data from 2 national health insurers and 4 regional integrated health systems).

EXPOSURES

COVID-19 or influenza (identified by hospital diagnosis or nucleic acid test).

MAIN OUTCOMES AND MEASURES

Hospital diagnosis of arterial thromboembolism (acute myocardial infarction or ischemic stroke) and venous thromboembolism (deep vein thrombosis or pulmonary embolism) within 90 days. Outcomes were ascertained through July 2019 for patients with influenza and through August 2021 for patients with COVID-19. Propensity scores with fine stratification were developed to account for differences between the influenza and COVID-19 cohorts. Weighted Cox regression was used to estimate the adjusted hazard ratios (HRs) for outcomes during each COVID-19 vaccine availability period vs the influenza period.

RESULTS

A total of 85 637 patients with COVID-19 (mean age, 72 [SD, 13.0] years; 50.5% were male) and 8269 with influenza (mean age, 72 [SD, 13.3] years; 45.0% were male) were included. The 90-day absolute risk of arterial thromboembolism was 14.4% (95% CI, 13.6%-15.2%) in patients with influenza vs 15.8% (95% CI, 15.5%-16.2%) in patients with COVID-19 before vaccine availability (risk difference, 1.4% [95% CI, 1.0%-2.3%]) and 16.3% (95% CI, 16.0%-16.6%) in patients with COVID-19 during vaccine availability (risk difference, 1.9% [95% CI, 1.1%-2.7%]). Compared with patients with influenza, the risk of arterial thromboembolism was not significantly higher among patients with COVID-19 before vaccine availability (adjusted HR, 1.04 [95% CI, 0.97-1.11]) or during vaccine availability (adjusted HR, 1.07 [95% CI, 1.00-1.14]). The 90-day absolute risk of venous thromboembolism was 5.3% (95% CI, 4.9%-5.8%) in patients with influenza vs 9.5% (95% CI, 9.2%-9.7%) in patients with COVID-19 before vaccine availability (risk difference, 4.1% [95% CI, 3.6%-4.7%]) and 10.9% (95% CI, 10.6%-11.1%) in patients with COVID-19 during vaccine availability (risk difference, 5.5% [95% CI, 5.0%-6.1%]). Compared with patients with influenza, the risk of venous thromboembolism was significantly higher among patients with COVID-19 before vaccine availability (adjusted HR, 1.60 [95% CI, 1.43-1.79]) and during vaccine availability (adjusted HR, 1.89 [95% CI, 1.68-2.12]).

CONCLUSIONS AND RELEVANCE

Based on data from a US public health surveillance system, hospitalization with COVID-19 before and during vaccine availability, vs hospitalization with influenza in 2018-2019, was significantly associated with a higher risk of venous thromboembolism within 90 days, but there was no significant difference in the risk of arterial thromboembolism within 90 days.

The Value of Federated Data Networks in Oncology: What Research Questions Do They Answer? Outcomes From a Systematic Literature Review

OBJECTIVES

Real-world evidence (RWE) plays an important role in addressing key research questions of interest to healthcare decision makers. Federated data networks (FDNs) apply novel technology to enable the conduct of RWE studies with multiple partners, without the need to share the individual partner's data set. A systematic review of the published literature was performed to determine which types of research questions can best be addressed through FDNs, specifically in the field of oncology.

METHODS

Systematic searches of MEDLINE and Embase were undertaken to identify the types of research questions that had been addressed in studies using FDNs. Additional information was retrieved about study characteristics, statistical methods, and the FDN itself.

RESULTS

In total, 40 publications were included where research questions on the following had been addressed (multiple categories possible): disease natural history (58%), safety surveillance (18%), treatment pathways (15%), comparative effectiveness (10%), and cost/resource use studies (3%)-13% of studies had to be left uncategorized. A total of 50% of the studies were run with data partners in networks of ≤5. The size of the networks ranged from 227 patients to >5 million patients. Statistical methods used included distributed learning and distributed regression methods.

CONCLUSIONS

Further work is needed to raise awareness of the important role that FDNs can play in leveraging readily available RWE to address key research questions of interest in cancer and the benefits to the research community in engaging in federated data initiatives with a long-term perspective.

Characteristics of new adult users of mepolizumab with asthma in the USA

BACKGROUND

In the USA, over 25 million people have asthma; 5%-10% of cases are severe. Mepolizumab (Nucala) is an interleukin-5 antagonist monoclonal antibody; it was approved by the FDA in 2015 as add-on maintenance treatment of severe asthma for patients aged ≥12 years with an eosinophilic phenotype.

OBJECTIVES

(1) Describe baseline demographic and clinical characteristics of new US adult mepolizumab users 2015-2019, (2) describe asthma medication use in the 12 months preceding initiation of and concomitant with mepolizumab and (3) assess mepolizumab adherence, persistence and discontinuation patterns in 12 months postinitiation.

METHODS

We conducted a new-user observational cohort study using data from Aetna, a CVS Health Company, HealthCore (Anthem), Harvard Pilgrim Healthcare, and IBM MarketScan Research Databases. Curated administrative claims data in the FDA Sentinel System common data model format and publicly available Sentinel analytical tools were used to query the databases. We included adults who initiated mepolizumab in 2015-2019 with an asthma diagnosis in the preceding 12 months and no evidence of cystic fibrosis. We examined age, sex, comorbid conditions, asthma medication use and severe asthma exacerbations.

RESULTS

We identified 3496 adults (mean age 54.2 years, SD 12.5 years) who initiated mepolizumab. In the 12 months before mepolizumab initiation, 22% had received inhaled corticosteroids, 46% had inhaled corticosteroid/long-acting beta agonists, 72.6% had leukotriene antagonists, 38% had long-acting muscarinic antagonist, 18% had omalizumab,<1% had reslizumab, dupilumab or benralizumab. In the previous 12 months, 70% had a diagnosis of allergic rhinitis, 32% had chronic obstructive pulmonary disease, 17% eosinophilia and 3% eosinophilic granulomatosis with polyangiitis. Further, 56% had an asthma-related ambulatory visit, 73%≥1 course of oral corticosteroids lasting 3-27 days, 10% an asthma-related emergency department visit and 22% an asthma-related hospitalisation. In the 12 months following initiation, the mean proportion of days covered was 70%, and reductions in the average mean dispensings of rescue oral corticosteriods (35%) and omalizumab (61%) were observed.

CONCLUSIONS

Adults with asthma treated with mepolizumab had varying levels of healthcare utilisation and we observed evidence of mepolizumab use in patients without severe asthma.

Safety of medicines and vaccines - building next generation capability

The systematic safety surveillance of real-world use of medicinal products and related activities (pharmacovigilance) started in earnest as a scientific field only in the 1960s. While developments have occurred over the past 50 years, adding to its complexity and sophistication, the extent to which some of these advances have positively impacted the capability for ensuring patient safety is questionable. We review how the conduct of safety surveillance has changed, highlight recent scientific advances, and argue how they need to be harnessed to enhance pharmacovigilance in the future. Specifically, we describe five changes that we believe should and will need to happen globally in the coming years: (i) better, more diverse data used for safety; (ii) the switch from manual activities to automation; (iii) removal of limited value, extraneous transactional activities and replacement with sharpened focus on scientific efforts to improve patient safety; (iv) patient-involved and focussed safety; and (v) personalised safety.

Measuring the Effectiveness of Real-World Evidence to Ensure Appropriate Impact

The value of real-world evidence (RWE) in medicines regulation and health technology assessment has been increasingly emphasized. Nevertheless, although RWE is increasingly used, there has been limited systematic evidence of its value. A recent study that examined the role and impact of RWE in regulatory assessments conducted through the European Medicines Agency provided such evidence. Results of the study demonstrated RWE was important to decision making, particularly for certain questions such as the quantification of adverse events, the evaluation of risk minimization measures, and the assessment of product usage. The study suggested, however, that in many of the assessments further RWE would have been valuable and concluded that RWE has, as yet, played a limited role in hypothesis generation and in the assessment of medication effectiveness. This study had been possible only because of the transparency of the European Medicines Agency decision making. Ensuring transparency of RWE evidence collection, study design and conduct, and of decision making based on this evidence will facilitate further development of the uses and value of RWE. Keywords: benefit-risk assessment; medicines regulation; real-world evidence; regulatory decision making.

Association between methylphenidate and risk of myocardial infarction: A multinational self-controlled case series study

PURPOSE

To investigate the association between use of methylphenidate and risk of myocardial infarction among Asians.

METHODS

We conducted a multinational self-controlled case series study using nationwide healthcare databases of South Korea (2002-2018), Taiwan (2004-2015), and Hong Kong (2001-2016). Of patients with myocardial infarction who were also prescribed methylphenidate within the observation period, methylphenidate use was classified into four mutually exclusive periods by each person-day: exposed (exposed to methylphenidate), pre-exposure (prior to the first methylphenidate prescription), washout (after the end of methylphenidate treatment), and baseline (unexposed to methylphenidate). Risk of myocardial infarction among the three periods of methylphenidate use was compared to the baseline period using conditional Poisson regression analysis to estimate incidence rate ratios (IRRs) with 95% confidence intervals (CIs).

RESULTS

We identified 2104, 484, and 30 patients from South Korea, Taiwan, and Hong Kong, respectively. Risk of myocardial infarction was the highest during the pre-exposure period in all three populations: South Korea, pre-exposure (IRR 3.17, 95% CI 3.04-3.32), exposed (1.05, 1.00-1.11), washout (1.92, 1.80-2.04); Taiwan, pre-exposure (1.97, 1.78-2.17), exposed (0.72, 0.65-0.80), washout (0.56, 0.46-0.68); Hong Kong, pre-exposure (18.09, 8.19-39.96), exposed (9.32, 3.44-25.28), washout (7.69, 1.72-34.41). Following stratification for age and sex, the trends remained analogous to the main findings across all three populations.

CONCLUSIONS

Although a positive association between initiating methylphenidate and the onset of myocardial infarction was observed, the risk was the highest in the period before its initiation. Thus, this multinational study suggests there was no causal relationship between methylphenidate and myocardial infarction among Asians.

Integrating Real-World Evidence in the Regulatory Decision-Making Process: A Systematic Analysis of Experiences in the US, EU, and China Using a Logic Model

Real world evidence (RWE) and real-world data (RWD) are drawing ever-increasing attention in the pharmaceutical industry and drug regulatory authorities (DRAs) all over the world due to their paramount role in supporting drug development and regulatory decision making. However, there is little systematic documentary analysis about how RWE was integrated for the use by the DRAs in evaluating new treatment approaches and monitoring post-market safety. This study aimed to analyze and discuss the integration of RWE into regulatory decision-making process from the perspective of DRAs. Different development strategies to develop and adopt RWE by the DRAs in the US, Europe, and China were reviewed and compared, and the challenges encountered were discussed. It was found that different strategies on development of RWE were applied by FDA, EMA, and NMPA. The extent to which RWE was adopted in China was relatively limited compared to that in the US and EU, which was highly related to the national pharmaceutical environment and development stages. A better understanding of the overall goals, inputs, activities, outputs, and outcomes in developing RWE will help inform actions to harness RWD and leverage RWE for better health care decisions.

Risk of Severe Abnormal Uterine Bleeding Associated with Rivaroxaban Compared with Apixaban, Dabigatran and Warfarin

INTRODUCTION

There have been reports of clinically relevant uterine bleeding events among women of reproductive age exposed to rivaroxaban.

OBJECTIVE

The aim of this study was to compare the risk of severe abnormal uterine bleeding (SAUB) resulting in transfusion or surgical intervention among women on rivaroxaban versus apixaban, dabigatran and warfarin.

METHODS

We conducted a retrospective cohort study in the FDA's Sentinel System (10/2010-09/2015) among females aged 18+ years with venous thromboembolism (VTE), or atrial flutter/fibrillation (AF) who newly initiated a direct oral anticoagulant (DOAC; rivaroxaban, apixaban, dabigatran) or warfarin. We followed women from dispensing date until the earliest of transfusion or surgery following vaginal bleeding, disenrollment, exposure or study end date, or recorded death. We estimated hazard ratios (HRs) using Cox proportional hazards regression via propensity score stratification. Four pairwise comparisons were conducted for each intervention.

RESULTS

Overall, there was an increased risk of surgical intervention with rivaroxaban when compared with dabigatran (HR 1.19; 95% CI 1.03-1.38), apixaban (1.23; 1.04-1.47), and warfarin (1.34; 1.22-1.47). No difference in risk for surgical intervention was observed for dabigatran-apixaban comparisons. Increased risk of transfusion was observed for rivaroxaban compared with dabigatran (1.49; 1.03-2.17) only. For patients with no gynecological history, rivaroxaban was associated with risk of surgical intervention compared with dabigatran (1.22; 1.05-1.42), apixaban (1.25; 1.04-1.49), and warfarin (1.36; 1.23-1.50).

CONCLUSION

Our study found increased SAUB risk with rivaroxaban use compared with other DOACs or warfarin. Increased risk with rivaroxaban was present among women without underlying gynecological conditions. Women on anticoagulant therapy should be aware of a risk of SAUB.

Validity of ICD-10-CM diagnoses to identify hospitalizations for serious infections among patients treated with biologic therapies

PURPOSE

Identifying hospitalizations for serious infections among patients dispensed biologic therapies within healthcare databases is important for post-marketing surveillance of these drugs. We determined the positive predictive value (PPV) of an ICD-10-CM-based diagnostic coding algorithm to identify hospitalization for serious infection among patients dispensed biologic therapy within the FDA's Sentinel Distributed Database.

METHODS

We identified health plan members who met the following algorithm criteria: (1) hospital ICD-10-CM discharge diagnosis of serious infection between July 1, 2016 and August 31, 2018; (2) either outpatient/emergency department infection diagnosis or outpatient antimicrobial treatment within 7 days prior to hospitalization; (3) inflammatory bowel disease, psoriasis, or rheumatological diagnosis within 1 year prior to hospitalization, and (4) were dispensed outpatient biologic therapy within 90 days prior to admission. Medical records were reviewed by infectious disease clinicians to adjudicate hospitalizations for serious infection. The PPV (95% confidence interval [CI]) for confirmed events was determined after further weighting by the prevalence of the type of serious infection in the database.

RESULTS

Among 223 selected health plan members who met the algorithm, 209 (93.7% [95% CI, 90.1%-96.9%]) were confirmed to have a hospitalization for serious infection. After weighting by the prevalence of the type of serious infection, the PPV of the ICD-10-CM algorithm identifying a hospitalization for serious infection was 80.2% (95% CI, 75.3%-84.7%).

CONCLUSIONS

The ICD-10-CM-based algorithm for hospitalization for serious infection among patients dispensed biologic therapies within the Sentinel Distributed Database had 80% PPV for confirmed events and could be considered for use within pharmacoepidemiologic studies.

Utilization of hydroxyprogesterone caproate among pregnancies with live birth deliveries in the sentinel distributed database

BACKGROUND

The U.S. Food and Drug Administration (FDA) approved Makena^® (hydroxyprogesterone caproate [HPC] injection) in February 2011 for reducing the risk of preterm birth (PTB) in women with a singleton pregnancy who had a history of singleton spontaneous PTB (sPTB). Makena was approved under accelerated approval and required a postmarketing study to verify its clinical benefits. However, the postmarketing trial (PROLONG) failed to verify Makena's clinical benefit to neonates and substantiate its effect on reducing the risk of recurrent PTB. This study examined the utilization of HPC, along with another progestogen (vaginal progesterone) used to reduce the risk of sPTB during pregnancy, to inform the landscape of HPC use in the United States.

METHODS

We included pregnant women aged 10-54 years with a live birth delivery from 1 January, 2008 to 31 December, 2018 in the Sentinel Distributed Database (SDD). We examined the prevalence of injectable HPC (Makena and its generics), compounded HPC, and vaginal progesterone use during the second and third trimesters during the study period. We also assessed the proportion of these HPC-exposed pregnancies with obstetrical conditions of interest as potential reasons for use: (1) history of preterm delivery; (2) cervical shortening in the current pregnancy; and (3) preterm labor in the current pregnancy.

RESULTS

We identified a total of 3,445,739 live-birth pregnancies (among 2.9 million women) between 2008 and 2018 in the SDD. Of these pregnancies, 6.5 per 1,000 pregnancies used injectable HPC, 2.3 per 1,000 pregnancies used compounded HPC, and 1.5 per 1,000 pregnancies used vaginal progesterone during the second and/or third trimesters. The yearly uptakeof pregnancies with injectable HPC use increased during the study period from 2.1 per 1,000 pregnancies in 2012 to 12.6 per 1,000 pregnancies in 2018; use of compounded HPC decreased from 3.3 per 1,000 pregnancies to 0.25 per 1,000 pregnancies over the same period. Of 16,524 pregnancies with injectable HPC use, 12,054 (73%) had at least one related obstetrical condition, including 6,439 (39%) with a recorded history of preterm delivery. In addition, 4,665 (28%) had a PTB recorded as the outcome for the current pregnancy.

CONCLUSIONS

We found modest use of HPC during the second and/or third trimesters among all live-birth pregnancies in SDD. The majority of pregnancies with injectable HPC use had at least one of three obstetrical indications of interest recorded before or during the pregnancy.

A COVID-19-ready public health surveillance system: The Food and Drug Administration's Sentinel System

The US Food and Drug Administration's Sentinel System was established in 2009 to use routinely collected electronic health data for improving the national capability to assess post‐market medical product safety. Over more than a decade, Sentinel has become an integral part of FDA's surveillance capabilities and has been used to conduct analyses that have contributed to regulatory decisions. FDA's role in the COVID‐19 pandemic response has necessitated an expansion and enhancement of Sentinel. Here we describe how the Sentinel System has supported FDA's response to the COVID‐19 pandemic. We highlight new capabilities developed, key data generated to date, and lessons learned, particularly with respect to working with inpatient electronic health record data.

Early in the pandemic, Sentinel developed a multi‐pronged approach to support FDA's anticipated data and analytic needs. It incorporated new data sources, created a rapidly refreshed database, developed protocols to assess the natural history of COVID‐19, validated a diagnosis‐code based algorithm for identifying patients with COVID‐19 in administrative claims data, and coordinated with other national and international initiatives.

Sentinel is poised to answer important questions about the natural history of COVID‐19 and is positioned to use this information to study the use, safety, and potentially the effectiveness of medical products used for COVID‐19 prevention and treatment.

Risk of Nonmelanoma Skin Cancer in Association With Use of Hydrochlorothiazide-Containing Products in the United States

Background

European studies reported an increased risk of nonmelanoma skin cancer associated with hydrochlorothiazide (HCTZ)-containing products. We examined the risks of basal cell carcinoma (BCC) and squamous cell carcinoma (SCC) associated with HCTZ compared with angiotensin-converting enzyme inhibitors (ACEIs) in a US population.

Methods

We conducted a retrospective cohort study in the US Food and Drug Administration's Sentinel System. From the date of HCTZ or ACEI dispensing, patients were followed until a SCC or BCC diagnosis requiring excision or topical chemotherapy treatment on or within 30 days after the diagnosis date or a censoring event. Using Cox proportional hazards regression models, we estimated the hazard ratios (HRs), overall and separately by age, sex, and race. We also examined site- and age-adjusted incidence rate ratios (IRRs) by cumulative HCTZ dose within the matched cohort.

Results

Among 5.2 million propensity-score matched HCTZ and ACEI users, the incidence rate (per 1000 person-years) of BCC was 2.78 and 2.82, respectively, and 1.66 and 1.60 for SCC. Overall, there was no difference in risk between HCTZ and ACEIs for BCC (HR = 0.99, 95% confidence interval [CI] = 0.97 to 1.00), but there was an increased risk for SCC (HR = 1.04, 95% CI = 1.02 to 1.06). HCTZ use was associated with higher risks of BCC (HR = 1.09, 95% CI = 1.07 to 1.11) and SCC (HR = 1.15, 95% CI = 1.12 to 1.17) among Caucasians. Cumulative HCTZ dose of 50 000 mg or more was associated with an increased risk of SCC in the overall population (IRR = 1.19, 95% CI = 1.05 to 1.35) and among Caucasians (IRR = 1.27, 95% CI = 1.10 to 1.47).

Conclusions

Among Caucasians, we identified small increased risks of BCC and SCC with HCTZ compared with ACEI. Appropriate risk mitigation strategies should be taken while using HCTZ.

A Framework (SOCRATex) for Hierarchical Annotation of Unstructured Electronic Health Records and Integration Into a Standardized Medical Database: Development and Usability Study

BACKGROUND

Although electronic health records (EHRs) have been widely used in secondary assessments, clinical documents are relatively less utilized owing to the lack of standardized clinical text frameworks across different institutions.

OBJECTIVE

This study aimed to develop a framework for processing unstructured clinical documents of EHRs and integration with standardized structured data.

METHODS

We developed a framework known as Staged Optimization of Curation, Regularization, and Annotation of clinical text (SOCRATex). SOCRATex has the following four aspects: (1) extracting clinical notes for the target population and preprocessing the data, (2) defining the annotation schema with a hierarchical structure, (3) performing document-level hierarchical annotation using the annotation schema, and (4) indexing annotations for a search engine system. To test the usability of the proposed framework, proof-of-concept studies were performed on EHRs. We defined three distinctive patient groups and extracted their clinical documents (ie, pathology reports, radiology reports, and admission notes). The documents were annotated and integrated into the Observational Medical Outcomes Partnership (OMOP)-common data model (CDM) database. The annotations were used for creating Cox proportional hazard models with different settings of clinical analyses to measure (1) all-cause mortality, (2) thyroid cancer recurrence, and (3) 30-day hospital readmission.

RESULTS

Overall, 1055 clinical documents of 953 patients were extracted and annotated using the defined annotation schemas. The generated annotations were indexed into an unstructured textual data repository. Using the annotations of pathology reports, we identified that node metastasis and lymphovascular tumor invasion were associated with all-cause mortality among colon and rectum cancer patients (both P=.02). The other analyses involving measuring thyroid cancer recurrence using radiology reports and 30-day hospital readmission using admission notes in depressive disorder patients also showed results consistent with previous findings.

CONCLUSIONS

We propose a framework for hierarchical annotation of textual data and integration into a standardized OMOP-CDM medical database. The proof-of-concept studies demonstrated that our framework can effectively process and integrate diverse clinical documents with standardized structured data for clinical research.

Using and improving distributed data networks to generate actionable evidence: the case of real-world outcomes in the Food and Drug Administration's Sentinel system

The US Food and Drug Administration (FDA) Sentinel System uses a distributed data network, a common data model, curated real-world data, and distributed analytic tools to generate evidence for FDA decision-making. Sentinel system needs include analytic flexibility, transparency, and reproducibility while protecting patient privacy. Based on over a decade of experience, a critical system limitation is the inability to identify enough medical conditions of interest in observational data to a satisfactory level of accuracy. Improving the system's ability to use computable phenotypes will require an "all of the above" approach that improves use of electronic health data while incorporating the growing array of complementary electronic health record data sources. FDA recently funded a Sentinel System Innovation Center and a Community Building and Outreach Center that will provide a platform for collaboration across disciplines to promote better use of real-world data for decision-making.

Physician decision support system for idiopathic sudden sensorineural hearing loss patients

BACKGROUND

Idiopathic sudden sensorineural hearing loss (ISSNHL) is an emergency disease, and its pathogenesis is still largely unknown, making it difficult to diagnose and develop a therapeutic strategy. To predict the treatment outcomes and further customize the treatment strategy, we used a physician decision support system that incorporates complex information from electronic health records. We first developed the infrastructure of the physician decision support system, including an integrated data warehouse and an automatic data de-identification workflow.

METHODS

We next conducted a cohort study to evaluate the treatment outcomes of 757 ISSNHL patients using the modified Siegel's criteria. The complete recovery (<25 dB) as a hearing outcome for ISSNHL patients was compared based on pretreatment hearing grades and disease onset with adjusted for age and sex after treatment initiation.

RESULTS

The results showed that a complete recovery hearing outcome based on pretreatment hearing grades and disease onset in consideration of age and sex was associated with a low risk of pretreatment hearing Grade 2 (26-45 dB) (adjusted odds ratio 12.3, 95% confidence interval [CI]: 4.8-31.3) and disease onset ≤7 days (adjusted odds ratio 13.9, 95% CI: 4.2-45.8), respectively. Hearing recovery outcomes among complete recovery and noncomplete recovery (>25 dB) subjects according to pretreatment hearing grades were 32.9% (Grade 2 or 26-45 dB HL), 25.4% (Grade 3 or 46-75 dB HL), 31.1% (Grade 4 or 76-90 dB), and 4.5% (Grade 5, or >90 dB HL) (p < 0.0001). Patients with pretreatment hearing Grade 2 who received treatment within ≤7 days of disease onset had the highest rate of complete recovery (32.9%, 23/70).

CONCLUSION

In summary, using the physician decision support system, we successfully identified two predictors, the pretreatment hearing Grade 2 (26-45 dB) and treatment within ≤7 days of disease onset, associated with the highest odds of achieving complete recovery (<25 dB) of hearing in patients with ISSNHL.

The FDA MyStudies app: a reusable platform for distributed clinical trials and real-world evidence studies

We developed a mobile application and secure patient data storage platform, FDA MyStudies, to address privacy, engagement, and extensibility challenges in mobile clinical research. The system extends the capabilities of the mobile frameworks Apple ResearchKit and ResearchStack through an intuitive front-end application and secure storage environment that can support health research studies. The platform supports single or multisite studies via role-based access and can be implemented within highly secure data environments. As a proof-of-concept, pregnant women participated in a descriptive study via the app in which data not routinely captured in electronic health records (EHR) were collected and linked with existing patient data to provide a more wholistic view of the patient and illustrate how patient data combined with EHR data could be used to support public health research.

Risk of Stroke and Bleeding in Atrial Fibrillation Treated with Apixaban Compared with Warfarin

BACKGROUND

A previous FDA study reported a favorable benefit risk for apixaban compared with warfarin for stroke prevention in older non-valvular atrial fibrillation (NVAF) patients (≥ 65 years). However, it remains unclear whether this favorable benefit risk persists in other populations including younger users. We examined if a similar benefit risk was observed in the Sentinel System and if it varied by age group.

OBJECTIVE

To examine the risk of ischemic stroke, gastrointestinal (GI) bleeding, and intracranial hemorrhage (ICH) in apixaban users compared with warfarin users in Sentinel Distributed Database (SDD).

DESIGN AND PARTICIPANTS

A retrospective new user cohort study was conducted among patients, 21 years and older initiating apixaban and warfarin for NVAF, between December 28, 2012, and June 30, 2018, in the SDD.

MAIN MEASURES

Cox proportional hazard regression was used to estimate the hazard ratios (HR) and 95% confidence intervals (95% CI) for each outcome (ischemic stroke, GI bleeding, and ICH) in propensity score matched apixaban users compared with the warfarin users. Subgroup analyses by age (21-64, 65-74, and 75+ years) were conducted.

KEY RESULTS

After matching, 55.3% and 58.4% (n = 55,038) of the apixaban and warfarin users were included in the main analysis. GI bleeding was the most common outcome. The HR (95% CI) for GI bleeding, ICH, and ischemic stroke in apixaban users compared with warfarin users were 0.57 (0.50-0.66), 0.53 (0.40-0.70), and 0.56 (0.45-0.71) respectively. The reduced risk of these outcomes in apixaban compared with warfarin users persisted across age groups.

CONCLUSION

In NVAF patients of all ages initiating either apixaban or warfarin for stroke prevention in the Sentinel System, apixaban was associated with a decreased risk of GI bleeding, ICH, and ischemic stroke compared with warfarin. Among patients less than 65 years of age, apixaban use was associated with a decreased risk of GI bleeding and ischemic stroke.

Developing a Standardized and Reusable Method to Link Distributed Health Plan Databases to the National Death Index: Methods Development Study Protocol

BACKGROUND

Certain medications may increase the risk of death or death from specific causes (eg, sudden cardiac death), but these risks may not be identified in premarket randomized trials. Having the capacity to examine death in postmarket safety surveillance activities is important to the US Food and Drug Administration's (FDA) mission to protect public health. Distributed networks of electronic health plan databases used by the FDA to conduct multicenter research or medical product safety surveillance studies often do not systematically include death or cause-of-death information.

OBJECTIVE

This study aims to develop reusable, generalizable methods for linking multiple health plan databases with the Centers for Disease Control and Prevention's National Death Index Plus (NDI+) data.

METHODS

We will develop efficient administrative workflows to facilitate multicenter institutional review board (IRB) review and approval within a distributed network of 6 health plans. The study will create a distributed NDI+ linkage process that avoids sharing of identifiable patient information between health plans or with a central coordinating center. We will develop standardized criteria for selecting and retaining NDI+ matches and methods for harmonizing linked information across multiple health plans. We will test our processes within a use case comprising users and nonusers of antiarrhythmic medications.

RESULTS

We will use the linked health plan and NDI+ data sets to estimate the incidences and incidence rates of mortality and specific causes of death within the study use case and compare the results with reported estimates. These comparisons provide an opportunity to assess the performance of the developed NDI+ linkage approach and lessons for future studies requiring NDI+ linkage in distributed database settings. This study is approved by the IRB at Harvard Pilgrim Health Care in Boston, MA. Results will be presented to the FDA at academic conferences and published in peer-reviewed journals.

CONCLUSIONS

This study will develop and test a reusable distributed NDI+ linkage approach with the goal of providing tested NDI+ linkage methods for use in future studies within distributed data networks. Having standardized and reusable methods for systematically obtaining death and cause-of-death information from NDI+ would enhance the FDA's ability to assess mortality-related safety questions in the postmarket, real-world setting.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/21811.

Artificial Intelligence, Real-World Automation and the Safety of Medicines

Despite huge technological advances in the capabilities to capture, store, link and analyse data electronically, there has been some but limited impact on routine pharmacovigilance. We discuss emerging research in the use of artificial intelligence, machine learning and automation across the pharmacovigilance lifecycle including pre-licensure. Reasons are provided on why adoption is challenging and we also provide a perspective on changes needed to accelerate adoption, and thereby improve patient safety. Last, we make clear that while technologies could be superimposed on existing pharmacovigilance processes for incremental improvements, these great societal advances in data and technology also provide us with a timely opportunity to reconsider everything we do in pharmacovigilance operations to maximise the benefit of these advances.

Distributed Regression Analysis Application in Large Distributed Data Networks: Analysis of Precision and Operational Performance

Background

A distributed data network approach combined with distributed regression analysis (DRA) can reduce the risk of disclosing sensitive individual and institutional information in multicenter studies. However, software that facilitates large-scale and efficient implementation of DRA is limited.

Objective

This study aimed to assess the precision and operational performance of a DRA application comprising a SAS-based DRA package and a file transfer workflow developed within the open-source distributed networking software PopMedNet in a horizontally partitioned distributed data network.

Methods

We executed the SAS-based DRA package to perform distributed linear, logistic, and Cox proportional hazards regression analysis on a real-world test case with 3 data partners. We used PopMedNet to iteratively and automatically transfer highly summarized information between the data partners and the analysis center. We compared the DRA results with the results from standard SAS procedures executed on the pooled individual-level dataset to evaluate the precision of the SAS-based DRA package. We computed the execution time of each step in the workflow to evaluate the operational performance of the PopMedNet-driven file transfer workflow.

Results

All DRA results were precise (<10⁻¹²), and DRA model fit curves were identical or similar to those obtained from the corresponding pooled individual-level data analyses. All regression models required less than 20 min for full end-to-end execution.

Conclusions

We integrated a SAS-based DRA package with PopMedNet and successfully tested the new capability within an active distributed data network. The study demonstrated the validity and feasibility of using DRA to enable more privacy-protecting analysis in multicenter studies.

Incidence of Serious Infections and Design of Utilization and Safety Studies for Biologic and Biosimilar Surveillance

BACKGROUND

There is a need for postmarketing evidence generation for novel biologics and biosimilars.

OBJECTIVE

To assess the feasibility, strengths, and limitations of the Biologics and Biosimilars Collective Intelligence Consortium (BBCIC) Distributed Research Network (DRN) to examine the utilization and comparative safety of immune-modulating agents among patients with autoimmune diseases.

METHODS

We conducted a retrospective cohort study among patients enrolled in health insurance plans participating in the BBCIC DRN between January 1, 2006, and September 30, 2015. Eligible patients were adult (≥18 years) new users of a disease-modifying nonbiologic and/or biologic agent with a prior diagnosis of rheumatoid arthritis (RA), other inflammatory conditions (psoriasis, psoriatic arthritis, ankylosing spondylitis), or inflammatory bowel disease (IBD). Follow-up started at treatment initiation and ended at the earliest of outcome occurrence (serious infection); treatment discontinuation; or switching, death, disenrollment, or end of study period. The study leveraged the FDA Sentinel System infrastructure for data management and analysis; descriptive statistics of patient characteristics and unadjusted incidence rates of study outcomes during follow-up were calculated.

RESULTS

Eligible patient drug episodes included 111,611 with RA (75% female), 61,050 with other inflammatory conditions (51% female), and 30,628 with IBD (52% female). Across all 3 cohorts, approximately half of the patient drug episodes initiated a biologic (50% in RA; 60% in psoriasis, psoriatic arthritis, ankylosing spondylitis; and 55% in IBD). The crude incidence rate of serious infection was 9.8 (9.5-10.0) cases per 100 person-years in RA, 7.1 (6.8-7.5) in other inflammatory conditions, and 14.2 (13.6-14.8) in IBD patients.

CONCLUSIONS

This study successfully identified large numbers of new users of biologics and produced results that were consistent with those from earlier published studies. The BBCIC DRN is a potential resource for surveillance of biologics.

DISCLOSURES

This study was funded by the Biologics and Biosimilars Collective Intelligence Consortium (BBCIC). HealthCore conducted this study in collaboration with Harvard Pilgrim Health Care. Zhang and Sridhar were employed by HealthCore at the time of this study. Haynes is employed by HealthCore funded by PCORI, the NIH, and the FDA. Barr and Eichelberger were employed by AMCP at the time of this study. Lockhart is employed by the BBCIC. Holmes and Clewell are employed by AbbVie. Accrott is an employee of and shareholder in Amgen. Marshall and Brown are employed by Harvard Pilgrim Health Care. Barr is a shareholder in Roche/Genentech. Curtis has received research grants from and consults with the following: Amgen, AbbVie, BMS, CORRONA, Lilly, Janssen, Myriad, Pfizer, Roche, Regeneron, and UCB. Brown has received research grants from GSK and Pfizer and consulting fees from Bayer, Roche, and Jazz Pharmaceuticals, along with funding from the Reagan-Udall Foundation for the FDA to conduct studies for medical product manufacturers, including Eli Lilly, Novartis, Abbvie, and Merck. Brown is also funded by PCORI, the NIH, and the FDA. McMahill-Walraven subcontracts with Harvard Pilgrim Health Care Institute for public health and safety surveillance distributed data network activtities and with the FDA, GSK, and Pfizer. She also reports fees from Reagan Udall Foundation for the FDA and the Patient Centered Outcomes Research Institute.

Translating claims-based CHA2 DS2 -VaSc and HAS-BLED to ICD-10-CM: Impacts of mapping strategies

PURPOSE

The CHA₂ DS₂ -VaSc and HAS-BLED risk scores are commonly used in the studies of oral anticoagulants (OACs). The best ways to map these scores to the International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) codes is unclear, as is how they perform in various types of OAC users. We aimed to assess the distributions of CHA₂ DS₂ -VaSc and HAS-BLED scores and C-statistics for outcome prediction in the ICD-10-CM era using different mapping strategies.

METHODS

We compared the distributions of CHA₂ DS₂ -VaSc and HAS-BLED scores from various mapping strategies in atrial fibrillation patients before, during, and after ICD-10-CM transition. We estimated the C-statistics predicting the 90-day risk of hospitalized stroke (for CHA₂ DS₂ -VaSc) or hospitalized bleeding (for HAS-BLED) in patients identified at least 6 months after the ICD-10-CM transition, overall and by anticoagulant type.

RESULTS

Forward-backward mapping produced higher CHA₂ DS₂ -VaSc and HAS-BLED scores in the ICD-10-CM era compared to the ICD-9-CM era: the mean difference was 0.074 (95% confidence interval 0.064-0.085) for CHA₂ DS₂ -VaSc and 0.055 (0.048-0.062) for HAS-BLED. Both scores had higher C-statistics in patients taking no OACs (0.697 [0.677-0.717] for CHA₂ DS₂ -VaSc; 0.719 [0.702-0.737] for HAS-BLED) or direct OACs (0.695 [0.654-0.735] for CHA₂ DS₂ -VaSc; 0.700 [0.673-0.728] for HAS-BLED) than those taking warfarin (0.655 [0.613-0.697] for CHA₂ DS₂ -VaSc; 0.663 [0.6320.695] for HAS-BLED).

CONCLUSIONS

Existing mapping strategies generally preserved the distributions of CHA₂ DS₂ -VaSc and HAS-BLED scores after ICD-10-CM transition. Both scores performed better in patients on no OACs or direct OACs than patients on warfarin.

Use of FDA's Sentinel System to Quantify Seizure Risk Immediately Following New Ranolazine Exposure

INTRODUCTION

Neurological complications including seizures have been reported with ranolazine. We sought to quantify the risk of seizure-related hospitalizations or emergency department events following ranolazine exposure in the Sentinel System (2006-2015).

STUDY DESIGN AND SETTING

Eligibility criteria were new use of ranolazine after 183 days washout period and absence of seizure diagnoses, anti-epileptic drugs, or seizure-related disorders during the baseline period.

RESULTS

Among 52,155 ranolazine users, we identified 28 seizures in the 1-32 days after new ranolazine dispensing: 12 occurring in days 1-10 (high-risk window), 11 in days 11-20 (moderate-risk window) and 5 in the control window (days 21-32). Assuming an equal likelihood of seizure events across the 32-day observation window, we estimate an attributable risk of 0.9 excess cases per 10,000 exposed users. Using a self-controlled risk interval design with exact logistic regression, seizures were elevated in the high-risk window (relative risk [RR] 2.88 (95% confidence interval [CI] 1.01-8.33) compared with the control window. No significant increased risk was observed in the moderate window. Half of the seizure cases had a diagnosis of renal disease, although seizure risk was not significant (RR 3.20 [CI 0.82-14.01]). A majority of patients in both risk windows were 75 years or older.

CONCLUSION

Our study suggests risk among younger ranolazine patients is rare. Given the imprecision of the risk estimates, we interpret the elevated seizure risk following ranolazine exposure with caution. Further analysis in a larger elderly population is warranted.

Descriptive Analysis of Long- and Intermediate-Acting Insulin and Key Safety Outcomes in Adults with Type 2 Diabetes Mellitus

BACKGROUND

As new biosimilar and follow-on insulins enter the market, more data are needed on safety, effectiveness, and patterns of use for these products to inform prescriber and patient decision-making regarding treatment. Additionally, data are needed regarding real-world patterns of use to inform future studies comparing the safety and effectiveness of bio-similars to already approved agents for diabetes treatment.

OBJECTIVE

To analyze the medication use patterns, adverse events, and availability of glycated hemoglobin (A1c) values for adult patients with type 2 diabetes mellitus (T2DM) who use long-acting insulin (LAI) or neutral protamine Hagedorn (NPH), an intermediate-acting insulin.

METHODS

We used the Biologics and Biosimilars Collective Intelligence Consortium's (BBCIC) distributed research network (DRN) for this descriptive analysis. The analysis time frame was January 1, 2011, to September 30, 2015, and included patients continuously insured for at least 183 days before the first date of a filled prescription for LAI or NPH insulin alone or with rapid- or short-acting insulin or sulfonylureas, whether newly starting insulin or switching to a different product. Insulin exposure episodes were the unit of analysis, and patients were classified in cohorts according to treatment. We followed patients until end of health plan enrollment or the end of the study period. We used occurrence of a study outcome, switch to another medication regimen, discontinuation of the current medication, or study end date to mark the end of an insulin episode. We describe demographics and availability of A1c values for analysis. Study outcomes included severe hypoglycemic events and major adverse cardiac events (MACE).

RESULTS

We identified 103,951 patients with T2DM from a database of 39.1 million patients with commercial or Medicare Advantage pharmacy and medical benefits, who contributed 279,533 unique insulin exposure episodes. Most episodes (89%) included patients using LAI, and 52% of patients contributed data to 2 or more exposure cohorts. Insulin episodes lasted an average of 3.5 months, and patients had an average follow-up of 8.6 months. The unadjusted rate of severe hypoglycemic events requiring medical attention was 96.9 per 10,000 patient-years at risk (10kPYR). The unadjusted incident MACE rate was 676.9 events per 10kPYR. 38,330 T2DM patients in the BBCIC DRN had a baseline A1c available, and of those, less than 50% had a follow-up A1c result.

CONCLUSIONS

Among patients with T2DM, our observed insulin patterns of use and rates of severe hypoglycemic outcomes and MACE are consistent with other studies. We noted a paucity of A1c results available, which implies that additional data sources may be needed to augment the BBCIC DRN.

DISCLOSURES

This study was coordinated and funded by the Biologics and Biosimilars Collective Intelligence Consortium (BBCIC) and represents the independent findings of the BBCIC Insulins Principal Investigator and the BBCIC Insulins Research Team. Lockhart is employed by the BBCIC and the Academy of Managed Care Pharmacy (AMCP). Eichelberger was employed by the BBCIC and AMCP at the time of this study. McMahill-Walraven is employed by Aetna, a CVS Health business. Panozzo, Marshall, and Brown are employed by Harvard Pilgrim Healthcare Institute. Aetna was reimbursed for data and analytic support from Harvard Pilgrim Healthcare Institute and the Reagan Udall Foundation for the U.S. Food and Drug Administration. Aetna receives external funding through research grants and subcontracts with Harvard Pilgrim Healthcare Institute, which are funded by the FDA, NIH, PCORI, BBCIC, Pfizer, and GSK; the Reagan-Udall Foundation for IMEDS; and PCORI for the ADAPTABLE Study. This work was previously presented as a poster at AMCP Nexus 2018; October 22-25, 2018; in Orlando, FL.

Harnessing the Biologics and Biosimilars Collective Intelligence Consortium to Evaluate Patterns of Care

INTRODUCTION

As clinical trials test efficacy rather than effectiveness of medications, real-world effectiveness data often vary from clinical trial data. Given the recent market entry of multiple biologics and biosimilars, a dedicated assessment of these diverse agents is needed to build the evidence base regarding efficacy and safety of innovator biologics and biosimilars.

PROGRAM DESCRIPTION

The Academy of Managed Care Pharmacy's Biologics and Biosimilars Collective Intelligence Consortium (BBCIC) was convened to address the lack of real-world, postmarket outcome evidence generation for innovator biologics and corresponding biosimilars. The BBCIC is a multistakeholder scientific research consortium whose participants prioritize topics and collaboratively conduct research studies. The BBCIC conducts a wide range of analyses, including population characterization, epidemiologic studies, and active observational studies, and develops best practices for conducting large-scale studies to provide real-world evidence.

OBSERVATIONS

Over the past 3 years, we undertook multiple descriptive analyses with the goal of characterizing data availability and demonstrating the feasibility and efficacy of using the BBCIC distributed research network (DRN), which includes commercial claims data from 2008-2018 covering approximately 100 million lives, with approximately 20 million active members in 2017 from 2 major U.S. health plans and 3 regional integrated delivery networks. We analyzed 4 medication classes of particular interest to biologics and biosimilars development: insulins, granulocyte colony-stimulating factors, erythropoietic-stimulating agents, and anti-inflammatories. We were able to identify exposures and user characteristics in all 4 categories. Herein we describe the successes and challenges of conducting some of our analyses, specifically among insulin users with type 1 diabetes mellitus.

IMPLICATIONS

Our results demonstrate the BBCIC DRN's ability to identify and characterize exposures, cohorts, and outcomes that can contribute to more sophisticated comparative surveillance of biosimilars and innovator biologics in the future. Additional linkages to laboratory data and a wider range of insurance carriers will further strengthen the BBCIC DRN.

DISCLOSURES

This study was coordinated and funded by the Biologics and Biosimilars Collective Intelligence Consortium (BBCIC) and represents the independent findings of the BBCIC Insulins Principal Investigator and the BBCIC Insulins Research Team. Lockhart is employed by the BBCIC; Eichelberger was employed by the BBCIC at the time of this study. McMahill-Walraven is employed by Aetna, a CVS Health business. Panozzo, Marshall, and Brown are employed by Harvard Pilgrim Healthcare Institute. Aetna receives external funding through research grants and subcontracts with Harvard Pilgrim Healthcare Institute, which are funded by the FDA, NIH, PCORI, BBCIC, Pfizer, and GSK; the Reagan-Udall Foundation for IMEDS; and PCORI for the ADAPTABLE Study. Aetna was reimbursed for data and analytic support from Harvard Pilgrim Healthcare Institute and the Reagan Udall Foundation for the U.S. Food and Drug Administration. This work was presented as a poster at AMCP Nexus 2018; October 22-25, 2018; in Orlando, FL.

Reproducing Protocol-Based Studies Using Parameterizable Tools-Comparison of Analytic Approaches Used by Two Medical Product Surveillance Networks

The US Sentinel System and the Canadian Network for Observational Drug Effect Studies (CNODES) are two medical product safety surveillance networks. Using Sentinel's preprogrammed, parameterizable analytic tools, we reproduced two protocol-based studies conducted by CNODES to assess the risks of acute pancreatitis and heart failure (HF) associated with the use of incretin-based drugs, compared with use of ≥ 2 oral hypoglycemic agents. Results from the replication new-user cohort analyses aligned with those from the CNODES nested case-control studies. The adjusted hazard ratios were 0.95 (0.81-1.12; vs. 1.03 (0.87-1.22) in CNODES) for acute pancreatitis and 0.91 (0.84-1.00; vs. 0.82 (0.67-1.00) in CNODES) for HF among patients without HF history. The CNODES's common protocol approach allows studies tailored to specific safety questions, whereas the Sentinel's common data model plus pretested program approach enables more rapid analysis. Despite these differences, it is possible to obtain comparable results using both approaches.

Choosing Among Common Data Models for Real-World Data Analyses Fit for Making Decisions About the Effectiveness of Medical Products

Many real-world data analyses use common data models (CDMs) to standardize terminologies for medication use, medical events and procedures, data structures, and interpretations of data to facilitate analyses across data sources. For decision makers, key aspects that influence the choice of a CDM may include (i) adaptability to a specific question; (ii) transparency to reproduce findings, assess validity, and instill confidence in findings; and (iii) ease and speed of use. Organizing CDMs preserve the original information from a data source and have maximum adaptability. Full mapping data models, or preconfigured rules systems, are easy to use, since all raw codes are mapped to medical constructs. Adaptive rule systems grow libraries of reusable measures that can easily adjust to preserve adaptability, expedite analyses, and ensure study-specific transparency.

Innovative Solutions for State Medicaid Programs to Leverage Their Data, Build Their Analytic Capacity, and Create Evidence-Based Policy

As states have embraced additional flexibility to change coverage of and payment for Medicaid services, they have also faced heightened expectations for delivering high-value care. Efforts to meet these new expectations have increased the need for rigorous, evidence-based policy, but states may face challenges finding the resources, capacity, and expertise to meet this need. By describing state-university partnerships in more than 20 states, this commentary describes innovative solutions for states that want to leverage their own data, build their analytic capacity, and create evidence-based policy. From an integrated web-based system to improve long-term care to evaluating the impact of permanent supportive housing placements on Medicaid utilization and spending, these state partnerships provide significant support to their state Medicaid programs. In 2017, these partnerships came together to create a distributed research network that supports multi-state analyses. The Medicaid Outcomes Distributed Research Network (MODRN) uses a common data model to examine Medicaid data across states, thereby increasing the analytic rigor of policy evaluations in Medicaid, and contributing to the development of a fully functioning Medicaid innovation laboratory.

Design and Refinement of a Data Quality Assessment Workflow for a Large Pediatric Research Network

Background:

Clinical data research networks (CDRNs) aggregate electronic health record data from multiple hospitals to enable large-scale research. A critical operation toward building a CDRN is conducting continual evaluations to optimize data quality. The key challenges include determining the assessment coverage on big datasets, handling data variability over time, and facilitating communication with data teams. This study presents the evolution of a systematic workflow for data quality assessment in CDRNs.

Implementation:

Using a specific CDRN as use case, the workflow was iteratively developed and packaged into a toolkit. The resultant toolkit comprises 685 data quality checks to identify any data quality issues, procedures to reconciliate with a history of known issues, and a contemporary GitHub-based reporting mechanism for organized tracking.

Results:

During the first two years of network development, the toolkit assisted in discovering over 800 data characteristics and resolving over 1400 programming errors. Longitudinal analysis indicated that the variability in time to resolution (15day mean, 24day IQR) is due to the underlying cause of the issue, perceived importance of the domain, and the complexity of assessment.

Conclusions:

In the absence of a formalized data quality framework, CDRNs continue to face challenges in data management and query fulfillment. The proposed data quality toolkit was empirically validated on a particular network, and is publicly available for other networks. While the toolkit is user-friendly and effective, the usage statistics indicated that the data quality process is very time-intensive and sufficient resources should be dedicated for investigating problems and optimizing data for research.

The Generalized Data Model for clinical research

BACKGROUND

Most healthcare data sources store information within their own unique schemas, making reliable and reproducible research challenging. Consequently, researchers have adopted various data models to improve the efficiency of research. Transforming and loading data into these models is a labor-intensive process that can alter the semantics of the original data. Therefore, we created a data model with a hierarchical structure that simplifies the transformation process and minimizes data alteration.

METHODS

There were two design goals in constructing the tables and table relationships for the Generalized Data Model (GDM). The first was to focus on clinical codes in their original vocabularies to retain the original semantic representation of the data. The second was to retain hierarchical information present in the original data while retaining provenance. The model was tested by transforming synthetic Medicare data; Surveillance, Epidemiology, and End Results data linked to Medicare claims; and electronic health records from the Clinical Practice Research Datalink. We also tested a subsequent transformation from the GDM into the Sentinel data model.

RESULTS

The resulting data model contains 19 tables, with the Clinical Codes, Contexts, and Collections tables serving as the core of the model, and containing most of the clinical, provenance, and hierarchical information. In addition, a Mapping table allows users to apply an arbitrarily complex set of relationships among vocabulary elements to facilitate automated analyses.

CONCLUSIONS

The GDM offers researchers a simpler process for transforming data, clear data provenance, and a path for users to transform their data into other data models. The GDM is designed to retain hierarchical relationships among data elements as well as the original semantic representation of the data, ensuring consistency in protocol implementation as part of a complete data pipeline for researchers.