Automated data-adaptive analytics for electronic healthcare data to study causal treatment effects

Development and validation of a distributed representation model of Japanese high-dimensional administrative claims data for clinical epidemiology studies

BACKGROUND

Unmeasured confounders pose challenges when observational data are analysed in comparative effectiveness studies. Integrating high-dimensional administrative claims data may help adjust for unmeasured confounders. We determined whether distributed representations can compress high-dimensional administrative claims data to adjust for unmeasured confounders.

METHOD

Using the Japanese Diagnosis Procedure Combination (DPC) database from 1291 hospitals (between April 2018 and March 2020), we applied the word2vec algorithm to create distributed representations for all medical codes. We focused on patients with heart failure (HF) and simulated four risk-adjustment models: 1, no adjustment; 2, adjusting for previously reported confounders; 3, adjusting for the sum of distributed representation weights of administrative claims data on the day of hospitalisation (novel method); and 4, a combination of models 2 and 3. We re-evaluated a previous study on the effect of early rehabilitation in patients with HF and compared these risk-adjustment methods (models 1-4).

RESULTS

Distributed representations were generated from the data of 15 998 963 in-patients, and 319 581 HF patients were identified. In the simulation study, Model 3 reduced the impact of unmeasured confounders and achieved better covariate balances than Model 1. Model 4 showed no increase in bias compared with the true model (Model 2) and was used as a reference model in the real-world application. When applied to a previous study, models 3 and 4 showed similar results.

CONCLUSION

Distributed representation can compress detailed administrative claims data and adjust for unmeasured confounders in comparative effectiveness studies.

Mapping the effectiveness and risks of GLP-1 receptor agonists

Glucagon-like peptide 1 receptor agonists (GLP-1RAs) are increasingly being used to treat diabetes and obesity. However, their effectiveness and risks have not yet been systematically evaluated in a comprehensive set of possible health outcomes. Here, we used the US Department of Veterans Affairs databases to build a cohort of people with diabetes who initiated GLP-1RA (n = 215,970) and compared them to those who initiated sulfonylureas (n = 159,465), dipeptidyl peptidase 4 (DPP4) inhibitors (n = 117,989) or sodium-glucose cotransporter-2 (SGLT2) inhibitors (n = 258,614), a control group composed of an equal proportion of individuals initiating sulfonylureas, DPP4 inhibitors and SGLT2 inhibitors (n = 536,068), and a control group of 1,203,097 individuals who continued use of non-GLP-1RA antihyperglycemics (usual care). We used a discovery approach to systematically map an atlas of the associations of GLP-1RA use versus each comparator with 175 health outcomes. Compared to usual care, GLP-1RA use was associated with a reduced risk of substance use and psychotic disorders, seizures, neurocognitive disorders (including Alzheimer's disease and dementia), coagulation disorders, cardiometabolic disorders, infectious illnesses and several respiratory conditions. There was an increased risk of gastrointestinal disorders, hypotension, syncope, arthritic disorders, nephrolithiasis, interstitial nephritis and drug-induced pancreatitis associated with GLP-1RA use compared to usual care. The results provide insights into the benefits and risks of GLP-1RAs and may be useful for informing clinical care and guiding research agendas.

A future of data-rich pharmacoepidemiology studies: transitioning to large-scale linked electronic health record + claims data

Electronic health record (EHR) data are seen as an important source for pharmacoepidemiology studies. In the US health-care system, EHR systems often identify only fragments of patients' health information across the care continuum, including primary care, specialist care, hospitalizations, and pharmacy dispensing. This leads to unobservable information in longitudinal evaluations of medication effects, causing unmeasured confounding, misclassification, and truncated follow-up times. A remedy is to link EHR data with longitudinal health insurance claims data, which record all encounters during a defined enrollment period across all care settings. Here we evaluate EHR and claims data sources in 3 aspects relevant to etiological studies of medical products: data continuity, data granularity, and data chronology. Reflecting on the strengths and limitations of EHR and insurance claims data, it becomes obvious that they complement each other. The combination of both will improve the validity of etiological studies and expand the range of questions that can be answered. As the research community transitions towards a future state with access to large-scale combined EHR + claims data, we outline analytical templates to improve the validity and broaden the scope of pharmacoepidemiology studies in the current environment where EHR data are available only for a subset of patients with claims data. This article is part of a Special Collection on Pharmacoepidemiology.

Natural language processing for scalable feature engineering and ultra-high-dimensional confounding adjustment in healthcare database studies

Background

To improve confounding control in healthcare database studies, data-driven algorithms may empirically identify and adjust for large numbers of pre-exposure variables that indirectly capture information on unmeasured confounding factors ('proxy' confounders). Current approaches for high-dimensional proxy adjustment do not leverage free-text notes from EHRs. Unsupervised natural language processing (NLP) technology can scale to generate large numbers of structured features from unstructured notes.

Objective

To assess the impact of supplementing claims data analyses with large numbers of NLP generated features for high-dimensional proxy adjustment.

Methods

We linked Medicare claims with EHR data to generate three cohorts comparing different classes of medications on the 6-month risk of cardiovascular outcomes. We used various NLP methods to generate structured features from free-text EHR notes and used LASSO regression to fit several PS models that included different covariate sets as candidate predictors. Covariate sets included features generated from claims data only, and claims data plus NLP-generated EHR features.

Results

Including both claims codes and NLP-generated EHR features as candidate predictors improved overall covariate balance with standardized differences being <0.1 for all variables. While overall balance improved, the impact on estimated treatment effects was more nuanced with adjustment for NLP-generated features moving effect estimates further in the expected direction in two of the empirical studies but had no impact on the third study.

Conclusion

Supplementing administrative claims with large numbers of NLP-generated features for ultra-high-dimensional proxy confounder adjustment improved overall covariate balance and may provide a modest benefit in terms of capturing confounder information.

Cardiovascular risk associated with glucagon-like peptide-1 receptor agonists versus other conventional glucose-lowering drugs in patients with type-2 diabetes: protocol for a nationwide observational comparative study in routine care

INTRODUCTION

Several cardiovascular outcome trials have been conducted to assess the cardiovascular safety and efficacy of glucagon-like peptide-1 receptor agonists (GLP1-RAs) on cardiorenal outcomes in patients with type-2 diabetes (T2D). However, the strict requirements of randomised controlled trials to avoid most confounding factors are at the expense of external validity. Using national real-world data, we aimed to evaluate the effectiveness of GLP-1RAs in association with metformin especially on cardiovascular events, hospitalisation for heart failure and all-cause death in comparison with other diabetes treatment schemes using dipeptidyl peptidase IV inhibitors, sulfonylureas/glinides or insulin also associated with metformin. Sodium-glucose transport protein 2 inhibitors (SGLT-2i) will be excluded as comparators, as this class of oral hypoglycaemic agents just started in 2020 to be marketed in France.

METHODS AND ANALYSIS

The Système National des Données de Santé is a comprehensive nationwide administrative healthcare database in France that covers approximately 67 million people.Several cohorts of adult patients with T2D initiating any GLP1-RA in dual or triple therapies, as recommended by the French Health authorities, will be identified in this database over the period 2016-2021. These cohorts will be defined by the combination of glucose-lowering drugs prescribed simultaneously with GLP1-RA and diabetes treatment received over a 6-month period before GLP1-RA initiation. They will be first matched with T2D controls (1:3 ratio) based on the year of drug initiation and treatment regimens before and simultaneously with GLP1-RA in the different selected cohorts. Comparative analyses will be conducted versus these control groups, adjusting for cardiovascular event history and a propensity score considering age, sex, area of residence, deprivation index, comorbidities, duration of diabetes, use of lipid-lowering drugs, anticoagulants, antiplatelet therapies and blood pressure-lowering therapies. Comparative analyses will be conducted versus these control groups, using a high-dimensional propensity scores method and fixed baseline characteristics. Treatment effects on the different outcomes measured will be estimated for each GLP1-RA group, through HR and their corresponding CIs (95% CI) using Cox regressions and/or competitive risk regressions when necessary.

ETHICS AND DISSEMINATION

The study has been approved by an independent ethics committee (Comité éthique et scientifique pour les recherches, les études et les évaluations dans le domaine de la santé, Paris, France; reference: 8699786, dated 2 June 2022) and has been registered with the French National Data Protection Commission (Commission Nationale de l'Informatique et des Libertés, Paris, France; reference: 922161, dated 26 June 2022). The findings of this study will be published in peer-reviewed scientific journals and presented at international conferences.

TRIAL REGISTRATION NUMBER

F20220803152803.

Comparative effectiveness and safety of single inhaler triple therapies for chronic obstructive pulmonary disease: new user cohort study

OBJECTIVE

To compare the effectiveness and safety of budesonide-glycopyrrolate-formoterol, a twice daily metered dose inhaler, and fluticasone-umeclidinium-vilanterol, a once daily dry powder inhaler, in patients with chronic obstructive pulmonary disease (COPD) treated in routine clinical practice.

DESIGN

New user cohort study.

SETTING

Longitudinal commercial US claims data.

PARTICIPANTS

New initiators of budesonide-glycopyrrolate-formoterol or fluticasone-umeclidinium-vilanterol between 1 January 2021 and 30 September 2023 who had a diagnosis of COPD and were aged 40 years or older.

MAIN OUTCOME MEASURES

In this 1:1 propensity score matched study, the main outcome measures were first moderate or severe COPD exacerbation (effectiveness) and first admission to hospital with pneumonia (safety) while on treatment. Potential confounders were measured in the 365 days before cohort entry and included in propensity scores. Hazard ratios and 95% confidence intervals (CIs) were estimated using a Cox proportional hazards regression model.

RESULTS

The study cohort included 20 388 propensity score matched pairs of new users initiating single inhaler triple therapy. Patients who received budesonide-glycopyrrolate-formoterol had a 9% higher incidence of first moderate or severe COPD exacerbation (hazard ratio 1.09 (95% CI 1.04 to 1.14); number needed to harm 38) compared with patients receiving fluticasone-umeclidinium-vilanterol and an identical incidence of first admission to hospital with pneumonia (1.00 (0.91 to 1.10)). The hazard of first moderate COPD exacerbation was 7% higher (1.07 (1.02 to 1.12); number needed to harm 54) and the hazard of first severe COPD exacerbation 29% higher (1.29 (1.12 to 1.48); number needed to harm 97) among those receiving budesonide-glycopyrrolate-formoterol compared to fluticasone-umeclidinium-vilanterol. Prespecified sensitivity analyses yielded similar findings to the primary analysis.

CONCLUSIONS

Budesonide-glycopyrrolate-formoterol was not associated with improved clinical outcomes compared with fluticasone-umeclidinium-vilanterol. Given the added climate impact of metered dose inhalers, health systems seeking to decrease use of these products may consider steps to promote further prescribing of fluticasone-umeclidinium-vilanterol compared with budesonide-glycopyrrolate-formoterol in people with COPD.

STUDY REGISTRATION

Center for Open Science Real World Evidence Registry (https://osf.io/6gdyp/).

Mismatch Repair Status and Surgical Outcomes in Localized Colorectal Cancer: A Nationwide Cohort Study

Objective

This study examined the association between deficient mismatch repair (dMMR) versus proficient MMR (pMMR) status and overall survival and disease-free survival in patients with localized colorectal cancer.

Background

Several distinctions exist between patients with dMMR and pMMR colorectal cancer. However, the impact on prognosis is yet to be investigated in large-scale cohort studies.

Methods

In this cohort study, we included patients who underwent curative-intent surgery for localized colorectal cancer between 2009 and 2020. Patients were identified in the Danish Colorectal Cancer Group database and patient-level data were extracted from 6 registry databases. After inclusion, patients with dMMR status were matched 1:1 to patients with pMMR status using an estimated propensity score.

Results

After matching, 5994 patients were included. The patients had a median age of 74 years and a median follow-up of 4.1 years. There was no significant association between mismatch repair (MMR) status and overall survival (hazard ratio, 0.91; 95% confidence interval [CI], 0.81-1.03) or disease-free survival (hazard ratio, 0.89; 95% CI, 0.78-1.01). However, the restricted 5-year mean disease-free survival time, calculated due to violation of the proportional hazards assumption, showed a significant absolute difference of 0.13 years (95% CI, 0.03-0.23; P = 0.01) in favor of the dMMR group.

Conclusions

No significant association with overall survival was found according to MMR status. dMMR status was, however, found to be associated with marginally improved disease-free survival compared to pMMR status in patients with localized colorectal cancer undergoing curative-intent surgery.

Targeted learning with an undersmoothed LASSO propensity score model for large-scale covariate adjustment in health-care database studies

Least absolute shrinkage and selection operator (LASSO) regression is widely used for large-scale propensity score (PS) estimation in health-care database studies. In these settings, previous work has shown that undersmoothing (overfitting) LASSO PS models can improve confounding control, but it can also cause problems of nonoverlap in covariate distributions. It remains unclear how to select the degree of undersmoothing when fitting large-scale LASSO PS models to improve confounding control while avoiding issues that can result from reduced covariate overlap. Here, we used simulations to evaluate the performance of using collaborative-controlled targeted learning to data-adaptively select the degree of undersmoothing when fitting large-scale PS models within both singly and doubly robust frameworks to reduce bias in causal estimators. Simulations showed that collaborative learning can data-adaptively select the degree of undersmoothing to reduce bias in estimated treatment effects. Results further showed that when fitting undersmoothed LASSO PS models, the use of cross-fitting was important for avoiding nonoverlap in covariate distributions and reducing bias in causal estimates.

Use of the French National Health Data System (SNDS) in pharmacoepidemiology: A systematic review in its maturation phase

AIM OF THE STUDY

The French National Health Data System (SNDS) comprises healthcare data that cover 99% of the population (over 67 million individuals) in France. The aim of this study was to present an overview of published pharmacoepidemiological studies using the SNDS in its maturation phase.

METHODS

We conducted a systematic literature review of original research articles in the Pubmed and EMBASE databases from January 2012 until August 2018.

RESULTS

A total of 316 full-text articles were included, with an annual increase over the study period. Only 16 records were excluded after screening because they did not involve the SNDS but other French healthcare databases. The study design was clearly reported in only 66% of studies of which 57% were retrospective cohorts and 22% cross-sectional studies. The reported study objectives were drug utilization (65%), safety (22%) and effectiveness (9%). Almost all ATC groups were studied but the most frequent ones concerned the nervous system in 149 studies (49%), cardiovascular system drugs in 104 studies (34%) and anti-infectives for systemic use in 50 studies (16%).

CONCLUSION

The SNDS is of growing interest for studies on drug use and safety, which could be conducted more in specific populations, including children, pregnant women and the elderly, as these populations are often not included in clinical trials.

Hydrochlorothiazide Use and Risk of Skin Cancer: A Population-Based Retrospective Cohort Study

PURPOSE

Hydrochlorothiazide (HCTZ) exposure has been linked to increased skin cancer in Caucasian (white) populations, especially squamous cell carcinoma (SCC), but not basal cell carcinoma (BCC). This study aimed to evaluate and compare skin cancer risks associated with HCTZ- and other antihypertensives use.

METHODS

This retrospective cohort study utilized Taiwan's National Health Insurance Research Database. We identified patients aged 20 years and older, newly receiving antihypertensive medications between 2004 and 2015. We calculated the medication possession ratio (MPR) for the first 2 years of treatment to determine patient eligibility and treatment classification, whereby only patients with MPR above 80% were included. These were subsequently categorized by the type of antihypertensives they received, namely HCTZ, other thiazide diuretics, non-thiazide diuretics or non-diuretic antihypertensives. Cox proportional hazards model was used to evaluate skin cancer risks, and these were then classified as SCC or BCC.

RESULTS

Our study included 41 086, 27 402, 19 613, and 856 782 patients receiving HCTZ, other thiazide diuretics, non-thiazide diuretics, and non-diuretic antihypertensives, respectively. We found BCC risks were similar when comparing HCTZ with other thiazides (adjusted hazard ratio: 0.84; 95% CI: 0.54-1.33), non-thiazide diuretics (0.93; 0.51-1.67), and non-diuretic antihypertensives (0.91; 0.66-1.26). We observed a higher SCC risk in the HCTZ group, compared to other thiazides (1.24; 0.74-2.08), non-thiazide diuretics (1.32; 0.70-2.51), and non-diuretic antihypertensives (1.23; 0.87-1.73), although the confidence intervals (CIs) were wide and crossed the null.

CONCLUSIONS

We concluded that skin cancer need not be of major concern to physicians when prescribing antihypertensives for an Asian population.

Long-term risk of psychiatric disorder and psychotropic prescription after SARS-CoV-2 infection among UK general population

Despite evidence indicating increased risk of psychiatric issues among COVID-19 survivors, questions persist about long-term mental health outcomes and the protective effect of vaccination. Using UK Biobank data, three cohorts were constructed: SARS-CoV-2 infection (n = 26,101), contemporary control with no evidence of infection (n = 380,337) and historical control predating the pandemic (n = 390,621). Compared with contemporary controls, infected participants had higher subsequent risks of incident mental health at 1 year (hazard ratio (HR): 1.54, 95% CI 1.42-1.67; P = 1.70 × 10-24; difference in incidence rate: 27.36, 95% CI 21.16-34.10 per 1,000 person-years), including psychotic, mood, anxiety, alcohol use and sleep disorders, and prescriptions for antipsychotics, antidepressants, benzodiazepines, mood stabilizers and opioids. Risks were higher for hospitalized individuals (2.17, 1.70-2.78; P = 5.80 × 10-10) than those not hospitalized (1.41, 1.30-1.53; P = 1.46 × 10-16), and were reduced in fully vaccinated people (0.97, 0.80-1.19; P = 0.799) compared with non-vaccinated or partially vaccinated individuals (1.64, 1.49-1.79; P = 4.95 × 10-26). Breakthrough infections showed similar risk of psychiatric diagnosis (0.91, 0.78-1.07; P = 0.278) but increased prescription risk (1.42, 1.00-2.02; P = 0.053) compared with uninfected controls. Early identification and treatment of psychiatric disorders in COVID-19 survivors, especially those severely affected or unvaccinated, should be a priority in the management of long COVID. With the accumulation of breakthrough infections in the post-pandemic era, the findings highlight the need for continued optimization of strategies to foster resilience and prevent escalation of subclinical mental health symptoms to severe disorders.

A methodological review of the high-dimensional propensity score in comparative-effectiveness and safety-of-interventions research finds incomplete reporting relative to algorithm development and robustness

OBJECTIVES

The use of secondary databases has become popular for evaluating the effectiveness and safety of interventions in real-life settings. However, the absence of important confounders in these databases is challenging. To address this issue, the high-dimensional propensity score (hdPS) algorithm was developed in 2009. This algorithm uses proxy variables for mitigating confounding by combining information available across several healthcare dimensions. This study assessed the methodology and reporting of the hdPS in comparative effectiveness and safety research.

STUDY DESIGN AND SETTING

In this methodological review, we searched PubMed and Google Scholar from July 2009 to May 2022 for studies that used the hdPS for evaluating the effectiveness or safety of healthcare interventions. Two reviewers independently extracted study characteristics and assessed how the hdPS was applied and reported. Risk of bias was evaluated with the Risk Of Bias In Non-randomised Studies - of Interventions (ROBINS-I) tool.

RESULTS

In total, 136 studies met the inclusion criteria; the median publication year was 2018 (Q1-Q3 2016-2020). The studies included 192 datasets, mostly North American databases (n = 132, 69%). The hdPS was used in primary analysis in 120 studies (88%). Dimensions were defined in 101 studies (74%), with a median of 5 (Q1-Q3 4-6) dimensions included. A median of 500 (Q1-Q3 200-500) empirically identified covariates were selected. Regarding hdPS reporting, only 11 studies (8%) reported all recommended items. Most studies (n = 81, 60%) had a moderate overall risk of bias.

CONCLUSION

There is room for improvement in the reporting of hdPS studies, especially regarding the transparency of methodological choices that underpin the construction of the hdPS.

Interpretable machine learning in predicting drug-induced liver injury among tuberculosis patients: model development and validation study

BACKGROUND

The objective of this research was to create and validate an interpretable prediction model for drug-induced liver injury (DILI) during tuberculosis (TB) treatment.

METHODS

A dataset of TB patients from Ningbo City was used to develop models employing the eXtreme Gradient Boosting (XGBoost), random forest (RF), and the least absolute shrinkage and selection operator (LASSO) logistic algorithms. The model's performance was evaluated through various metrics, including the area under the receiver operating characteristic curve (AUROC) and the area under the precision recall curve (AUPR) alongside the decision curve. The Shapley Additive exPlanations (SHAP) method was used to interpret the variable contributions of the superior model.

RESULTS

A total of 7,071 TB patients were identified from the regional healthcare dataset. The study cohort consisted of individuals with a median age of 47 years, 68.0% of whom were male, and 16.3% developed DILI. We utilized part of the high dimensional propensity score (HDPS) method to identify relevant variables and obtained a total of 424 variables. From these, 37 variables were selected for inclusion in a logistic model using LASSO. The dataset was then split into training and validation sets according to a 7:3 ratio. In the validation dataset, the XGBoost model displayed improved overall performance, with an AUROC of 0.89, an AUPR of 0.75, an F1 score of 0.57, and a Brier score of 0.07. Both SHAP analysis and XGBoost model highlighted the contribution of baseline liver-related ailments such as DILI, drug-induced hepatitis (DIH), and fatty liver disease (FLD). Age, alanine transaminase (ALT), and total bilirubin (Tbil) were also linked to DILI status.

CONCLUSION

XGBoost demonstrates improved predictive performance compared to RF and LASSO logistic in this study. Moreover, the introduction of the SHAP method enhances the clinical understanding and potential application of the model. For further research, external validation and more detailed feature integration are necessary.

Concepts of Designing and Implementing Pharmacoepidemiology Studies on the Safety of Systemic Treatments in Dermatology Practice

The U.S. Food and Drug Administration and clinical guidelines use evidence from pharmacoepidemiology studies to inform prescribing decisions and fill evidence gaps left by randomized controlled trials (RCTs). The long-term safety and infrequent adverse reactions are not well-understood when RCTs are short and involve few patients, as is the case for most systemic immunomodulating drugs in dermatology. A better understanding of the design and implementation of pharmacoepidemiology studies will help practitioners assess the accuracy of etiologic findings and use them with confidence in clinical practice. Conducting pharmacoepidemiology studies follows a structured approach, which we discuss in this article: (i) a design layer connects the research question with the appropriate study design, and considering which hypothetical RCT one ideally would want to conduct reduces inadvertent investigator errors; (ii) a measurement layer transforms longitudinal patient-level data into variables that identify the study population, patient characteristics, treatment, and outcomes; and (iii) the analysis focuses on the causal treatment effect estimation. The review and interpretation of pharmacoepidemiology studies should consider issues beyond a typical review of RCTs, chiefly the lack of baseline randomization and the use of secondary data. Well-designed and well-conducted pharmacoepidemiologic studies complement dermatology practice with critical information on prescribing systemic medications.

An application of the Causal Roadmap in two safety monitoring case studies: Causal inference and outcome prediction using electronic health record data

Background

Real-world data, such as administrative claims and electronic health records, are increasingly used for safety monitoring and to help guide regulatory decision-making. In these settings, it is important to document analytic decisions transparently and objectively to assess and ensure that analyses meet their intended goals.

Methods

The Causal Roadmap is an established framework that can guide and document analytic decisions through each step of the analytic pipeline, which will help investigators generate high-quality real-world evidence.

Results

In this paper, we illustrate the utility of the Causal Roadmap using two case studies previously led by workgroups sponsored by the Sentinel Initiative - a program for actively monitoring the safety of regulated medical products. Each case example focuses on different aspects of the analytic pipeline for drug safety monitoring. The first case study shows how the Causal Roadmap encourages transparency, reproducibility, and objective decision-making for causal analyses. The second case study highlights how this framework can guide analytic decisions beyond inference on causal parameters, improving outcome ascertainment in clinical phenotyping.

Conclusion

These examples provide a structured framework for implementing the Causal Roadmap in safety surveillance and guide transparent, reproducible, and objective analysis.

Comparative effectiveness of SGLT2 inhibitors, GLP-1 receptor agonists, DPP-4 inhibitors, and sulfonylureas on risk of major adverse cardiovascular events: emulation of a randomised target trial using electronic health records

BACKGROUND

Randomised clinical trials showed that compared with placebo, SGLT2 inhibitors and GLP-1 receptor agonists reduced risk of adverse cardiovascular events. The evidence base for the older antihyperglycaemic drug classes (DPP-4 inhibitors and sulfonylureas) is generally less well developed. Because most randomised trials evaluated one antihyperglycaemic medication versus placebo, a head-to-head comparative effectiveness analysis of the newer drug classes (SGLT2 inhibitors vs GLP-1 receptor agonists) or newer (SGLT2 inhibitors or GLP-1 receptor agonists) versus older (DPP-4 inhibitors or sulfonylureas) drug classes on risk of major adverse cardiovascular events (MACE) is not available. In this study, we aimed to evaluate the comparative effectiveness of incident use of SGLT2 inhibitors, GLP-1 receptor agonists, DPP-4 inhibitors, or sulfonylureas on risk of MACE.

METHODS

We first specified the protocol of a four-arm randomised pragmatic clinical trial and then emulated it using the health-care databases of the US Department of Veterans Affairs. We built a cohort of metformin users with incident use of SGLT2 inhibitors, GLP-1 receptor agonists, DPP-4 inhibitors, or sulfonylureas between Oct 1, 2016 and Sept 30, 2021, and followed up until Dec 31, 2022. We used the overlap weighting approach to balance the treatment groups using a battery of predefined variables and a set of algorithmically selected variables from high-dimensional data domains. Both intention-to-treat and per-protocol analyses (the latter estimated the effect of maintained use of the antihyperglycaemic throughout follow-up) were conducted to estimate risk of MACE-defined as a composite endpoint of stroke, myocardial infarction, and all-cause mortality.

FINDINGS

The final cohort consisted of 283 998 new users of SGLT2 inhibitors (n=46 516), GLP-1 receptor agonists (n=26 038), DPP-4 inhibitors (n=55 310), or sulfonylureas (n=156 134). In intention-to-treat analyses, compared with sulfonylureas, SGLT2 inhibitors, GLP-1 receptor agonists, and DPP-4 inhibitors were associated with lower risk of MACE (hazard ratio [HR] 0·77 [95% CI 0·74-0.80], 0·78 [0·74-0·81), and 0·90 [0·86-0.93], respectively). Both SGLT2 inhibitors and GLP-1 receptor agonists were associated with a lower risk of MACE when compared with DPP-4 inhibitors (HR 0·86 [0·82-0·89] and 0·86 [0·82-0·90], respectively). The risk of MACE between SGLT2 inhibitors and GLP-1 receptor agonists yielded an HR of 0·99 (0·94-1·04). In per-protocol analyses, compared with sulfonylureas, SGLT2 inhibitors, GLP1 receptor agonists, and DPP-4 inhibitors were associated with reduced risk of MACE (HR 0·77 [95% CI 0·73-0·82], 0·77 [0·72-0·82], and 0·88 [0·83-0·93], respectively). Both SGLT2 inhibitors and GLP-1 receptor agonists were associated with a lower risk of MACE when compared with DPP-4 inhibitors (HR 0·88 [0·83-0·93] and 0·88 [0·82-0·93], respectively). The risk of MACE between SGLT2 inhibitors and GLP-1 receptor agonists yielded an HR of 1·01 (0·94-1·07).

INTERPRETATION

Both SGLT2 inhibitors and GLP-1 receptor agonists were associated with reduced risk of MACE compared with DPP-4 inhibitors or sulfonylureas. DPP-4 inhibitors were associated with reduced risk of MACE compared with sulfonylureas. There was no statistically significant difference in risk of MACE between SGLT2 inhibitors and GLP-1 receptor agonists. The results provide evidence of the real-world comparative effectiveness of the four most commonly used second-line antihyperglycaemics and could guide choice of antihyperglycaemic therapy.

FUNDING

US Department of Veterans Affairs and the American Society of Nephrology.

The impact of time to surgery on oncological outcomes in stage I-III dMMR colon cancer - A nationwide cohort study

INTRODUCTION

One of the considerations when investigating neoadjuvant interventions is the prolonging of time from diagnosis to curative surgery (i.e. the treatment interval [TI]). The aim of this study was to investigate the association between the length of TI and overall survival and disease-free survival in patients with deficient mismatch repair (dMMR) colon cancer.

MATERIALS AND METHODS

This retrospective propensity score-adjusted study included all patients of ≥18 years of age undergoing elective curative surgery for stage I-III, dMMR colon cancer. Data were extracted from four Danish patient databases. Outcomes were investigated in groups with TIs of ≤14 days versus >14 days. Propensity scores were computed using all demographics, diagnoses and measurements. Matching was done in a 1:1 ratio.

RESULTS

A total of 4130 patients were included in the study with a mean age of 73.8 years and a median follow-up time of 43.9 months. After matching, 2794 patients were included in the analysis of overall survival. No significant difference in overall survival was seen between patients with TIs of ≤14 days versus >14 days (hazard ratio [HR], 0.97; 95% confidence interval [CI], 0.81-1.17; p = 0.78). In the analysis of disease-free survival, 1798 patients were included after matching. This showed no significant difference between patients with TIs of ≤14 days versus >14 days (HR, 0.85; 95% CI, 0.69-1.06; p = 0.14).

CONCLUSION

No associations were found between TI and overall survival and disease-free survival in patients with stage I-III, dMMR colon cancer undergoing elective curative surgery.

Comparative Effectiveness and Safety of Generic Versus Brand-Name Fluticasone-Salmeterol to Treat Chronic Obstructive Pulmonary Disease

BACKGROUND

In 2019, the U.S. Food and Drug Administration (FDA) approved the first generic maintenance inhaler for asthma and chronic obstructive pulmonary disease (COPD). The inhaler, Wixela Inhub (fluticasone-salmeterol; Viatris), is a substitutable version of the dry powder inhaler Advair Diskus (fluticasone-salmeterol; GlaxoSmithKline). When approving complex generic products like inhalers, the FDA applies a special "weight-of-evidence" approach. In this case, manufacturers were required to perform a randomized controlled trial in patients with asthma but not COPD, although the product received approval for both indications.

OBJECTIVE

To compare the effectiveness and safety of generic (Wixela Inhub) and brand-name (Advair Diskus) fluticasone-salmeterol among patients with COPD treated in routine care.

DESIGN

A 1:1 propensity score-matched cohort study.

SETTING

A large, longitudinal health care database.

PATIENTS

Adults older than 40 years with a diagnosis of COPD.

MEASUREMENTS

Incidence of first moderate or severe COPD exacerbation (effectiveness outcome) and incidence of first pneumonia hospitalization (safety outcome) in the 365 days after cohort entry.

RESULTS

Among 45 369 patients (27 305 Advair Diskus users and 18 064 Wixela Inhub users), 10 012 matched pairs were identified for the primary analysis. Compared with Advair Diskus use, Wixela Inhub use was associated with a nearly identical incidence of first moderate or severe COPD exacerbation (hazard ratio [HR], 0.97 [95% CI, 0.90 to 1.04]) and first pneumonia hospitalization (HR, 0.99 [CI, 0.86 to 1.15]).

LIMITATIONS

Follow-up times were short, reflecting real-world clinical practice. The possibility of residual confounding cannot be completely excluded.

CONCLUSION

Use of generic and brand-name fluticasone-salmeterol was associated with similar outcomes among patients with COPD treated in routine practice.

PRIMARY FUNDING SOURCE

National Heart, Lung, and Blood Institute.

Adaptability of High Dimensional Propensity Score Procedure in the Transition from ICD-9 to ICD-10 in the US Healthcare System

Background

High-Dimensional Propensity Score procedure (HDPS) is a data-driven approach to assist control for confounding in pharmacoepidemiologic research. The transition to the International Classification of Disease (ICD-9/10) in the US health system may pose uncertainty in applying the HDPS procedure.

Methods

We assembled a base cohort of patients in MarketScan^® Commercial Claims Database who had newly initiated celecoxib or traditional NSAIDs to compare gastrointestinal bleeding risk. We then created bootstrapped hypothetical cohorts from the base cohort with predefined patient selection patterns from the ICD eras. Three strategies for HDPS deployment were tested: 1) split the cohort by ICD era, deploy HDPS twice, and pool the relative risks (pooled RR), 2) consider codes from each ICD era as a separate data dimension and deploy HDPS in the entire cohort (data dimensions) and 3) map ICD codes from both eras to Clinical Classifications Software (CCS) concepts before deploying HDPS in the entire cohort (CCS mapping). We calculated percent bias and root-mean-squared error to compare the strategies.

Results

A similar bias reduction was observed in cohorts where patient selection pattern from each ICD era was comparable between the exposure groups. In the presence of considerable disparity in patient selection, we observed a bimodal distribution of propensity scores in the data dimensions strategy, indicating instrument-like covariates. Moreover, the CCS mapping strategy resulted in at least 30% less bias than pooled RR and data dimensions strategies (RMSE: 0.14, 0.19, 0.21, respectively) in this scenario.

Conclusion

Mapping ICD codes to a stable terminology like CCS serves as a helpful strategy to reduce residual bias when deploying HDPS in pharmacoepidemiologic studies spanning both ICD eras.

Evaluating the Performance of High-Dimensional Propensity Scores Compared with Standard Propensity Scores for Comparing Antihypertensive Therapies in the CPRD GOLD Database

INTRODUCTION

Propensity score (PS) matching is widely used in medical record studies to create balanced treatment groups, but relies on prior knowledge of confounding factors. High-dimensional PS (hdPS) is a semi-automated algorithm that selects variables with the highest potential for confounding from medical databases. The objective of this study was to evaluate performance of hdPS and PS when used to compare antihypertensive therapies in the UK clinical practice research datalink (CPRD) GOLD database.

METHODS

Patients initiating antihypertensive treatment with either monotherapy or bitherapy were extracted from the CPRD GOLD database. Simulated datasets were generated using plasmode simulations with a marginal hazard ratio (HRm) of 1.29 for bitherapy versus monotherapy for reaching blood pressure control at 3 months. Either 16 or 36 known covariates were forced into the PS and hdPS models, and 200 additional variables were automatically selected for hdPS. Sensitivity analyses were conducted to assess the impact of removing known confounders from the database on hdPS performance.

RESULTS

With 36 known covariates, the estimated HRm (RMSE) was 1.31 (0.05) for hdPS and 1.30 (0.04) for PS matching; the crude HR was 0.68 (0.61). Using 16 known covariates, the estimated HRm (RMSE) was 1.23 (0.10) and 1.09 (0.20) for hdPS and PS, respectively. Performance of hdPS was not compromised when known confounders were removed from the database.

RESULTS ON REAL DATA

With 49 investigator-selected covariates, the HR was 1.18 (95% CI 1.10; 1.26) for PS and 1.33 (95% CI 1.22; 1.46) for hdPS. Both methods yielded the same conclusion, suggesting superiority of bitherapy over monotherapy for time to blood pressure control.

CONCLUSION

HdPS can identify proxies for missing confounders, thereby having an advantage over PS in case of unobserved covariates. Both PS and hdPS showed superiority of bitherapy over monotherapy for reaching blood pressure control.

High-dimensional propensity scores for empirical covariate selection in secondary database studies: Planning, implementation, and reporting

Real-world evidence used for regulatory, payer, and clinical decision-making requires principled epidemiology in design and analysis, applying methods to minimize confounding given the lack of randomization. One technique to deal with potential confounding is propensity score (PS) analysis, which allows for the adjustment for measured preexposure covariates. Since its first publication in 2009, the high-dimensional propensity score (hdPS) method has emerged as an approach that extends traditional PS covariate selection to include large numbers of covariates that may reduce confounding bias in the analysis of healthcare databases. hdPS is an automated, data-driven analytic approach for covariate selection that empirically identifies preexposure variables and proxies to include in the PS model. This article provides an overview of the hdPS approach and recommendations on the planning, implementation, and reporting of hdPS used for causal treatment-effect estimations in longitudinal healthcare databases. We supply a checklist with key considerations as a supportive decision tool to aid investigators in the implementation and transparent reporting of hdPS techniques, and to aid decision-makers unfamiliar with hdPS in the understanding and interpretation of studies employing this approach. This article is endorsed by the International Society for Pharmacoepidemiology.

Abiraterone acetate versus docetaxel for metastatic castration-resistant prostate cancer: a cohort study within the French Nationwide Claims Database

OBJECTIVES

To conduct the direct comparison of abiraterone acetate and docetaxel for first-line treatment of metastatic castration-resistant prostate cancer (mCRPC) in real-life settings.

METHODS

Data were extracted from the French nationwide claims database (SNDS) on all men aged ≥40 years starting first-line treatment with abiraterone acetate or docetaxel for mCRPC in 2014. A high-dimensional propensity score including 100 baseline characteristics was used to match patients of both groups and form two comparative cohorts. Three-year overall survival and treatment discontinuation-free survival were determined using Kaplan-Meier analysis.

RESULTS

In 2014, 2,444 patients started abiraterone for treatment of mCRPC and 1,214 started docetaxel. After trimming and matching, 716 patients were available in each group. Median overall survival tended to be longer in the abiraterone acetate cohort (23.8 months, 95% confidence interval = [21.5; 26.0]) than in the docetaxel cohort (20.3 [18.4; 21.6] months). Survival at 36 months was 34.6% for abiraterone acetate and 27.9% for docetaxel (p = 0.0027). Treatment discontinuation-free median was longer in the abiraterone acetate cohort compared to the docetaxel cohort (10.8 [10.1; 11.7] versus 7.4 [7.0; 8.0] months).

CONCLUSION

The findings underline the interest of oral abiraterone acetate over intravenous docetaxel as the first-line treatment option in mCRPC.

Utility of automated data-adaptive propensity score method for confounding by indication in comparative effectiveness study in real world Medicare and registry data

BACKGROUND

Confounding by indication is a serious threat to comparative studies using real world data. We assessed the utility of automated data-adaptive analytic approach for confounding adjustment when both claims and clinical registry data are available.

METHODS

We used a comparative study example of carotid artery stenting (CAS) vs. carotid endarterectomy (CEA) in 2005-2008 when CAS was only indicated for patients with high surgical risk. We included Medicare beneficiaries linked to the Society for Vascular Surgery's Vascular Registry >65 years old undergoing CAS/CEA. We compared hazard ratios (HRs) for death while adjusting for confounding by combining various 1) Propensity score (PS) modeling strategies (investigator-specified [IS-PS] vs. automated data-adaptive [ada-PS]); 2) data sources (claims-only, registry-only and claims-plus-registry); and 3) PS adjustment approaches (matching vs. quintiles-adjustment with/without trimming). An HR of 1.0 was used as a benchmark effect estimate based on CREST trial.

RESULTS

The cohort included 1,999 CAS and 3,255 CEA patients (mean age 76). CAS patients were more likely symptomatic and at high surgical risk, and experienced higher mortality (crude HR = 1.82 for CAS vs. CEA). HRs from PS-quintile adjustment without trimming were 1.48 and 1.52 for claims-only IS-PS and ada-PS, 1.51 and 1.42 for registry-only IS-PS and ada-PS, and 1.34 and 1.23 for claims-plus-registry IS-PS and ada-PS, respectively. Estimates from other PS adjustment approaches showed similar patterns.

CONCLUSIONS

In a comparative effectiveness study of CAS vs. CEA with strong confounding by indication, ada-PS performed better than IS-PS in general, but both claims and registry data were needed to adequately control for bias.

Evaluation of propensity score methods for causal inference with high-dimensional covariates

In recent work, researchers have paid considerable attention to the estimation of causal effects in observational studies with a large number of covariates, which makes the unconfoundedness assumption plausible. In this paper, we review propensity score (PS) methods developed in high-dimensional settings and broadly group them into model-based methods that extend models for prediction to causal inference and balance-based methods that combine covariate balancing constraints. We conducted systematic simulation experiments to evaluate these two types of methods, and studied whether the use of balancing constraints further improved estimation performance. Our comparison methods were post-double-selection (PDS), double-index PS (DiPS), outcome-adaptive LASSO (OAL), group LASSO and doubly robust estimation (GLiDeR), high-dimensional covariate balancing PS (hdCBPS), regularized calibrated estimators (RCAL) and approximate residual balancing method (balanceHD). For the four model-based methods, simulation studies showed that GLiDeR was the most stable approach, with high estimation accuracy and precision, followed by PDS, OAL and DiPS. For balance-based methods, hdCBPS performed similarly to GLiDeR in terms of accuracy, and outperformed balanceHD and RCAL. These findings imply that PS methods do not benefit appreciably from covariate balancing constraints in high-dimensional settings. In conclusion, we recommend the preferential use of GLiDeR and hdCBPS approaches for estimating causal effects in high-dimensional settings; however, further studies on the construction of valid confidence intervals are required.

Synthetic Negative Controls: Using Simulation to Screen Large-scale Propensity Score Analyses

The propensity score has become a standard tool to control for large numbers of variables in healthcare database studies. However, little has been written on the challenge of comparing large-scale propensity score analyses that use different methods for confounder selection and adjustment. In these settings, balance diagnostics are useful but do not inform researchers on which variables balance should be assessed or quantify the impact of residual covariate imbalance on bias. Here, we propose a framework to supplement balance diagnostics when comparing large-scale propensity score analyses. Instead of focusing on results from any single analysis, we suggest conducting and reporting results for many analytic choices and using both balance diagnostics and synthetically generated control studies to screen analyses that show signals of bias caused by measured confounding. To generate synthetic datasets, the framework does not require simulating the outcome-generating process. In healthcare database studies, outcome events are often rare, making it difficult to identify and model all predictors of the outcome to simulate a confounding structure closely resembling the given study. Therefore, the framework uses a model for treatment assignment to divide the comparator population into pseudo-treatment groups where covariate differences resemble those in the study cohort. The partially simulated datasets have a confounding structure approximating the study population under the null (synthetic negative control studies). The framework is used to screen analyses that likely violate partial exchangeability due to lack of control for measured confounding. We illustrate the framework using simulations and an empirical example.

Machine learning for improving high-dimensional proxy confounder adjustment in healthcare database studies: an overview of the current literature

Controlling for large numbers of variables that collectively serve as 'proxies' for unmeasured factors can often improve confounding control in pharmacoepidemiologic studies utilizing administrative healthcare databases. There is a growing body of evidence showing that data-driven machine learning algorithms for high-dimensional proxy confounder adjustment can supplement investigator-specified variables to improve confounding control compared to adjustment based on investigator-specified variables alone. Consequently, there has been a recent focus on the development of data-driven methods for high-dimensional proxy confounder adjustment. In this paper, we discuss the considerations underpinning three areas for data-driven high-dimensional proxy confounder adjustment: 1) feature generation-transforming raw data into covariates (or features) to be used for proxy adjustment; 2) covariate prioritization, selection and adjustment; and 3) diagnostic assessment. We survey current approaches and recent advancements within each area, including the most widely used approach to proxy confounder adjustment in healthcare database studies (the high-dimensional propensity score or hdPS). We also discuss limitations of the hdPS and outline recent advancements that incorporate the principles of proxy adjustment with machine learning extensions to improve performance. We further discuss challenges and avenues of future development within each area. This manuscript is endorsed by the International Society for Pharmacoepidemiology (ISPE). This article is protected by copyright. All rights reserved.

Marginal structural models using calibrated weights with SuperLearner: application to type II diabetes cohort

As different scientific disciplines begin to converge on machine learning for causal inference, we demonstrate the application of machine learning algorithms in the context of longitudinal causal estimation using electronic health records. Our aim is to formulate a marginal structural model for estimating diabetes care provisions in which we envisioned hypothetical (i.e. counterfactual) dynamic treatment regimes using a combination of drug therapies to manage diabetes: metformin, sulfonylurea and SGLT-2i. The binary outcome of diabetes care provisions was defined using a composite measure of chronic disease prevention and screening elements [27] including (i) primary care visit, (ii) blood pressure, (iii) weight, (iv) hemoglobin A1c, (v) lipid, (vi) ACR, (vii) eGFR and (viii) statin medication. We used several statistical learning algorithms to describe causal relationships between the prescription of three common classes of diabetes medications and quality of diabetes care using the electronic health records contained in National Diabetes Repository. In particular, we generated an ensemble of statistical learning algorithms using the SuperLearner framework based on the following base learners: (i) least absolute shrinkage and selection operator, (ii) ridge regression, (iii) elastic net, (iv) random forest, (v) gradient boosting machines, and (vi) neural network. Each statistical learning algorithm was fitted using the pseudo-population generated from the marginalization of the time-dependent confounding process. Covariate balance was assessed using the longitudinal (i.e. cumulative-time product) stabilized weights with calibrated restrictions. Our results indicated that the treatment drop-in cohorts (with respect to metformin, sulfonylurea and SGLT-2i) may have improved diabetes care provisions in relation to treatment naive (i.e. no treatment) cohort. As a clinical utility, we hope that this article will facilitate discussions around the prevention of adverse chronic outcomes associated with type II diabetes through the improvement of diabetes care provisions in primary care.

Reduction of the Cycle Time in the Biopsies Diagnosis Through a Simulation Based on the Box Müller Algorithm

Anatomic pathology services study disease in hospitals on the basis of macroscopic and microscopic examination of organs and tissues. The focus of this research investigation was on improving clinical biopsy diagnosis times through simulation based on the Box-Muller algorithm to reduce the waiting time in the diagnosis of clinical biopsies. The data were provided by a hospital in San José (Costa Rica). They covered 5 years and showed waiting times for a pathological diagnosis that for some biopsies were close to 120 days. The correlation between the main causes identified and the cycle time in the biopsy diagnostic process was defined. A statistical analysis of the variables most representative of the process and of the waiting times was carried out. It followed the DMAIC structure (Define, Measure, Analyse, Improve, Control) for the continuous improvement of processes. Two of the activities of the process were identified as being the main bottlenecks. Their processing times had a normal distribution, for which reason a Box-Muller algorithm was used to generate the simulation model. The results showed that waiting times for a diagnosis can be reduced to 3 days, for a productive capacity of 8 000 biopsies per annum, optimizing the logistics performance of health care.

Improving In-Hospital Care For Older Adults: A Mixed Methods Study Protocol to Evaluate a System-Wide Sub-Acute Care Intervention in Canada

Introduction:

Acute care hospitals often inadequately prepare older adults to transition back to the community. Interventions that seek to improve this transition process are usually evaluated using healthcare use outcomes (e.g., hospital re-visit rates) only, and do not gather provider and patient perspectives about strategies to better integrate care. This protocol describes how we will use complementary research approaches to evaluate an in-hospital sub-acute care (SAC) intervention, designed to better prepare and transition older adults home.

Methods:

In three sequential research phases, we will assess (1) SAC transition pathways and effectiveness using administrative data, (2) provider fidelity to SAC core practices using chart audits, and (3) SAC implementation outcomes (e.g., facilitators and barriers to success, strategies to better integrate care) using provider and patient interviews.

Results:

Findings from each phase will be combined to determine SAC effectiveness and efficiency; to assess intervention components and implementation processes that ‘work’ or require modification; and to identify provider and patient suggestions for improving care integration, both while patients are hospitalized and to some extent after they transition back home.

Discussion:

This protocol helps to establish a blueprint for comprehensively evaluating interventions conducted in complex care settings using complementary research approaches and data sources.

Transparency of high-dimensional propensity score analyses: guidance for diagnostics and reporting

Purpose

The high‐dimensional propensity score (HDPS) is a semi‐automated procedure for confounder identification, prioritisation and adjustment in large healthcare databases that requires investigators to specify data dimensions, prioritisation strategy and tuning parameters. In practice, reporting of these decisions is inconsistent and this can undermine the transparency, and reproducibility of results obtained. We illustrate reporting tools, graphical displays and sensitivity analyses to increase transparency and facilitate evaluation of the robustness of analyses involving HDPS.

Methods

Using a study from the UK Clinical Practice Research Datalink that implemented HDPS we demonstrate the application of the proposed recommendations.

Results

We identify seven considerations surrounding the implementation of HDPS, such as the identification of data dimensions, method for code prioritisation and number of variables selected. Graphical diagnostic tools include assessing the balance of key confounders before and after adjusting for empirically selected HDPS covariates and the identification of potentially influential covariates. Sensitivity analyses include varying the number of covariates selected and assessing the impact of covariates behaving empirically as instrumental variables. In our example, results were robust to both the number of covariates selected and the inclusion of potentially influential covariates. Furthermore, our HDPS models achieved good balance in key confounders.

Conclusions

The data‐adaptive approach of HDPS and the resulting benefits have led to its popularity as a method for confounder adjustment in pharmacoepidemiological studies. Reporting of HDPS analyses in practice may be improved by the considerations and tools proposed here to increase the transparency and reproducibility of study results.

Applying Machine Learning in Distributed Data Networks for Pharmacoepidemiologic and Pharmacovigilance Studies: Opportunities, Challenges, and Considerations

Increasing availability of electronic health databases capturing real-world experiences with medical products has garnered much interest in their use for pharmacoepidemiologic and pharmacovigilance studies. The traditional practice of having numerous groups use single databases to accomplish similar tasks and address common questions about medical products can be made more efficient through well-coordinated multi-database studies, greatly facilitated through distributed data network (DDN) architectures. Access to larger amounts of electronic health data within DDNs has created a growing interest in using data-adaptive machine learning (ML) techniques that can automatically model complex associations in high-dimensional data with minimal human guidance. However, the siloed storage and diverse nature of the databases in DDNs create unique challenges for using ML. In this paper, we discuss opportunities, challenges, and considerations for applying ML in DDNs for pharmacoepidemiologic and pharmacovigilance studies. We first discuss major types of activities performed by DDNs and how ML may be used. Next, we discuss practical data-related factors influencing how DDNs work in practice. We then combine these discussions and jointly consider how opportunities for ML are affected by practical data-related factors for DDNs, leading to several challenges. We present different approaches for addressing these challenges and highlight efforts that real-world DDNs have taken or are currently taking to help mitigate them. Despite these challenges, the time is ripe for the emerging interest to use ML in DDNs, and the utility of these data-adaptive modeling techniques in pharmacoepidemiologic and pharmacovigilance studies will likely continue to increase in the coming years.

A Comparison of Confounder Selection and Adjustment Methods for Estimating Causal Effects Using Large Healthcare Databases

PURPOSE

Confounding adjustment is required to estimate the effect of an exposure on an outcome in observational studies. However, variable selection and unmeasured confounding are particularly challenging when analyzing large healthcare data. Machine learning methods may help address these challenges. The objective was to evaluate the capacity of such methods to select confounders and reduce unmeasured confounding bias.

METHODS

A simulation study with known true effects was conducted. Completely synthetic and partially synthetic data incorporating real large healthcare data were generated. We compared Bayesian Adjustment for Confounding, Generalized Bayesian Causal Effect Estimation, Group Lasso and Doubly Robust Estimation, high-dimensional propensity score, and scalable collaborative targeted maximum likelihood algorithms. For the high-dimensional propensity score, two adjustment approaches targeting the effect in the whole population were considered: full matching and inverse probability weighting.

RESULTS

In scenarios without hidden confounders, most methods were essentially unbiased. The bias and variance of the high-dimensional propensity score varied considerably according to the number of variables selected by the algorithm. In scenarios with hidden confounders, substantial bias reduction was achieved by using machine learning methods to identify proxies as compared to adjusting only by observed confounders. High-dimensional propensity score and Group Lasso performed poorly in the partially synthetic simulation. Bayesian Adjustment for Confounding, Generalized Bayesian Causal Effect Estimation, and scalable collaborative targeted maximum likelihood algorithms performed particularly well.

CONCLUSIONS

Machine learning can help to identify measured confounders in large healthcare databases. They can also capitalize on proxies of unmeasured confounders to substantially reduce residual confounding bias. This article is protected by copyright. All rights reserved.

Broadening the reach of the FDA Sentinel system: A roadmap for integrating electronic health record data in a causal analysis framework

The Sentinel System is a major component of the United States Food and Drug Administration's (FDA) approach to active medical product safety surveillance. While Sentinel has historically relied on large quantities of health insurance claims data, leveraging longitudinal electronic health records (EHRs) that contain more detailed clinical information, as structured and unstructured features, may address some of the current gaps in capabilities. We identify key challenges when using EHR data to investigate medical product safety in a scalable and accelerated way, outline potential solutions, and describe the Sentinel Innovation Center's initiatives to put solutions into practice by expanding and strengthening the existing system with a query-ready, large-scale data infrastructure of linked EHR and claims data. We describe our initiatives in four strategic priority areas: (1) data infrastructure, (2) feature engineering, (3) causal inference, and (4) detection analytics, with the goal of incorporating emerging data science innovations to maximize the utility of EHR data for medical product safety surveillance.

Validity of a model using routinely collected data for identifying infections following gastric, colorectal, and liver cancer surgeries

PURPOSE

Validating outcome measures is a prerequisite for using administrative databases for comparative effectiveness research. Although the Japanese Diagnosis Procedure Combination database is widely used in surgical studies, the outcome measure for postsurgical infection has not been validated. We developed a model to identify postsurgical infections using the routinely-collected Diagnosis Procedure Combination data.

METHODS

We retrospectively identified inpatients who underwent surgery for gastric, colorectal, or liver cancer between April 2016 and March 2018 at four hospitals. Chart reviews were conducted to identify postsurgical infections. We used bootstrap analysis with backwards variable elimination to select independent variables from routinely-collected diagnosis and procedure data. Selected variables were used to create a score predicting the chart review-identified infections, and the performance of the score was tested.

RESULTS

Among the 756 eligible patients, 102 patients (13%) had postoperative infections. Three variables were identified as predictors: diagnosis of infectious disease recorded as a complication arising after admission, addition of an intravenous antibiotic, and bacterial microscopy or culture. The prediction model had a C-statistic of 0.891 and pseudo-R² of 0.380. A cut-off of 1 point of the score showed a sensitivity of 92% and specificity of 71%, and a cut-off of 2 points showed a sensitivity of 77% and specificity of 91%.

CONCLUSIONS

Our model using routinely-collected administrative data accurately identified postoperative infections. Further external validation would lead to the application of the model for research using administrative databases. This article is protected by copyright. All rights reserved.

Risk of COPD Exacerbations Associated with Statins versus Fibrates: A New User, Active Comparison, and High-Dimensional Propensity Score Matched Cohort Study

BACKGROUND

Several observational studies have found that statins may materially decrease the risk of chronic obstructive pulmonary disease (COPD) exacerbations. However, most of these studies used a prevalent user, non-user comparison approach, which may lead to overestimation of the clinical benefits of statins. We aimed to explore the risk of COPD exacerbations associated with statins with a new user, active comparison approach to address potential methodological concerns. We selected fibrates, another class of lipid-lowering agents, as the reference group because no evidence suggests that fibrates have an effect on COPD exacerbations.

METHODS

We identified patients with COPD who initiated statins or fibrates from a nationwide Taiwanese database. Patients were followed from cohort entry to the earliest of the following: hospitalization for COPD exacerbations, death, end of the data, or 180 days after cohort entry. Stratified Cox regression models were used to estimate hazard ratios (HRs) and 95% confidence intervals (CIs) of COPD exacerbations comparing statins with fibrates after variable-ratio propensity score (PS) matching and high-dimensional PS (hd-PS) matching, respectively.

RESULTS

We identified a total of 134,909 eligible patients (110,726 initiated statins; 24,183 initiated fibrates); 1979 experienced COPD exacerbations during follow-up. The HRs were 1.10 (95% CI, 0.96 to 1.26) after PS matching and 1.08 (95% CI, 0.94 to 1.24) after hd-PS matching. The results did not differ materially by type of statins and patient characteristic and did not change with longer follow-up durations.

CONCLUSION

This large-scale, population-based cohort study did not show that use of statins was associated with a reduced risk of acute exacerbations in patients with COPD using state-of-the-art pharmacoepidemiologic approaches. The findings emphasize the importance of applying appropriate methodology in exploring statin effectiveness in real-world settings.

Conducting Real-world Evidence Studies on the Clinical Outcomes of Diabetes Treatments

Real-world evidence (RWE), the understanding of treatment effectiveness in clinical practice generated from longitudinal patient-level data from the routine operation of the healthcare system, is thought to complement evidence on the efficacy of medications from randomized controlled trials (RCTs). RWE studies follow a structured approach. (1) A design layer decides on the study design, which is driven by the study question and refined by a medically informed target population, patient-informed outcomes, and biologically informed effect windows. Imagining the randomized trial we would ideally perform before designing an RWE study in its likeness reduces bias; the new-user active comparator cohort design has proven useful in many RWE studies of diabetes treatments. (2) A measurement layer transforms the longitudinal patient-level data stream into variables that identify the study population, the pre-exposure patient characteristics, the treatment, and the treatment-emergent outcomes. Working with secondary data increases the measurement complexity compared to primary data collection that we find in most RCTs. (3) An analysis layer focuses on the causal treatment effect estimation. Propensity score analyses have gained in popularity to minimize confounding in healthcare database analyses. Well-understood investigator errors, like immortal time bias, adjustment for causal intermediates, or reverse causation, should be avoided. To increase reproducibility of RWE findings, studies require full implementation transparency. This article integrates state-of-the-art knowledge on how to conduct and review RWE studies on diabetes treatments to maximize study validity and ultimately increased confidence in RWE-based decision making.

Reflection on modern methods: building causal evidence within high-dimensional molecular epidemiological studies of moderate size

This commentary provides a practical perspective on epidemiological analysis within a single high-dimensional study of moderate size to consider a causal question. In this setting, non-causal confounding is important. This occurs when a factor is a determinant of outcome and the underlying association between exposure and the factor is non-causal. That is, the association arises due to chance, confounding or other bias rather than reflecting that exposure and the factor are causally related. In particular, the influence of technical processing factors must be accounted for by pre-processing measures to remove artefact or to control for these factors such as batch run. Work steps include the evaluation of alternative non-causal explanations for observed exposure-disease associations and strategies to obtain the highest level of causal inference possible within the study. A systematic approach is required to work through a question set and obtain insights on not only the exposure-disease association but also the multifactorial causal structure of the underlying data where possible. The appropriate inclusion of molecular findings will enhance the quest to better understand multifactorial disease causation in modern observational epidemiological studies.

Real-world evidence of bariatric surgery and cardiovascular benefits using electronic health records data: A lesson in bias

AIM

To reproduce and correct studies on bariatric surgery and the reduction in major adverse cardiovascular events (MACE) among patients with obesity and type 2 diabetes (T2D).

METHODS

We used electronic healthcare records (EHR) from in and outpatient facilities around the United States to identify a cohort of patients with T2D, aged 18 to 80 years and with a body mass index (BMI) of 30 kg/m² or higher undergoing bariatric surgery. We compared against hip/knee arthroplasty to establish an active comparison group that reduced bias from differential information and confounding. The main outcome was six-point MACE. Pre-exposure characteristics were adjusted in propensity score (PS) models with 1:2 matching plus high-dimensional PS matching.

RESULTS

After a range of exclusions, the final cohort included 344 bariatric surgery patients (65% female; mean age 58 years) and 551 PS-matched patients undergoing arthroplasty (65% female; 59 years). Median follow-up was 2.5 years in both groups. Bariatric surgery patients showed a sustained 20% weight reduction and an HbA1c reduction by 1% point. We found no benefits of bariatric surgery for six-point MACE (HR = 0.99; 95% CI 0.76-1.30). We observed known increases in risks for vitamin B₁₂ deficiency anaemia (HR = 3.06; 1.10-8.49) and cholelithiasis (HR = 1.72; 0.94-3.13).

CONCLUSIONS

This real-world evidence study found reductions in HbA1c and BMI following bariatric surgery similar to trials, and no meaningful cardiovascular benefit compatible with the underpowered trials but in contrast to earlier EHR studies. We showed how information bias typical in EHR analyses and confounding may cause substantial bias.

Effectiveness of angiotensin-neprilysin inhibitor treatment versus renin-angiotensin system blockade in older adults with heart failure in clinical care

OBJECTIVE

To evaluate the effectiveness of angiotensin receptor-neprilysin inhibitor (ARNI) versus renin-angiotensin system (RAS) blockade alone in older adults with heart failure with reduced ejection fraction (HFrEF).

METHODS

We conducted a cohort study using US Medicare fee-for-service claims data (2014-2017). Patients with HFrEF ≥65 years were identified in two cohorts: (1) initiators of ARNI or RAS blockade alone (ACE inhibitor, ACEI; or angiotensin receptor blocker, ARB) and (2) switchers from an ACEI to either ARNI or ARB. HR with 95% CI from Cox proportional hazard regression and 1-year restricted mean survival time (RMST) difference with 95% CI were calculated for a composite outcome of time to first worsening heart failure event or all-cause mortality after adjustment for 71 pre-exposure characteristics through propensity score fine-stratification weighting. All analyses of initiator and switcher cohorts were conducted separately and then combined using fixed effects.

RESULTS

51 208 patients with a mean age of 76 years were included, with 16 193 in the ARNI group. Adjusted HRs comparing ARNI with RAS blockade alone were 0.92 (95% CI 0.84 to 1.00) among initiators and 0.79 (95% CI 0.74 to 0.85) among switchers, with a combined estimate of 0.84 (95% CI 0.80 to 0.89). Adjusted 1-year RMST difference (95% CI) was 4 days in the initiator cohort (-1 to 9) and 12 days (8 to 17) in the switcher cohort, resulting in a pooled estimate of 9 days (6 to 12) favouring ARNI.

CONCLUSION

ARNI treatment was associated with lower risk of a composite effectiveness endpoint compared with RAS blockade alone in older adults with HFrEF.

Deep Learning-based Propensity Scores for Confounding Control in Comparative Effectiveness Research: A Large-scale, Real-world Data Study

BACKGROUND

Due to the non-randomized nature of real-world data, prognostic factors need to be balanced, which is often done by propensity scores (PSs). This study aimed to investigate whether autoencoders, which are unsupervised deep learning architectures, might be leveraged to compute PS.

METHODS

We selected patient-level data of 128,368 first-line treated cancer patients from the Flatiron Health EHR-derived de-identified database. We trained an autoencoder architecture to learn a lower-dimensional patient representation, which we used to compute PS. To compare the performance of an autoencoder-based PS with established methods, we performed a simulation study. We assessed the balancing and adjustment performance using standardized mean differences, root mean square errors (RMSE), percent bias, and confidence interval coverage. To illustrate the application of the autoencoder-based PS, we emulated the PRONOUNCE trial by applying the trial's protocol elements within an observational database setting, comparing two chemotherapy regimens.

RESULTS

All methods but the manual variable selection approach led to well-balanced cohorts with average standardized mean differences <0.1. LASSO yielded on average the lowest deviation of resulting estimates (RMSE 0.0205) followed by the autoencoder approach (RMSE 0.0248). Altering the hyperparameter setup in sensitivity analysis, the autoencoder approach led to similar results as LASSO (RMSE 0.0203 and 0.0205, respectively). In the case study, all methods provided a similar conclusion with point estimates clustered around the null (e.g., HRautoencoder 1.01 [95% confidence interval = 0.80, 1.27] vs. HRPRONOUNCE 1.07 [0.83, 1.36]).

CONCLUSIONS

Autoencoder-based PS computation was a feasible approach to control for confounding but did not perform better than some established approaches like LASSO.

High-dimensional propensity scores improved the control of indication bias in surgical comparative effectiveness studies

OBJECTIVES

The objective of the study is to evaluate the performance of high-dimensional propensity scores (hdPSs) for controlling indication bias as compared with propensity scores (PSs) in surgical comparative effectiveness studies.

STUDY DESIGN AND SETTING

Patients who underwent interventional transcatheter aortic valve implantation (TAVI) or surgical aortic valve replacement (SAVR) between 2013 and 2017 were included from the French nationwide hospitals. At each hospital level, matched pairs of patients who underwent TAVI and SAVR were formed using PSs, considering 20 patient baseline characteristics, and hdPSs, considering the same patient characteristics and 300 additional variables from procedure and diagnosis codes the year before surgery. We compared death, reoperation, and stroke up to 3 years between TAVI and SAVR using Cox or Fine and Gray models.

RESULTS

Before matching, 12 of 20 patient characteristics were imbalanced between the included patients who underwent TAVI and SAVR. No significant imbalance persisted after matching with both methods. Hazard ratio of 1-year death, reoperation, and stroke was 1.3 [1.1; 1.4], 1.6 [1.1; 2.4], and 1.4 [1.2; 1.7] for TAVI relative to SAVR with PSs (n = 9,498 pairs) and 1.1 [1.0; 1.3], 1.3 [0.8; 2.0], and 1.3 [1.0; 1.6] with hdPSs (n = 7,157).

CONCLUSION

HdPS estimations were more consistent with results seen in randomized controlled trials. The HdPS is a promising alternative for the PS to control indication bias in comparative studies of surgical procedures.

Artificial Intelligence, Real-World Automation and the Safety of Medicines

Despite huge technological advances in the capabilities to capture, store, link and analyse data electronically, there has been some but limited impact on routine pharmacovigilance. We discuss emerging research in the use of artificial intelligence, machine learning and automation across the pharmacovigilance lifecycle including pre-licensure. Reasons are provided on why adoption is challenging and we also provide a perspective on changes needed to accelerate adoption, and thereby improve patient safety. Last, we make clear that while technologies could be superimposed on existing pharmacovigilance processes for incremental improvements, these great societal advances in data and technology also provide us with a timely opportunity to reconsider everything we do in pharmacovigilance operations to maximise the benefit of these advances.

Implementing high-dimensional propensity score principles to improve confounder adjustment in UK electronic health records

PURPOSE

Recent evidence from US claims data suggests use of high-dimensional propensity score (hd-PS) methods improve adjustment for confounding in non-randomised studies of interventions. However, it is unclear how best to apply hd-PS principles outside their original setting, given important differences between claims data and electronic health records (EHRs). We aimed to implement the hd-PS in the setting of United Kingdom (UK) EHRs.

METHODS

We studied the interaction between clopidogrel and proton pump inhibitors (PPIs). Whilst previous observational studies suggested an interaction (with reduced effect of clopidogrel), case-only, genetic and randomised trial approaches showed no interaction, strongly suggesting the original observational findings were subject to confounding. We derived a cohort of clopidogrel users from the UK Clinical Practice Research Datalink linked with the Myocardial Ischaemia National Audit Project. Analyses estimated the hazard ratio (HR) for myocardial infarction (MI) comparing PPI users with non-users using a Cox model adjusting for confounders. To reflect unique characteristics of UK EHRs, we varied the application of hd-PS principles including the level of grouping within coding systems and adapting the assessment of code recurrence. Results were compared with traditional analyses.

RESULTS

Twenty-four thousand four hundred and seventy-one patients took clopidogrel, of whom 9111 were prescribed a PPI. Traditional PS approaches obtained a HR for the association between PPI use and MI of 1.17 (95% CI: 1.00-1.35). Applying hd-PS modifications resulted in estimates closer to the expected null (HR 1.00; 95% CI: 0.78-1.28).

CONCLUSIONS

hd-PS provided improved adjustment for confounding compared with other approaches, suggesting hd-PS can be usefully applied in UK EHRs.

Propensity score methods in real-world epidemiology: A practical guide for first-time users

Real-world epidemiology gives us the unique opportunity to observe large numbers of people, and the actions and events that characterize their encounters with healthcare providers. However, the heterogeneity and sheer diversity of the population and healthcare systems makes it impossible for researchers to compare "like with like" when attempting to draw causal inferences about interventions and outcomes. The critical issue in epidemiological datasets relates to high risk of bias due to confounders that stem from baseline differences between groups. Propensity score (PS) techniques are statistical approaches that have been used to tackle potential imbalance in the comparison groups. The PS is the estimated probability (based on measured baseline covariates) that the patient receives a particular intervention. Patients that share similar PS will most likely have the same distributions of underlying covariates included in the PS. Implementation of PS methods may achieve better balance of covariates, but there is no consensus on the best way of capturing all relevant confounders for incorporation into the PS model. Should covariates be selected by clinical or epidemiological experts, or would data-driven algorithms (machine learning) offer more efficient and reliable methods of estimating PS and controlling for confounding? The PS can be incorporated into the analysis in different ways, each with its own strengths and limitations, and researchers must choose the best fit for their study objectives. PS methods are particularly advantageous in situations where there are large numbers of measured covariates but relatively few outcome events captured in healthcare administrative databases.

Big Data - How to Realize the Promise

The increasing volume and complexity of data now being captured across multiple settings and devices offers the opportunity to deliver a better characterization of diseases, treatments, and the performance of medicinal products in individual healthcare systems. Such data sources, commonly labeled as big data, are generally large, accumulating rapidly, and incorporate multiple data types and forms. Determining the acceptability of these data to support regulatory decisions demands an understanding of data provenance and quality in addition to confirming the validity of new approaches and methods for processing and analyzing these data. The Heads of Agencies and the European Medicines Agency Joint Big Data Taskforce was established to consider these issues from the regulatory perspective. This review reflects the thinking from its first phase and describes the big data landscape from a regulatory perspective and the challenges to be addressed in order that regulators can know when and how to have confidence in the evidence generated from big datasets.

What Can We Learn About Drug Safety and Other Effects in the Era of Electronic Health Records and Big Data That We Would Not Be Able to Learn From Classic Epidemiology?

As more and more health systems have converted to the use of electronic health records, the amount of searchable and analyzable data is exploding. This includes not just provider or laboratory created data but also data collected by instruments, personal devices, and patients themselves, among others. This has led to more attention being paid to the analysis of these data to answer previously unaddressed questions. This is especially important given the number of therapies previously found to be beneficial in clinical trials that are currently being re-scrutinized. Because there are orders of magnitude more information contained in these data sets, a fundamentally different approach needs to be taken to their processing and analysis and the generation of knowledge. Health care and medicine are drivers of this phenomenon and will ultimately be the main beneficiaries. Concurrently, many different types of questions can now be asked using these data sets. Research groups have become increasingly active in mining large data sets, including nationwide health care databases, to learn about associations of medication use and various unrelated diseases such as cancer. Given the recent increase in research activity in this area, its promise to radically change clinical research, and the relative lack of widespread knowledge about its potential and advances, we surveyed the available literature to understand the strengths and limitations of these new tools. We also outline new databases and techniques that are available to researchers worldwide, with special focus on work pertaining to the broad and rapid monitoring of drug safety and secondary effects.

Pharmacovigilance - The next chapter

The discovery and quantification of adverse drug reactions has long relied on the careful analysis of spontaneously reported cases. Causality assessment (imputation) was a fundamental feature of individual case report analysis. This was complemented by analysis of aggregated cases, and of disproportionality analyses in spontaneous reports databases. In the absence of more specific information sources, these have resulted in the discovery of many new adverse reactions, altering drug information. It has led to the withdrawal from the market of many drugs, but its use for risk quantification remains fraught with uncertainty. The recent access to population-wide claims or electronic health records databases have confirmed for spontaneous reporting a predominant role in hypothesis generation for serious adverse drug reactions, notably those that result in hospital admission or death. In these cases, the events are identifiable at the population level, and can be quantified precisely using the tools of modern pharmacoepidemiology, to generate specific benefit-risk analyses. Spontaneous reporting remains irreplaceable in signal and alert generation in drug safety, despite its inherent limitations. For signal strengthening and assessment, more systematic and quantitative methods should be sought, such as claims databases for reactions resulting in hospital admissions.

Estimates of all cause mortality and cause specific mortality associated with proton pump inhibitors among US veterans: cohort study

OBJECTIVE

To estimate all cause mortality and cause specific mortality among patients taking proton pump inhibitors (PPIs).

DESIGN

Longitudinal observational cohort study.

SETTING

US Department of Veterans Affairs.

PARTICIPANTS

New users of PPIs (n=157 625) or H2 blockers (n=56 842).

MAIN OUTCOME MEASURES

All cause mortality and cause specific mortality associated with taking PPIs (values reported as number of attributable deaths per 1000 patients taking PPIs).

RESULTS

There were 45.20 excess deaths (95% confidence interval 28.20 to 61.40) per 1000 patients taking PPIs. Circulatory system diseases (number of attributable deaths per 1000 patients taking PPIs 17.47, 95% confidence interval 5.47 to 28.80), neoplasms (12.94, 1.24 to 24.28), infectious and parasitic diseases (4.20, 1.57 to 7.02), and genitourinary system diseases (6.25, 3.22 to 9.24) were associated with taking PPIs. There was a graded relation between cumulative duration of PPI exposure and the risk of all cause mortality and death due to circulatory system diseases, neoplasms, and genitourinary system diseases. Analyses of subcauses of death suggested that taking PPIs was associated with an excess mortality due to cardiovascular disease (15.48, 5.02 to 25.19) and chronic kidney disease (4.19, 1.56 to 6.58). Among patients without documented indication for acid suppression drugs (n=116 377), taking PPIs was associated with an excess mortality due to cardiovascular disease (22.91, 11.89 to 33.57), chronic kidney disease (4.74, 1.53 to 8.05), and upper gastrointestinal cancer (3.12, 0.91 to 5.44). Formal interaction analyses suggested that the risk of death due to these subcauses was not modified by a history of cardiovascular disease, chronic kidney disease, or upper gastrointestinal cancer. Taking PPIs was not associated with an excess burden of transportation related mortality and death due to peptic ulcer disease (as negative outcome controls).

CONCLUSIONS

Taking PPIs is associated with a small excess of cause specific mortality including death due to cardiovascular disease, chronic kidney disease, and upper gastrointestinal cancer. The burden was also observed in patients without an indication for PPI use. Heightened vigilance in the use of PPI may be warranted.