Machine learning in the estimation of causal effects: targeted minimum loss-based estimation and double/debiased machine learning

The modern alchemy of clinical pathology: turning the output of microbiology laboratory operations into gold

The clinical microbiology laboratory generates a huge amount of high-quality data that play a vital role in clinical care. With proper extraction, cleaning, analysis, and validation pipelines, these data can serve multiple other purposes that include supporting laboratory operations, understanding local epidemiology, informing hospital-specific policies, and public health surveillance. In this review, I use one of the core activities of the microbiology laboratory, antimicrobial susceptibility testing (AST), to illustrate several potential applications of next-generation data analytics. The first involves continuous monitoring of commercial AST systems using comparisons of minimum inhibitory concentration (MIC) distributions over time to trigger re-verification when statistically significant differences are detected. An extension of this is temporal analysis of joint MIC distributions to understand performance for multidrug-resistant organisms. More sophisticated analyses involve linking microbiologic data to clinical metadata to gain insight into the clinical validity of AST data and to inform treatment policies. The elements of a robust, validated analysis engine using routine data streams already exist, but numerous challenges must be overcome to make it a reality. Most importantly, it will require the sustained collaboration and advocacy of hospital leadership, microbiologists, clinicians, antimicrobial stewardship, data scientists, and regulatory agencies. Though no small feat, achieving this vision would provide an important resource for microbiology laboratories facing a rapidly evolving practice landscape and further cement its role as an integral part of a learning health system.

Frameworks for estimating causal effects in observational settings: comparing confounder adjustment and instrumental variables

To estimate causal effects, analysts performing observational studies in health settings utilize several strategies to mitigate bias due to confounding by indication. There are two broad classes of approaches for these purposes: use of confounders and instrumental variables (IVs). Because such approaches are largely characterized by untestable assumptions, analysts must operate under an indefinite paradigm that these methods will work imperfectly. In this tutorial, we formalize a set of general principles and heuristics for estimating causal effects in the two approaches when the assumptions are potentially violated. This crucially requires reframing the process of observational studies as hypothesizing potential scenarios where the estimates from one approach are less inconsistent than the other. While most of our discussion of methodology centers around the linear setting, we touch upon complexities in non-linear settings and flexible procedures such as target minimum loss-based estimation and double machine learning. To demonstrate the application of our principles, we investigate the use of donepezil off-label for mild cognitive impairment. We compare and contrast results from confounder and IV methods, traditional and flexible, within our analysis and to a similar observational study and clinical trial.

Screening of antibacterial compounds with novel structure from the FDA approved drugs using machine learning methods

Bacterial infection is one of the most important factors affecting the human life span. Elderly people are more harmed by bacterial infections due to their deficits in immunity. Because of the lack of new antibiotics in recent years, bacterial resistance has increasingly become a serious problem globally. In this study, an antibacterial compound predictor was constructed using the support vector machines and random forest methods and the data of the active and inactive antibacterial compounds from the ChEMBL database. The results showed that both models have excellent prediction performance (mean accuracy >0.9 and mean AUC >0.9 for the two models). We used the predictor to screen potential antibacterial compounds from FDA-approved drugs in the DrugBank database. The screening results showed that 1087 small-molecule drugs have potential antibacterial activity and 154 of them are FDA-approved antibacterial drugs, which accounts for 76.2% of the approved antibacterial drugs collected in this study. Through molecular fingerprint similarity analysis and common substructure analysis, we screened 8 predicted antibacterial small-molecule compounds with novel structures compared with known antibacterial drugs, and 5 of them are widely used in the treatment of various tumors. This study provides a new insight for predicting antibacterial compounds by using approved drugs, the predicted compounds might be used to treat bacterial infections and extend lifespan.

Designing efficient randomized trials: power and sample size calculation when using semiparametric efficient estimators

Trials enroll a large number of subjects in order to attain power, making them expensive and time-consuming. Sample size calculations are often performed with the assumption of an unadjusted analysis, even if the trial analysis plan specifies a more efficient estimator (e.g. ANCOVA). This leads to conservative estimates of required sample sizes and an opportunity for savings. Here we show that a relatively simple formula can be used to estimate the power of any two-arm, single-timepoint trial analyzed with a semiparametric efficient estimator, regardless of the domain of the outcome or kind of treatment effect (e.g. odds ratio, mean difference). Since an efficient estimator attains the minimum possible asymptotic variance, this allows for the design of trials that are as small as possible while still attaining design power and control of type I error. The required sample size calculation is parsimonious and requires the analyst to provide only a small number of population parameters. We verify in simulation that the large-sample properties of trials designed this way attain their nominal values. Lastly, we demonstrate how to use this formula in the "design" (and subsequent reanalysis) of a real randomized trial and show that fewer subjects are required to attain the same design power when a semiparametric efficient estimator is accounted for at the design stage.

Applications of Machine Learning to the Problem of Antimicrobial Resistance: an Emerging Model for Translational Research

Antimicrobial resistance (AMR) remains one of the most challenging phenomena of modern medicine. Machine learning (ML) is a subfield of artificial intelligence that focuses on the development of algorithms that learn how to accurately predict outcome variables using large sets of predictor variables that are typically not hand selected and are minimally curated. Models are parameterized using a training data set and then applied to a test data set on which predictive performance is evaluated. The application of ML algorithms to the problem of AMR has garnered increasing interest in the past 5 years due to the exponential growth of experimental and clinical data, heavy investment in computational capacity, improvements in algorithm performance, and increasing urgency for innovative approaches to reducing the burden of disease. Here, we review the current state of research at the intersection of ML and AMR with an emphasis on three domains of work. The first is the prediction of AMR using genomic data. The second is the use of ML to gain insight into the cellular functions disrupted by antibiotics, which forms the basis for understanding mechanisms of action and developing novel anti-infectives. The third focuses on the application of ML for antimicrobial stewardship using data extracted from the electronic health record. Although the use of ML for understanding, diagnosing, treating, and preventing AMR is still in its infancy, the continued growth of data and interest ensures it will become an important tool for future translational research programs.

The Unique Impact of COVID-19 on Human Gut Microbiome Research

The coronavirus (COVID-19) pandemic has disrupted clinical trials globally, with unique implications for research into the human gut microbiome. In this mini-review, we explore the direct and indirect influences of the pandemic on the gut microbiome and how these can affect research and clinical trials. We explore the direct bidirectional relationships between the COVID-19 virus and the gut and lung microbiomes. We then consider the significant indirect effects of the pandemic, such as repeated lockdowns, increased hand hygiene, and changes to mood and diet, that could all lead to longstanding changes to the gut microbiome at an individual and a population level. Together, these changes may affect long term microbiome research, both in observational as well as in population studies, requiring urgent attention. Finally, we explore the unique implications for clinical trials using faecal microbiota transplants (FMT), which are increasingly investigated as potential treatments for a range of diseases. The pandemic introduces new barriers to participation in trials, while the direct and indirect effects laid out above can present a confounding factor. This affects recruitment and sample size, as well as study design and statistical analyses. Therefore, the potential impact of the pandemic on gut microbiome research is significant and needs to be specifically addressed by the research community and funders.

G-computation and machine learning for estimating the causal effects of binary exposure statuses on binary outcomes

In clinical research, there is a growing interest in the use of propensity score-based methods to estimate causal effects. G-computation is an alternative because of its high statistical power. Machine learning is also increasingly used because of its possible robustness to model misspecification. In this paper, we aimed to propose an approach that combines machine learning and G-computation when both the outcome and the exposure status are binary and is able to deal with small samples. We evaluated the performances of several methods, including penalized logistic regressions, a neural network, a support vector machine, boosted classification and regression trees, and a super learner through simulations. We proposed six different scenarios characterised by various sample sizes, numbers of covariates and relationships between covariates, exposure statuses, and outcomes. We have also illustrated the application of these methods, in which they were used to estimate the efficacy of barbiturates prescribed during the first 24 h of an episode of intracranial hypertension. In the context of GC, for estimating the individual outcome probabilities in two counterfactual worlds, we reported that the super learner tended to outperform the other approaches in terms of both bias and variance, especially for small sample sizes. The support vector machine performed well, but its mean bias was slightly higher than that of the super learner. In the investigated scenarios, G-computation associated with the super learner was a performant method for drawing causal inferences, even from small sample sizes.

Regulatory oversight, causal inference, and safe and effective health care machine learning

In recent years, the applications of Machine Learning (ML) in the health care delivery setting have grown to become both abundant and compelling. Regulators have taken notice of these developments and the U.S. Food and Drug Administration (FDA) has been engaging actively in thinking about how best to facilitate safe and effective use. Although the scope of its oversight for software-driven products is limited, if FDA takes the lead in promoting and facilitating appropriate applications of causal inference as a part of ML development, that leadership is likely to have implications well beyond regulated products.

On the Convergence of Epidemiology, Biostatistics, and Data Science

Epidemiology, biostatistics, and data science are broad disciplines that incorporate a variety of substantive areas. Common among them is a focus on quantitative approaches for solving intricate problems. When the substantive area is health and health care, the overlap is further cemented. Researchers in these disciplines are fluent in statistics, data management and analysis, and health and medicine, to name but a few competencies. Yet there are important and perhaps mutually exclusive attributes of these fields that warrant a tighter integration. For example, epidemiologists receive substantial training in the science of study design, measurement, and the art of causal inference. Biostatisticians are well versed in the theory and application of methodological techniques, as well as the design and conduct of public health research. Data scientists receive equivalently rigorous training in computational and visualization approaches for high-dimensional data. Compared to data scientists, epidemiologists and biostatisticians may have less expertise in computer science and informatics, while data scientists may benefit from a working knowledge of study design and causal inference. Collaboration and cross-training offer the opportunity to share and learn of the constructs, frameworks, theories, and methods of these fields with the goal of offering fresh and innovate perspectives for tackling challenging problems in health and health care. In this article, we first describe the evolution of these fields focusing on their convergence in the era of electronic health data, notably electronic medical records (EMRs). Next we present how a collaborative team may design, analyze, and implement an EMR-based study. Finally, we review the curricula at leading epidemiology, biostatistics, and data science training programs, identifying gaps and offering suggestions for the fields moving forward.