Thyroid Cancer Polygenic Risk Score Improves Classification of Thyroid Nodules as Benign or Malignant

CONTEXT

Thyroid nodule ultrasound-based risk stratification schemas rely on the presence of high-risk sonographic features. However, some malignant thyroid nodules have benign appearance on thyroid ultrasound. New methods for thyroid nodule risk assessment are needed.

OBJECTIVE

We investigated polygenic risk score (PRS) accounting for inherited thyroid cancer risk combined with ultrasound-based analysis for improved thyroid nodule risk assessment.

METHODS

The convolutional neural network classifier was trained on thyroid ultrasound still images and cine clips from 621 thyroid nodules. Phenome-wide association study (PheWAS) and PRS PheWAS were used to optimize PRS for distinguishing benign and malignant nodules. PRS was evaluated in 73 346 participants in the Colorado Center for Personalized Medicine Biobank.

RESULTS

When the deep learning model output was combined with thyroid cancer PRS and genetic ancestry estimates, the area under the receiver operating characteristic curve (AUROC) of the benign vs malignant thyroid nodule classifier increased from 0.83 to 0.89 (DeLong, P value = .007). The combined deep learning and genetic classifier achieved a clinically relevant sensitivity of 0.95, 95% CI [0.88-0.99], specificity of 0.63 [0.55-0.70], and positive and negative predictive values of 0.47 [0.41-0.58] and 0.97 [0.92-0.99], respectively. AUROC improvement was consistent in European ancestry-stratified analysis (0.83 and 0.87 for deep learning and deep learning combined with PRS classifiers, respectively). Elevated PRS was associated with a greater risk of thyroid cancer structural disease recurrence (ordinal logistic regression, P value = .002).

CONCLUSION

Augmenting ultrasound-based risk assessment with PRS improves diagnostic accuracy.

Stratification of Pediatric COVID-19 Cases Using Inflammatory Biomarker Profiling and Machine Learning

While pediatric COVID-19 is rarely severe, a small fraction of children infected with SARS-CoV-2 go on to develop multisystem inflammatory syndrome (MIS-C), with substantial morbidity. An objective method with high specificity and high sensitivity to identify current or imminent MIS-C in children infected with SARS-CoV-2 is highly desirable. The aim was to learn about an interpretable novel cytokine/chemokine assay panel providing such an objective classification. This retrospective study was conducted on four groups of pediatric patients seen at multiple sites of Texas Children's Hospital, Houston, TX who consented to provide blood samples to our COVID-19 Biorepository. Standard laboratory markers of inflammation and a novel cytokine/chemokine array were measured in blood samples of all patients. Group 1 consisted of 72 COVID-19, 70 MIS-C and 63 uninfected control patients seen between May 2020 and January 2021 and predominantly infected with pre-alpha variants. Group 2 consisted of 29 COVID-19 and 43 MIS-C patients seen between January and May 2021 infected predominantly with the alpha variant. Group 3 consisted of 30 COVID-19 and 32 MIS-C patients seen between August and October 2021 infected with alpha and/or delta variants. Group 4 consisted of 20 COVID-19 and 46 MIS-C patients seen between October 2021 andJanuary 2022 infected with delta and/or omicron variants. Group 1 was used to train an L1-regularized logistic regression model which was tested using five-fold cross validation, and then separately validated against the remaining naïve groups. The area under receiver operating curve (AUROC) and F1-score were used to quantify the performance of the cytokine/chemokine assay-based classifier. Standard laboratory markers predict MIS-C with a five-fold cross-validated AUROC of 0.86 ± 0.05 and an F1 score of 0.78 ± 0.07, while the cytokine/chemokine panel predicted MIS-C with a five-fold cross-validated AUROC of 0.95 ± 0.02 and an F1 score of 0.91 ± 0.04, with only sixteen of the forty-five cytokines/chemokines sufficient to achieve this performance. Tested on Group 2 the cytokine/chemokine panel yielded AUROC = 0.98 and F1 = 0.93, on Group 3 it yielded AUROC = 0.89 and F1 = 0.89, and on Group 4 AUROC = 0.99 and F1 = 0.97. Adding standard laboratory markers to the cytokine/chemokine panel did not improve performance. A top-10 subset of these 16 cytokines achieves equivalent performance on the validation data sets. Our findings demonstrate that a sixteen-cytokine/chemokine panel as well as the top ten subset provides a highly sensitive, and specific method to identify MIS-C in patients infected with SARS-CoV-2 of all the major variants identified to date.

Stratification of Pediatric COVID-19 cases by inflammatory biomarker profiling and machine learning

An objective method to identify imminent or current Multi-Inflammatory Syndrome in Children (MIS-C) infected with SARS-CoV-2 is highly desirable. The aims was to define an algorithmically interpreted novel cytokine/chemokine assay panel providing such an objective classification. This study was conducted on 4 groups of patients seen at multiple sites of Texas Children's Hospital, Houston, TX who consented to provide blood samples to our COVID-19 Biorepository. Standard laboratory markers of inflammation and a novel cytokine/chemokine array were measured in blood samples of all patients. Group 1 consisted of 72 COVID-19, 66 MIS-C and 63 uninfected control patients seen between May 2020 and January 2021 and predominantly infected with pre-alpha variants. Group 2 consisted of 29 COVID-19 and 43 MIS-C patients seen between January-May 2021 infected predominantly with the alpha variant. Group 3 consisted of 30 COVID-19 and 32 MIS-C patients seen between August-October 2021 infected with alpha and/or delta variants. Group 4 consisted of 20 COVID-19 and 46 MIS-C patients seen between October 2021-January 2022 infected with delta and/or omicron variants. Group 1 was used to train a L1-regularized logistic regression model which was validated using 5-fold cross validation, and then separately validated against the remaining naïve groups. The area under receiver operating curve (AUROC) and F1-score were used to quantify the performance of the algorithmically interpreted cytokine/chemokine assay panel. Standard laboratory markers predict MIS-C with a 5-fold cross-validated AUROC of 0.86 ± 0.05 and an F1 score of 0.78 ± 0.07, while the cytokine/chemokine panel predicted MIS-C with a 5-fold cross-validated AUROC of 0.95 ± 0.02 and an F1 score of 0.91 ± 0.04, with only sixteen of the forty-five cytokines/chemokines sufficient to achieve this performance. Tested on Group 2 the cytokine/chemokine panel yielded AUROC =0.98, F1=0.93, on Group 3 it yielded AUROC=0.89, F1 = 0.89, and on Group 4 AUROC= 0.99, F1= 0.97). Adding standard laboratory markers to the cytokine/chemokine panel did not improve performance. A top-10 subset of these 16 cytokines achieves equivalent performance on the validation data sets. Our findings demonstrate that a sixteen-cytokine/chemokine panel as well as the top ten subset provides a sensitive, specific method to identify MIS-C in patients infected with SARS-CoV-2 of all the major variants identified to date.

RF11 | PSAT234 Deep learning analysis of thyroid nodule ultrasound images has high sensitivity and negative predictive value to rule-out thyroid cancer

Purpose

To evaluate deep learning analysis of thyroid nodule ultrasound images as a rule-out test for thyroid malignancy.

Methods

Supervised deep learning (DL) classifier of thyroid nodules was trained on 32,545 thyroid US images from 621 nodules representing all major benign and malignant types of thyroid lesions and tested on an independent set of 145 nodules collected at a different healthcare system in the United States. The Big Transfer BiT-M ResNet-50×1 convolutional neural net architecture was modified to contain 3, 4, 6 and 3 PreActBottleneck units per block 1 through 4. Weights pretrained on the ImageNet-21k dataset were loaded and weights for blocks 3 and 4 were fine-tuned for the binary classification task of distinguishing benign and malignant thyroid nodules.

Results

The deep learning thyroid nodule classifier achieved an area under receiver operating characteristic curve (AUROC) of 0.889 on five-fold cross-validation. The AUROC improved when images were scaled by nodule size and six randomly selected cine clip frames were added to the training set per epoch. GradCAM class activation heatmaps revealed that microcalcifications and spongiform appearance were reliably recognized by the classifier as malignant and benign features, respectively. Spongiform nodules were found to be benign even when microcystic spaces constituted less than 50% of nodule volume.

To investigate the clinical relevance of the benign vs. malignant classifier, the binary classification threshold for the probability of malignancy generated by model was set at 7% to achieve sensitivity and negative predictive value (NPV) comparable to that of the fine needle aspiration biopsy (FNA). At this threshold, cross-validated deep-learning model achieved a sensitivity of 90%, specificity of 63%, positive predictive value (PPV) of 46% and negative predictive value of 94%. When tested on an independent image set that includes 18 classic papillary thyroid cancers (PTC), 5 follicular variant PTC, 4 medullary thyroid cancers, 3 follicular thyroid cancers (FTC), and 1 Hurthle cell thyroid cancer, the DL classifier achieved AUROC of 0.88, sensitivity of 97%, specificity of 61%, PPV of 40% and NPV of 99%. A single minimally-invasive FTC that had no suspicious features on thyroid ultrasound was incorrectly classified as benign.

Conclusions

This study demonstrates that the ultrasound-based deep-learning classifier of thyroid nodules achieves sensitivity and negative predictive value comparable to that of thyroid fine needle aspiration (FNA). Clinicians may use this tool to augment clinical judgment when determining whether to perform FNA procedures.

Presentation: Saturday, June 11, 2022 1:00 p.m. - 3:00 p.m., Saturday, June 11, 2022 1:06 p.m. - 1:11 p.m.

Leveraging big data bioinformatics approaches to extract knowledge from Staphylococcus aureus public omics data

Staphylococcus aureus is a notorious pathogen posing challenges in the medical industry due to drug resistance and biofilm formation. The horizon of knowledge on S. aureus pathogenesis has expanded with the advancement of data-driven bioinformatics techniques. Mining information from sequenced genomes and their expression data is an economic approach that alleviates wastage of resources and redundancy in experiments. The current review covers how big data bioinformatics has been used in the analysis of S. aureus from publicly available -omics data to uncover mechanisms of infection and inhibition. Particularly, advances in the past two decades in biomarker discovery, host responses, phenotype identification, consolidation of information, and drug development are discussed highlighting the challenges and shortcomings. Overall, the review summarizes the diverse aspects of scrupulous re-analysis of S. aureus proteomic and transcriptomic expression datasets retrieved from public repositories in terms of the efforts taken, benefits offered, and follow-up actions. The detailed review thus serves as a reference and aid for (i) Computational biologists by briefing the approaches utilized for bacterial omics re-analysis concerning S. aureus and (ii) Experimental biologists by elucidating the potential of bioinformatics in biological research to generate reliable postulates in a prompt and economical manner.

Improving Pharmacovigilance Signal Detection from Clinical Notes with Locality Sensitive Neural Concept Embeddings

Although pharmaceutical products undergo clinical trials to profile efficacy and safety, some adverse drug reactions (ADRs) are only discovered after release to market. Post-market drug safety surveillance - pharmacovigilance - leverages information from various sources to proactively identify such ADRs. Clinical notes are one source of observational data that could assist this process, but their inherent complexity can obfuscate possible ADR signals. In previous research, embeddings trained on observational reports have improved detection of such signals over commonly used statistical measures. Moreover, neural embedding methods which further encode juxtapositional information have shown promise on analogical retrieval tasks, suggesting proximity-based alternatives to document-level modeling for signal detection. This work uses natural language processing and locality sensitive neural embeddings to increase ADR signal recovery from clinical notes, with AUCs of ~0.63-0.71. Constituting a ~50% increase over baselines, our method sets the state-of-the-art for these reference standards when solely leveraging clinical notes.

Predicting Drug Blood-Brain Barrier Penetration with Adverse Event Report Embeddings

Adverse event reports (AER) are widely used for post-market drug safety surveillance and drug repurposing, with the assumption that drugs with similar side-effects may have similar therapeutic effects also. In this study, we used distributed representations of drugs derived from the Food and Drug Administration (FDA) AER system using aer2vec, a method of representing AER, with drug embeddings emerging from a neural network trained to predict the probability of adverse drug effects given observed drugs. We combined these representations with molecular features to predict permeability of the blood-brain barrier to drugs, a prerequisite to their application to treat conditions of the central nervous system. Across multiple machine learning classifiers, the addition of distributed representations improved performance over prior methods using drug-drug similarity estimates derived from discrete representations of AER system data. Embedding-based approaches outperformed those using discrete statistics, with improvements in absolute AUC of 5% and 9%, corresponding to improvements of 9% and 13% over performance with molecular features only. Performance was retained when reducing embedding dimensions from 500 to 6, indicating that they are neither attributable to overfitting, nor to a difference in the number of trainable parameters. These results indicate that aer2vec distributed representations carry information that is valuable for drug repurposing.

Augmenting aer2vec: Enriching distributed representations of adverse event report data with orthographic and lexical information

Adverse Drug Events (ADEs) are prevalent, costly, and sometimes preventable. Post-marketing drug surveillance aims to monitor ADEs that occur after a drug is released to market. Reports of such ADEs are aggregated by reporting systems, such as the Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS). In this paper, we consider the topic of how best to represent data derived from reports in FAERS for the purpose of detecting post-marketing surveillance signals, in order to inform regulatory decision making. In our previous work, we developed aer2vec, a method for deriving distributed representations (concept embeddings) of drugs and side effects from ADE reports, establishing the utility of distributional information for pharmacovigilance signal detection. In this paper, we advance this line of research further by evaluating the utility of encoding orthographic and lexical information. We do so by adapting two Natural Language Processing methods, subword embedding and vector retrofitting, which were developed to encode such information into word embeddings. Models were compared for their ability to distinguish between positive and negative examples in a set of manually curated drug/ADE relationships, with both aer2vec enhancements offering advantages in performances over baseline models, and best performance obtained when retrofitting and subword embeddings were applied in concert. In addition, this work demonstrates that models leveraging distributed representations do not require extensive manual preprocessing to perform well on this pharmacovigilance signal detection task, and may even benefit from information that would otherwise be lost during the normalization and standardization process.

Population stratification enables modeling effects of reopening policies on mortality and hospitalization rates

OBJECTIVE

Study the impact of local policies on near-future hospitalization and mortality rates.

MATERIALS AND METHODS

We introduce a novel risk-stratified SIR-HCD model that introduces new variables to model the dynamics of low-contact (e.g., work from home) and high-contact (e.g., work on-site) subpopulations while sharing parameters to control their respective R₀(t) over time. We test our model on data of daily reported hospitalizations and cumulative mortality of COVID-19 in Harris County, Texas, from May 1, 2020, until October 4, 2020, collected from multiple sources (USA FACTS, U.S. Bureau of Labor Statistics, Southeast Texas Regional Advisory Council COVID-19 report, TMC daily news, and Johns Hopkins University county-level mortality reporting).

RESULTS

We evaluated our model's forecasting accuracy in Harris County, TX (the most populated county in the Greater Houston area) during Phase-I and Phase-II reopening. Not only does our model outperform other competing models, but it also supports counterfactual analysis to simulate the impact of future policies in a local setting, which is unique among existing approaches.

DISCUSSION

Mortality and hospitalization rates are significantly impacted by local quarantine and reopening policies. Existing models do not directly account for the effect of these policies on infection, hospitalization, and death rates in an explicit and explainable manner. Our work is an attempt to improve prediction of these trends by incorporating this information into the model, thus supporting decision-making.

CONCLUSION

Our work is a timely effort to attempt to model the dynamics of pandemics under the influence of local policies.

Domain-driven models yield better predictions at lower cost than reservoir computers in Lorenz systems

Recent advances in computing algorithms and hardware have rekindled interest in developing high-accuracy, low-cost surrogate models for simulating physical systems. The idea is to replace expensive numerical integration of complex coupled partial differential equations at fine time scales performed on supercomputers, with machine-learned surrogates that efficiently and accurately forecast future system states using data sampled from the underlying system. One particularly popular technique being explored within the weather and climate modelling community is the echo state network (ESN), an attractive alternative to other well-known deep learning architectures. Using the classical Lorenz 63 system, and the three tier multi-scale Lorenz 96 system (Thornes T, Duben P, Palmer T. 2017 Q. J. R. Meteorol. Soc. 143, 897-908. (doi:10.1002/qj.2974)) as benchmarks, we realize that previously studied state-of-the-art ESNs operate in two distinct regimes, corresponding to low and high spectral radius (LSR/HSR) for the sparse, randomly generated, reservoir recurrence matrix. Using knowledge of the mathematical structure of the Lorenz systems along with systematic ablation and hyperparameter sensitivity analyses, we show that state-of-the-art LSR-ESNs reduce to a polynomial regression model which we call Domain-Driven Regularized Regression (D2R2). Interestingly, D2R2 is a generalization of the well-known SINDy algorithm (Brunton SL, Proctor JL, Kutz JN. 2016 Proc. Natl Acad. Sci. USA 113, 3932-3937. (doi:10.1073/pnas.1517384113)). We also show experimentally that LSR-ESNs (Chattopadhyay A, Hassanzadeh P, Subramanian D. 2019 (http://arxiv.org/abs/1906.08829)) outperform HSR ESNs (Pathak J, Hunt B, Girvan M, Lu Z, Ott E. 2018 Phys. Rev. Lett. 120, 024102. (doi:10.1103/PhysRevLett.120.024102)) while D2R2 dominates both approaches. A significant goal in constructing surrogates is to cope with barriers to scaling in weather prediction and simulation of dynamical systems that are imposed by time and energy consumption in supercomputers. Inexact computing has emerged as a novel approach to helping with scaling. In this paper, we evaluate the performance of three models (LSR-ESN, HSR-ESN and D2R2) by varying the precision or word size of the computation as our inexactness-controlling parameter. For precisions of 64, 32 and 16 bits, we show that, surprisingly, the least expensive D2R2 method yields the most robust results and the greatest savings compared to ESNs. Specifically, D2R2 achieves 68 × in computational savings, with an additional 2 × if precision reductions are also employed, outperforming ESN variants by a large margin. This article is part of the theme issue 'Machine learning for weather and climate modelling'.

Predicting Adverse Drug-Drug Interactions with Neural Embedding of Semantic Predications

The identification of drug-drug interactions (DDIs) is important for patient safety; yet, compared to other pharmacovigilance work, a limited amount of research has been conducted in this space. Recent work has successfully applied a method of deriving distributed vector representations from structured biomedical knowledge, known as Embedding of Semantic Predications (ESP), to the problem of predicting individual drug side effects. In the current paper we extend this work by applying ESP to the problem of predicting polypharmacy side-effects for particular drug combinations, building on a recent reconceptualization of this problem as a network of drug nodes connected by side effect edges. We evaluate ESP embeddings derived from the resulting graph on a side-effect prediction task against a previously reported graph convolutional neural network approach, using the same data and evaluation methods. We demonstrate that ESP models perform better, while being faster to train, more re-usable, and significantly simpler.

aer2vec: Distributed Representations of Adverse Event Reporting System Data as a Means to Identify Drug/Side-Effect Associations

Adverse event report (AER) data are a key source of signal for post marketing drug surveillance. The standard methodology to analyze AER data applies disproportionality metrics, which estimate the strength of drug/side-effect associations from discrete counts of their occurrence at report level. However, in other domains, improvements in predictive modeling accuracy have been obtained through representation learning, where discrete features are replaced by distributed representations learned from unlabeled data. This paper describes aer2vec, a novel representational approach for AER data in which concept embeddings emerge from neural networks trained to predict drug/side-effect co-occurrence. Trained models are evaluated for their utility in identifying drug/side-effect relationships, with improvements over disproportionality metrics in most cases. In addition, we evaluate the utility of an otherwise-untapped resource in the Food and Drug Administration (FDA) AER system - reporter designations of suspected causality - and find that incorporating this information enhances performance of all models evaluated.

Complementing Observational Signals with Literature-Derived Distributed Representations for Post-Marketing Drug Surveillance

INTRODUCTION

As a result of the well documented limitations of data collected by spontaneous reporting systems (SRS), such as bias and under-reporting, a number of authors have evaluated the utility of other data sources for the purpose of pharmacovigilance, including the biomedical literature. Previous work has demonstrated the utility of literature-derived distributed representations (concept embeddings) with machine learning for the purpose of drug side-effect prediction. In terms of data sources, these methods are complementary, observing drug safety from two different perspectives (knowledge extracted from the literature and statistics from SRS data). However, the combined utility of these pharmacovigilance methods has yet to be evaluated.

OBJECTIVE

This research investigates the utility of directly or indirectly combining an observational signal from SRS with literature-derived distributed representations into a single feature vector or in an ensemble approach for downstream machine learning (logistic regression).

METHODS

Leveraging a recently developed representation scheme, concept embeddings were generated from relational connections extracted from the literature and composed to represent drug and associated adverse reactions, as defined by two reference standards of positive (likely causal) and negative (no causal evidence) pairs. Embeddings were presented with and without common measures of observational signal from SRS sources to logistic regressors, and performance was evaluated with the receiver operating characteristic (ROC) area under the curve (AUC) metric.

RESULTS

ROC AUC performance with these composite models improves up to ≈ 20% over SRS-based disproportionality metrics alone and exceeds the best prior results reported in the literature when models leverage both sources of information.

CONCLUSIONS

Results from this study support the hypothesis that knowledge extracted from the literature can enhance the performance of SRS-based methods (and vice versa). Across reference sets, using literature and SRS information together performed better than using either source alone, providing strong support for the complementary nature of these approaches to post-marketing drug surveillance.

RNA-seq analysis reveals resistome genes and signalling pathway associated with vancomycin-intermediate Staphylococcus aureus

Context

Vancomycin-intermediate Staphylococcus aureus remains one of the most prevalent multidrug-resistant pathogens causing healthcare infections that are difficult to treat.

Aims

This study uses a comprehensive computational analysis to systematically investigate various gene expression profiles of resistant and sensitive S. aureus strains on exposure to antibiotics.

Settings and Design

The transcriptional changes leading to the development of multiple antibiotic resistance were examined by an integrative analysis of nine differential expression experiments under selected conditions of vancomycin-intermediate and -sensitive strains for four different antibiotics using publicly available RNA-Seq datasets.

Materials and Methods

For each antibiotic, three experimental conditions for expression analysis were selected to identify those genes that are particularly involved in the development of resistance. The results were further scrutinised to generate a resistome that can be analysed for their role in the development or adaptation to antibiotic resistance.

Results

The 99 genes in the resistome are then compiled to create a multiple drug resistome of 25 known and novel genes identified to play a part in antibiotic resistance. The inclusion of agr genes and associated virulence factors in the identified resistome supports the role of agr quorum sensing system in multiple drug resistance. In addition, enrichment analysis also identified the kyoto encyclopedia of genes and genomes (KEGG) pathways - quorum sensing and two-component system pathways - in the resistome gene set.

Conclusion

Further studies on understanding the role of the identified molecular targets such as SAA6008_00181, SAA6008_01127, agrA, agrC and coa in adapting to the pressure of antibiotics at sub-inhibitory concentrations can help in learning the molecular mechanisms causing resistance to the pathogens as well as finding other potential therapeutics.

APOE Genotype Influences Postprandial Blood Pressure after High Fat Feeding in Older Adults

OBJECTIVE

Postprandial hypotension (PPH) is a common phenomenon among older adults. The degree to which individuals experience PPH is related to cerebrovascular risk factors and the presence of neurodegenerative diseases such as Alzheimer's disease (AD). Carrier status of the E4 allele of the apolipoprotein E (APOE) gene is a risk factor for AD and influences a variety of responses to metabolic and dietary interventions. However, it is unknown whether APOE genotype influences the risk of PPH and whether type of meal can mediate that response.

DESIGN

Acute meal study with a crossover design.

PARTICIPANTS

32 cognitively healthy older adults with (n=18) and without (n=14) E4+ carrier status.

INTERVENTION

As a part of an ongoing meal study we examined the postprandial blood pressure response after ingestion of a high carbohydrate (HCM) and high fat meal (HFM).

MEASUREMENTS

Blood pressure measurements were taken at 7 time points and change scores, area under the curve (AUC) scores were calculated. Data were analyzed by repeated measures ANOVA as well as Pearson correlation.

RESULTS

Both meals produced a sustained drop in systolic (SBP) and diastolic (DBP) blood pressure, with 37.5% of participants meeting criteria for PPH. Participants carrying the E4+ risk gene experienced a larger decrease in SBP than E4- participants, and this was significantly different after the HFM (E4+ AUC = -30.8 ± 7.6, E4- AUC = -0.2 ± 8.7, p=0.015). Increasing age was associated with a larger drop in postprandial blood pressure but only for the E4+ group after the HFM (p=0.002).

CONCLUSIONS

These data suggest that E4+ individuals experience a greater postprandial blood pressure response particularly following high fat feeding, and this effect becomes more pronounced with age. The prevalence of PPH may play a role in the development of AD and may be mediated by diet.

Learning predictive models of drug side-effect relationships from distributed representations of literature-derived semantic predications

Objective

The aim of this work is to leverage relational information extracted from biomedical literature using a novel synthesis of unsupervised pretraining, representational composition, and supervised machine learning for drug safety monitoring.

Methods

Using ≈80 million concept-relationship-concept triples extracted from the literature using the SemRep Natural Language Processing system, distributed vector representations (embeddings) were generated for concepts as functions of their relationships utilizing two unsupervised representational approaches. Embeddings for drugs and side effects of interest from two widely used reference standards were then composed to generate embeddings of drug/side-effect pairs, which were used as input for supervised machine learning. This methodology was developed and evaluated using cross-validation strategies and compared to contemporary approaches. To qualitatively assess generalization, models trained on the Observational Medical Outcomes Partnership (OMOP) drug/side-effect reference set were evaluated against a list of ≈1100 drugs from an online database.

Results

The employed method improved performance over previous approaches. Cross-validation results advance the state of the art (AUC 0.96; F1 0.90 and AUC 0.95; F1 0.84 across the two sets), outperforming methods utilizing literature and/or spontaneous reporting system data. Examination of predictions for unseen drug/side-effect pairs indicates the ability of these methods to generalize, with over tenfold label support enrichment in the top 100 predictions versus the bottom 100 predictions.

Discussion and Conclusion

Our methods can assist the pharmacovigilance process using information from the biomedical literature. Unsupervised pretraining generates a rich relationship-based representational foundation for machine learning techniques to classify drugs in the context of a putative side effect, given known examples.

Classification-by-Analogy: Using Vector Representations of Implicit Relationships to Identify Plausibly Causal Drug/Side-effect Relationships

An important aspect of post-marketing drug surveillance involves identifying potential side-effects utilizing adverse drug event (ADE) reporting systems and/or Electronic Health Records. These data are noisy, necessitating identified drug/ADE associations be manually reviewed - a human-intensive process that scales poorly with large numbers of possibly dangerous associations and rapid growth of biomedical literature. Recent work has employed Literature Based Discovery methods that exploit implicit relationships between biomedical entities within the literature to estimate the plausibility of drug/ADE connections. We extend this work by evaluating machine learning classifiers applied to high-dimensional vector representations of relationships extracted from the literature as a means to identify substantiated drug/ADE connections. Using a curated reference standard, we show applying classifiers to such representations improves performance (+≈37%AUC) over previous approaches. These trained systems reproduce outcomes of the manual literature review process used to create the reference standard, but further research is required to establish their generalizability.

Pregnancy after diabetes obesity surgery (PADOS): Qualitative study of pre-pregnancy care

Half of all bariatric surgical procedures are in women of childbearing age. Surgery may improve fertility yet exacerbate nutritional deficiencies, that may be disadvantageous to the fetus. A frequently encountered subgroup of obese women have type 2 diabetes. The health risks, to both mother and child, of diabetes in pregnancy are well described including 4.7× risk of stillbirth and 2× risk of congenital abnormality. What is not clear is whether bariatric surgery mitigates or complicates the health consequences of women with obesity and diabetes in pregnancy. In addition the influence of the type of surgery, the optimal interval between surgery and conception and evidence based preconception recommendations are unknown. This study complements wider research aiming to inform optimal management of this patient population. Obese diabetic women require clear guidance regarding pregnancy planning after surgery. This study will develop an understanding of the barriers and facilitators (psychological, behavioural, attitudinal and nutritional) to achieving effective pre-pregnancy health and care in women with type 2 diabetes who have undergone metabolic surgery. Currently women's perception of fertility issues and risks after bariatric surgery is unknown and thus a qualitative interpretive paradigm was chosen. Interviews with the target population will explore decision-making processes; experience regarding metabolic surgery and perceived pregnancy risk. Interviews with a broad range of health professionals involved in bariatric care will include rationale for selected surgical procedure and post surgery referral processes e.g. contraceptive care. This will advance understanding of how to provide targeted support and monitoring.

Pregnancy after diabetes obesity surgery (PADOS): Incidence and outcomes

Half of all bariatric surgical procedures are in women of childbearing age but it remains unclear whether surgery is suitable for women who subsequently conceive: specifically the relative risks and benefits of potential nutrient deficiencies versus weight reduction. We will present data collected from Clinical Practice Research Databases on the maternal and fetal outcomes of pregnancies complicated either by obesity or previous bariatric surgery (BS). Two groups, matched to obese controls for BMI pre-BS and post-BS (at the time of ante-natal booking) will be compared. In this way, the effect of BS on pregnancy outcomes may be examined, independent of its effect on weight. A sub-group of women with antecedent Type 2 diabetes (T2DM) will allow for investigation of the additional impact and persistence of this co-morbidity. This builds upon pilot data collected from a retrospective cohort of women (18-45years) undergoing laparoscopic roux-en-Y (RYGB) surgery over a 24-month period (n=218). After exclusions and loss to follow up, data from 111 patients were analysed; 81 (73%) had conceived prior to RYGB, 20 (18%) became pregnant post RYGB and a further 22 patients (20%) were trying to conceive at the time of data collection. Three women had T2DM which resolved post BS. A suggestion of greater miscarriage risk prior to surgery in this sub-group will be confirmed as more women are recruited. Pregnancy is a frequent desire/occurrence after BS. This database study will advance understanding of the maternal and fetal outcomes of such pregnancies and inform antenatal care.

Graph-based signal integration for high-throughput phenotyping

BACKGROUND

Electronic Health Records aggregated in Clinical Data Warehouses (CDWs) promise to revolutionize Comparative Effectiveness Research and suggest new avenues of research. However, the effectiveness of CDWs is diminished by the lack of properly labeled data. We present a novel approach that integrates knowledge from the CDW, the biomedical literature, and the Unified Medical Language System (UMLS) to perform high-throughput phenotyping. In this paper, we automatically construct a graphical knowledge model and then use it to phenotype breast cancer patients. We compare the performance of this approach to using MetaMap when labeling records.

RESULTS

MetaMap's overall accuracy at identifying breast cancer patients was 51.1% (n=428); recall=85.4%, precision=26.2%, and F1=40.1%. Our unsupervised graph-based high-throughput phenotyping had accuracy of 84.1%; recall=46.3%, precision=61.2%, and F1=52.8%.

CONCLUSIONS

We conclude that our approach is a promising alternative for unsupervised high-throughput phenotyping.

Model averaging strategies for structure learning in Bayesian networks with limited data

BACKGROUND

Considerable progress has been made on algorithms for learning the structure of Bayesian networks from data. Model averaging by using bootstrap replicates with feature selection by thresholding is a widely used solution for learning features with high confidence. Yet, in the context of limited data many questions remain unanswered. What scoring functions are most effective for model averaging? Does the bias arising from the discreteness of the bootstrap significantly affect learning performance? Is it better to pick the single best network or to average multiple networks learnt from each bootstrap resample? How should thresholds for learning statistically significant features be selected?

RESULTS

The best scoring functions are Dirichlet Prior Scoring Metric with small λ and the Bayesian Dirichlet metric. Correcting the bias arising from the discreteness of the bootstrap worsens learning performance. It is better to pick the single best network learnt from each bootstrap resample. We describe a permutation based method for determining significance thresholds for feature selection in bagged models. We show that in contexts with limited data, Bayesian bagging using the Dirichlet Prior Scoring Metric (DPSM) is the most effective learning strategy, and that modifying the scoring function to penalize complex networks hampers model averaging. We establish these results using a systematic study of two well-known benchmarks, specifically ALARM and INSURANCE. We also apply our network construction method to gene expression data from the Cancer Genome Atlas Glioblastoma multiforme dataset and show that survival is related to clinical covariates age and gender and clusters for interferon induced genes and growth inhibition genes.

CONCLUSIONS

For small data sets, our approach performs significantly better than previously published methods.

Modeling the role of basal ganglia in saccade generation: is the indirect pathway the explorer?

We model the role played by the Basal Ganglia (BG) in the generation of voluntary saccadic eye movements. The BG model explicitly represents key nuclei like the striatum (caudate), Substantia Nigra pars reticulata (SNr) and compata (SNc), the Subthalamic Nucleus (STN), the two pallidal nuclei and Superior Colliculus. The model is cast within the Reinforcement Learning (RL) framework, with the dopamine representing the temporal difference error, the striatum serving as the critic, and the indirect pathway playing the role of the explorer. Performance of the model is evaluated on a set of tasks such as feature and conjunction searches, directional selectivity and a successive saccade task. Behavioral phenomena such as independence of search time on number of distractors in feature search and linear increase in search time with number of distractors in conjunction search are observed. It is also seen that saccadic reaction times are longer and search efficiency is impaired on diminished BG contribution, which corroborates with reported data obtained from Parkinson's Disease (PD) patients.

New predictive models of heart failure mortality using time-series measurements and ensemble models

BACKGROUND

Morbidity and mortality rates associated with heart failure remain high. A wide variety of demographic and clinical factors as well as biomarkers are associated with increased mortality rates. Despite this, most multivariate predictive models for heart failure mortality have predictive accuracies characterized by a C-statistic (area under the receiver operating curve) of ≈0.74.

METHODS AND RESULTS

We analyzed data on 963 patients enrolled in the Vesnarinone Evaluation of Survival Trial (VEST), including circulating levels of 2 cytokines (tumor necrosis factor and interleukin-6) and their receptors sampled at baseline and at 8, 16, and 24 weeks. We built multivariate logistic regression models by using standard clinical variables and time-series of cytokine and cytokine receptor levels, using independent components analysis to handle collinearity among cytokine measurements, and L2-penalized stepwise regression for variable selection. We also built ensemble models with these data, using gentle boosting. Our multivariate logistic regression model using time-series cytokine measurements predicts 1-year mortality rates significantly better (P=0.001) than the baseline model, with a C-statistic of 0.81±0.03. Without the cytokines, the baseline model has a C-statistic of 0.73±0.03, and, with only baseline cytokine and cytokine receptor levels added, the model has a C-statistic of 0.74±0.04. An ensemble model of 100 decision stumps with serial cytokine measurements has a significantly better (P=0.04) C-statistic of 0.84±0.02. An ensemble model with baseline cytokine data and without the serial measurements has a C-statistic of 0.74±0.04.

CONCLUSIONS

Significant gains in accuracy of one year mortality prediction in chronic heart failure can be obtained by using logistic regression models that incorporate serial measurements of biomarkers such as cytokine and cytokine receptor levels. Ensemble models capture inherent variability in large patient populations, and boost predictive accuracy through the use of time-series measurements.

Persistence and adherence in the treatment of overactive bladder syndrome with anticholinergic therapy: a systematic review of the literature

Overactive bladder syndrome (OAB) is a chronic condition that has an impact on patients' daily activities and health-related quality of life (HRQL). Anticholinergic therapy is often prescribed following insufficient results with behaviour modification alone; however, rates of treatment discontinuation are consistently high. This study systematically reviewed persistence and adherence data in patients with OAB treated with anticholinergic therapy. A search focused on the intersection of OAB, persistence/adherence, and anticholinergic therapy was conducted in MEDLINE and EMBASE. Articles published after 1998 were reviewed and selected for inclusion based on prespecified criteria. A total of 147 articles and two abstracts were included in the review. Results from 12-week clinical trials showed high rates of discontinuation, ranging from 4% to 31% and 5% to 20% in treatment and placebo groups, respectively. Unsurprisingly, rates of discontinuation found in medical claims studies were substantially higher, with 43% to 83% of patients discontinuing medication within the first 30 days and rates continuing to rise over time. Findings from medical claims studies also suggest that over half of patients never refill their initial prescription and that adherence levels tend to be low, with mean/median medication possession ratio (MPR) values ranging from 0.30 to 0.83. The low levels of persistence and adherence documented in this review reveal cause for concern about the balance between the efficacy and tolerability of anticholinergic agents. Strategies should be identified to increase persistence and adherence. New agents and non-pharmacologic alternatives with good efficacy and minimal side effects should be explored.

MEDRank: using graph-based concept ranking to index biomedical texts

BACKGROUND

As the volume of biomedical text increases exponentially, automatic indexing becomes increasingly important. However, existing approaches do not distinguish central (or core) concepts from concepts that were mentioned in passing. We focus on the problem of indexing MEDLINE records, a process that is currently performed by highly trained humans at the National Library of Medicine (NLM). NLM indexers are assisted by a system called the Medical Text Indexer (MTI) that suggests candidate indexing terms.

OBJECTIVE

To improve the ability of MTI to select the core terms in MEDLINE abstracts. These core concepts are deemed to be most important and are designated as "major headings" by MEDLINE indexers. We introduce and evaluate a graph-based indexing methodology called MEDRank that generates concept graphs from biomedical text and then ranks the concepts within these graphs to identify the most important ones.

METHODS

We insert a MEDRank step into the MTI and compare MTI's output with and without MEDRank to the MEDLINE indexers' selected terms for a sample of 11,803 PubMed Central articles. We also tested whether human raters prefer terms generated by the MEDLINE indexers, MTI without MEDRank, and MTI with MEDRank for a sample of 36 PubMed Central articles.

RESULTS

MEDRank improved recall of major headings designated by 30% over MTI without MEDRank (0.489 vs. 0.376). Overall recall was only slightly (6.5%) higher (0.490 vs. 0.460) as was F(2) (3%, 0.408 vs. 0.396). However, overall precision was 3.9% lower (0.268 vs. 0.279). Human raters preferred terms generated by MTI with MEDRank over terms generated by MTI without MEDRank (by an average of 1.00 more term per article), and preferred terms generated by MTI with MEDRank and the MEDLINE indexers at the same rate.

CONCLUSIONS

The addition of MEDRank to MTI significantly improved the retrieval of core concepts in MEDLINE abstracts and more closely matched human expectations compared to MTI without MEDRank. In addition, MEDRank slightly improved overall recall and F(2).

Modeling basal ganglia for understanding Parkinsonian reaching movements

We present a computational model that highlights the role of basal ganglia (BG) in generating simple reaching movements. The model is cast within the reinforcement learning (RL) framework with correspondence between RL components and neuroanatomy as follows: dopamine signal of substantia nigra pars compacta as the temporal difference error, striatum as the substrate for the critic, and the motor cortex as the actor. A key feature of this neurobiological interpretation is our hypothesis that the indirect pathway is the explorer. Chaotic activity, originating from the indirect pathway part of the model, drives the wandering, exploratory movements of the arm. Thus, the direct pathway subserves exploitation, while the indirect pathway subserves exploration. The motor cortex becomes more and more independent of the corrective influence of BG as training progresses. Reaching trajectories show diminishing variability with training. Reaching movements associated with Parkinson's disease (PD) are simulated by reducing dopamine and degrading the complexity of indirect pathway dynamics by switching it from chaotic to periodic behavior. Under the simulated PD conditions, the arm exhibits PD motor symptoms like tremor, bradykinesia and undershooting. The model echoes the notion that PD is a dynamical disease.

AI Theory and Practice: A Discussion on Hard Challenges and Opportunities Ahead

The Microsoft Research Faculty Summit brought together eight experts in different areas of AI to share their thoughts about the key challenges ahead in theory and/or practice in the broad constellation of artificial intelligence.  This article summarizes their conversation.

New components of the Dictyostelium PKA pathway revealed by Bayesian analysis of expression data

Background

Identifying candidate genes in genetic networks is important for understanding regulation and biological function. Large gene expression datasets contain relevant information about genetic networks, but mining the data is not a trivial task. Algorithms that infer Bayesian networks from expression data are powerful tools for learning complex genetic networks, since they can incorporate prior knowledge and uncover higher-order dependencies among genes. However, these algorithms are computationally demanding, so novel techniques that allow targeted exploration for discovering new members of known pathways are essential.

Results

Here we describe a Bayesian network approach that addresses a specific network within a large dataset to discover new components. Our algorithm draws individual genes from a large gene-expression repository, and ranks them as potential members of a known pathway. We apply this method to discover new components of the cAMP-dependent protein kinase (PKA) pathway, a central regulator of Dictyostelium discoideum development. The PKA network is well studied in D. discoideum but the transcriptional networks that regulate PKA activity and the transcriptional outcomes of PKA function are largely unknown. Most of the genes highly ranked by our method encode either known components of the PKA pathway or are good candidates. We tested 5 uncharacterized highly ranked genes by creating mutant strains and identified a candidate cAMP-response element-binding protein, yet undiscovered in D. discoideum, and a histidine kinase, a candidate upstream regulator of PKA activity.

Conclusions

The single-gene expansion method is useful in identifying new components of known pathways. The method takes advantage of the Bayesian framework to incorporate prior biological knowledge and discovers higher-order dependencies among genes while greatly reducing the computational resources required to process high-throughput datasets.

Who evacuates when hurricanes approach? The role of risk, information, and location

OBJECTIVE

This article offers an expanded perspective on evacuation decision making during severe weather. In particular, this work focuses on uncovering determinants of individual evacuation decisions.

METHODS

We draw on a survey conducted in 2005 of residents in the eight-county Houston metropolitan area after Hurricane Rita made landfall on September 24, 2005.

RESULTS

We find that evacuation decisions are influenced by a heterogeneous set of parameters, including perceived risk from wind, influence of media and neighbors, and awareness of evacuation zone, that are often at variance with one of the primary measures of risk used by public officials to order or recommend an evacuation (i.e., storm surge). We further find that perceived risk and its influence on evacuation behavior is a local phenomenon more readily communicated by and among individuals who share the same geography, as is the case with residents living inside and outside official risk areas.

CONCLUSIONS

Who evacuates and why is partially dependent on where one lives because perceptions of risk are not uniformly shared across the area threatened by an approaching hurricane and the same sources and content of information do not have the same effect on evacuation behavior. Hence, efforts to persuade residential populations about risk and when, where, and how to evacuate or shelter in place should originate in the neighborhood rather than emanating from blanket statements from the media or public officials. Our findings also raise important policy questions (included in the discussion section) that require further study and consideration by those responsible with organizing and implementing evacuation plans.

Learning robust cell signalling models from high throughput proteomic data

We propose a framework for learning robust Bayesian network models of cell signalling from high-throughput proteomic data. We show that model averaging using Bayesian bootstrap resampling generates more robust structures than procedures that learn structures using all of the data. We also develop an algorithm for ranking the importance of network features using bootstrap resample data. We apply our algorithms to derive the T-cell signalling network from the flow cytometry data of Sachs et al. (2005). Our learning algorithm has identified, with high confidence, several new crosstalk mechanisms in the T-cell signalling network. Many of them have already been confirmed experimentally in the recent literature and six new crosstalk mechanisms await experimental validation.

Predicting altered pathways using extendable scaffolds

Many diseases, especially solid tumors, involve the disruption or deregulation of cellular processes. Most current work using gene expression and other high-throughput data, simply list a set of differentially expressed genes. We propose a new method, PAPES (predicting altered pathways using extendable scaffolds), to computationally reverse-engineer models of biological systems. We use sets of genes that occur in a known biological pathway to construct component process models. We then compose these models to build larger scale networks that capture interactions among pathways. We show that we can learn process modifications in two coupled metabolic pathways in prostate cancer cells.

Rapid spread of penicillin-resistant Streptococcus pneumoniae among high-risk hospital inpatients and the role of molecular typing in outbreak confirmation

This study describes an outbreak of penicillin-resistant Streptococcus pneumoniae among patients on an ear, nose and throat (ENT) ward. Over a period of 10 days, S. pneumoniae [penicillin minimum inhibitory concentration (MIC) 0.75] was isolated from a total of seven patients with symptoms and signs of lower respiratory tract infection. Standard source isolation was implemented and the ward was closed to admissions and discharges. Screening of nasopharyngeal secretions was undertaken on all patients and staff contacts. Three patients (of eight possible contacts) and none of the staff (47 screened) were identified as colonized with the same strain. Nasal mupirocin and oral rifampicin were administered to carriers. Confirmation of the outbreak was achieved rapidly using a contemporary molecular typing method (pulsed-field gel electrophoresis) and timely introduction of infection control measures successfully contained the outbreak. Implications for pneumococcal vaccination are discussed.

Analysis of the binding of p53 to DNAs containing mismatched and bulged bases

The tumor suppressor protein p53 modulates cellular response to DNA damage by a variety of mechanisms that may include direct recognition of some forms of primary DNA damage. Linear 49-base pair duplex DNAs were constructed containing all possible single-base mismatches as well as a 3-cytosine bulge. Filter binding and gel retardation assays revealed that the affinity of p53 for a number of these lesions was equal to or greater than that of the human mismatch repair complex, hMSH2-hMSH6, under the same binding conditions. However, other mismatches including G/T, which is bound strongly by hMSH2-hMSH6, were poorly recognized by p53. The general order of affinity of p53 was greatest for a 3-cytosine bulge followed by A/G and C/C mismatches, then C/T and G/T mismatches, and finally all the other mismatches.

Mechanism of topology simplification by type II DNA topoisomerases

Type II DNA topoisomerases actively reduce the fractions of knotted and catenated circular DNA below thermodynamic equilibrium values. To explain this surprising finding, we designed a model in which topoisomerases introduce a sharp bend in DNA. Because the enzymes have a specific orientation relative to the bend, they act like Maxwell's demon, providing unidirectional strand passage. Quantitative analysis of the model by computer simulations proved that it can explain much of the experimental data. The required sharp DNA bend was demonstrated by a greatly increased cyclization of short DNA fragments from topoisomerase binding and by direct visualization with electron microscopy.

Statistical methods for the objective design of screening procedures for macromolecular crystallization

The crystallization of a new macromolecule is still very much a trial-and-error process. As is well known, it requires the search of a large parameter space of experimental settings to find the relatively few idiosyncratic conditions that lead to diffraction-quality crystals. Crystallographers have developed a variety of screens to help identify initial crystallization conditions, including those based on systematic grids, incomplete factorial and sparse-matrix approaches. These are somewhat subjectively formulated based on accumulated data from past crystallization experiments. Ideally, one would prefer as objective a procedure as possible; however, that requires objective methods that incorporate a broad source of crystallization data. The Biological Macromolecular Crystallization Database (BMCD), a repository of all published crystallization conditions, is an obvious source of this data. This database has been augmented with a hierarchical classification of the macromolecules contained in the BMCD as well as extensive data on the additives used with them. A statistical analysis of the augmented BMCD shows the existence of significant correlations between families of macromolecules and the experimental conditions under which they crystallize. This in turn leads to a Bayesian technique for determining the probability of success of a set of experimental conditions based on the data in the BMCD as well as facts about a macromolecule known prior to crystallization. This has been incorporated into software that enables users to rank experimental conditions for new macromolecules generated by a dense partial factorial design. Finally, an additional advantage of the software described here is that it also facilitates the accumulation of the data required for improving the accuracy of estimation of the probabilities of success - knowledge of the conditions which lead to failure of crystallization.

Optimizing for reduced code space using genetic algorithms

Code space is a critical issue facing designers of software for embedded systems. Many traditional compiler optimizations are designed to reduce the execution time of compiled code, but not necessarily the size of the compiled code. Further, different results can be achieved by running some optimizations more than once and changing the order in which optimizations are applied. Register allocation only complicates matters, as the interactions between different optimizations can cause more spill code to be generated. The compiler for embedded systems, then, must take care to use the best sequence of optimizations to minimize code space.Since much of the code for embedded systems is compiled once and then burned into ROM, the software designer will often tolerate much longer compile times in the hope of reducing the size of the compiled code. We take advantage of this by using a genetic algorithm to find optimization sequences that generate small object codes. The solutions generated by this algorithm are compared to solutions found using a fixed optimization sequence and solutions found by testing random optimization sequences. Based on the results found by the genetic algorithm, a new fixed sequence is developed to reduce code size. Finally, we explore the idea of using different optimization sequences for different modules and functions of the same program.