The center for causal discovery of biomedical knowledge from big data

Challenges in defining thresholds for health effects: some considerations for asbestos and silica

This paper summarizes several presentations in the Thresholds in Epidemiology and Risk Assessment session at the Monticello III conference. These presentations described evidence regarding thresholds for particles, including asbestos and silica, and cancer (e.g., mesothelioma) and noncancer (e.g., silicosis) endpoints. In the case of exposure to various types of particles and malignancy, it is clear that even though a linear non-threshold model has often been assumed, experimental and theoretical support for thresholds exist (e.g., through particle clearance, repair mechanisms, and various other aspects of the carcinogenic process). For mesothelioma and exposure to elongate mineral particles (EMPs), there remains controversy concerning the epidemiological demonstration of thresholds. However, using data from the Québec mining cohort studies, it was shown that a "practical" threshold exists for chrysotile exposure and mesothelioma. It was also noted that, in such evaluations, measurement error in diagnosis and exposure assessment needs to be incorporated into risk analyses. Researchers were also encouraged to use biobanks that collect specimens and data on mesothelioma to more precisely define cases of mesothelioma and possible variants for cases of all ages, and trends that may help define background rates and distinguish those mesotheliomas related to EMP exposures from those that are not, as well as other factors that support or define thresholds. New statistical approaches have been developed for identifying and quantifying exposure thresholds, an example of which is described for respirable crystalline silica (RCS) exposure and silicosis risk. Finally, the application of Artificial Intelligence (AI) to considering the multiple factors influencing risk and thresholds may prove useful.

Graphical modeling of causal factors associated with the postoperative survival of esophageal cancer subjects

PURPOSE

To clarify the causal relationship between factors contributing to the postoperative survival of patients with esophageal cancer.

METHODS

A cohort of 195 patients who underwent surgery for esophageal cancer between 2008 and 2021 was used in the study. All patients had preoperative chest computed tomography (CT) and positron emission tomography-CT (PET-CT) scans prior to receiving any treatment. From these images, high throughput and quantitative radiomic features, tumor features, and various body composition features were automatically extracted. Causal relationships among these image features, patient demographics, and other clinicopathological variables were analyzed and visualized using a novel score-based directed graph called "Grouped Greedy Equivalence Search" (GGES) while taking prior knowledge into consideration. After supplementing and screening the causal variables, the intervention do-calculus adjustment (IDA) scores were calculated to determine the degree of impact of each variable on survival. Based on this IDA score, a GGES prediction formula was generated. Ten-fold cross-validation was used to assess the performance of the models. The prediction results were evaluated using the R-Squared Score (R2 score).

RESULTS

The final causal graphical model was formed by two PET-based image variables, ten body composition variables, four pathological variables, four demographic variables, two tumor variables, and one radiological variable (Percentile 10). Intramuscular fat mass was found to have the most impact on overall survival month. Percentile 10 and overall TNM (T: tumor, N: nodes, M: metastasis) stage were identified as direct causes of overall survival (month). The GGES casual model outperformed GES in regression prediction (R2  = 0.251) (p < 0.05) and was able to avoid unreasonable causality that may contradict common sense.

CONCLUSION

The GGES causal model can provide a reliable and straightforward representation of the intricate causal relationships among the variables that impact the postoperative survival of patients with esophageal cancer.

Evaluating Large Language Models in Extracting Cognitive Exam Dates and Scores

Importance

Large language models (LLMs) are crucial for medical tasks. Ensuring their reliability is vital to avoid false results. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR.

Objective

Evaluate ChatGPT and LlaMA-2 performance in extracting MMSE and CDR scores, including their associated dates.

Methods

Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation.

Results

For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date.

Conclusions

In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

A Prospective Cohort Protocol for the Remnant Investigation in Sepsis Study

BACKGROUND

Sepsis is a common and deadly syndrome, accounting for more than 11 million deaths annually. To mature a deeper understanding of the host and pathogen mechanisms contributing to poor outcomes in sepsis, and thereby possibly inform new therapeutic targets, sophisticated, and expensive biorepositories are typically required. We propose that remnant biospecimens are an alternative for mechanistic sepsis research, although the viability and scientific value of such remnants are unknown.

METHODS AND RESULTS

The Remnant Biospecimen Investigation in Sepsis study is a prospective cohort study of 225 adults (age ≥ 18 yr) presenting to the emergency department with community sepsis, defined as sepsis-3 criteria within 6 hours of arrival. The primary objective was to determine the scientific value of a remnant biospecimen repository in sepsis linked to clinical phenotyping in the electronic health record. We will study candidate multiomic readouts of sepsis biology, governed by a conceptual model, and determine the precision, accuracy, integrity, and comparability of proteins, small molecules, lipids, and pathogen sequencing in remnant biospecimens compared with paired biospecimens obtained according to research protocols. Paired biospecimens will include plasma from sodium-heparin, EDTA, sodium fluoride, and citrate tubes.

CONCLUSIONS

The study has received approval from the University of Pittsburgh Human Research Protection Office (Study 21120013). Recruitment began on October 25, 2022, with planned release of primary results anticipated in 2024. Results will be made available to the public, the funders, critical care societies, laboratory medicine scientists, and other researchers.

A Graph-Based Approach to Identify Factors Contributing to Postoperative Lung Cancer Recurrence among Patients with Non-Small-Cell Lung Cancer

The accurate identification of the preoperative factors impacting postoperative cancer recurrence is crucial for optimizing neoadjuvant and adjuvant therapies and guiding follow-up treatment plans. We modeled the causal relationship between radiographical features derived from CT scans and the clinicopathologic factors associated with postoperative lung cancer recurrence and recurrence-free survival. A retrospective cohort of 363 non-small-cell lung cancer (NSCLC) patients who underwent lung resections with a minimum 5-year follow-up was analyzed. Body composition tissues and tumor features were quantified based on preoperative whole-body CT scans (acquired as a component of PET-CT scans) and chest CT scans, respectively. A novel causal graphical model was used to visualize the causal relationship between these factors. Variables were assessed using the intervention do-calculus adjustment (IDA) score. Direct predictors for recurrence-free survival included smoking history, T-stage, height, and intramuscular fat mass. Subcutaneous fat mass, visceral fat volume, and bone mass exerted the greatest influence on the model. For recurrence, the most significant variables were visceral fat volume, subcutaneous fat volume, and bone mass. Pathologic variables contributed to the recurrence model, with bone mass, TNM stage, and weight being the most important. Body composition, particularly adipose tissue distribution, significantly and causally impacted both recurrence and recurrence-free survival through interconnected relationships with other variables.

Advances in Conceptual and Methodological Issues in Symptom Cluster Research: A 20-Year Perspective

Two conceptual approaches are used to evaluate symptom clusters: "clustering" symptoms (ie, variable-centered analytic approach) and "clustering" patients (ie, person-centered analytic approach). However, these methods are not used consistently and conceptual clarity is needed. Given the emergence of novel methods to evaluate symptom clusters, a review of the conceptual basis for older and newer analytic methods is warranted. Therefore, this article will review the conceptual basis for symptom cluster research; compare and contrast the conceptual basis for using variable-centered versus patient-centered analytic approaches in symptom cluster research; review their strengths and weaknesses; and compare their applications in symptom cluster research.

A mechanistic framework for cardiometabolic and coronary artery diseases

Coronary atherosclerosis results from the delicate interplay of genetic and exogenous risk factors, principally taking place in metabolic organs and the arterial wall. Here we show that 224 gene-regulatory coexpression networks (GRNs) identified by integrating genetic and clinical data from patients with (n = 600) and without (n = 250) coronary artery disease (CAD) with RNA-seq data from seven disease-relevant tissues in the Stockholm-Tartu Atherosclerosis Reverse Network Engineering Task (STARNET) study largely capture this delicate interplay, explaining >54% of CAD heritability. Within 89 cross-tissue GRNs associated with clinical severity of CAD, 374 endocrine factors facilitated inter-organ interactions, primarily along an axis from adipose tissue to the liver (n = 152). This axis was independently replicated in genetically diverse mouse strains and by injection of recombinant forms of adipose endocrine factors (EPDR1, FCN2, FSTL3 and LBP) that markedly altered blood lipid and glucose levels in mice. Altogether, the STARNET database and the associated GRN browser (http://starnet.mssm.edu) provide a multiorgan framework for exploration of the molecular interplay between cardiometabolic disorders and CAD.

Big Data to Knowledge: Application of Machine Learning to Predictive Modeling of Therapeutic Response in Cancer

Background

In recent years, the availability of high throughput technologies, establishment of large molecular patient data repositories, and advancement in computing power and storage have allowed elucidation of complex mechanisms implicated in therapeutic response in cancer patients. The breadth and depth of such data, alongside experimental noise and missing values, requires a sophisticated human-machine interaction that would allow effective learning from complex data and accurate forecasting of future outcomes, ideally embedded in the core of machine learning design.

Objective

In this review, we will discuss machine learning techniques utilized for modeling of treatment response in cancer, including Random Forests, support vector machines, neural networks, and linear and logistic regression. We will overview their mathematical foundations and discuss their limitations and alternative approaches in light of their application to therapeutic response modeling in cancer.

Conclusion

We hypothesize that the increase in the number of patient profiles and potential temporal monitoring of patient data will define even more complex techniques, such as deep learning and causal analysis, as central players in therapeutic response modeling.

Artificial Intelligence in Clinical Health Care Applications: Viewpoint

The idea of artificial intelligence (AI) has a long history. It turned out, however, that reaching intelligence at human levels is more complicated than originally anticipated. Currently, we are experiencing a renewed interest in AI, fueled by an enormous increase in computing power and an even larger increase in data, in combination with improved AI technologies like deep learning. Healthcare is considered the next domain to be revolutionized by artificial intelligence. While AI approaches are excellently suited to develop certain algorithms, for biomedical applications there are specific challenges. We propose six recommendations—the 6Rs—to improve AI projects in the biomedical space, especially clinical health care, and to facilitate communication between AI scientists and medical doctors: (1) Relevant and well-defined clinical question first; (2) Right data (ie, representative and of good quality); (3) Ratio between number of patients and their variables should fit the AI method; (4) Relationship between data and ground truth should be as direct and causal as possible; (5) Regulatory ready; enabling validation; and (6) Right AI method.

Causality on longitudinal data: Stable specification search in constrained structural equation modeling

A typical problem in causal modeling is the instability of model structure learning, i.e., small changes in finite data can result in completely different optimal models. The present work introduces a novel causal modeling algorithm for longitudinal data, that is robust for finite samples based on recent advances in stability selection using subsampling and selection algorithms. Our approach uses exploratory search but allows incorporation of prior knowledge, e.g., the absence of a particular causal relationship between two specific variables. We represent causal relationships using structural equation models. Models are scored along two objectives: the model fit and the model complexity. Since both objectives are often conflicting, we apply a multi-objective evolutionary algorithm to search for Pareto optimal models. To handle the instability of small finite data samples, we repeatedly subsample the data and select those substructures (from the optimal models) that are both stable and parsimonious. These substructures can be visualized through a causal graph. Our more exploratory approach achieves at least comparable performance as, but often a significant improvement over state-of-the-art alternative approaches on a simulated data set with a known ground truth. We also present the results of our method on three real-world longitudinal data sets on chronic fatigue syndrome, Alzheimer disease, and chronic kidney disease. The findings obtained with our approach are generally in line with results from more hypothesis-driven analyses in earlier studies and suggest some novel relationships that deserve further research.

Discovering hidden knowledge through auditing clinical diagnostic knowledge bases

OBJECTIVE

Evaluate potential for data mining auditing techniques to identify hidden concepts in diagnostic knowledge bases (KB). Improving completeness enhances KB applications such as differential diagnosis and patient case simulation.

MATERIALS AND METHODS

Authors used unsupervised (Pearson's correlation - PC, Kendall's correlation - KC, and a heuristic algorithm - HA) methods to identify existing and discover new finding-finding interrelationships ("properties") in the INTERNIST-1/QMR KB. Authors estimated KB maintenance efficiency gains (effort reduction) of the approaches.

RESULTS

The methods discovered new properties at 95% CI rates of [0.1%, 5.4%] (PC), [2.8%, 12.5%] (KC), and [5.6%, 18.8%] (HA). Estimated manual effort reduction for HA-assisted determination of new properties was approximately 50-fold.

CONCLUSION

Data mining can provide an efficient supplement to ensuring the completeness of finding-finding interdependencies in diagnostic knowledge bases. Authors' findings should be applicable to other diagnostic systems that record finding frequencies within diseases (e.g., DXplain, ISABEL).

Integration of pan-cancer transcriptomics with RPPA proteomics reveals mechanisms of epithelial-mesenchymal transition

Integrating data from multiple regulatory layers across cancer types could elucidate additional mechanisms of oncogenesis. Using antibody-based protein profiling of 736 cancer cell lines, along with matching transcriptomic data, we show that pan-cancer bimodality in the amounts of mRNA, protein, and protein phosphorylation reveals mechanisms related to the epithelial-mesenchymal transition (EMT). Based on the bimodal expression of E-cadherin, we define an EMT signature consisting of 239 genes, many of which were not previously associated with EMT. By querying gene expression signatures collected from cancer cell lines after small-molecule perturbations, we identify enrichment for histone deacetylase (HDAC) inhibitors as inducers of EMT, and kinase inhibitors as mesenchymal-to-epithelial transition (MET) promoters. Causal modeling of protein-based signaling identifies putative drivers of EMT. In conclusion, integrative analysis of pan-cancer proteomic and transcriptomic data reveals key regulatory mechanisms of oncogenic transformation.

Profiling molecular and phenotypic characteristics of large collections of cancer cell lines can be used to identify distinct and common oncogenic pathways across cancer types. So far, most large-scale data obtained from cancer cell lines have been at the genomic, transcriptomic, and phenotypic levels. Recently, high-quality data at the level of cell signaling through protein abundances and phosphorylation sites has become available. By integrating this newly generated protein data with prior transcriptomic data, and by visualizing all cancer cell lines using dimensionality reduction techniques, pan-cancer cell lines are strikingly shown to organize into a gradient of epithelial to mesenchymal types. Interestingly, many of the measured proteins and transcripts display bimodality; the expression of genes, proteins, and protein phosphorylations is either high or low, strongly suggesting that they act as molecular switches. Focusing on further characterizing molecular switches of epithelial-mesenchymal transitions, we identify candidate regulators and small molecules that can induce or reverse such transition, as well as potential causal relationships between proteins. Since the mesenchymal state of tumors is known to be associated with metastasis and later-stage cancer development, better understanding the regulatory mechanisms of epithelial-to-mesenchymal transition can lead to improved targeted therapeutics.

Envisioning the future of 'big data' biomedicine

Through the increasing availability of more efficient data collection procedures, biomedical scientists are now confronting ever larger sets of data, often finding themselves struggling to process and interpret what they have gathered. This, while still more data continues to accumulate. This torrent of biomedical information necessitates creative thinking about how the data are being generated, how they might be best managed, analyzed, and eventually how they can be transformed into further scientific understanding for improving patient care. Recognizing this as a major challenge, the National Institutes of Health (NIH) has spearheaded the "Big Data to Knowledge" (BD2K) program - the agency's most ambitious biomedical informatics effort ever undertaken to date. In this commentary, we describe how the NIH has taken on "big data" science head-on, how a consortium of leading research centers are developing the means for handling large-scale data, and how such activities are being marshalled for the training of a new generation of biomedical data scientists. All in all, the NIH BD2K program seeks to position data science at the heart of 21st Century biomedical research.

2016 Year-in-Review of Clinical and Consumer Informatics: Analysis and Visualization of Keywords and Topics

Objectives

The objective of this study was to review and visualize the medical informatics field over the previous 12 months according to the frequencies of keywords and topics in papers published in the top four journals in the field and in Healthcare Informatics Research (HIR), an official journal of the Korean Society of Medical Informatics.

Methods

A six-person team conducted an extensive review of the literature on clinical and consumer informatics. The literature was searched using keywords employed in the American Medical Informatics Association year-in-review process and organized into 14 topics used in that process. Data were analyzed using word clouds, social network analysis, and association rules.

Results

The literature search yielded 370 references and 1,123 unique keywords. ‘Electronic Health Record’ (EHR) (78.6%) was the most frequently appearing keyword in the articles published in the five studied journals, followed by ‘telemedicine’ (2.1%). EHR (37.6%) was also the most frequently studied topic area, followed by clinical informatics (12.0%). However, ‘telemedicine’ (17.0%) was the most frequently appearing keyword in articles published in HIR, followed by ‘telecommunications’ (4.5%). Telemedicine (47.1%) was the most frequently studied topic area, followed by EHR (14.7%).

Conclusions

The study findings reflect the Korean government's efforts to introduce telemedicine into the Korean healthcare system and reactions to this from the stakeholders associated with telemedicine.

TCGA Expedition: A Data Acquisition and Management System for TCGA Data

Background

The Cancer Genome Atlas Project (TCGA) is a National Cancer Institute effort to profile at least 500 cases of 20 different tumor types using genomic platforms and to make these data, both raw and processed, available to all researchers. TCGA data are currently over 1.2 Petabyte in size and include whole genome sequence (WGS), whole exome sequence, methylation, RNA expression, proteomic, and clinical datasets. Publicly accessible TCGA data are released through public portals, but many challenges exist in navigating and using data obtained from these sites. We developed TCGA Expedition to support the research community focused on computational methods for cancer research. Data obtained, versioned, and archived using TCGA Expedition supports command line access at high-performance computing facilities as well as some functionality with third party tools. For a subset of TCGA data collected at University of Pittsburgh, we also re-associate TCGA data with de-identified data from the electronic health records. Here we describe the software as well as the architecture of our repository, methods for loading of TCGA data to multiple platforms, and security and regulatory controls that conform to federal best practices.

Results

TCGA Expedition software consists of a set of scripts written in Bash, Python and Java that download, extract, harmonize, version and store all TCGA data and metadata. The software generates a versioned, participant- and sample-centered, local TCGA data directory with metadata structures that directly reference the local data files as well as the original data files. The software supports flexible searches of the data via a web portal, user-centric data tracking tools, and data provenance tools. Using this software, we created a collaborative repository, the Pittsburgh Genome Resource Repository (PGRR) that enabled investigators at our institution to work with all TCGA data formats, and to interrogate these data with analysis pipelines, and associated tools. WGS data are especially challenging for individual investigators to use, due to issues with downloading, storage, and processing; having locally accessible WGS BAM files has proven invaluable.

Conclusion

Our open-source, freely available TCGA Expedition software can be used to create a local collaborative infrastructure for acquiring, managing, and analyzing TCGA data and other large public datasets.

A Perspective on Implementing a Quantitative Systems Pharmacology Platform for Drug Discovery and the Advancement of Personalized Medicine

Drug candidates exhibiting well-defined pharmacokinetic and pharmacodynamic profiles that are otherwise safe often fail to demonstrate proof-of-concept in phase II and III trials. Innovation in drug discovery and development has been identified as a critical need for improving the efficiency of drug discovery, especially through collaborations between academia, government agencies, and industry. To address the innovation challenge, we describe a comprehensive, unbiased, integrated, and iterative quantitative systems pharmacology (QSP)-driven drug discovery and development strategy and platform that we have implemented at the University of Pittsburgh Drug Discovery Institute. Intrinsic to QSP is its integrated use of multiscale experimental and computational methods to identify mechanisms of disease progression and to test predicted therapeutic strategies likely to achieve clinical validation for appropriate subpopulations of patients. The QSP platform can address biological heterogeneity and anticipate the evolution of resistance mechanisms, which are major challenges for drug development. The implementation of this platform is dedicated to gaining an understanding of mechanism(s) of disease progression to enable the identification of novel therapeutic strategies as well as repurposing drugs. The QSP platform will help promote the paradigm shift from reactive population-based medicine to proactive personalized medicine by focusing on the patient as the starting and the end point.