Systematic auditing is essential to debiasing machine learning in biology

Evidence-based personalised medicine in critical care: a framework for quantifying and applying individualised treatment effects in patients who are critically ill

Clinicians aim to provide treatments that will result in the best outcome for each patient. Ideally, treatment decisions are based on evidence from randomised clinical trials. Randomised trials conventionally report an aggregated difference in outcomes between patients in each group, known as an average treatment effect. However, the actual effect of treatment on outcomes (treatment response) can vary considerably between individuals, and can differ substantially from the average treatment effect. This variation in response to treatment between patients-heterogeneity of treatment effect-is particularly important in critical care because common critical care syndromes (eg, sepsis and acute respiratory distress syndrome) are clinically and biologically heterogeneous. Statistical approaches have been developed to analyse heterogeneity of treatment effect and predict individualised treatment effects for each patient. In this Review, we outline a framework for deriving and validating individualised treatment effects and identify challenges to applying individualised treatment effect estimates to inform treatment decisions in clinical care.

Bioinformatic Resources for Exploring Human-virus Protein-protein Interactions Based on Binding Modes

Historically, there have been many outbreaks of viral diseases that have continued to claim millions of lives. Research on human-virus protein-protein interactions (PPIs) is vital to understanding the principles of human-virus relationships, providing an essential foundation for developing virus control strategies to combat diseases. The rapidly accumulating data on human-virus PPIs offer unprecedented opportunities for bioinformatics research around human-virus PPIs. However, available detailed analyses and summaries to help use these resources systematically and efficiently are lacking. Here, we comprehensively review the bioinformatic resources used in human-virus PPI research, and discuss and compare their functions, performance, and limitations. This review aims to provide researchers with a bioinformatic toolbox that will hopefully better facilitate the exploration of human-virus PPIs based on binding modes.

Advancing age grading techniques for Glossina morsitans morsitans, vectors of African trypanosomiasis, through mid-infrared spectroscopy and machine learning

Tsetse are the insects responsible for transmitting African trypanosomes, which cause sleeping sickness in humans and animal trypanosomiasis in wildlife and livestock. Knowing the age of these flies is important when assessing the effectiveness of vector control programs and modelling disease risk. Current methods to assess fly age are, however, labour-intensive, slow, and often inaccurate as skilled personnel are in short supply. Mid-infrared spectroscopy (MIRS), a fast and cost-effective tool to accurately estimate several biological traits of insects, offers a promising alternative. This is achieved by characterising the biochemical composition of the insect cuticle using infrared light coupled with machine-learning (ML) algorithms to estimate the traits of interest. We tested the performance of MIRS in estimating tsetse sex and age for the first-time using spectra obtained from their cuticle. We used 541 insectary-reared Glossina m. morsitans of two different age groups for males (5 and 7 weeks) and three age groups for females (3 days, 5 weeks, and 7 weeks). Spectra were collected from the head, thorax, and abdomen of each sample. ML models differentiated between male and female flies with a 96% accuracy and predicted the age group with 94% and 87% accuracy for males and females, respectively. The key infrared regions important for discriminating sex and age classification were characteristic of lipid and protein content. Our results support the use of MIRS as a rapid and accurate way to identify tsetse sex and age with minimal pre-processing. Further validation using wild-caught tsetse could pave the way for this technique to be implemented as a routine surveillance tool in vector control programmes.

The METRIC-framework for assessing data quality for trustworthy AI in medicine: a systematic review

The adoption of machine learning (ML) and, more specifically, deep learning (DL) applications into all major areas of our lives is underway. The development of trustworthy AI is especially important in medicine due to the large implications for patients' lives. While trustworthiness concerns various aspects including ethical, transparency and safety requirements, we focus on the importance of data quality (training/test) in DL. Since data quality dictates the behaviour of ML products, evaluating data quality will play a key part in the regulatory approval of medical ML products. We perform a systematic review following PRISMA guidelines using the databases Web of Science, PubMed and ACM Digital Library. We identify 5408 studies, out of which 120 records fulfil our eligibility criteria. From this literature, we synthesise the existing knowledge on data quality frameworks and combine it with the perspective of ML applications in medicine. As a result, we propose the METRIC-framework, a specialised data quality framework for medical training data comprising 15 awareness dimensions, along which developers of medical ML applications should investigate the content of a dataset. This knowledge helps to reduce biases as a major source of unfairness, increase robustness, facilitate interpretability and thus lays the foundation for trustworthy AI in medicine. The METRIC-framework may serve as a base for systematically assessing training datasets, establishing reference datasets, and designing test datasets which has the potential to accelerate the approval of medical ML products.

Pioneering noninvasive colorectal cancer detection with an AI-enhanced breath volatilomics platform

Background: The sensitivity and specificity of current breath biomarkers are often inadequate for effective cancer screening, particularly in colorectal cancer (CRC). While a few exhaled biomarkers in CRC exhibit high specificity, they lack the requisite sensitivity for early-stage detection, thereby limiting improvements in patient survival rates. Methods: In this study, we developed an advanced Mass Spectrometry-based volatilomics platform, complemented by an enhanced breath sampler. The platform integrates artificial intelligence (AI)-assisted algorithms to detect multiple volatile organic compounds (VOCs) biomarkers in human breath. Subsequently, we applied this platform to analyze 364 clinical CRC and normal exhaled samples. Results: The diagnostic signatures, including 2-methyl, octane, and butyric acid, generated by the platform effectively discriminated CRC patients from normal controls with high sensitivity (89.7%), specificity (86.8%), and accuracy (AUC = 0.91). Furthermore, the metastatic signature correctly identified over 50% of metastatic patients who tested negative for carcinoembryonic antigen (CEA). Fecal validation indicated that elevated breath biomarkers correlated with an inflammatory response guided by Bacteroides fragilis in CRC. Conclusion: This study introduces a sophisticated AI-aided Mass Spectrometry-based platform capable of identifying novel and feasible breath biomarkers for early-stage CRC detection. The promising results position the platform as an efficient noninvasive screening test for clinical applications, offering potential advancements in early detection and improved survival rates for CRC patients.

Good machine learning practices: Learnings from the modern pharmaceutical discovery enterprise

Machine Learning (ML) and Artificial Intelligence (AI) have become an integral part of the drug discovery and development value chain. Many teams in the pharmaceutical industry nevertheless report the challenges associated with the timely, cost effective and meaningful delivery of ML and AI powered solutions for their scientists. We sought to better understand what these challenges were and how to overcome them by performing an industry wide assessment of the practices in AI and Machine Learning. Here we report results of the systematic business analysis of the personas in the modern pharmaceutical discovery enterprise in relation to their work with the AI and ML technologies. We identify 23 common business problems that individuals in these roles face when they encounter AI and ML technologies at work, and describe best practices (Good Machine Learning Practices) that address these issues.

Node-degree aware edge sampling mitigates inflated classification performance in biomedical random walk-based graph representation learning

Motivation

Graph representation learning is a family of related approaches that learn low-dimensional vector representations of nodes and other graph elements called embeddings. Embeddings approximate characteristics of the graph and can be used for a variety of machine-learning tasks such as novel edge prediction. For many biomedical applications, partial knowledge exists about positive edges that represent relationships between pairs of entities, but little to no knowledge is available about negative edges that represent the explicit lack of a relationship between two nodes. For this reason, classification procedures are forced to assume that the vast majority of unlabeled edges are negative. Existing approaches to sampling negative edges for training and evaluating classifiers do so by uniformly sampling pairs of nodes.

Results

We show here that this sampling strategy typically leads to sets of positive and negative examples with imbalanced node degree distributions. Using representative heterogeneous biomedical knowledge graph and random walk-based graph machine learning, we show that this strategy substantially impacts classification performance. If users of graph machine-learning models apply the models to prioritize examples that are drawn from approximately the same distribution as the positive examples are, then performance of models as estimated in the validation phase may be artificially inflated. We present a degree-aware node sampling approach that mitigates this effect and is simple to implement.

Availability and implementation

Our code and data are publicly available at https://github.com/monarch-initiative/negativeExampleSelection.

Accelerating the Discovery and Design of Antimicrobial Peptides with Artificial Intelligence

Peptides modulate many processes of human physiology targeting ion channels, protein receptors, or enzymes. They represent valuable starting points for the development of new biologics against communicable and non-communicable disorders. However, turning native peptide ligands into druggable materials requires high selectivity and efficacy, predictable metabolism, and good safety profiles. Machine learning models have gradually emerged as cost-effective and time-saving solutions to predict and generate new proteins with optimal properties. In this chapter, we will discuss the evolution and applications of predictive modeling and generative modeling to discover and design safe and effective antimicrobial peptides. We will also present their current limitations and suggest future research directions, applicable to peptide drug design campaigns.

How is Big Data reshaping preclinical aging research?

The exponential scientific and technological progress during the past 30 years has favored the comprehensive characterization of aging processes with their multivariate nature, leading to the advent of Big Data in preclinical aging research. Spanning from molecular omics to organism-level deep phenotyping, Big Data demands large computational resources for storage and analysis, as well as new analytical tools and conceptual frameworks to gain novel insights leading to discovery. Systems biology has emerged as a paradigm that utilizes Big Data to gain insightful information enabling a better understanding of living organisms, visualized as multilayered networks of interacting molecules, cells, tissues and organs at different spatiotemporal scales. In this framework, where aging, health and disease represent emergent states from an evolving dynamic complex system, context given by, for example, strain, sex and feeding times, becomes paramount for defining the biological trajectory of an organism. Using bioinformatics and artificial intelligence, the systems biology approach is leading to remarkable advances in our understanding of the underlying mechanism of aging biology and assisting in creative experimental study designs in animal models. Future in-depth knowledge acquisition will depend on the ability to fully integrate information from different spatiotemporal scales in organisms, which will probably require the adoption of theories and methods from the field of complex systems. Here we review state-of-the-art approaches in preclinical research, with a focus on rodent models, that are leading to conceptual and/or technical advances in leveraging Big Data to understand basic aging biology and its full translational potential.

Enabling interpretable machine learning for biological data with reliability scores

Machine learning tools have proven useful across biological disciplines, allowing researchers to draw conclusions from large datasets, and opening up new opportunities for interpreting complex and heterogeneous biological data. Alongside the rapid growth of machine learning, there have also been growing pains: some models that appear to perform well have later been revealed to rely on features of the data that are artifactual or biased; this feeds into the general criticism that machine learning models are designed to optimize model performance over the creation of new biological insights. A natural question arises: how do we develop machine learning models that are inherently interpretable or explainable? In this manuscript, we describe the SWIF(r) reliability score (SRS), a method building on the SWIF(r) generative framework that reflects the trustworthiness of the classification of a specific instance. The concept of the reliability score has the potential to generalize to other machine learning methods. We demonstrate the utility of the SRS when faced with common challenges in machine learning including: 1) an unknown class present in testing data that was not present in training data, 2) systemic mismatch between training and testing data, and 3) instances of testing data that have missing values for some attributes. We explore these applications of the SRS using a range of biological datasets, from agricultural data on seed morphology, to 22 quantitative traits in the UK Biobank, and population genetic simulations and 1000 Genomes Project data. With each of these examples, we demonstrate how the SRS can allow researchers to interrogate their data and training approach thoroughly, and to pair their domain-specific knowledge with powerful machine-learning frameworks. We also compare the SRS to related tools for outlier and novelty detection, and find that it has comparable performance, with the advantage of being able to operate when some data are missing. The SRS, and the broader discussion of interpretable scientific machine learning, will aid researchers in the biological machine learning space as they seek to harness the power of machine learning without sacrificing rigor and biological insight.

Challenges in translational machine learning

Machine learning (ML) algorithms are increasingly being used to help implement clinical decision support systems. In this new field, we define as "translational machine learning", joint efforts and strong communication between data scientists and clinicians help to span the gap between ML and its adoption in the clinic. These collaborations also improve interpretability and trust in translational ML methods and ultimately aim to result in generalizable and reproducible models. To help clinicians and bioinformaticians refine their translational ML pipelines, we review the steps from model building to the use of ML in the clinic. We discuss experimental setup, computational analysis, interpretability and reproducibility, and emphasize the challenges involved. We highly advise collaboration and data sharing between consortia and institutes to build multi-centric cohorts that facilitate ML methodologies that generalize across centers. In the end, we hope that this review provides a way to streamline translational ML and helps to tackle the challenges that come with it.

Systematic indication extension for drugs using patient stratification insights generated by combinatorial analytics

Indication extension or repositioning of drugs can, if done well, provide a faster, cheaper, and derisked route to the approval of new therapies, creating new options to address pockets of unmet medical need for patients and offering the potential for significant commercial and clinical benefits. We look at the promises and challenges of different repositioning strategies and the disease insights and scalability that new high-resolution patient stratification methodologies can bring. This is exemplified by a systematic analysis of all development candidates and on-market drugs, which identified 477 indication extension opportunities across 30 chronic disease areas, each supported by patient stratification biomarkers. This illustrates the potential that new artificial intelligence (AI) and combinatorial analytics methods have to enhance the rate and cost of innovation across the drug discovery industry.

Navigating the pitfalls of applying machine learning in genomics

The scale of genetic, epigenomic, transcriptomic, cheminformatic and proteomic data available today, coupled with easy-to-use machine learning (ML) toolkits, has propelled the application of supervised learning in genomics research. However, the assumptions behind the statistical models and performance evaluations in ML software frequently are not met in biological systems. In this Review, we illustrate the impact of several common pitfalls encountered when applying supervised ML in genomics. We explore how the structure of genomics data can bias performance evaluations and predictions. To address the challenges associated with applying cutting-edge ML methods to genomics, we describe solutions and appropriate use cases where ML modelling shows great potential.

Modeling in systems biology: Causal understanding before prediction?

Babur et al. (2021) developed the CausalPath tool to infer causal signaling interactions in high-throughput proteomics data that may foster mechanical understanding from large-scale biological datasets.

Protein sequence design with deep generative models

Protein engineering seeks to identify protein sequences with optimized properties. When guided by machine learning, protein sequence generation methods can draw on prior knowledge and experimental efforts to improve this process. In this review, we highlight recent applications of machine learning to generate protein sequences, focusing on the emerging field of deep generative methods.

Deep Automation Bias: How to Tackle a Wicked Problem of AI?

The increasing use of AI in different societal contexts intensified the debate on risks, ethical problems and bias. Accordingly, promising research activities focus on debiasing to strengthen fairness, accountability and transparency in machine learning. There is, though, a tendency to fix societal and ethical issues with technical solutions that may cause additional, wicked problems. Alternative analytical approaches are thus needed to avoid this and to comprehend how societal and ethical issues occur in AI systems. Despite various forms of bias, ultimately, risks result from eventual rule conflicts between the AI system behavior due to feature complexity and user practices with limited options for scrutiny. Hence, although different forms of bias can occur, automation is their common ground. The paper highlights the role of automation and explains why deep automation bias (DAB) is a metarisk of AI. Based on former work it elaborates the main influencing factors and develops a heuristic model for assessing DAB-related risks in AI systems. This model aims at raising problem awareness and training on the sociotechnical risks resulting from AI-based automation and contributes to improving the general explicability of AI systems beyond technical issues.