PyMC: a modern, and comprehensive probabilistic programming framework in Python

Bayesian Mechanistic Inference, Statistical Mechanics, and a New Era for Monte Carlo

On the one hand, much of computational chemistry is concerned with "bottom-up" calculations which elucidate observable behavior starting from exact or approximated physical laws, a paradigm exemplified by typical quantum mechanical calculations and molecular dynamics simulations. On the other hand, "top down" computations aiming to formulate mathematical models consistent with observed data, e.g., parametrizing force fields, binding or kinetic models, have been of interest for decades but recently have grown in sophistication with the use of Bayesian inference (BI). Standard BI provides an estimation of parameter values, uncertainties, and correlations among parameters. Used for "model selection," BI can also distinguish between model structures such as the presence or absence of individual states and transitions. Fortunately for physical scientists, BI can be formulated within a statistical mechanics framework, and indeed, BI has led to a resurgence of interest in Monte Carlo (MC) algorithms, many of which have been directly adapted from or inspired by physical strategies. Certain MC algorithms─notably procedures using an "infinite temperature" reference state─can be successful in a 5-20 parameter BI context which would be unworkable in molecular spaces of 103 coordinates and more. This Review provides a pedagogical introduction to BI and reviews key aspects of BI through a physical lens, setting the computations in terms of energy landscapes and free energy calculations and describing promising sampling algorithms. Statistical mechanics and basic probability theory also provide a reference for understanding intrinsic limitations of Bayesian inference with regard to model selection and the choice of priors.

Emotional State Transitions in Trauma-Exposed Individuals With and Without Posttraumatic Stress Disorder

Importance

Posttraumatic stress disorder (PTSD) is marked by the contrasting symptoms of hyperemotional reactivity and emotional numbing (ie, reduced emotional reactivity). Comprehending the mechanism that governs the transition between neutral and negative emotional states is crucial for developing targeted therapeutic strategies.

Objectives

To explore whether individuals with PTSD experience a more pronounced shift between neutral and negative emotional states and how the intensity of emotional numbing symptoms impacts this shift.

Design, Setting, and Participants

This cross-sectional study used hierarchical bayesian modeling to fit a 5-parameter logistic regression to analyze the valence ratings of images. The aim was to compare the curve's slope between groups and explore its association with the severity of emotional numbing symptoms. The study was conducted online, using 35 images with a valence range from highly negative to neutral. The rating of these images was used to assess the emotional responses of the participants. The study recruited trauma-exposed individuals (witnessed or experienced life-threatening incident, violent assault, or someone being killed) between January 17 and March 8, 2023. Participants completed the PTSD Checklist for the Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) (DSM-5) (PCL-5).

Exposure

On the basis of DSM-5 criteria (endorsing at least 1 symptom from clusters B and C and 2 from D and E), participants were categorized as having probable PTSD (pPTSD) or as trauma-exposed controls (TECs).

Main Outcomes and Measures

The main outcome was the slope parameter (b) of the logistic curve fitted to the valence rating. The slope parameter indicates the rate at which emotional response intensity changes with stimulus valence, reflecting how quickly the transition occurs between neutral and negatively valenced states. The secondary outcome was the association between emotional numbing (PCL-5 items 12-14) and the slope parameter.

Results

A total of 1440 trauma-exposed individuals were included. The pPTSD group (n = 445) was younger (mean [SD] age, 36.1 [10.9] years) compared with the TEC group (mean [SD] age, 41.5 [13.3] years; P < .001). Sex distribution (427 women in the TEC group vs 230 in the pPTSD group) did not significantly differ between groups (P = .67). The pPTSD group exhibited a steeper slope (mean slope difference, -0.255; 89% highest posterior density [HPD], -0.340 to -0.171) compared with the controls. Across all individuals (n = 1440), a robust association was found between the slope and emotional numbing severity (mean [SD] additive value, 0.100 [0.031]; 89% HPD, 0.051-0.15). Additional analysis controlling for age confirmed the association between emotional numbing and transition sharpness (mean [SD] additive value, 0.108 [0.032]; 89% HPD, 0.056-0.159), without evidence of an age-related association (mean [SD] additive value, 0.031 [0.033]; 89% HPD, -0.022 to 0.083).

Conclusions and Relevance

These findings support that individuals with PTSD undergo rapid transitions between neutral and negative emotional states, a phenomenon intensified by the severity of emotional numbing symptoms. Therapeutic interventions aimed at moderating these swift emotional transitions could potentially alleviate PTSD symptoms.

Differences in Discounting Behavior and Brain Responses for Food and Money Reward

Most neuroeconomic research seeks to understand how value influences decision-making. The influence of reward type is less well understood. We used functional magnetic resonance imaging (fMRI) to investigate delay discounting of primary (i.e., food) and secondary rewards (i.e., money) in 28 healthy, normal-weighted participants (mean age = 26.77; 18 females). To decipher differences in discounting behavior between reward types, we compared how well-different option-based statistical models (exponential, hyperbolic discounting) and attribute-wise heuristic choice models (intertemporal choice heuristic, dual reasoning and implicit framework theory, trade-off model) captured the reward-specific discounting behavior. Contrary to our hypothesis of different strategies for different rewards, we observed comparable discounting behavior for money and food (i.e., exponential discounting). Higher k values for food discounting suggest that individuals decide more impulsive if confronted with food. The fMRI revealed that money discounting was associated with enhanced activity in the right dorsolateral prefrontal cortex, involved in executive control; the right dorsal striatum, associated with reward processing; and the left hippocampus, involved in memory encoding/retrieval. Food discounting, instead, was associated with higher activity in the left temporoparietal junction suggesting social reinforcement of food decisions. Although our findings do not confirm our hypothesis of different discounting strategies for different reward types, they are in line with the notion that reward types have a significant influence on impulsivity with primary rewards leading to more impulsive choices.

Estimation of Peptide Helicity from Circular Dichroism Using the Ensemble Model

An established method for the quantitation of the helix content in peptides using circular dichroism (CD) relies on the linear spectroscopic model. This model assumes an average value of the helix-length correction for all peptide conformers, irrespective of the length of the helical segment. Here we assess the validity of this approximation and introduce a more physically realistic ensemble-based analysis of the CD signal in which the length correction is assigned specifically to each ensemble conformer. We demonstrate that the linear model underestimates peptide helicity, with the difference depending on the ensemble composition. We developed a computer program that implements the ensemble model to estimate the peptide helicity. Using this model and the CD data set covering a broad range of helicities, we recalibrate CD baseline parameters and redetermine helix-coil parameters for the alanine-rich peptide. We show that the ensemble model leverages small differences in signal between conformers to extract more information from the experimental data, enabling the determination of several poorly defined quantities, such as the nucleation constant and heat capacity change associated with helix folding. Overall, the presented ensemble-based treatment of the CD signal, together with the recalibrated values of the spectroscopic baseline parameters, provides a coherent framework for the analysis of the peptide helix content.

Understanding the impact of numerical solvers on inference for differential equation models

Most ordinary differential equation (ODE) models used to describe biological or physical systems must be solved approximately using numerical methods. Perniciously, even those solvers that seem sufficiently accurate for the forward problem, i.e. for obtaining an accurate simulation, might not be sufficiently accurate for the inverse problem, i.e. for inferring the model parameters from data. We show that for both fixed step and adaptive step ODE solvers, solving the forward problem with insufficient accuracy can distort likelihood surfaces, which might become jagged, causing inference algorithms to get stuck in local 'phantom' optima. We demonstrate that biases in inference arising from numerical approximation of ODEs are potentially most severe in systems involving low noise and rapid nonlinear dynamics. We reanalyse an ODE change point model previously fit to the COVID-19 outbreak in Germany and show the effect of the step size on simulation and inference results. We then fit a more complicated rainfall run-off model to hydrological data and illustrate the importance of tuning solver tolerances to avoid distorted likelihood surfaces. Our results indicate that, when performing inference for ODE model parameters, adaptive step size solver tolerances must be set cautiously and likelihood surfaces should be inspected for characteristic signs of numerical issues.

Transfer Learning Bayesian Optimization to Design Competitor DNA Molecules for Use in Diagnostic Assays

With the rise in engineered biomolecular devices, there is an increased need for tailor-made biological sequences. Often, many similar biological sequences need to be made for a specific application meaning numerous, sometimes prohibitively expensive, lab experiments are necessary for their optimization. This paper presents a transfer learning design of experiments workflow to make this development feasible. By combining a transfer learning surrogate model with Bayesian optimization, we show how the total number of experiments can be reduced by sharing information between optimization tasks. We demonstrate the reduction in the number of experiments using data from the development of DNA competitors for use in an amplification-based diagnostic assay. We use cross-validation to compare the predictive accuracy of different transfer learning models, and then compare the performance of the models for both single objective and penalized optimization tasks.

"High-throughput screening of catalytically active inclusion bodies using laboratory automation and Bayesian optimization"

BACKGROUND

In recent years, the production of inclusion bodies that retain substantial catalytic activity was demonstrated. These catalytically active inclusion bodies (CatIBs) are formed by genetic fusion of an aggregation-inducing tag to a gene of interest via short linker polypeptides. The resulting CatIBs are known for their easy and cost-efficient production, recyclability as well as their improved stability. Recent studies have outlined the cooperative effects of linker and aggregation-inducing tag on CatIB activities. However, no a priori prediction is possible so far to indicate the best combination thereof. Consequently, extensive screening is required to find the best performing CatIB variant.

RESULTS

In this work, a semi-automated cloning workflow was implemented and used for fast generation of 63 CatIB variants with glucose dehydrogenase of Bacillus subtilis (BsGDH). Furthermore, the variant BsGDH-PT-CBDCell was used to develop, optimize and validate an automated CatIB screening workflow, enhancing the analysis of many CatIB candidates in parallel. Compared to previous studies with CatIBs, important optimization steps include the exclusion of plate position effects in the BioLector by changing the cultivation temperature. For the overall workflow including strain construction, the manual workload could be reduced from 59 to 7 h for 48 variants (88%). After demonstration of high reproducibility with 1.9% relative standard deviation across 42 biological replicates, the workflow was performed in combination with a Bayesian process model and Thompson sampling. While the process model is crucial to derive key performance indicators of CatIBs, Thompson sampling serves as a strategy to balance exploitation and exploration in screening procedures. Our methodology allowed analysis of 63 BsGDH-CatIB variants within only three batch experiments. Because of the high likelihood of TDoT-PT-BsGDH being the best CatIB performer, it was selected in 50 biological replicates during the three screening rounds, much more than other, low-performing variants.

CONCLUSIONS

At the current state of knowledge, every new enzyme requires screening for different linker/aggregation-inducing tag combinations. For this purpose, the presented CatIB toolbox facilitates fast and simplified construction and screening procedures. The methodology thus assists in finding the best CatIB producer from large libraries in short time, rendering possible automated Design-Build-Test-Learn cycles to generate structure/function learnings.

copepodTCR: Identification of Antigen-Specific T Cell Receptors with combinatorial peptide pooling

T cell receptor (TCR) repertoire diversity enables the orchestration of antigen-specific immune responses against the vast space of possible pathogens. Identifying TCR/antigen binding pairs from the large TCR repertoire and antigen space is crucial for biomedical research. Here, we introduce copepodTCR, an open-access tool for the design and interpretation of high-throughput experimental assays to determine TCR specificity. copepodTCR implements a combinatorial peptide pooling scheme for efficient experimental testing of T cell responses against large overlapping peptide libraries, useful for "deorphaning" TCRs of unknown specificity. The scheme detects experimental errors and, coupled with a hierarchical Bayesian model for unbiased results interpretation, identifies the response-eliciting peptide for a TCR of interest out of hundreds of peptides tested using a simple experimental set-up. We experimentally validated our approach on a library of 253 overlapping peptides covering the SARS-CoV-2 spike protein. We provide experimental guides for efficient design of larger screens covering thousands of peptides which will be crucial for the identification of antigen-specific T cells and their targets from limited clinical material.

Supercritical and homogenous transmission of monkeypox in the capital of China

Starting from May 31, 2023, the local transmission of monkeypox (Mpox) in mainland China began in Beijing. Till now, the transmission characteristics have not been explored. Based on the daily Mpox incidence data in the first 3 weeks of Beijing (from May 31 to June 21, 2023), we employed the instant-individual heterogeneity transmission model to simultaneously calculate the effective reproduction number (Re ) and the degree of heterogeneity (k) of the Beijing epidemic. We additionally simulated the monthly infection size in Beijing from July to November and compared with the reported data to project subsequent transmission dynamics. We estimated Re to be 1.68 (95% highest posterior density [HPD]: 1.12-2.41), and k to be 2.57 [95% HPD: 0.54-83.88], suggesting the transmission of Mpox in Beijing was supercritical and didn't have considerable transmission heterogeneity. We projected that Re fell in the range of 0.95-1.0 from July to November, highlighting more efforts needed to further reduce the Mpox transmissibility. Our findings revealed supercritical and homogeneous transmission of the Mpox epidemic in Beijing. Our results could serve as a reference for understanding and predicting the ongoing Mpox transmission in other regions of China and evaluating the effect of control measures.

Predicting the Length of Stay of Cardiac Patients Based on Pre-Operative Variables-Bayesian Models vs. Machine Learning Models

Length of stay (LoS) prediction is deemed important for a medical institution's operational and logistical efficiency. Sound estimates of a patient's stay increase clinical preparedness and reduce aberrations. Various statistical methods and techniques are used to quantify and predict the LoS of a patient based on pre-operative clinical features. This study evaluates and compares the results of Bayesian (simple Bayesian regression and hierarchical Bayesian regression) models and machine learning (ML) regression models against multiple evaluation metrics for the problem of LoS prediction of cardiac patients admitted to Tabba Heart Institute, Karachi, Pakistan (THI) between 2015 and 2020. In addition, the study also presents the use of hierarchical Bayesian regression to account for data variability and skewness without homogenizing the data (by removing outliers). LoS estimates from the hierarchical Bayesian regression model resulted in a root mean squared error (RMSE) and mean absolute error (MAE) of 1.49 and 1.16, respectively. Simple Bayesian regression (without hierarchy) achieved an RMSE and MAE of 3.36 and 2.05, respectively. The average RMSE and MAE of ML models remained at 3.36 and 1.98, respectively.

Spatial and temporal heterogeneity alter the cost of plasticity in Pristionchus pacificus

Phenotypic plasticity, the ability of a single genotype to produce distinct phenotypes under different environmental conditions, has become a leading concept in ecology and evolutionary biology, with the most extreme examples being the formation of alternative phenotypes (polyphenisms). However, several aspects associated with phenotypic plasticity remain controversial, such as the existence of associated costs. While already predicted by some of the pioneers of plasticity research, i.e. Schmalhausen and Bradshaw, experimental and theoretical approaches have provided limited support for the costs of plasticity. In experimental studies, one common restriction is the measurement of all relevant parameters over long time periods. Similarly, theoretical studies rarely use modelling approaches that incorporate specific experimentally-derived fitness parameters. Therefore, the existence of the costs of plasticity remains disputed. Here, we provide an integrative approach to understand the cost of adaptive plasticity and its ecological ramifications, by combining laboratory data from the nematode plasticity model system Pristionchus pacificus with a stage-structured population model. Taking advantage of measurements of two isogenic strains grown on two distinct diets, we illustrate how spatial and temporal heterogeneity with regard to the distribution of resources on a metapopulation can alter the outcome of the competition and alleviate the realized cost of plasticity.

Comprehensive Screening of a Light-Inducible Split Cre Recombinase with Domain Insertion Profiling

Splitting proteins with light- or chemically inducible dimers provides a mechanism for post-translational control of protein function. However, current methods for engineering stimulus-responsive split proteins often require significant protein engineering expertise and the laborious screening of individual constructs. To address this challenge, we use a pooled library approach that enables rapid generation and screening of nearly all possible split protein constructs in parallel, where results can be read out by using sequencing. We perform our method on Cre recombinase with optogenetic dimers as a proof of concept, resulting in comprehensive data on the split sites throughout the protein. To improve the accuracy in predicting split protein behavior, we develop a Bayesian computational approach to contextualize errors inherent to experimental procedures. Overall, our method provides a streamlined approach for achieving inducible post-translational control of a protein of interest.

SHIP: identifying antimicrobial resistance gene transfer between plasmids

MOTIVATION

Plasmids are carriers for antimicrobial resistance (AMR) genes and can exchange genetic material with other structures, contributing to the spread of AMR. There is no reliable approach to identify the transfer of AMR genes across plasmids. This is mainly due to the absence of a method to assess the phylogenetic distance of plasmids, as they show large DNA sequence variability. Identifying and quantifying such transfer can provide novel insight into the role of small mobile elements and resistant plasmid regions in the spread of AMR.

RESULTS

We developed SHIP, a novel method to quantify plasmid similarity based on the dynamics of plasmid evolution. This allowed us to find conserved fragments containing AMR genes in structurally different and phylogenetically distant plasmids, which is evidence for lateral transfer. Our results show that regions carrying AMR genes are highly mobilizable between plasmids through transposons, integrons, and recombination events, and contribute to the spread of AMR. Identified transferred fragments include a multi-resistant complex class 1 integron in Escherichia coli and Klebsiella pneumoniae, and a region encoding tetracycline resistance transferred through recombination in Enterococcus faecalis.

AVAILABILITY AND IMPLEMENTATION

The code developed in this work is available at https://github.com/AbeelLab/plasmidHGT.

Interpretable neural architecture search and transfer learning for understanding CRISPR/Cas9 off-target enzymatic reactions

Finely-tuned enzymatic pathways control cellular processes, and their dysregulation can lead to disease. Creating predictive and interpretable models for these pathways is challenging because of the complexity of the pathways and of the cellular and genomic contexts. Here we introduce Elektrum, a deep learning framework which addresses these challenges with data-driven and biophysically interpretable models for determining the kinetics of biochemical systems. First, it uses in vitro kinetic assays to rapidly hypothesize an ensemble of high-quality Kinetically Interpretable Neural Networks (KINNs) that predict reaction rates. It then employs a novel transfer learning step, where the KINNs are inserted as intermediary layers into deeper convolutional neural networks, fine-tuning the predictions for reaction-dependent in vivo outcomes. Elektrum makes effective use of the limited, but clean in vitro data and the complex, yet plentiful in vivo data that captures cellular context. We apply Elektrum to predict CRISPR-Cas9 off-target editing probabilities and demonstrate that Elektrum achieves state-of-the-art performance, regularizes neural network architectures, and maintains physical interpretability.

Associations of HIV and iron status with gut microbiota composition, gut inflammation and gut integrity in South African school-age children: a two-way factorial case-control study

BACKGROUND

Human immunodeficiency virus (HIV) and iron deficiency (ID) affect many African children. Both HIV and iron status interact with gut microbiota composition and related biomarkers. The study's aim was to determine the associations of HIV and iron status with gut microbiota composition, gut inflammation and gut integrity in South African school-age children.

METHODS

In this two-way factorial case-control study, 8- to 13-year-old children were enrolled into four groups based on their HIV and iron status: (1) With HIV (HIV+) and ID (n = 43), (2) HIV+ and iron-sufficient nonanaemic (n = 41), (3) without HIV (HIV-) and ID (n = 44) and (4) HIV- and iron-sufficient nonanaemic (n = 38). HIV+ children were virally suppressed (<50 HIV RNA copies/ml) on antiretroviral therapy (ART). Microbial composition of faecal samples (16S rRNA sequencing) and markers of gut inflammation (faecal calprotectin) and gut integrity (plasma intestinal fatty acid-binding protein [I-FABP]) were assessed.

RESULTS

Faecal calprotectin was higher in ID versus iron-sufficient nonanaemic children (p = 0.007). I-FABP did not significantly differ by HIV or iron status. ART-treated HIV (redundancy analysis [RDA] R2  = 0.009, p = 0.029) and age (RDA R2  = 0.013 p = 0.004) explained the variance in the gut microbiota across the four groups. Probabilistic models showed that the relative abundance of the butyrate-producing genera Anaerostipes and Anaerotruncus was lower in ID versus iron-sufficient children. Fusicatenibacter was lower in HIV+ and in ID children versus their respective counterparts. The prevalence of the inflammation-associated genus Megamonas was 42% higher in children with both HIV and ID versus HIV- and iron-sufficient nonanaemic counterparts.

CONCLUSIONS

In our sample of 8- to 13-year-old virally suppressed HIV+ and HIV- children with or without ID, ID was associated with increased gut inflammation and changes in the relative abundance of specific microbiota. Moreover, in HIV+ children, ID had a cumulative effect that further shifted the gut microbiota to an unfavourable composition.

Optimizing combination therapy in a murine model of HER2+ breast cancer

Human epidermal growth factor receptor 2 positive (HER2+) breast cancer is frequently treated with drugs that target the HER2 receptor, such as trastuzumab, in combination with chemotherapy, such as doxorubicin. However, an open problem in treatment design is to determine the therapeutic regimen that optimally combines these two treatments to yield optimal tumor control. Working with data quantifying temporal changes in tumor volume due to different trastuzumab and doxorubicin treatment protocols in a murine model of human HER2+ breast cancer, we propose a complete framework for model development, calibration, selection, and treatment optimization to find the optimal treatment protocol. Through different assumptions for the drug-tumor interactions, we propose ten different models to characterize the dynamic relationship between tumor volume and drug availability, as well as the drug-drug interaction. Using a Bayesian framework, each of these models are calibrated to the dataset and the model with the highest Bayesian information criterion weight is selected to represent the biological system. The selected model captures the inhibition of trastuzumab due to pre-treatment with doxorubicin, as well as the increase in doxorubicin efficacy due to pre-treatment with trastuzumab. We then apply optimal control theory (OCT) to this model to identify two optimal treatment protocols. In the first optimized protocol, we fix the maximum dosage for doxorubicin and trastuzumab to be the same as the maximum dose delivered experimentally, while trying to minimize tumor burden. Within this constraint, optimal control theory indicates the optimal regimen is to first deliver two doses of trastuzumab on days 35 and 36, followed by two doses of doxorubicin on days 37 and 38. This protocol predicts an additional 45% reduction in tumor burden compared to that achieved with the experimentally delivered regimen. In the second optimized protocol we fix the tumor control to be the same as that obtained experimentally, and attempt to reduce the doxorubicin dose. Within this constraint, the optimal regimen is the same as the first optimized protocol but uses only 43% of the doxorubicin dose used experimentally. This protocol predicts tumor control equivalent to that achieved experimentally. These results strongly suggest the utility of mathematical modeling and optimal control theory for identifying therapeutic regimens maximizing efficacy and minimizing toxicity.

BayesLDM: A Domain-specific Modeling Language for Probabilistic Modeling of Longitudinal Data

In this paper we present BayesLDM, a library for Bayesian longitudinal data modeling consisting of a high-level modeling language with specific features for modeling complex multivariate time series data coupled with a compiler that can produce optimized probabilistic program code for performing inference in the specified model. BayesLDM supports modeling of Bayesian network models with a specific focus on the efficient, declarative specification of dynamic Bayesian Networks (DBNs). The BayesLDM compiler combines a model specification with inspection of available data and outputs code for performing Bayesian inference for unknown model parameters while simultaneously handling missing data. These capabilities have the potential to significantly accelerate iterative modeling workflows in domains that involve the analysis of complex longitudinal data by abstracting away the process of producing computationally efficient probabilistic inference code. We describe the BayesLDM system components, evaluate the efficiency of representation and inference optimizations and provide an illustrative example of the application of the system to analyzing heterogeneous and partially observed mobile health data.

Utilising Surrogate Models to Approximate Cardiac Potentials when Solving Inverse Problems via Bayesian Techniques

Solving inverse problems is computationally expensive, if not infeasible, under specific scenarios. For example, many forward solutions are required when solving inverse problems using Bayesian techniques. In this work, a novel inference protocol is established, that can be used to infer the cardiac bidomain conductivities and the cardiac fibre rotation angle (bidomain parameters). This protocol uses a surrogate model, developed using generalised polynomial chaos techniques, to approximate cardiac potentials on a multi-electrode array. The resulting surrogate model is used in conjunction with Bayesian inference techniques to infer the bidomain parameters. A lower-order surrogate model (order three) can effectively characterise the influence of the extracellular conductivities and fibre rotation on the cardiac potentials; however, it is recommended that a higher-order surrogate model expansion of order seven be used to adequately characterise the influence of the intracellular conductivities as well. This seventh order surrogate model was successfully used to infer the extracellular conductivities and fibre rotation angle from a single set of synthetically generated noisy experimental potentials, while the intracellular conductivities were unable to be retrieved accurately under this scenario.